Deric Bownds

Chapter 8

Perceiving Mind

This chapter, the beginning of Part III of this book, launches our exploration of how the different agencies of our minds work and how they are assembled into a coherent whole. These agencies arise from the evolutionary and developmental histories discussed in Parts I and II. We have noted many examples of the modularity of our brain functions but have addressed only a fraction of the experimental evidence. This chapter and Chapters 9 through 11 deal with our everyday experiences of sensing, acting, having emotions, and using language, respectively. Breaking up the discussion into these chunks helps us sort things out somewhat, even though these faculties interact through so many feedback loops that to view them as separate is really an oversimplification.

Ecology of Sensing and Acting

What animals like ourselves are able to perceive is shaped by a long history of adaptation and co-evolution with a particular environment, a particular ecology. We have receptors for visible (to us) light between about 400 and 700 nanometers in wavelength. We are unable to see patterns of ultraviolet light visible to the moth, we do not have receptors for the infrared light that some snakes use to sense heat given off by prey, and we do not have the acute long-distance vision of a hawk. We cannot hear the sound of a dog whistle or follow the scent of an escaping prisoner. Compared to most mammals, our sense of smell is quite poor. On an evolutionary time scale, we, like other animal species, have developed sensing and acting routines that are suitable for our particular physical and social niches. From the flood of physical information that impinges on us---light, sound, smell, and so on---we generate the perceptions we need to answer the question "What do I do next?" In humans, as in the most simple animals, perception serves action. 1 The two are so intimately linked that it is a bit arbitrary to present them in two separate chapters.



The ultimate purpose of our perceptions is to make it more likely that we will act in ways that help transmit our genes to future generations. Perception serves action.


Faculties such as visual perception and movement are linked in a continuous loop both during the developmental wiring of the brain and in the adult brain. Motion informs and refines vision, which informs and refines motion. Vision has little meaning apart from its use in behavior. One striking example of this has been provided by experiments in which two groups of newborn kittens were treated differently. 2 One group could move about normally, and each of its members was harnessed to a simple carriage and basket that contained a member of the second group. The kittens in the second group had the same sort of visual input as the first, but they were passive---unable to test the relationship between their own movement and the environment. After a few weeks of this treatment, the first group behaved normally, but those who had been carried around failed several visual tests, such as avoiding steep drops. This experiment suggests that gaining knowledge of how to act in response to what is seen requires testing by the visual guidance of action and that such knowledge cannot be obtained just by passively extracting visual features from the environment. The elaborate reciprocal testing among developing brain, body, and external environment installs the filters and interpreters of our sensory world that are described in this chapter.

Perception Focuses on Change

Perception involves much more than just the brain, with its picture-like images and talking-to-oneself thoughts. What we know of external reality we learn through the way that reality affects our entire bodies. The outside world is registered in terms of the internal modifications it causes as we move about, changing the state of our viscera, our skin, and our autonomic nervous system, as well as our central nervous system. Our brains register what is happening "out there" as a set of rapidly flickering maps spread across our somatosensory and motor cortexes, as well as less topographical representations of our viscera in the brainstem and hypothalamus. One grounding reference for a sense of "self" appears to be a constantly renewed primordial representation of the body in action. 3

Our sensory systems are most interested in rapid changes in our environment, for it is these that we must respond to in order to survive. Think of how quickly you react when an unexpected shadow suddenly falls across your path. It is characteristic of our sensory receptors that dramatic alterations in a stimulus, such as onset and offset, are detected more easily than equal changes along a continuum. Recall the times when you have turned on a three-way lamp and then turned if off again. The sequence would have been something like: darkness -> 50 -> 100 -> 150 watts of light -> darkness. You notice dark -> 50 watts as a big effect; 50 -> 100 and 100 -> 150 seem much less; off is again a big effect. Similarly, you have a very clear response to first touching your finger to your forearm. If you then apply a slowly increasing steady pressure, and occasionally press harder briefly to notice the amount of change you can just detect, the brief press must become larger as the background pressure is increased. Each bit of stimulus has made your skin receptors less responsive to the next bit of stimulus. The system "resets itself" to a new level of sensitivity (in other words, it adapts) so that its response is appropriate to the background.

Our perception of a stimulus is seldom absolute but, rather, depends on what has gone before. For example, if you move your hands among hot, cold, and tepid bowls of water, tepid feels warm or cold depending on which direction you come from. (The same sort of demonstration can be done with a set of three increasing weights. 4 ) Although we are good at perceiving changes on the time scale of these moment-to-moment behaviors, we usually don't notice changes that are much slower. You might not be aware, for example, that air pollution in your city has increased slowly over a period of several years, until a friend who visits after being away for a long time says, "Boy, the air is a lot more polluted now than when I left." 5

Perception Is Filtered and Directed by Many Factors

Most of the sea of sound, odor, gravity, and light in which we are immersed is known to our brain/body but not explicitly perceived. What we are aware of perceiving is only a small fraction of the information registered by our sensory organs and processed apart from consciousness. In fact, imaging studies have revealed brain regions that are responsive to novelty outside our awareness. 6 Numerous psychological studies have documented that unconscious perceptions can influence subsequent cognitive and affective reactions. Our perceptions are formed as primary sensations pass through an elaborate series of filters and interpreters installed by our evolution and our individual development, as well as by the attention, arousal, and expectations at hand. 7 What we perceive is a mixture of what is there and what we expect.

The filtering processes of our brains help to guarantee that what gets through is relevant to the moment (when we are hungry and we look for restaurants, not gas stations, as we drive along). Anticipating a particular kind of stimulus actually increases blood flow to the part of the brain that processes that sensory modality. 8 Our perceptions can be heightened or suppressed by powerful ascending pathways that rise like a fountain from the brainstem and shower the cortex with arousal-regulating neurotransmitters such as serotonin, acetylcholine, dopamine, norepinephrine (also called noradrenaline), and histamine. 9 Our alertness and attention can be enhanced by drugs that stimulate norepinephrine release in the brain. 10 Brain imaging studies reveal locations in the cortex that are components of networks of attention and that also regulate orientation and vigilance. 11



Our stream of conscious awareness is just a fraction of a larger intelligence that selects what perceptions make it into awareness---an intelligence of which we are usually unaware.


As an example of filters and censors that can act on our vision and language, 12 consider being asked to finish the words S_X, SHI_, and F__K. If you reported six, shin, and fork, was the censorship bias in the reception or the response? Probably a mixture of both. Think of all the parts of your cortex that were lighting up during this exercise! Or consider the "cocktail party effect." When several conversations are going on around you, and you hear your name mentioned in one, you attend to that conversation immediately and tune out the others. Filters and censors like these ones suggest the presence of a feedback loop between cognitive processes of the frontal lobe and the more primary sensory cortexes that are registering all of the input.

Our assumptions and expectations can color our interpretation of what we hear and see. Consider the statements "I went to the nudist play" versus "I went to the new display" or "the prince of whales" versus "the prince of Wales." In this case, context enables us to differentiate between phrases that sound identical. Figure 8-1 offers an example of a shifting visual interpretation. In seeing the object shown on the left, we make the most simple assumption, that it is a hexagon, unless prior expectations tell us that it is another view of the object on the right.

Figure 8-1

Drawings that illustrate the distinction between sensation and perception. The figure on the left is usually taken as an octagon, but it could be interpreted as the cube on the right rotated to a different position. The cube on the right, which is usually called the Necker cube, can be seen as having its top surface receding either to the left or to the right.

Distinguishing Between Sensation and Perception

Look again at the object on the right in Figure 8-1. Most people initially register this experience as looking at the top of a square with the lower set of vertical lines defining a surface that stands nearest the observer. However, if we stare at the figure long enough, it will flip so that we seem to be looking up at the bottom of the box, with the same vertical lines now defining its back side. Of course, neither the object itself nor its image on our retinas is changing. This is only one of many examples that can be used to demonstrate the distinction between sensation (the direct sensory recording) and perception (what we take that recording to mean). Recall that in Chapter 2 we discussed the evolutionary origins of this distinction, as more complicated animals began to separate the pathways of "what is happening to me" (sensation) from "what is happening out there" (perception).

Such distinctions are now being observed at the level of the firing of single nerve cells whose activity can be correlated with the way an object is perceived. 13 It seems most likely that the rules for sorting out ambiguous visual images or linguistic utterances are formulated during the postnatal development of our brain circuitry. 14 It is very unlikely that an a priori set of rules that evolved as adaptations to our ancestral paleolithic environment could anticipate and accommodate the visual and linguistic ambiguities generated by our modern surroundings. Perception of three-dimensional forms requires experience and learning. 15 The sophisticated scene and feature analyses carried out by our visual brains (discussed below) are apparently constructed during development, as top-down processes determine which work best in meeting changing environmental demands.

A more dramatic demonstration of the brain's ability to generate perception distinct from sensation can be observed in blind humans by attaching to their backs a patch driven by a video camera that the subject controls, a camera whose output to the patch causes the stimulation of multiple points on the skin. Each point represents one small area of the image captured by the camera. Within hours, some subjects can learn to recognize common objects such as cups and telephones, to point accurately in space, to judge distance, and finally to use perspective and parallax to perceive external objects in a stable three-dimensional world. The patterns projected onto the skin develop no such "visual" content unless the individual is behaviorally active, directing the video camera via head, hand, or body movements. After a few hours, the person no longer interprets the skin sensations as being of the body but projects them into the space being explored by the body-directed "gaze" of the video camera. What develops is not necessarily understanding but a strategy for responding appropriately. Test subjects do not locate objects as lying up against their skin---any more than those of us with vision locate objects as lying up against the retina of our eyes. Instead, they perceive objects as being out there in space. Thus a tactile sensation of "what is happening to me" is converted into a vision-like perception of "what is happening out there." 16 More recently another form of sensory substitution, the conversion of visual images to sounds, has proven partially successful. 17



A vivid demonstration of the plasticity of sensation and perception comes from experiments in which blind subjects can learn to "see with their skin."


Perception of Spatial Relationships

Our brains use context to shape our perception of relationships. Consider the example shown in Figure 8-2. The two central circles are the same size, but they look different because of the relative size of the surrounding circles. In this case, there is a fascinating corollary: Although we consciously experience the central circles as differing in size, if we are asked to pick up the center discs, our grip immediately matches the true size of the objects. This demonstrates that the visual pathways in our brain that are required for skilled actions are separate from those that mediate our conscious perception. 18 These two pathways are described in more detail below.

Figure 8-2

Our subjective experience of the size of the middle circle is influenced by the size of the surrounding circles.

Perceptions frequently do not correspond to reality. Our stored images of familiar objects can be so strong that we see or hear what we think should be present rather that what actually is present. We (and all other animals) can act on the basis of caricatures of reality. Just a few aspects of a stimulus are usually sufficient to enable us (and induce us) to identify an object. (We are not unlike the male stickleback, a fish that will attack any object that vaguely resembles the red belly of another male.) We base our perception on best bets about what is out there. The point of have perceptions is not to be a perfect physical measuring device; it is to participate promptly in appropriate behaviors.



If you are ready to see a black ace of spades or a red ace of hearts, when a red ace of spades is shown, you are likely to see one of the two choices you have set for yourself rather than the card actually before you.


These examples illustrate that sensing is seldom naive but, rather, depends on context and comparison with a history of similar sensing. 19 The brain specializes in rough-and-ready detection of what's probably important, not necessarily what is really there, and it is willing to make a lot of mistakes. We can extrapolate from such simple examples to our more complicated behaviors. Bias and hunches born of our history influence the scanning mechanisms we use to watch out for changes or danger in the environment. In people who are constantly apprehensive, those mechanisms seem to have taken on a chronic and inappropriate negative bias. Even the most benign events are interpreted as threatening. The converse of this situation is the Pollyanna syndrome, where negative input is denied. Humans, as individuals and groups, frequently deny the relevance of information they don't want to deal with. On a larger scale, we see this response in the muted reactions that many people have to ethnic genocidal wars and to the environmental and population crises.

Visual Systems

The five major sensory systems (vision, olfaction, audition, sensation, and gustation) not only process separate sensory modalities but also extensively interact with and influence each other. For example, sound can alter our perception of visual motion, the auditory cortex is activated during silent lip reading, and vision and touch are closely linked in analyzing objects in our surroundings. We'll focus on vision here, though. 20 Humans are visual animals, and at least 60 percent of our cerebral cortexes are involved in some aspect of processing visual information. In the visual system we face directly the question "Where does it all come together---where is the I?" We study vision from the top down (psychological experiments) and from the bottom up (electrophysiological experiments), and the area where the two have not yet met provides some fascinating puzzles. Here we are touching on something not fully understood: the link between neuronal firing and our conscious experience.

Not only does vision interact with our other senses, but it also forms an ensemble with movements of body muscles in which perception and action are linked. Vision involves much more than passive perception. Contractions of muscles that move the iris, rotate the eyes, control the face muscles around the eyes, contact neck muscles, and bring about other somatic movements can all be integral components of the experience of vision.

Self-Experiment: A Look at Vision and Its Muscular Correlates

We can use a simple exercise to illustrate the linkage of vision to muscle movements and tension. The following is best done if you can have someone else read the instructions to you, but it will probably work if you read through the instructions yourself, paying attention to the places where you are asked to pause before continuing. Take it slowly; don't rush.

Start by trying to focus and concentrate very hard on the following line, imagining that you will have to reproduce it from memory and that you are given only 10 seconds to learn and remember the characters:


Now, stop. Move your forehead muscles. Are you frowning? Expand and contract the muscles around your eyes. What is their position now, and what was it when you were looking at the characters? Open and shut your jaw gently. Are the muscles loose or slightly clenched? Shrug your shoulders up and down. Are they relaxed or slightly elevated? Move your head slightly forward and back to stretch the muscles along the back of your neck. How tense are they? Now note your breathing. Take a deep breath, let it out, and see what pace of breathing seems natural as you resume.

You are likely to note numerous muscle contractions of which you are not usually aware. These patterns of contraction go along with your paying close, focused attention to a stimulus such as the characters you memorized.

Now, let's move on to another set of instructions, but try to remember the patterns of muscle tension that you just experienced. Read the following three instructions several times so that you can remember them, but do not read further. When you have the instructions in mind, you can continue with the experiment.

1. Rub your hands together to warm them, and then place your palms lightly over your eyes. Leave them there for at least 30 seconds. Let your eye muscles and breathing relax.

2. Place your hands in your lap, and, in your mind's eye, imagine a horizontal bar and slowly look back and forth from one end of the bar to the other. Then imagine a vertical bar, and do the same.

3. After several minutes of doing this, slowly open your eyes to receive the whole visual world without reaching out to focus on any particular item.

Once you have read these steps several times, do what they say before reading any further! Only then should you continue reading.

Now that you have taken all three steps, consider this question: How does your visual experience right now compare with what you were feeling just after you were asked you to focus on and remember the characters in the previous exercise? Are you more aware of the edges of your visual world? Does your vision feel more "open" than before? Check through the muscles you reviewed above. Do they feel harder or softer?

Some individuals who have completed this exercise with spoken instructions have reported that their visual world was very fuzzy, or even disappeared entirely, for a moment. This is similar to the experience most of us have had while listening to a boring sermon or lecture: We notice with a start that the visual scene was gone for a moment. If our eyes become completely still, the image of the external world on our retinas becomes stationary and we are temporarily blind. Seeing requires rapid saccadic movement of the eye muscles---that is, a constant darting about of the eyes that causes corresponding shifts of the visual field on the retinas. Visual sensing requires scanning and noting differences. We imagine we see a whole visual scene, but in fact our brains are attending to a series of rapid snapshots taken at each point where the eye rests as it shifts about. The point of the imagined movements along the horizontal and vertical bars was to get you to relax the chronic tonus of the muscles that usually move the eyes, in an effort to neutralize this scanning mechanism.

Perhaps you noted a difference between your focused vision at the start of the exercise and your more open vision at the end. A check of the muscles mentioned above would then illustrate some of the muscular correlates of vision focused on the details of a single object versus vision opened up to take in the whole visual world. In the former case, light is directed toward a small central region of our retinas called the fovea, which contains a high concentration of color-sensitive cone cells and is responsible for high-resolution vision (see Figure 8-3, below). In the latter case, the periphery of the retina is also engaged to take in the whole visual world, including its edges. Foveal vision is often correlated with the tensing of facial muscles and the contraction of flexor muscles along the front of the torso. The relaxing exercises were meant to engage input from more of the retina, letting more of the peripheral visual field that stimulates the edges of the retina be accessible to notice. This more open vision can correlate with relaxation of facial muscles and the front flexor muscles of the torso.



Some athletes who must keep track of complicated movements over their entire visual field report cultivating a more open, or holistic, seeing of the sort we are getting at here. If instead, they focus, they must move their heads to see as much, and the act of focusing inhibits the flexibility of somatic muscle movement. This kind of visual experience may account for some athletes reporting that they enter a trance ("a zone") during periods of peak performance. The feeling is that an entity largely outside of conscious awareness has taken over and is running the show.


Our Visual Brain

Now let's consider our visual brain, the subject of the most intense contemporary research in cognitive neuroscience. 21 Visual sensation and perception are the best understood of our mental faculties. 22 This system manages the ordinary but amazing feat of keeping our perceptions of external objects constant as we move about, even as the size, shape, color, and brightness of the objects projected onto our retinas is constantly changing. The visual system must receive a flickering, two-dimensional image from the retinas and render this image into some sort of three-dimensional version that can guide appropriate action.

Our elaborate visual cortexes are the latest installment in the evolution of seeing. They follow upon the transitions we discussed earlier: from light-sensitive patches of membrane to image-forming eyes that send information to complex neural networks. 23 Studying these cortexes in the cat and macaque monkey has revealed much of what we know about our own central visual pathways. Although we have been evolving separately from the macaque monkey for more than 15 million years, and thus must be cautious in comparing our visual brains, many areas of functional specialization seem to be similar in both. 24



Humans are primarily visual animals, relying much less than most other vertebrates on touch, smell, and hearing.


The visual system has been studied more intensively than other parts of the brain for several reasons. For one thing, it is relatively easy to describe exactly the input to the visual system---the form, intensity, motion, and color of light that hits our eyes and is then transmitted to our retinas. (It is much harder to describe the quantity of an odor we inhale and to pinpoint exactly where it acts.) Another reason why interest has focused on the visual system is that the anatomy of the structures involved in visual processing is better understood than that of any other part of the brain. A series of relay stations can be followed from the retina to deep in the brain. At each stage of the relay, one can sample the activities of adjacent nerve cells and find that they are spread out in a topographical representation of the external world. In the occipital cortex, for example, this representation has become distorted, as though you had taken the retinal image on a piece of paper and crumpled it up to make it fit on the foldings and twistings of the cortex.

The next section offers a brief description of visual pathways and the processing of visual information in parallel streams that deal with different aspects of visual images, such as form, motion, and color. This will set the stage for us to consider of how visual perception is accomplished.

Visual Information Is Processed in Parallel Streams

The lens of our eye projects an inverted image of the external world onto the retina, the thin film of nerve cells at the back of the eye (see Figure 8-3). This film contains millions of light-sensitive photoreceptor cells much like the grains in a photographic emulsion, and the pattern of their excitation corresponds precisely to the patterns being viewed. The photoreceptor cells then "talk" to a second layer of cells (bipolar cells), which in turn talk to a third layer of cells (ganglion cells) whose axons form the optic nerve that carries visual information toward the center of our brains. This nerve is a bundle or cable containing about a million ganglion cell axons. Electrical recordings show that each of these axons responds best to discrete small spots of light (or darkness); thus they are sometimes called spot detectors. Such recordings also show that the ganglion cells from which these axons derive specialize to report different kinds of stimuli.

Figure 8-3

Anatomy of the human eye and retina. (a) Like the lens of a camera, the lens of the eye casts an inverted image of external objects onto the retina. The image of the object on which we are most directly focusing falls on a central part of the retina called the fovea, which contains color-sensitive photoreceptors (cones) and is most important in daylight vision. The scene surrounding this central object projects to more peripheral parts of the retina that contain fewer cones and more rod photoreceptors. Rod cells are important in vision in dim light. (b) The retina itself is a thin tissue consisting of three layers of nerve cells, and light passes through these layers before actually striking the photoreceptors. Axons of the ganglion cells form the optic nerve, which carries information from retina to brain. Also illustrated are a few of the cells that can transmit information sideways in the retina.

Some ganglion cells with relatively large cell bodies (magnocellular cells, M) are especially sensitive to changes in a light stimulus and ultimately feed information to parts of the visual cortex that specialize in analyzing motion, location, size, and spatial relationships. Other ganglion cells with smaller cell bodies (parvocellular cells, P) specialize in color and high contrast, and their information is finally used by cortical areas that are involved in object recognition. The distinction between M and P pathways arises at the first information relay between photoreceptor cells and bipolar cells and then persists into the visual areas of the brain. You will encounter further references to M and P pathways later in this chapter.

As the optic nerves from both eyes travel toward the center of the head, they follow the pathway outlined in Figure 8-4, meeting each other in a crossing called the optic chiasm and then proceeding via the lateral geniculate nucleus to the V1 areas of the right and left hemispheres. (Look back at the view of these pathways shown in Figure 7-1.) Area V1 of the left visual cortex, which reports on the right visual world, and area V1 of the right visual cortex, which reports on the left visual world, would be just under your left and right forefingers if you placed them at the middle of the back of your head at about the level of the top of your ears. Recall from Chapter 7 that the two halves of our visual world are reconnected and integrated by means of the corpus callosum, the large bundle of nerve fibers that communicates information between the two hemispheres. We have already noted that cutting this bundle in some patients with intractable epilepsy caused them to become individuals who see the two halves of their visual world with two different "selves."

Figure 8-4

Information flows from retina to brain. Axons that report from the left sides of our two retinas, and thus relay information about the right-hand side of our visual world (our right visual field), proceed back toward our left hemispheres, where they connect to the left lateral geniculate nucleus, a part of the thalamus that is the brain's first nerve relay station for visual input. The axons that report from the right sides of our retinas (our left visual field) do the opposite, sending information about the left visual field to the right lateral geniculate nucleus. Alternating layers of the lateral geniculate structures get input from one or the other eye, and each of the layers is specialized to project information for either the M or the P pathway onto the visual cortex. 25 Information about the left and right halves of our visual world remains segregated as the two lateral geniculate nuclei then send information back to area V1 of the occipital cortex, the primary visual cortex. In this drawing, the optic nerves and lateral geniculate nuclei are shown larger than their actual sizes for clarity.



Certain developmental dyslexics who do poorly in tests that require rapid visual processing show abnormalities in the lateral geniculate magnocellular layers that process fast, low-contrast visual information. Also, imaging studies of such individuals show reduced activity in higher visual centers that process motion information. Some dyslexics have difficulty not only with rapid visual stimuli but also in processing sounds, which suggests a more general deficit in processing of rapidly changing sensory stimuli.


Form, Motion, and Color

Axons that have arrived at area V1 from the lateral geniculate nucleus make connections on neurons in the middle of the six cell layers of the cortex (at layer 4), and at this point they are still spot detectors, responding best to small spots of light or darkness just as ganglion cells of the retina and cells in the lateral geniculate nucleus do. However, a fascinating transformation can be observed by exploring the activity of cells in nearby layers of the cortex. An electrode inserted perpendicular to the cortex (into layers either higher or lower than layer 4) reveals cells that fire best not in response to a spot of light but, rather, when a small patch of retina is stimulated by a line of one particular orientation. Although the exact wiring that turns spot detectors into "line detectors" is not known, a simple hunch is probably the correct one. If a number of spot detectors arranged in a straight line (that is, cells that are ultimately getting information from a line of receptor cells on the retina) all "reported" to a single following cell, they would constitute a line detector. As the electrode that found the line detector is moved up and down through the cortical layers, it encounters cells with more complex properties. Some cells respond best not to lines of light but to dark-light edges; some distinguish whether those edges are moving from right to left or from left to right. Others respond best to right-angle corners between light and dark.



As we move into the cortex, we find cells that respond to more complex stimuli. Instead of being sensitive to spots, cells fire in response to lines of a particular orientation, movement from a specific direction, and edges of light or dark. The components of our visual world are being constructed one piece at a time.


The plot thickens as the electrode is removed and inserted into an adjacent area of cortex. Now the preferred orientation for the line detectors has shifted slightly, as well as that for the more complicated cells. Moving across the cortex and lowering the recording electrode, we encounter a series of "orientation columns," each about 20--50 microns (micrometers) across, and each column's orientation preference is slightly different from the preferred orientation of the preceding column. These columns are driven mainly by one eye, but after the electrode has sampled across the cortex for 200--500 microns, columns are encountered that prefer the other eye. Thus mapping the surface of the visual cortex reveals stripes or bands of cortex 200--500 microns across that are driven mainly by one or the other eye. These orientation and occular dominance columns are shown in Figure 8-5. Finally, the cortex contains islands of cells that respond to the color of the stimulus.

The surface of area V1 of the cortex, then, is a rich mosaic of partially segregated subcompartments that contain, for each spot on the retina and in the visual world, cells whose firing corresponds to all possible orientations of the stimulus, whether the stimulus is presented to the right or left visual world, whether it is colored, whether the stimulus is rapid with low light contrast or slow with high contrast, whether the disparity of the same stimulus on the two retinas gives clues about its distance or depth, and so on.

Figure 8-5

Orientation and ocular dominance columns of the primary visual cortex, area V1. For simplicity, this drawing represents the domains as rectangular in shape, but they actually are arranged in pinwheel configurations.

Area V1 in the back of the cortex acts as a post office, sending information forward to an adjacent area of cortex, V2, and then (both directly and via V2) to the areas V3, V4, and V5 that occupy folds, or crenulations, in the cortex a small distance toward the front of the brain (Figure 8-6). Lesions in area V1, usually caused by strokes, result in a complete block of the ability to acquire visual information consciously. Both electrical recordings and brain imaging studies show that areas V3, V4, and V5 deal with distinctively different aspects of the visual input. Area V5 (which is also referred to as the MT area) is most active in response to moving stimuli. Brain lesions in this area can cause akinetopsia, the inability to distinguish moving forms, without loss of color or form perception. Area V5 responds poorly to moving stimuli in dyslexic subjects who show deficits in reading and phonological awareness. 26 A small area adjacent to V5, called MST, processes the visual motion, or optic flow, that results from movement of the observer through its environment.



Recall sitting in a stationary train as another train slowly pulled past on an adjacent track and then feeling that your own train was moving backward after the other train passed. This is called a motion after-effect illusion. You may also have looked at a waterfall for a period of time and then, on looking to the side, observed that trees in the adjacent forest appeared to be moving upward. When normal subjects are presented with this motion after-effect, known as the waterfall illusion, they report seeing motion even though the stimulus is stationary, and at the same time their V5 areas are very active. 27 This is an intriguing result, because it begins to provide us with a neural correlate of one of our subjective experiences.


Figure 8-6

Location of different visual areas, shown using the macaque monkey brain, wherein they have been mapped with the most precision.

Area V4 is most reactive to form-with-color input, such as a Mondrian painting. Lesions in this area can cause achromatopsia, the inability to distinguish between colors while form perception remains intact. Firing of neurons in area V4 can be correlated with visual attention directed either toward an area of space or toward a particular object. 28 Surgical excision of area V4 in the rhesus monkey causes deficits in the recognition of objects that have been transformed either in their size, degree of occlusion, or amount of contour information provided. 29 In general, increasingly complex functions are spared as lesions occur in consecutively higher parts of the human visual system. 30 These areas correspond to several dissociable levels of conscious vision: phenomenal vision, object vision, and object recognition.

It seems that visual information goes from the post offices of areas V1 and V2 into a variety of pigeonholes where it is analyzed piecemeal. There are at least 32 visual areas in the macaque monkey, comprising approximately 50 percent of the cortex, and about half of these contain maps of the external world that are reported by the retina. As information flows toward association cortex and the front of the brain, visual information mixes with other sensory input (such as sound and somatosensory stimuli) and motor systems. There is such a melding of visual input with resulting motor output that it is difficult to say where it all comes together before action is taken. Action can be initiated well before visual processing is complete. As information flows from sensory input toward motor output, we would expect to find stages at which responses to stimuli began to correspond to the task about to be performed, rather than to the sensory input itself. Indeed, the activity of cells in the posterior parietal cortex of monkeys can correspond to actions about to be performed. 31

The "What" and "Where" systems.

The prevailing idea is that the distinction between magnocellular (M) and parvocellular (P) pathways persists as information flows from area V1 toward higher visual centers. 32 The M pathway carries information about where a visual stimulus is and about its spatial relationships, size, and motion. This pathway projects from V3 (dynamic form) and V5 (motion) in a dorsal information stream toward the parietal lobe, where motor and somatosensory cortices are located (Figure 8-7). Cells in this stream respond rapidly to changes in stimuli, are not sensitive to color, have low resolution, report information from relatively large areas of the retina, and are very sensitive to changes in contrast (light intensity changes at edges). This is what one might expect for a system that is ultimately responsible for rapid guidance of muscular movements in response to visual stimuli. It is probably most appropriate to view this dorsal pathway as a visuomotor control pathway, not just a sensory or perceptual pathway. There is no clear demarcation of where vision ends and action begins.

Figure 8-7

Dorsal (where) and ventral (what) streams of information.

The P pathway proceeds mainly from V3 and V4 in the direction of the temporal lobe (the inferotemporal cortex), sending a ventral stream of information that is most concerned with what a stimulus is. Cells in this pathway have high resolution (that is, they report information from a very small area of the retina), respond to color, are slower than M cells in responding to changes in stimuli, and are not very sensitive to changes in contrast. These are properties that seem appropriate for object recognition, and in fact, a part of the temporal cortex appears to be specialized for the detection of faces (see below). 33

The separation of these two major streams of information suggests that information about the identity of objects can be computed separately from the information needed to react to that stimulus. Occipital-parietal-temporal association areas might call up various memories relevant to a stimulus (such as the fact that apples can be used to make pies), and decisions related to the stimulus would also involve the frontal lobes. The dissociation of object-processing and spatial-processing domains persists into the prefrontal cortex in primates. 34



The ventral "what" system (temporal lobe) might tell you that an object is an apple; the dorsal "where" system (parietal lobe) would tell you where it is and how to shape your hand to pick it up. 35


The distinction between dorsal and ventral streams has been studied most carefully in macaque monkeys, with their approximately 30 different visual areas. Imaging studies have so far differentiated ten human cortical visual areas with corresponding properties. Of course, the distinctions between the M and P, parietal and temporal, where and what pathways are surely not so clear and tidy as the simplified picture being presented here. There is segregation of information, but these pathways also interact continuously. This account does not cover further information streams that may sketch boundaries and paint in their color, nor does it describe how different information streams might converge and recombine.

Visual Pathways Outside the Cortex

Axons from the retina also pass to at least ten other regions, including the pretectum and superior colliculus of the midbrain, areas that are important in regulating eye movements guided by visual input. Visual projections such as these, that do not pass through the cortex, may underlie the phenomenon of blindsight. Subjects with lesions in the primary visual areas have scotomas, or blind zones, in their visual fields. They cannot verbally report on visual stimuli in these areas, but when asked to guess whether a stick (that they "can't see") is held horizontally or vertically, they answer correctly. And they can direct their hand to where a point of light is flashed in their blind area. 36 This is an example of sensation being flawed while perception is partially intact.



You can get a feel for what blindsight might be like by performing a simple exercise. Try looking about the room and then closing your eyes. Visual sensation will cease, but for a while at least, your knowledge of the room will persist, and you will be able to reach in the right direction for something. This is not surprising. But imagine what it would be like if you were to keep your eyes permanently closed and still had continuously updated knowledge of the position and shapes of objects as though you had just closed your eyes. You would feel that your perception was not legitimate, much as a patient with blindsight does, because you would not have ongoing involvement in the sensation of seeing.


Blindsight can be produced in normal subjects by a clever visual stimulus that places a square-shaped array of paired dots moving past each other in differing locations within a larger square array of similarly paired dots moving at right angles to the first. The dots are so closely superimposed that an observer does not consciously perceive two groups of dots. Still, subjects "guess" the location of the smaller set of moving dots correctly. 37 The stimulus is one that is projected only poorly from area V1 to higher visual areas, which suggests that activation of these higher areas is required for subjective awareness of a stimulus. 38 This correct performance in the absence of awareness might utilize subcortical pathways. There are many other examples, in both normal and brain-damaged subjects, demonstrating nonconscious or implicit knowledge of stimuli that cannot, however, be perceived explicitly or recollected consciously. 39 This observation suggests that, in addition to the modules for explicit sensing and reporting, other distinct modules capable of sensory processing can still do their work when the first set of modules is out of order.

Ensembles of Cells Encode Faces and Other Icons

Are there specific dedicated cells whose firing corresponds to a single complex image (such as your grandmother's face)? Apparently not. It is clear that cells in our inferior temporal cortex are active in facial recognition, but stimuli are represented by patterns across ensembles of cells, not by the firing of single cells. 40 Face-selective cells are found to be members of ensembles for coding faces rather than detectors for a particular face. 41 In the superior temporal sulcus, different populations of cells, located in patches 3--5 millimeters across, are selective for different views of the face and head. These cells may support a parallel analysis of different characteristic views of the head that combine to support view-independent (object-centered) recognition. Some neurons in the inferior temporal cortex can learn to respond to all views of previously unfamiliar objects, but most respond to only one view. 42 Activities of these cells may be important as "social attention" signals, indicating where other persons are directing their attention. Such a process is required for social interactions.

Activation of face-sensitive cells can correlate with activity in both the hippocampus (declarative memory) and the amygdala (involved in emotional memory). Positron emission tomography (PET) scan studies show that large areas of temporal and occipital cortex are active during the processing of facial information. 43 When subjects are asked to visualize internally the face of a famous person, the face area of the temporal cortex becomes more active. Magnetic resonance imaging (MRI) studies using binocular rivalry show how activity in the face area can correlate with visual awareness: A subject whose left eye is shown a picture of a face and whose right eye is shown a picture of a house reports switching---experiencing one and then the other every few seconds. During intervals in which the face is reported, the face area becomes relatively more active, and when the house is reported, the region surrounding the hippocampus that is involved in spatial recognition becomes more active.



Brain lesions, particularly to the right hemisphere's temporal lobe, can cause face-recognition impairments, or prosopagnosias, while sparing the ability to recognize most other objects. 44 Such a case is described by Oliver Sacks in his book The Man Who Mistook His Wife for a Hat. The man in question had lost his ability to recognize faces, so he had to use recollection of the dress his wife was wearing to recognize her at a party.


Electrical recordings from monkey cortex have recently shown that columns of cells in the inferior temporal cortex are selective for a range of similar stimuli. One intriguing set of experiments started with a large collection of toy animals, vegetables, and natural objects as visual stimuli and then progressively simplified and abstracted the features necessary for eliciting the strongest responses of single neurons. 45 Columns responsive to colored bars, hand shapes, ovals with gradients of luminosity, face shapes, T-shapes and other iconic stimuli were found.

The inferior temporal cortex appears to use ensemble coding, with no one cell representing the detailed information being processed. Rather, the world we perceive emerges from the coordinated firing of sets of interconnected neurons. 46 Examples of ensemble coding occur at all levels of the visual system. Color perception is an emergent property of a "committee" of photoreceptors. Depth perception is an emergent property of disparity-detecting cells in area V4 whose firing reflects how the position of an object on the retina changes as its distance from the eye changes. Individual cells are not sharply tuned for objects, say 8.0--8.2 meters away, but we can make this distinction by averaging the responses of large numbers of similar cells. Such estimates improve with the square root of the number of cells involved (such that quadrupling the number of cells doubles the precision). Similarly, any single cell in area V5 might not track the velocity of a visual stimulus with great accuracy, but the average response of a large number of these cells connected together might be very accurate indeed. We can resolve spatial differences in images on our retina that are smaller than the separation of the photoreceptors, presumably because the reports of many receptors on the retinal position of a stimulus are averaged.

Information Streams Flow Forward, Backward, and Sideways

Work on the visual system is refining the simplistic classical distinction, still widely used in teaching physiology, between what goes into the brain (sensations, perceptions) and what comes out (actions). Lurking behind this idea was the incorrect metaphor of a clear chain of command: an input stream, then someplace where "it all comes together for decision making," and then an output stream. The initial stages of visual and other sensory input are indeed hierarchical and involve mainly the flow of information from peripheral receptors into the brain. (For example, our brains do not send information back to our retinas.) Information flow in the final stages of motor output also is mainly one-way. In between these more peripheral functions, however, visual information is flowing in the dorsal and ventral streams mentioned above into 30--50 different areas where it mixes with signals from other sensory areas and motor areas, with lots of cross-talk and feedback.

Even at the early stage of the lateral geniculate nucleus, which is usually described as a simple relay station passing information from retina to cortex, feedback connections predominate. The lateral geniculate nucleus gets only 20 percent of its input from the retina; the other 80 percent is feedback from the visual cortex and input from lower centers in the brains such as those that control eye movement. Recordings from the lateral geniculate body or area V1 in live animals show that the responses are very context-sensitive, changing with body tilt or posture or with auditory stimulation. Resonating neuronal ensembles link vision, posture, sound, and so on. Meaning resides in complex patterns of activity among the numerous units that make up these networks.



There are so many reciprocal and cooperative interactions between visual and other systems that the situation is better described as a cocktail party than as a chain of command.


The projection of a later stage of processing back onto an earlier one is referred to as top-down, re-entry, or re-entrant processing. This re-entrant input is often more diffuse than the incoming information was. Thus the forward projections of V1 and V2 to V3 and V5 are very specific; thin stripes of V2 cells project to V4, whereas thick stripes of cells project to V3 and V5. Yet the return (re-entrant) input to V1 and V2 from the specialized visual areas is very broad. This input apparently allows V1 and V2, which are largely devoted to simple line detectors, to represent complex features that require feedback from higher cortical areas. Thus correlates of attention and figure-ground separation have been found in recordings from single cells and small groups of cells in areas V1 and V2. 47 One example comes from cells that respond to the illusory edges of the invisible triangle (the Kanizsa triangle) shown in Figure 8-8. Cells in areas V1 and V2 that respond to an illusory contour or line at a border where no contrast exists must be receiving feedback from higher visual areas that have defined the whole figure. 48

Figure 8-8

The Kazinski triangle. Cells are found in areas V1 and V2 that can respond to the illusory edges formed by these shapes.

The effect of top-down processing can be readily observed in both monkeys and humans when attention is focused on a particular aspect of a stimulus, such as its color, motion, or identity. Higher centers in the frontal lobes apparently feed back to enhance the activity of cells and areas dealing with that aspect of the stimulus, while suppressing others. 49 Thus instructions to pay attention to the motion, shape, or color of a stimulus enhance the activity of V5, V4, and the temporal lobe "what" pathway, respectively. 50

Another example of a correlation between a sophisticated visual perception and neural activity comes from neurons in areas V1, V2, and V4 whose activity follows perception during binocular rivalry experiments. Monkeys were trained to report their perceived orientation of a grating stimulus by pressing levers, and then gratings of different orientations were shown to the two eyes. Many cells, particularly in area V4, showed activity that correlated with the perceptual dominance of one, and suppression of the other, stimulus. 51 When recordings moved on to the inferior temporal cortex, activity of nearly all the cells reflected the dominant perception. 52

A linking of perception and action at the single-cell level has also been observed. 53 Monkeys can be trained to respond behaviorally to a set of light points moving together in a fixed direction in a field also occupied by randomly moving points (the human observer doesn't notice the coherent movement of the subset; everything just seems to be random). Activity in the V5 region is recorded that corresponds to this discrimination. Microstimulation of the circuit that encodes this direction of motion elicits the trained behavioral response (without stimulating the retina). This work has been carried even further to show that the responses of some cells in the frontal lobe hold information about intended behavioral movement before it is actually carried out.

Re-entrant, or top-down, processing presumably underlies the activation of areas V1 and V2 that is observed in magnetic resonance and PET imaging studies when a visual stimulus is recalled. 54 The areas activated by a visual stimulus can increase their activity during mental recall of the same stimulus. Brain imaging studies show that both motor areas and area V1 can become active when subjects are asked to rotate imagined visual objects. Numerous fields in parietal, temporal, and frontal cortex also become active during manipulation of images. 55 PET studies show differential recruitment of the temporal ("what") and parietal ("where") pathways when subjects are asked to imagine an object or to imagine tracing a pathway between two familiar street locations. Damage to the right frontal lobes can compromise tasks that involve mental imagery in the (imagined) left visual field. 56



The fact that knowledge stored in higher centers can activate primary visual cortex might partially explain why our visual perception can be biased to report what we want to see. In the fascinating case of Charles Bonnet syndrome, damage to early parts of the visual pathway results in a blind spot in some part of the visual field. Subjects report vivid visual hallucinations confined entirely to the region of this blind spot. This suggests that the hallucinations may be generated by feedback from higher cortical centers---feedback that can no longer be compared with what is actually present in the subject's visual world.


As one might expect, the "what" and "where" systems are selectively compromised by damage in the temporal and parietal lobes, respectively. For example, a car accident victim with a lesion between the part of the brain that compiles images coming from the eyes and the library of images in memory could not recognize what he saw but could still draw on his library of images. He could not name an asparagus shown to him but could draw an asparagus when asked. His ability to recognize human faces was unaffected by the brain damage, which implies a separate system for face recognition. 57 What emerges from all these facts is a picture not only of the bottom-up flow of information but also of top-down projection of higher brain areas back to earlier visual areas. 58

A Binding Process Underlies Visual Perception

How are different aspects of the visual stimulus organized into perception? This is called the binding problem. Complex visual images are being processed by parallel pathways that specialize in color, form, motion, solidity, and so on. How does the brain associate the work being independently done by separate cortical regions? Different properties such as shape, color, and motion must be "bound" to the objects they characterize. An object and its motion must be distinguished from its background. These problems are among the most crucial in cognitive neuroscience. 59 The binding problem is not just a theoretical worry. Some patients with bilateral parietal lobe damage miscombine colors and shapes and are unable to judge relative or absolute visual locations. Spatial information associated with the dorsal "where" pathway has been compromised. 60

Our visual consciousness starts building as our eyes dart about over the surface of a viewed object to generate a series of very rapid, focused snapshots. These are the saccadic eye movements mentioned in the discussion of the first self-experiment in this chapter. The information gathered during a saccade is delivered to a scanning and processing mechanism in the visual cortex; the process is complete in about 50 milliseconds. This mechanism registers distinctive boundaries that are created by elementary properties of brightness, color, or line orientation. We rapidly and automatically process the property of subjective closure of forms. 61 A pop-out effect occurs for letters and digits: A letter target is detected more efficiently among digits than among letters, and vice versa. Because letter recognition and digit recognition are not innate, it is likely that the customary separate groupings of letters and digits in the environment leads to their separate encoding. For postal workers who routinely see letters and digits together, letters do not pop out so much from digits. 62



Our experience of a continuous visual world is an illusion. It could be arranged, by attaching a tracking device to your cornea, that small fragments of a text (from a whole written page) are displayed only during the brief times during which you are actually directing a saccade toward their position on the page. An observer not connected to this device would see only transient flickers of word fragments darting about the page, but you would experience a continuous page of text. 63


The scanning system encodes useful elementary properties of the overall scene (such as texture, color, orientation, size, and direction) into feature maps in different brain regions. 64 An abstracting function may then compose key aspects of the image that distinguish the object of attention from its surround---a master or saliency map is formed. Thus the early stages of the visual pathway provide a faithful representation of the retinal image, whereas later stages of processing hold edited representations of the visual world modified to suit the immediate interest and goals of the viewer. 65 If a key part needs further examination, then the brain refers to the individual feature maps.

How are salient features bound together into a master map? One suggestion is that visual attention may be mediated by subcortical structures such as the pulvinar, claustrum, and superior colliculus, as well as by the prefrontal cortex. 66 Bursts of correlated action potentials in these structures may represent the saliency map. Evidence suggests that neurons in the visual cortex that are activated by the same object in the world tend to discharge synchronously. 67 According to this theory, separate populations of cells---say, those responding to aspects of the color, shape, texture, motion, smell, and sound of a stimulus like a bus, along with those that hold memories of buses experienced in the past---all send out nervous impulses at the same rate for a fraction of a second. As the neurons all fire together, sometimes oscillating about a central frequency, the perception of the bus is created in the network. 68 The binding is a matter not of where but of when. Thus the synthesis of the visual image in the brain would depend not only on the simultaneous activity of cells in the different specialized visual areas but also on the temporal synchrony of their responses. 69 This synchronous activity would include areas such as V1 and V2 with which the specialized visual areas are reciprocally connected. 70 To link visual perception and action, it would be necessary to synchronize also with motor areas of the cortex, and such synchronization has been observed. 71 Synchronization of the firings of large ensembles of neurons could be important whether or not those firings were occurring in a rhythmic or oscillatory pattern. (Investigators' excitement about the joining of nerve cells in rhythmic oscillations being a possible answer to the binding problem has been muted somewhat by the discovery that the oscillations measured in the visual areas of the cat brain are not observed in visual areas of the monkey brain.)

The visual percept might not reside in any given visual area, even if that area is critical for certain features of that visual image, but might rather result from ongoing activity in several connected visual areas. It seems likely that the simultaneous activity of many visual areas is necessary for conscious visual perception. 72 Stimuli do not reach visual awareness unless all these areas are stimulated, even if signals bypass area V1 to reach the specialized visual areas indirectly, as is observed in blindsight. 73

Neural Correlates of Visual Consciousness

A goal of current research is to determine the neural correlates of our experience of seeing---our visual consciousness. It seems most likely that visual awareness and planning involve the frontal cortexes, which receive connections from the higher, specialized visual areas such as V4 and V5, but not directly from area V1. 74 This argues against activity in area V1 by itself being the neural correlate of images we are aware of. 75 It is possible that there are specific neurons, normally dependent on input from many areas, whose firing gives rise to the current content of our visual (or other) consciousness. If this is the case, one might be able---with some not-yet-invented technique---to stimulate these cells directly and generate in a human subject the same subjective experience that is associated with their normal input and firing. 76


We are at the threshold of understanding how the array of modular processors that first transform and analyze different aspects of our sensory world---such as visual motion, form, depth, and color---proceed to interact and generate our conscious experience of a visual percept. Progress in understanding the perceptual worlds of hearing, tasting, smelling, and body movements is not so advanced. The great variety of line detectors, shape detectors, icon detectors, figure-surround discriminators, and so on represent parts of the solution to the central problem posed by the physical environments in which the visual brain evolved: how to process visual information in a way that leads to effective action in the world, action ultimately in the service of reproduction. Sensory information coming into our brains is arranged hierarchically at first but soon merges into regions of association cortex that specialize in "what" or "where" information---regions that blend sensory modalities and mix with motor, premotor, and prefrontal areas. These higher areas send information back to earlier stages of processing, and cross-talk between different information streams contributes to the binding together of percepts. The initiation of action, then, has a fuzzy origin that becomes more structured and hierarchical as actual movement repertoires are executed.

Higher vertebrates in particular use the past as well as the present in interpreting current sensory input. What has been seen in the past can influence what is seen in the present. Experience during development of the brain and as in adulthood leads to the assembling of a whole repertoire of assumptions and interpretations that have proved useful in the past. These most often facilitate perception and actions, but they also can bias and confuse our responses if a past interpretation is not appropriate to the present. As we will see in the next chapter, systems that regulate body movement also acquire training and habits, which, though they are indispensable for skilled action, can also impede appropriate responses to novel situations.

Questions for Thought

1. One approach to understanding perception is to ask, "What kind of machine would a good physicist or engineer build to render inside the head an accurate representation of the sights, sounds, and so on out there in the real world?" Another approach is to say, "Forget the omniscient engineer. What perceptual mechanisms have in fact been cobbled together during evolution to guide action in the real world, in the service of passing on our genes?" What do you think are the merits of each of these two approaches? Do they suggest different research strategies?

2. Many experiments demonstrate that most of our perceptions are not naive but are informed by a previous history of sensing that imposes biases and interpretations on what we think we perceive. Why do you think our brains would have been designed this way? What are the advantages and disadvantages? How might you try to determine whether such biases are learned or innate?

3. Would a lesion in your right inferior temporal lobe be expected to influence how you pick up an object with your left hand? If so, how?

4. Visual information flows from higher visual areas back to lower visual areas such as V1. What are some ideas about the role such top-down processing might play?

Suggestions for Further General Reading

Ornstein, R. 1991. The Evolution of Consciousness. New York: Prentice Hall. Several examples of the relativity of perception cited in this chapter were taken from this book, which is an engaging account of some of the topics covered in this and subsequent chapters.

Goleman, D. 1986. Vital Lies, Simple Truths: The Psychology of Self Deception. New York: Simon & Schuster. An accessible account of psychological studies that demonstrate how perceptual filters, self-deceptions, self-images, and the like are constructed. Several examples used in this chapter were taken from this book.

Crick, F. 1994. The Astonishing Hypothesis: The Scientific Search for the Soul. New York: Scribner. This book, like the book by Zeki, cited below, gives a description of information processing by the visual system.

Pinker, S. 1997. How the Mind Works. New York: Norton. Chapter 4 of this book includes an interesting discussion of how the visual mind solves problems of shape, space, and perspective.

Reading on More Advanced or Specialized Topics

Zeki, S. 1993. A Vision of the Brain. Oxford, England: Blackwell Scientific Publications.

Gazzaniga, M.S. (ed). 1995. The Cognitive Neurosciences. Cambridge, MA: M.I.T. Press.

The next two volumes are collections of brief review articles that cover essentially all of the areas of modern cognitive neuroscience. They have material relevant to all the chapters in Part III of this book. Together with Gazzaniga's cognitive neuroscience textbook and the citations on the Web site for this book, they provide more detailed primary references on the observations reported in this chapter.

Squire, L.R., & Kosslyn, S.M. 1998. Findings and Current Opinion in Cognitive Neuroscience. Cambridge, MA: M.I.T. Press.

Gazzaniga, M.S., Ivry, R.B., & Mangun, G.R. 1998. Cognitive Neuroscience---The Biology of the Mind. New York: Norton.

1. Pathways in the brain are tuned to processing relevant natural stimuli, such as natural sounds (Nelken et al,, 1999) or rapidly approaching objects (Rind and Simmons, 1999).

2. Held and Hein, 1963.

3. Damasio, 1994, pg. 230.

4. These examples are from Ornstein, 1991, pp 108-109.

5. Ornstein and Ehrlich, 1989, pg. 39.

6. Berns et al., 1997.

7. One hallmark of schizophrenia is an inability of the brain to filter out irrelevant sounds and other sensations (Grady, 1997)

8. Posner, 1995.

9. Greenfield, 1995, Ch. 7-8, discusses these systems.

10. Smith and Nutt, 1996.

11. Posner and Raichle, 1994, Ch. 7.

12. Goleman,1986

13. Leopold and Logothetis, 1996.

14. Purves and Andrews, 1997.

15. Sinha and Poggio, 1996.

16. Humphrey, 1992, Ch. 10.

17. Zimmer, 1993

18. Aglioti et al., 1995.

19. The way in which filters, self deceptions, self images, etc. are constructed is covered in a book by Goleman (1986), Vital Lies, Simple Truths. It is the most accessible and readable account of human psychology that I have seen. Some relaxation of ordinary filters on awareness can be accomplished by meditation techniques. ( Strauch,R., 1989; Deikman,1966, reprinted in Ornstein, 1973; Varela et al, 1992.

20. Bear et al., 1996, Ch. 8-12, provides a clear introduction to these topics.

21. Bownds and Arshavksy, 1995.

22. More comprehensive and detailed accounts of visual processing than I provide here can be found in Zeki, 1993, Crick, 1994, and Gazzaniga (ed.), 1995. See also Kandel et al, 1991, chapters, 28-31, Bear et al, 1996, Ch. 8-9, and Van Essen ( 1992)

23. There is a beautiful discussion of the evolution of eyes in the chapter "The Forty-fold Path to Enlightenment" in Dawkins, 1996.

24. Kaas, 1995.

25. A class of developmental dyslexics who do poorly in tests requiring rapid visual processing show abnormalities in the lateral geniculate magnocellular layers that process fast, low contrast visual information. Livingston et al., 1991; Walsh, 1995. Some dyslexics have difficulty not only with rapid visual stimuli, but also in processing sounds, suggesting a more general deficit in processing of rapidly changing sensory stimuli. Merzenich et al., 1996; Stein and Walsh, 1997.

26. Eden et al., 1996. Some dyslexics appear to have a general deficit in processing rapid signals, whether visual or auditory. Merzenich et al., 1996, report that training exercises can improve temporal processing.

27. Tootell et al, 1995.

28. Newsome, 1996.

29. Schiller, 1995.

30. Stoerig, 1996.

31. Carpenter, 1997.

32. The parallel pathways of visual information flow are described in Kandel et al, 1991, Ch. 30; Kosslyn and Koenig, 1992, pg. 55, Maunsell, 1992, and Goodale and Milner, 1992. I am giving a truncated version of the story. Van Essen and DeYoe, 1995, describe pieces that I have left out giving a picture of how complex the convergence, divergence, crosstalk, and feedback is at each level of the hierarchy. Hilgetag et al., 1996, argue that the anatomy permits many alternative descriptions of the visual hierarchy.

33. The distinction between dorsal and ventral streams has been studied most carefully in macaque monkeys, in which approximately 30 different visual areas have been noted. Imaging studies have so far differentiated ten human cortical visual areas with corresponding properties. Tootell et al, 1996.

34. Wilson et al., 1993

35. The distinctions between the M and P, parietal and temporal, where and what pathways are surely not as clear and tidy as I describe them here. Zeki, 1993, argues that they are an oversimplification. There is segregation of information, but these pathways also interact continuously. I have not covered the distinction, which persists to higher levels, between the interblob pathway that may sketch boundaries and the blob system that may paint in their color. Ship, 1995, gives a review of how different information streams might converge and recombine.

36. For a recent review, see Weiskrantz, 1996.

37. Kolb and Braun, 1995.

38. Koch and Braun, 1996.

39. This is reviewed in Schacter, 1992.

40. An issue of the Philos.Trans.R.Soc.Lond.[Biol.] (series B, Vol. 335, 1992, pp. 1-128) contains articles relevant to this point.

41. Young, M.P., 1992

42. Logothetis et al., 1995.

43. Tovee, M.J. 1995, gives a useful review of brain processing of information about faces, and also discusses how "special" face recognition is, compared with other classes of objects (Tovee, 1998). Gauthier et al, 1999, argue that the "face areas" are involved in acquiring expertise at distinguishing novel objects.

44. Young,1992.

45. Fujita et al ( 1992, reviewed in Stryker, 1992. Young, 1993 has a useful summary figure. Further reference are: Chelazzi et al, 1993, on the inferior temporal cortex in visual search. and Tanaka, 1993, on neuronal mechanisms of object recognition.

46. Calvin, 1990, pp. 138-146

47. Articles on illusory figures and real neurons are: Wickelgren, 1992;, Grosof et al, 1993;Davis and Driver, 1994 and Albright, 1995.

48. One case history relates the story of an artist who had a cerebral vascular accident that caused a restricted scotoma (blind spot) of part of V1 and V2. This patient could not fill in the subjective contours in the Kanizsa triangle, suggesting that the kinds of cells in V2 that respond to subjective contours were damaged. Zeki, 1993, pg. 316

49. A recent review of this and other work is given by Barinaga, 1997.

50. See Posner and Gilbert, 1999, for attentional modulation of primary visual cortex by higher levels. Behrmann and Haimson, 1999; and Lee et al., 1999, describe how attention how attention activates a winner-take-all competition among visual filters that is modulated by both stimulus-driven and goal-driven factors.

51. Leopold and Logothetis, 1996.

52. Sheinberg and Logothetis, 1997.

53. A series of experiments in the area have been done by Newsome and his collaborators, see Salzman and Newsome, 1994.

54. Le Bihan et al., 1993;Kosslyn et al, 1995. Interestingly, transcranial magnetic stimulationtargeted at the primary visual cortex, area 17, impairs both seeing an object and internally visualizing it, and has corresponding effects on the PET images obtained. This suggests that area 17 is necessary for both processes (Kosslyn et al., 1999.) In fact, a confined transcranial pulse applied to the primary visual cortex can make a hole appear in an otherwise uniform grid stimulus, a temporary `blind spot.' (Kamitani and Shimojo, 1999.)

55. Kosslyn's book, 1994, Ch.10, 11, provides a account of the many subsystems involved in mental imagery.

56. Guariglia et al., 1993.

57. Detailed accounts of the multiple brain areas activated during imagery are given in Roland, 1993, Posner and Raichle, 1994, and Kosslyn, 1994.

58. Miyashita, 1995.

59. Treisman, 1996, provides a review of the binding problem.

60. Friedman-Hill et al., 1995

61. As one example, four open circles in a field of closed circles are detected effortlessly, but seeing four closed circles in a field of open ones is much more difficult, and requires element by element scrutiny. This is the subject of an article by Williams and Julez, 1992.

62. Polk and Farah, 1995.

63. See Miller and Bockisch, 1997, for a discussion of saccade mechanisms. Snowden, 1999, gives a description of how we actually work with an extremely sparse representation of the visual information we thing we are seeing.

64. Auditory and other sensory systems also use pre-attentive sensory memory. See Tiitinen et al., 1994.

65. Maunsell, 1995.

66. Hardcastle, 1994, provides a useful review.

67. Stryker, M.P., 1989; Engel et al, 1992; Konig et al., 1995;Singer, 1995.

68. The excitement about oscillations being a possible answer to the "binding" problem has been muted somewhat by finding that the oscillations measured in the visual areas of the cat brain are not observed in visual areas of the monkey's brain ( Young et al, 1992). See also Tovee and Rolls, 1992.

69. See Singer, 1999; Rodriguez et al., 1999, and Tallon-Baudry and Bertrand, 1999, for how long distance patterns of synchronization collelate with the perception of faces, and correlation of visual and tactile stimuli.

70. The importance of synchronous neural firing as a solution to the binding problem is far from clear. See Golledge et al, 1996, for discussion.

71. Roelfsema et al, 1997.

72. Damasio, 1995; Srinivasan et al, 1999, have shown that in binocular rivalry experiments where a human observer can report being conscious only only one of two incongruent images at a given moment, perception of each of these images can be correlated with the synchronous activity of large populatons of cortical neurons.

73. Zeki,1993, pg. 355.

74. See Lumer and Rees, 1999: and Farah and Aguirre, 1999, on MRI and PET data showing ocvariation of visual perception with activity in visual and prefrontal cortex. Kastner et al., 1999, show that frontal and parietal areas show larger increases in activity than visual areas when there is expectation of a visual stimlulus in an area other than the one being directly attended to.

75. Crick and Koch, 1995; Koch and Braun, 1996.

76. Koch, 1996.

Home   |   Mind Blog   |   Lectures & Writings   |   Music   |   About Me   |   Contact

©Deric Bownds • All Rights Reserved
Counter 1