Lecture Notes for February 3 and February 10

We began by reviewing the approaches of Helmholtz, Kant, Aristotle, and Gibson.

We then discussed another question by Hellar concerning the role of higher level cognition and representation in particular mental imagery, which intuitively seems highly similar and therefore relevant to picture perception.

One intuitive explanation of tactile perception which was voiced in class was that when the blind perceive pictures they are forming an internal mentally visible imaged that is "looked at" by the eye of the mind.
We discussed in class how this intuitive and pervasive answer may reflect an introspective bias of sighed people. This lead into a discussion about mental imagery.

We have evidence for the work of Kosslyn and others that images are like pictures. This evidence consists of studies involving mental rotation and scanning experiments. However, there is evidence that mental images are not like pictures. Images are indefinite in a way pictures are not. We cannot perceptually disembed from mental images they way we can from pictures. We also cannot do aspect shifts on mental images the way we can on pictures. In short, we do not seem to be able to do spontaneous perceptual restructuring with mental images the way we can with pictures.

We also noted that pictures are not adequate representations for a lot of important aspects of cognition. Pictures cannot represent negation, cannot represent statement involving "all", pictures are inherently ambiguous, indefinite in what they are representing, cannot represent many properties pictorially, e.g. heat. So for all these reason we should be careful in thinking there is a simple relationship between cognition and pictorial representation.

All of this goes to show that the intuitive relationship between pictures, cognition, and mental images is very problematic. We need to do a more careful analysis. So we turned to the question of how do pictures represent the world.

I pointed out that given some of the difficulties we had discussed above that the question needed to be broken down into smaller questions.
We turned to the questions " what do lines represent?" and "how do lines represent?’

Why start with lines? We noted that lines are sufficient for pictures, i.e., one can make a good picture using only outlines.

Although lines are not completely necessary for a picture, they are often present and play a crucial role in a picture.

Lines are often the basis of our first recognizable pictures.
We use outline pictures when we want to represent something quickly or to represent the important fact or features of some object or situation.
Yet what is a line? Not in a geometrical sense but in a psychological, i.e., representational sense.

There is an intuitive answer of how outline pictures work.

A line is a brightness contrast that stand for a similar brightness contrast in the world.

This explanation seems very obvious yet it would have the consequence that pictures would inherently be visible in nature.

However there are some significant problems with this intuitive explanation. I asked the students to draw a triangle in their notes while I drew one on the board. We agreed that their pictures and mine we the same. But then we noted that my line on the board went dark to light (white chalk) to dark, while there line went light to dark back to light again. Our pictures are the same yet how can that be if the brightness contours are opposite?

Another problem is that one can replace a continuous line with a dotted line or a line of small Xs. Yet these lines are not really lines, but the pictures that use them are successful nevertheless. Yet such dotted lines and X lines do not directly map onto any brightness pattern in the world.
Notice also that every line actually has two sides yet this line stands for one edge. In fact, the two sides of a line can create different pictures but we often do not notice this.

One way of explaining the opposite brightness contours working the same, and the two-sided line standing for a single edge is Kennedy’s 1993 axis theory. This claims that perception of a line actually involves an single axis that is abstracted from the brightness information and this single abstract axis, that falls someone between the two sides of line, stands a single edge in the world.

Notice that such axes need not be generated by brightness contrast because they are abstract in nature. It also possible that this could reflect amodal representation of space.

Yet what about the dotted line or the X line. Kennedy 2000 proposed alignment theory to account for such depiction. With the dots or Xs we use the locus of each dot or X and its set of relationships to its neighbors.
Notice how Kantian this explanation is. We seem to be imposing a pattern on things.

I put a circle of 8 even spaced dots on the board and everyone spontaneous saw a circle. Yet if I remove four dots I can reveal a square. But we do not see this square, nor do we see it as two squares intersecting or overlapping. So although technically the depiction is ambiguous there is no ambiguity in the perception. As Arnheim 1974, 1996 argued that some shapes or gestalts are preferred or imposed on perception.

There was much discuss and debate around this point.

So when you think of it much is happening in picture perception of outline drawings that is quite removed from sensing light.

Axes are abstracted from complex elements in order to stand for single edges, and non-continuous elements are aligned so as to stand for continuous features of the world.

Both of these processes look very amodal, at least multimodal, in nature. There is no reason in principle why these processes could not also be found in tactile pictures.

Yet a possible Helmholtzian style of reply might be that all of this points to how constructed pictorial perception is. Outline pictures involve learning to associate certain perception pattern with more cognitive knowledge. The features of a picture are not similar to their depicted features of the world. Instead they are more like symbols that stand via convention for aspects of the world.

After all, when we learn to read we learn that visual patterns stand via convention for cognitive knowledge. Perhaps outline pictures work in the same way. We learn and important set of conventions how pictorial elements stand for features of the world. And as the Stroop effect shows these conventions can become so automatic and habitual that we see them, e.g. we automatically see as visual pattern as a word. Picture perception could work in this way. It would explain how outlines work even thought they are very dissimilar from the world they represent.
This is a powerful explanation that is deserving of intellectual respect.
Yet there is good evidence against it. There are two kinds of evidence. One is that line drawing can be correctly perceived by people unfamiliar with pictures. The second is that we cannot make a line stand for important aspects of pictures even with great familiarity.

Kennedy and Fox 1977 showed that blind perceived dot patterns as we do. For example 4 dots perceived as a square. 6/8 subjects perceived the 8 even spaced dots as a circle and all blind subjects perceived 16 evenly spaced dots as a circle.

D’Anguilli et. Al. 1998 compared congenitally totally blind children 8-13 and blindfolded sighted children matched in age for recognition of raised line pictures of common objects. Some of the sighted children had their hands guided by the experimenter. Out of ten pictures the mean recognition rate of the unguided sighed was .7, of the guided sighted it was 2.8, and the blind rate was 3.6.

With little practice the guided sighted group performance approached that of the blind.

Kennedy 2000 concluded: "Evidently, it could be that the major distinction between the blind and the sighted initially, so as recognition of the kind of pictures of relatively common objects … is simply that the blind are more skilled in tactile exploration. If so, in many respects the internal picture processing skill serving recognition is essentially the same in the blind and sighted."

Hochberb and Brooks 1962 as well as DeLoache, Strausss and Maynard 1979 found that outline drawing where lines stand for changes in relief ( change of slant, depth) can be recognized by children less than 20 months old without any training in pictorial conventions.

Kennedy and Silvers 1974 found that cave art from opposite ends of the globe used outline to stand for the same kind of kind of relief features, i.e., change of depth and slant.

Kennedy and Ross 1975 found that the Songe, a small tribe of people of Papua New Guinea who have no pictures in their culture, could recognize an outline drawing showing depth etc.

Yet, on the other side we cannot make outlines stand for anything we could picture.

We cannot make outline drawings stand for chiarascuro information. Demonstrated this to the class.

Notice how our familiarity with faces or even with a particular face does not help. People do not seem to be able to acquire the skill of seeing outline drawings of chiarascurro patterns as pictures.
So what do lines stand for?

As Kennedy argues in 2003 lines stand for the borders of flat and curved surfaces, essentially edges of surfaces, and surfaces are most of what we perceive.

Surfaces, and edges are tangible as well as visible, hence outline drawings should be possible for the blind, even the congenitally blind.
So, while brightness contour theory and convention theory predict that blind people cannot do pictures we have another account that predicts that the blind can produce and perceive pictures.

Kennedy in fact argues that the blind and capable of many important aspects of pictorial representation. The most significant variable that affects performance is general encouragement in picture production.
Yet the history of research on visual impairment in the visual arts has not generally been encouraging.

Hayhoe 2003 points out that some of the earliest research drew very negative conclusion.

1932 von Senden published an influential report based on studying subjects who were blind but had subsequently been cured through surgery. Because of the spatial performance of such people he concluded that the sense of touch had no conception of objects or space. This seemed to confirm the earlier empiricist theory of Locke and Berkeley who had predicted that such individuals would not be able to perceive objects and space.

Revesz (1950) reviewed von Senden’s work and concluded that some of the case studies were unreliable. He concluded that touch provide some (probably unique) form of spatial perception.

More recent studies of recovery from visual impairment by Gregory 1974 and Sachs(1995) that such individual can draw recognizable pictures. Yet they do exhibit signs of confusion about their visual information.

However, although this approach has a famous philosophical heritage, it does not seem to be the correct test for the thesis that perception is based on modal specific learning This whole approach is filled with significant confounds.

What followed was a discussion of the various confounds in this form of study. The fact that something is multimodal or amodal does not imply that it can instantly and effortlessly transfer from one sense to the other. There seems to also be a confound between knowledge and experience, i.e., someone may have spatial experience but not have the needed perceptual skills or linguist knowledge to perform certain tasks or answr certain question.. There may be a confound due to the traumatic nature of the change to their life. There may be pressure from new social expectations., etc. Several of these confounds as well as others were discussed at length.

I then argued that Hayhoe’s review was very valuable but his final criticisms seem misplaced. He recommends moving " down the stairs of academic’s ivory tower towards practicality needs to continue further," and he goes on to claim that research should move "toward the understanding of creativity and the motivation of students."

Yet studies of creativity and motivation are not in a state that could reliably be applied to this whole area of study in any clear or determinate way. It is not clear how turning to the creativity and motivational research would be helpful or specifically relevant.

In fact, as we shall see there are still some very philosophical/theoretical issues in dispute regarding tactile pictures. See the Hopkins article.

We need to maintain a critical stance about what is going on in picture perception by the visually impaired.