Optics
and haptics: The picture
John M. Kennedy and Igor Juricevic
Psychology Department
University of Toronto
Acknowledgements: Dejan
Todorovic, Christoph Landerer, Josef Perner, Juan Bai, Shadi Edalati.
Abstract
Pictures are tactile as
well as visual. Outline pictures stand for the same kinds of surface features
in touch and vision. Vantage point geometry is used by blind and sighted perceivers
in pictures. Limits of pictures may be comparable for the blind and sighted,
and transcended in useful ways.
Introduction
In keeping with a conference on the multimodality of human communication, the
purpose of this paper is to show that some aspects of pictures are tangible
as well as visual. Many means of communication are often considered to be exclusively
visual or auditory, but the exclusionary view requires radical revision, the
evidence from pictures tells us.
While they are often considered to be creatures of vision, pictures are amenable
to touch in many ways. While that broad interpretation of the evidence is unmistakable,
the full extent to which pictures belong in more than one sense is still unclear.
What requires explanation is not just tactile pictures. We still need to give
a proper account of visible pictures. Further, like touch, vision brings idiosyncratic
features into service when triggered by pictures. In principle each and every
means of communication is likely multimodal, but also in practice each can evoke
unique features of a sense to some degree, much as a translation can keep the
gist of a message but modify some connotations.
Examples from Tracy
It may be best to begin with examples. First consider some pictures drawn at
our behest by Tracy, a blind woman, at the Museum of Modern Art, New York, in
January 2002. Tracy was blind early in life, having steadily lost vision in
infancy to the point of total loss at age 24 months, too early to learn to draw
using vision.
Tracy's drawings were made with a ballpoint pen on a plastic sheet resting on
a rubberized board, a kit which produces raised lines when the pen writes on
it and makes the sheet pucker. Conveniently, the long thin lines of puckers
are raised on the side from which pressure is applied.
Tracy was asked to draw a cube balanced on a point with three faces towards
her. The orientation of the cube was demonstrated to her with an 8.5cm wide
wooden cube. She initially drew two quadrilaterals attached along one side,
but then indicated this was not satisfactory and began again. She proceeded
to depict the cube by first drawing a Y shape, standing for the front vertex
of the cube. She completed her drawing by adding to the Y to show the three
faces of the cube in front of her. She produced three quadrilaterals. They were
formed by adding two lines, in V shapes, to each adjacent pair of lines in the
Y. The added V shapes stood for the rear edges of the faces of the cube. The
drawing reveals good use of a vantage point. It also suggests Tracy can project
features of an object at various orientations and depths onto a flat surface.
To test her use of a vantage point and projection further, Tracy was asked to
draw three objects (a cube, a cone and a ball) set around a rectangle, and to
make the drawing from 5 different vantage points. This is known as Piaget's
three-mountains task (Heller and Kennedy, 1990). The vantage points were, first,
her own location, second, a location on the opposite side of the rectangle,
third, a location on what was the left side of the rectangle from her and fourth,
one on the right side. A final one was directly above the rectangle. Tracy drew
the rectangle and put the three objects on it, to left, right and middle correctly
for all of the 5 vantage points. For example, from her vantage point, she drew
the cube (depicted by a square) upper left, the ball (depicted by a circle)
in the lower middle, and the cone (depicted by a dot for the pointed top, with
a triangle filled in with arcs below the dot) to the right and half as far up
the page as the cube. Also, the further away an object, the higher up in the
picture plane she drew it. In her drawing to show the array from the vantage
point of someone sitting opposite her, the arrangement was perfectly left-right
and top-down reversed.
Interestingly for the vantage point "above" the rectangle (and no longer level
with the tops of the objects) she said that since the vantage point is high,
the array of objects should be drawn smaller, and she did so. The array "from
above" was drawn as 4.2 cm on a side, smaller than 5.6cm for the corresponding
side in the drawing made to depict the array for an observer on the right, which
was the drawing made just previously. The distribution of the forms is precisely
in keeping with the changing vantage point. Further, what is further in distance
from the observer is shown by drawing the form higher in the picture plane.
The mention of smaller size with increased distance "from above" suggests polar
projection. However, the rectangle on which the objects sat was always depicted
by a rectangle, suggesting parallel projection and similar shapes.
To probe her use of polar projection, Tracy was asked to draw two rows of three
glasses evenly spaced and receding directly away from her across the table top.
That is, if each row were to be extended in straight lines, she would have been
sitting midway between. All the glasses were the same size. She drew the glasses
as U shapes. She used two large U shapes (2.1cm tall and 3.5 cm apart) for the
pair of glasses near the observer, two smaller U shapes ( 1.4 cm tall and 1.9
cm apart) spaced closer together for the pair of glasses halfway across the
table and two still-smaller U shapes (.6cm tall and 1.1cm apart), spaced even
closer together, for the glasses furthest away on the table. The further the
glass, the higher the U on the picture plane.
In the drawing of the rows of glasses, Tracy uses convergence. The drawing uses
decreasing sizes of objects and decreasing spaces to deal with increase in distance.
It is of interesting that the space from the base of the glasses in the lowest
row of glasses to the bases in the middle row is 3.5 cm, but the distance from
the bases of the middle row to the bases of the upper row is merely 3.0cm. This
diminution with distance is in keeping with polar perspective.
Tracy's drawings include many features richly deserving comment. Her lines show
corners where two surfaces meet, and may show the occluding borders of flat
and curved surfaces. Her raised lines are small ridges with two sides, on an
otherwise flat surface. Each ridge depicts a single border formed by changes
in depth and slant. She considers the vantage point of the observer, only depicting
the surfaces of the object that face the observer. She uses line shapes similar
to the shapes of the object's surface boundaries, such as squares for surfaces
of cubes. Her shapes may involve projection from occluding boundaries, as in
using quadrilaterals for the faces of a cube, and circles for spheres and triangles
for cones, though alternatively, these circles and triangles might be cross-sections.
Tracy's devices
It is especially intriguing that Tracy uses features of projection to the observer.
One kind of projection is a kind of parallel projection in which rectangular
surfaces of the object are depicted by similar shapes in the outline drawing.
A second is polar perspective, in which smaller size in both the left-right
and vertical dimension means increasing distance. Tracy's drawings of the cube
and the three-mountains may be in either kind of projection. The rows-of-glasses
drawing uses aspects of polar projection.
Both polar and parallel projection allow height in the picture plane to depict
depth. The higher the object in direction from the observer the higher it is
depicted in the picture plane. As a result, besides preserving the left and
right order of the three "mountains", the picture preserves their vertical directions
from the vantage point.
Tracy has been interested in drawing since she was young. She reports that she
is very largely self-taught. She tackles new pictorial problems such as drawing
two rows of glasses without seeking answers from others, and her solutions are
ones she devises on her own initiative. Tracy is representative of congenitally
and early blind people, interested in drawing, with largely self-taught abilities,
who have participated in our drawing studies in several countries (Kennedy,
1993), though Tracy is at the upper end of the drawing-development spectrum.
If we trace the development of drawing to the advanced level Tracy exemplifies,
the evidence suggests that we will find that blind and sighted children are
remarkably alike in three ways: use of outline, use of shapes similar to the
shapes of faces of objects, and use of projection (parallel and polar). Communication
with pictures is indeed multimodal.
Theory
Tracy's abilities with outline and projection systems may surprise many. Her
drawings call for new theories of touch, outline and projection.
Pictures are about the sources of our perceptual inputs. They represent those
sources by replicating aspects of those inputs: optics and what is tangible.
Just as vision is the psychological science of the optic input, haptics is the
science of what is tangible. Vision is to light what haptics is to resistance.
"Touch" is the informal term for a sensory system included in haptics. Since
light and resistance are quite distinct, many commentators have considered that
representations making sense in one domain would be foreign to the other (Arnheim,
1990; Gibson, 1962; Hopkins, 1998, 2000). However, Tracy's pictures, and pictures
from other blind individuals (Kennedy and Bai, 2002; Millar, 1994; Eriksson,
1998; Lopes, 1996, 1997) undermine that view. But how can outline operate similarly
in vision and touch?
Light fills an angle at the eye. Within the angle, it varies in intensity and
spectral composition, or wavelength. These properties vary with time. In contrast,
the variables of tactile resistance in time are pressure, or variation in weight,
and area of skin covered. The variables of optics and haptics are not fully
commensurate, far from it. Granted, angle subtended at the eye converts to area,
by means of the amount of retina covered by the optic input. But intensity of
weight (pressure) is not directly related to intensity of light in physics,
and spectral composition has no sibling in resistance. The conclusion that seems
to follow is that many creatures of vision should have no equivalent in touch.
The size of the object providing the visual input is not directly related to
the size of the retinal area covered, since much depends on the distance of
the object from the eye. In contrast, what presses on the skin is the actual
object itself, not a projection from the object. One might think therefore that
vision is distal and touch is proximal. Pictures are like transparent windows
on a distal world. But feeling the surface of a picture in raised form is not
like looking at a pictured world beyond the depicting surface. It cannot generate
an impression of a panorama beyond, one might conclude.
But of course the influences on vision and touch are not entirely incommensurate.
The sweeping claim about differences between the distal and the proximal is
much too restrictive. The key is variation in time. Using the proximal, touch
operates across time to discover the distal relations between objects. Touch
is not just a matter of sitting at a desk and feeling a surface with some raised
markings on it. Touch is more than the domain of a few finger movements. We
take time to walk around and touch things. Touch investigates space around us.
Touch provides information as we get up from the desk, walk around the room,
wander around the house, and step into the garden and along a path. Touch involves
a point of contact changing across time. Also, at any given moment, touch has
a point of outreach, a base from which exploratory movements are made. So touch
uses the proximal at any instant but discovers the distal across time, and thereby
the directions of things from many vantage points (Millar and Al-Attar, 2002).
Presumably these directions include the directions of objects from each other.
A tactile picture can perhaps be taken as an indicator of the directions of
things from a vantage point. This may require some sophistication on the part
of the observer, but nevertheless be quite feasible for most adults. If so,
touch offers a base for Tracy to employ forms of projection.
Operating across time what touch finds is the surfaces of extended objects.
Only a fraction of the surface is touched at any instant, but what haptics registers
is the surfaces and their size and orientation. Some are large and some small.
Some face vantage points from which the observer reaches out, and some face
away, so we need to reach around occluders. The border between what faces a
vantage point and what does not is often quite precise, as in the case of corners
of a cube. It can be precise too for rounded objects if the observer considers
what faces a particular small point in space. An undulating surface that we
feel in an extended movement occupies a set of directions from our location
at any given moment. This set of directions is part of the group of impressions
of the corrugated distal surface stretching away from us. The part to do with
direction from a point may come to the fore and be useful for some observers
exploring tactile pictures on some occasions.
Line
Borders of surfaces are shown by Tracy in her line drawings of cubes, balls
and the like. In these uses of outline Tracy emulates what has been present
in visual art since Cro-Magnon simple line sketches 50,000 years old (Kennedy,
1993)
Cave artists discovered several uses of line in pictures, the core being the
ability to stand for edges of surfaces. The same uses are deployed by line in
pictures for vision today. Further cave artists discovered continuous surface
edges could be shown by discontinuous hatch markings in pictures, that is lines
with gaps in them. In addition, lines in cave art pictures are often formed
by grooves. These have two sides, much like Tracy's ridge line but in reverse.
In cross-section, a groove is like the two hillsides of a rift valley with a
valley floor in-between. The two sloping hillsides are plain to see. But though
there two sides, the referent of the sunken groove in the outline drawing is
a single border, and like Tracy's raised ridge line with its two sides, means
a single change of depth or slant.
Just as lines in pictures can be of several kinds -- black pencil lines, white
chalk lines, ridges, grooves -- there are many visual inputs from the real world
being depicted that create impressions of borders. Consider the many kinds and
how they relate to the surface edges which lines can depict.
What creates visual borders has to do with luminance, colour, stereo and motion.
An intensity contrast between solid black and white is a luminance change, and
readily gives rise to perception of clear, sharp, continuous borders. Visual
area V1 handles apparent orientation of such borders and could play a key role
in detecting many of their properties. V1 also has subareas relevant to purely
binocular borders. Instructively, V1 stereo cells help support perception of
apparently-continuous borders linking visual elements that are physically spatially
separate. That is the input has gaps but the stereo percept has an illusory
contour that appears continuous. These seemingly-continuous borders readily
look like the edges of surfaces. Likewise, area V2 helps produce a strikingly-continuous
kind of visual border, called a purely subjective contour, from spatially-separate
luminance borders. Given a monocular, static pattern, with suitable luminance
variations, V2 performs this feat of extending continuous illusory contours
across gaps, thereby linking physically-present contours. The illusory contours
resemble infinitely thin wires, or they can appear like occluding edges of a
single foreground surface, or corners formed by two abutting surfaces. Areas
of visual processing beyond V2 likely contribute to the illusory contour looking
like a surface edge.
Small-scale motions of luminance borders can also be grouped in regions. A swarm
of insects, or a school of fish, contain individual items making small scale
motions. Given sufficient density, they suggest a body of a definite shape,
but one without continuous borders. A key brain area for small-scale motion
is V3. The point to stress is that these motions can give rise to an impression
of linked or grouped visual elements that are actually spatially separate. Further,
the motions of spatially separate elements can produce subjectively-continuous
contours if elements accrete and delete at a border. The subjectively-continuous
linkages may seem to be corners, or occluding boundaries of surfaces, depending
on the motions around them, much as a herd of individual animals moving up and
over the brow of a hill could define the hilltop as the occluding boundary of
a rounded surface, or a crowd of skiers vanishing behind a mound can define
the white snow-covered mound's perimeter against an equally-white snow-covered
ski-slope.
Another kind of border is made from a pure colour change. These strongly influence
V4, and can be modestly effective at suggesting surface edges. Another area
(V5) may be able to group elements and make visual borders that look like surface
edges if the visual input is large-scale motions of individual luminance borders.
To test for large-scale motion perception without small-scale motion, an element
can vanish in one visual location and burst into view at another visual location,
as if passing through a tunnel. The locations would only be only a degree or
so removed in small-scale motion. Also, to obtain clear apparent borders from
large-scale motions, the element may need relatively straight contours and sharp
corners.
In sum, each of the cortical areas V1 to V5 may contribute information about
borders of surfaces, including the perception of change of surface slant at
the border (a corner), or change of depth at the border (an occlusion edge).
An implication of occlusion and corners being evident is that each border-making
cortical region may contribute towards impressions of surfaces indicating the
observer's vantage point, and as a result they may be governed by a geometry
of depth to some extent.
Borders and vantage points
It is an open question how V1 to V5 not only define visual borders but also
use the geometry of vantage points and projection to handle depth, slant, occlusion
and the observer's locus that is implied. Indeed, every single visual area relevant
to visual borders may use a projective geometry such as perspective. After all,
polar perspective is the exact geometry of the variation in depth of objects
from a point. It is clear that a projective geometry is important in getting
information in stereo vision. Physically, the same basic geometry applies to
motion in depth, since motion of the observer through an object-filled environment
yields a field of objects changing their directions around the observer.
Each visual kind of border may trigger a brain area with its own version of
projective geometry. Or, alternatively, they may feed a set of borders like
a flat spiderweb to central regions of the brain. Perhaps only in the innermost
core of the central perceptual computer do the individual surface features begin
to trigger perception of the perspectival relations between them. Only finally
and centrally do they yield perception of a landscape. The issue remains to
be settled. Indeed, how generally vision uses the properties of polar projection
is still unsettled (Kennedy and Juricevic, 2002). The most likely account would
be that each of V1 to V5 contains its own perspective geometry, but the geometries
are similar in major principles, only disagreeing on matters of precision. One
way to test this would be to show pictures of squares, with varying amounts
of foreshortening, triggering V areas selectively. Each area might accept a
certain range of foreshortening, but the range might differ from V1 to V5.
Just as vision involves brain regions reacting to different kinds of borders,
haptics may be served by distinct brain areas specialized for different aspects
of tactile borders.
Much like the visual cortex, the cortical regions for touch can be divided into
a primary input-reception region, S1 (with several specialized layers), an adjoining
region S2 to which S1 feeds signals, and further regions to which S2 sends signals.
The regions are highly interconnected. Much like the visual system, touch is
organized hierarchically, with more complex pattern processing following on
the initial and simpler processing, and sending feedback to the simpler processing
stages.
Here, we will only present possible specialties to illustrate the argument.
Borders on the skin between regions with different pressure may be served by
some cortical area we shall call SX with cells that respond to the specific
textures in the regions. The next stage, SY, might contain cells responding
to the border itself, that is the changes at a border. Kennedy (2000) describes
an assembly of cells that can detect such borders. SY might also be especially
sensitive to different directions of motion on the skin. Vibrations (pressure
variations in time) may have different effects in SX and SY, and borders between
them. Attention to a skin region may affect SX's sensitivity much more than
SY. Anticipation of a stimulus may actually decrease sensitivity in SX more
than SY, and play a role in tuning the cells for effects of active exploratory
motion (Chapman, 1994). Certainly, some tuning to information about locations
in space being attained successively because of motions of joints, tendons and
muscles is especially relevant to sensorimotor input. Theory of the relation
between touch, attention and motor control holds that touch is part of a system
that incorporates modulation in sensitivity due to predictable inputs from active
motion, and modulation in input across time due to movement of the stimulus
(Chapman, 1994). Further, touch incorporates cells with selective sensitivity
to motions in particular directions, much like vision's regions. But how sophisticated
is the analysis of space in S1, S2 and adjoining regions? It is hardly credible
that the initial cortical analysis of tactile input would contain information
about surfaces and perspective variations. But this kind of analysis must be
performed somewhere that is open to tactile input.
Tactile information is related to motion of the body, information about the
direction of targets, and input from surfaces (D'Angiulli and Kennedy, 2001).
This means the S1 and S2 input from a few pressures at the fingertip as we palpate
(or our toes as we walk) at a given moment is related to an integrated pattern
from the bodily platform carrying and controlling the fingers (and toes). It
is this mobile platform and its postures that contains spatial information writ
large.
Vision begins with variations in light and produces percepts of surfaces. In
physical dimensions, the input is not commensurate with the product. They are
of different kinds, physically. Touch too begins with variations, though for
touch these are pressure variations related to postural information. But it
also yields impressions of surfaces. Touch, like vision, deals with edges, slant
and depth, providing percepts of surfaces and their borders. As in vision, the
currency of touch is the orientation of surfaces in space, including change
in slant and depth, over which touch can range. Pressure in input, as it is
related to mobile postures of the body, is largely exchanged for surfaces and
their locations in the percept. The products of the haptic and visual systems
have much in common. Vision's products imply the observer's vantage point, and
so too do touch's. As in vision, a tactile concave corner involves an observer
enclosed by surfaces, an occluding edge implies a front surface facing the observer
and a rear surface is facing away.
If touch's products in perception imply an observer, they also imply projection
of some kind (Kennedy and Bai, 2002). But what kind? A front surface faces toward
the vantage point, but does this involve polar or parallel projection? Either
one fits the bill. Likewise, recognizing that the rear surface faces away from
the observer could entail polar or parallel projection. Tracy may deploy different
projection geometries, polar and parallel. It seems likely that she appreciates
some aspects of a variety of projection systems, and the issue then for her
is what is suitable on a given occasion. Of course, her savvy with projection
systems is normally used in the real world. What is challenging in drawing is
how to bring that knowledge to bear on projection onto a flat surface. Tracy,
like most sighted people, is still working out the implications of the use of
a projection surface. What is evident is that she has ready use of a vantage
point, projection of similar shapes, and is able at times to apply polar projection
to the overall size of objects and the spaces between them. She realizes that
in principle further distance implies convergence. The change in distance means
a difference in the relative directions of two objects, with angular subtense
shrinking. Likewise the difference in direction of two parts of an object, such
as the left and right sides, lessens with distance.
Outline and links
In addition to speculation on the origin and geometry of borders, there is a
key matter of representation to be addressed. Both visual and tactile borders
can be depicted by lines. How is this done? What is most enlightening is just
how different the line in the perceptual machinery can be from the physical
input on a page that supports it. The input line can have two sides and yet
what is depicted is one change of depth or slant. The input line can be made
of dots with spaces between, but the referent's border can be depicted as continuous.
Evidently there is a function in perception which treats some property present
in many kinds of lines as also present in many kinds of surface borders.
A swarm of insects has a border, though the insects are separate elements. One
way to define the border is to say the low-frequency variations (the sorts present
in a defocused picture) indicate the rough layout of the swarm. High-frequency
variations are highly localized ones, like the continuous, abrupt borders in
hard-edged graphics. Should we say an outline provides a high-frequency border?
Perhaps not. An outline picture can be made of fine dots, far removed from continuous
hard-edge graphics. Alas, outline is also far removed from the low frequency
variation that can describe the location of a swarm, since a single line of
fine dots will not show up in a low-frequency (defocused) image.
One intriguing possibility is that tiny elements such as dots can define an
area enclosed by the line, and the area itself can be taken as a low-frequency
variation. Dots define lines, lines demark areas, and areas are interpreted
separately from dots or lines. This successive-step model is a promising beginning.
It goes through a stage in which spatial links between dots are processed, captures
some high-frequency input (continuous abrupt borders), before accepting these
as boundaries of regions, and then defines regions as broad or defocused swathes
of the input. This model could be applied to touch as well as vision, since
touch can detect dots and note the relation between them. While this is a promising
start, it cannot be the key story however. The defocused end-stage is not the
proper terminus. Outline readily shows precise spatial features such as nostrils
and earlobes in a drawing of a head, or twigs in a drawing of a tree. These
details are lost in defocused images. Therefore the kind of level of analysis
needed is the one evident when we consider short links between tiny dots. A
further caution to note is that the links themselves can be the representational
element, as in a drawing of a twig. The regions bounded by the links are relevant
to occlusion edges and corners, but we should not forget twigs, stick figures,
wires and the like where the line itself depicts a linear foreground referent.
A useful feature of the discussion of links is that links between dots are lines
without thickness. The dots they join may have appreciable width, but the link
is widthless. This may help us understand the function in perception which treats
the width of a line, between its two sides, as immaterial. In outline drawing,
two sides of a line show one surface edge. When we look at a cross-hatched line
in an outline drawing, we accept what is discontinuous as equivalent to something
continuous. The links between the hatchings are what does the work.
Perception's procedure for finding widthless links may be an asymmetric function.
We do not usually treat one side as equivalent to many, or what is continuous
as equivalent to what is discontinuous. In standard practice, the individual
contours and dots in the perceptual input trigger an impression of particular
relations between the group of elements, a particular impression of linkages
we might call the gestalt of a line. This is an asymmetric function because
the elements trigger a grouping function, and this resulting group or gestalt
does not trigger perception of an element.
The gestalt of the set of linked elements can have the form of a single linear
group. But grouping alone is not enough. The linear gestalt is the form which
is in common to all outlines. It is this which acts as an outline and supports
the perception of corners and occlusions. The gestalt function operates in both
vision and touch, clearly, since drawings by the blind resemble drawings by
the sighted. That is, it is the linear grouping produced in both touch and vision
is a continuous link between separate elements even if the physical input is
discontinuous, one dot visually spaced apart from the other, or each dot touched
one at a time by the exploring hand.
One way to achieve apparent continuity in touch is for the tactile gestalt line
to appear as continuous, much as a subjective contour from V2, stereo or motion
can look continuous. Alternatively, the dots can simply be grouped as a line,
much as three equally-spaced dots can appear as vertices of an isosceles triangle
without the triangle sides appearing as faint lines. That is, the modus operandi
of a gestalt line may be to allow detection of the gaps between the dots but
have the gaps deemed to be irrelevant, as if the dots are taken to be information
for a continuous referent. The dots represent continuity but do not constitute
it. The dots in this case mark the location of a continuous contour, but the
contour does not appear, only its location. In short, we need to develop theory,
first, on grouping and machinery in perception that can handle relevance and
irrelevance in different ways that might produce gestalt lines, and second on
how the location of a gestalt line in turn might then be taken to be a representation
of a continuous surface edge. At present all we wish to do is point out the
issues that need to be solved.
Perspective
Besides outline, Tracy's pictures have in common with pictures for the sighted
vantage point geometries. In particular, her pictures include use of convergence,
that is diminished sizes of forms depicting objects, and foreshortened distances
between the forms. These features make eminent sense to vision, but here they
are in her tactile pictures. Evidently a general theory of projection onto a
picture surface is needed, one more general than vision alone. In searching
for such a theory, caution is advisable. While much may be in common to vision
and touch so far as perspective is concerned, much may also be unique to vision
or touch. Consider some distinctive factors, some to do with the angles subtended
by the objects in a scene around us, and some to do with linear extents on a
picture surface showing a scene.
As objects recede from us, the angle they subtend at the eye diminishes. Artists
contend with this fact in drawing landscapes. It is telling that artists often
have to resort to procedures such as holding up paintbrushes to estimate the
relative visual angles of two objects with any accuracy. The reason is we are
often mislead in judging the visual angles unaided unless the objects are close
together in direction from our vantage point. The further apart the objects
are in direction the more error creeps in. Vision often regards two equal visual
angles (a dimension measured in degrees) as strikingly unequal, for reasons
to do with the linear size of objects (a dimension measured in centimeters).
Likewise, a more-distant object (like our left hand, held off to our left) that
is making a small angle at our eye can seem to subtend the same angle as a nearby
object (like our right hand, held off to our right) since it is the same size
in cm, even though the near one subtends say twice the angle.
In sum, when subjects look at objects their impressions of angle-subtended are
contaminated by optical information for linear size (Juricevic and Kennedy,
2001a). There is no known direct equivalent for this in touch. When we point
at objects from afar, the farther the object the smaller the subtense of the
top and bottom of the object so far as touch is concerned (Kennedy, 1993). It
seems unlikely that in comparing the angles subtended by two objects, touch
would benefit from having the two objects in the same direction, and thereby
escape a bias towards linear size ratios. Vision's accuracy depends on the objects
being close in direction, even if the objects are far apart in distance from
us. Touch is likely to have distance as a key influence on accuracy, but not
proximity in direction. Touch would readily compare two objects 180 degrees
apart, simultaneously exploring the two objects, one per hand.
Binocular vision obtains excellent information about linear size and distance
of objects. It responds to differences in the visual angles projected by an
object to the two eyes. The differences in visual angles constitute information
about linear sizes. Binocular vision obtains one percept, but bases it on two
different visual angles. Since the two visual angles are not available in the
final percept, it may be no surprise that judgments of the relative visual angles
of objects are poorer in binocular vision than monocular vision. They are especially
strongly biased towards linear size information (Kennedy and Juricevic, 1999).
Vision makes use of many relations produced by polar projection. To detect the
relative sizes of objects vision may make use of "meridian rules." The visual
scene contains many similar objects and we can assess trios of these by seeing
how the tops and bottoms line up. The imaginary line formed by aligning parts
of two parallel objects in a scene we can call "a meridian." For example, three
vertical columns on a flat surface may have tops in a line and bottoms in a
line. If the two lines converge to the horizon of the flat surface, then the
three objects must all be the same size. If all the bottoms line up but the
top of one is head and shoulders above the line joining the other two, it is
taller. This kind of "lining up" of bases of objects on a plain defines the
meridians of the landscape. follows that if the line joining the bases of two
columns, and the line joining their tops converge to the horizon the objects
are the same size (Sedgwick, 2001). Informative meridians can be chosen freely
from any objects and provide information about the relative size of the objects.
It may be that the lining up of a trio of parallel objects (or a pair and convergence
to the horizon), not only gives us the information that they are the same linear
size, but can bias vision to some extent towards the misleading impression that
their visual angles are similar.
Vision takes in a sweep of 360 degrees around us. But when we come to draw,
or compare visual angles of objects in pictures, we select a fraction of the
full panorama. In particular, we treat some objects as to the fore (directly
in front of us) and others as to the side (left and right of the objects to
the fore). In comparing visual angles of objects, our skill with pictures and
projection may intrude. We may envisage the objects projecting onto a pictorial
screen, as if we were preparing to sketch the object on a window through which
we can see the object. The window to the fore is likely perpendicular to the
line from the observer to the objects. A screen to the side may be an extension
of the screen to the fore, or be turned so as to be perpendicular to the line
of sight. In either case, it may be further from us than the forescreen, and
compensate for the diminution of the object with distance. In judging the object's
visual angle, we may act as if we are influenced by the size of the object's
projection on the screen. We may act as if we estimate the fraction of the screen
covered by the vertical extent of the object, for example. The upshot would
be the visual angle of objects to the side would be overestimated by vision.
Indeed, objects to the side (and more distant than objects to the fore) are
seen as having relatively larger visual angles than is correct (Juricevic and
Kennedy, 2001b). Asked to judge the relative visual angle of objects to the
fore and to the side, we make judgments that approach the linear size ratio
of the two objects.
Apparent size of pictorial elements
Linear size's influence on apparent visual angle likely plays a role when we
try to perceive the elements on a picture surface. Vision scans the lines on
picture surface from a distance. Touch explores them with direct contact. The
elements are distal for vision, and proximal for touch. Does this mean vision
is subject to error and touch must be accurate? Far from it! Touch has geometrical
illusions much like vision (Millar and Al-Attar, 2002). That is, opportunities
for misreading the elements are present for both senses. Perspective patterns
on the surface may influence attempts to detect the actual lines on the surface.
The picture may be taken as a setting for a element and to the extent it depicts
a scene the detection of the line element may be compromised. "Crosstalk" is
a term Sedgwick (in press) uses for this source of illusion.
A point to note carefully here is that crosstalk may differ from sense to sense.
Both senses may use one set of geometrical principles, but once the main function
is achieved there is room for different ancillary effects. Vision has striking
impressions from perspective pictures, and touch may not achieve the same precision
of depth perception from pictures. Rather, touch is likely able to emphasize
direction, where vision may emphasize depth, for example. Vision is strongly
affected by the optical information for the linear size of a depicted object.
In crosstalk, this information affects the apparent visual angle of an object
in a scene. The optical information for the apparent length of lines on a page
is in many respects the same as the optical information for the visual angles
of the object depicted by the line. Hence, the optical information for the linear
size of a depicted object may influence the apparent length of the lines on
a picture surface, and vice versa -- the optical information for the size of
an element on a picture surface may affect the apparent size of the depicted
object. Since subjects misjudge visual angles, influenced by linear size information,
they should misjudge the size of the pictorial elements too. However, the frame
of the picture and the pictorial surface texture may help limit errors in judging
the element's linear size. If so, the crosstalk may particularly strongly affect
the depicted object's apparent size. The element's size may affect the apparent
size of the referent.
The pictorial elements are just parts of pages that correspond to visual angles
projected by object parts. For every misjudgment of visual angle there should
be a comparable error of pictorial element judgment. Just as separation in direction
allows the linear-size effect on visual angle to grow in vision, the illusion
of differences in size on a picture surface is enhanced by separation (Juricevic
and Kennedy, 2001b). Vision may be more prone to this error from depicted linear
size than touch, and show the effect of separation in direction more strongly
than touch.
Touch and imagined projection screens
In one respect projection screens may be considered alike by vision and touch.
When we draw objects, we often try to draw the form that would be present on
a projection screen, and we envisage a screen at some location between us and
the object, on which the object is projected. Often the screen we imagine is
on the normal to the line joining our vantage point to the object. That is,
objects to the side and objects to the fore are not imagined projecting onto
one single continuous flat screen. But when we draw, we usually have no such
choice. Typically, we are drawing on one continuous flat surface. The consequence
of the mismatch between our imagined screen and our drawing surface's orientation
would be too much use of foreshortening in the drawing (Kennedy and Juricevic,
2002). Consider: All surfaces parallel to a flat continuous surface project
without foreshortening in the sense that perfect squares and circles project
as squares and circles. Let us align the front faces of several cubes. Consider
drawing the panorama of these objects on one flat picture. The front faces are
parallel to the imagined screen for an object to the fore. They are drawn as
squares, and not foreshortened. But front faces on cubes considerably off to
our side are now tilted so far as our line of sight to them is concerned. If
so, they would be drawn as foreshortened on the panorama by mistake.
We suggest this same error of overuse of foreshortening will be made by those
using polar projection in touch as well as vision. The blind can envision forms
of projection. But they may be as likely as the sighted to entertain a projection
screen perpendicular to the line joining their vantage point to the object.
A cube set much to their side, and a cube directly in front of their vantage
point, will seem to present "front faces." Let the cube to the side have a face
in the same plane as the cube to the fore. That face for the cube to the fore
will seem to be a front face. But the aligned face will not seem to be a front
face for a cube much to the side. The face deemed to be the front face will
be closer to perpendicular to the line joining the cube to the observer's vantage
point.
In sum, the typical blind and sighted subject who turns to polar projective
drawing has only a very partial understanding of projection. It seems likely
that Tracy's pictures are similar to many a sighted adult's level of skill.
Also, perceptual borders are triggered by different kinds of input. Do each
of these follow polar and parallel geometry in their processing? The lesson
from Tracy's drawings is that touch as much as vision -- very different kinds
of input -- may be subject to roughly similar geometrical principles. It seems
that apparent shape may be similar despite major differences in input. The importance
of the vertical, left and right, facing towards and away from, the location
of the observer, the role of depth and angle subtended -- all of these may play
similar roles in coping with diverse inputs.
Limits and tropes
It may be helpful to mention briefly the limits of a system such as outline
or projective geometry. Line may be used similarly in vision and touch, and
many aspects of projection to a vantage point may arise in both senses. But
the implication is that many of the limits of one sense may be akin to the limits
of the other. If so, this provides the conditions for tropes in human communication
to cut across sighted and blindness status.
In language we might say "the man had a heart of stone" when we are searching
for literal words to describe the man's lack of fellow-feeling. We overcome
our verbal limitations by using a metaphor. Similarly when we find we cannot
draw the wind, or smell or pain or motion we may be tempted to turn to a pictorial
metaphor. We may draw the spokes of a wheel as transformed into concentric circles.
We may draw paths of sounds bouncing around a room to convey echoes. In kind,
we may draw the stony man as metallic.
These pictures we may consider to be going beyond a limit for literal depiction.
Intriguingly, such pictures beyond the limits can be devised by the blind and
the devices they invent are interpretable by the sighted. The implication is
that devices that lie beyond the range of literal re-presentation can employ
apt features. The features have some literal function, just as the word "stone"
has its literal use, but some of the features can be put to use in a setting
in which the referent of a literal term does not belong. In the new setting,
the device refers to some of the relevant features of the object, and a pictorial
metaphor is born, interpretable by the blind or the sighted (Kennedy and Merkas,
2000).
Summary
Pictures are a form of representation that can be haptic as well as optic. Line
in visual and tactile pictures can depict depth and slant features of surfaces.
Some aspects of the observer's vantage point and projective geometries are available
to the blind and the sighted, and can be expressed in pictures. The limits of
pictures provide opportunities for tropes intelligible by both the blind and
sighted. Communication by pictures can use more than one modality. The implications
are diverse and call for new theories not only of pictures but also of vision
and touch.
References