Translator: Leyre Bastyr
Reviewer: Rhonda Jacobs People with synaesthesia
experience the ordinary world in quite extraordinary ways. For some people,
as they're listening to my words now, each word is its own, distinct colour, perhaps projected in their mind's eye
or floating like a ticker-tape in space. For other people,
when they listen to music, it's not just an auditory experience, it's an animated visual spectacle
of dancing shapes, colours and textures. For some people, such as Carol Steen, in this picture, when she has acupuncture or feels touch or pain in her body,
she experiences dynamic floating commas, in this case bright red
against a deep velvety background. So synaesthesia seems
very extraordinary to us, but to people with synaesthesia,
that is the only reality they know. This is how they experience the world, and for them this is a perfectly normal
way of experiencing things. It isn't just about vision. In this example, this is James Wannerton's
depiction of the London Underground. So for him, every word that he hears,
thinks about or reads is like an ebb and flow
of flavours and tastes that are subjectively
appearing on his tongue. So some train stations might taste
like apple pie or celery, and so on. And to him, this is how he experiences
things since childhood. For other people,
they may have auditory experiences. Can we have this movie, please? I don't know whether people can hear this? Well, this is a silent movie,
so you shouldn't be hearing it. But for some people with synaesthesia, this creates a whooshing,
shooshing sound, for instance, or a change in pitch, very dynamic. So for these people,
everyday life is very noisy, whether or not they're literally
hearing something or not. More formally we can think of synaesthesia
as a kind of joining of the senses, in which they experience something extra
tagged on to that which most people do. So in this case, they're having
an extra auditory experience on top of a more typical
visual experience. Synaesthesia turns out to be quite common. You all know somebody with synaesthesia,
but you probably don't know who it is unless you've asked them,
'What colour is this piece of music?' Synaesthesia
is a real biological condition. It runs in families, although the particular associations
don't necessarily. So, for instance, somebody who experiences
synaesthetic taste may have family members
who experience synaesthetic colour. So it's a general disposition here, and we know that there is
a genetic basis to synaesthesia through studies of the genome. Although we don't fully understand
what these genes are doing. They're almost certainly affecting
the way that the brain matures because synaesthetes
have differences in their brain, both in terms of gray matter -
they've got more gray matter density in, say, parts of the brain
to do with seeing. So parts of the brain seeing colour,
they have more of this. And similarly with regards
to connectivity - they have more white matter connectivity
between different regions of the brain. So they might, for instance,
connect the auditory parts of the brain with the visual parts of the brain
in a way that other people don't normally. And I say 'don't normally', but there's one claim in that we all
had synaesthesia at one point of our life and that's when we looked like this -
when we were infants. So one suggestion is that
we're all born with synaesthesia and most people lose it
as part of the normal maturation process. But synaesthetes, due to
their different genetic composition, retain these roots
that link together the senses. And we may not all fully lose
synaesthesia entirely; we're all able
to link our senses together, and synaesthesia reveals the rules by which we can understand
the links between vision and music. And this has important implications
for arts, for instance. So nobody knows whether Kandinsky
was a synaesthete or not. Some people say he is,
some people say he wasn't; but he knew about synaesthesia. Synaesthesia was well documented
at the time he was producing his artwork. And what Kandinsky wanted is that people
should understand his artwork not solely through the visual medium, but they should understand it
more as a multi-sensory kind of gestalt that encompass all the different senses. So I'm just going to do a little test now, and I'm just going to play
two different sounds, and I want you to think which sound
goes with this Kandinsky painting. He didn't to this experiment,
by the way, this is just me. So if we could listen sound one, please. (Discordant violin music) And sound two. (Harmonious violin music) There's no right or wrong answer, yet most of people
tend to go with sound one here, that's more discordant,
whereas sound two is more harmonious. So again, this might sound trivial,
but actually, why is that? What are these particular rules? And it turns out that
there's a whole set of rules for linking vision and music together, and we can analyse this by looking
at the experiences of synaesthetes, and a lot of them are present
from the early stages in infants too. So we know that
they're not purely cultural. If anything, the way
that our senses interact might be influencing our culture
rather than vice versa. So, for instance, if you're a synaesthete, listening to a high-pitched sound
relative to a low-pitched sound, a high-pitched sound
is going to be brighter, it's going to be smaller, it's going to be higher up in space and it's going to be
more jaggedy, for instance. And most synaesthetes
tend to have these principles, although they might differ
in exactly how it's experienced. So we've done various experiments
with these particular kinds of stimuli. So one of the things that we've done
is that we've taken these descriptions of how synaesthetes see the world,
and we've tweaked them in some way. So, for instance, we can simply
just rotate it left to right, or flip it top to bottom. We can change the colour - so if a synaesthete experiences
a yellow disc doing this, we can flip that colour
to, say, red, or green, or brown, or something like this. And what we can then do
is play the pair of movies to people and ask them which image,
which movie goes best with the sound. And what you find is that people
who don't have synaesthesia, they choose the synaesthetic
original movie as being the best representation, or as being the most aesthetically
pleasing one that goes with it. So people who don't have synaesthesia
can tune into these properties that synaesthetes naturally experience
within their mind's eye. We've also got people
who don't have synaesthesia to try and draw from their own experiences what it might be like
to experience synaesthesia. And, again, if we show those experiences
with a true synaesthete, people prefer the true
synaesthe's experience here relative to the one
that somebody's tried to recreate. So the synaesthete probably has a richer,
more vivid kind of vocabulary for matching this, but other people
use the same rules as well. What does that mean
for creativity and art? Are synaesthetes more creative? Well, we've done
formal measures of creativity and showed that
there does seem to be some trends for synaesthetes
to score better on some tests. What is perhaps more striking is that
if we look at occupations of synaesthetes, or their hobbies, we find that they tend to gravitate
more towards the arts. So these unusual experiences
are probably very beautiful for them, and it probably inspires them
to create art. But it might also affect their brain
in ways that actually is helpful. So we know that synaesthetes
are better at distinguishing between different colours
that to most people look very similar. They can tell them apart much better. So that would be a raw ability
that would help in the world of art. This is one of the artists
I've worked with who lives near me in Brighton. She's a portrait artist, but rather than painting people's faces,
she paints people's voices. So this is from a CD, but she sometimes
does interviews with people and paints their voices
as she's having a conversation with them. So this a very novel concept
that's inspired by her unique experience. So that's visual experiences.
What about touch or pain? So when I was a lecturer at UCL,
I sent an email out saying: Here's this thing called synaesthesia,
does anybody think that they have this? And I got a reply back saying, 'Well I don't know
whether this is synaesthesia, but whenever I see
somebody being touched, I feel touch on my own body.' So what she reports when looking
at this movie on the left is that when seeing a finger
going up and down somebody's face, such as this, she would say, 'I feel
a tactile sensation on my right cheek', as if you're looking in a mirror
of that particular person. They tend not to report
tactile experiences when looking at objects touched
although there's some variability in that. And what we find is that
when we look in the brains of people who are watching this kind of image here
seeing other people being touched, this is not just activating
the visual parts of the brain. So this is a purely visual stimulus, but the brain does not interpret it
solely in visual ways. And this happens both whether you have synaesthesia
or whether you don't. So you activate parts of the brain that are involved
in perceiving touch or pain. So it's almost as if you are embodying
what that other person is feeling. But for a synaesthete,
you experience that consciously, and for other people
it's more an implicit embodiment. Synaesthetes also use
some of these same mechanisms, but they use it in a somewhat
different way, drawing on different brain regions that probably enable them
to experience this consciously. What does this mean for art and for other kinds of constructs
such as empathy? What it means is that mapping
between the visual sense and the bodily senses
is something we do naturally, and it may enable us,
for instance in this image, to empathise with people
in pain, and so on. But this could also be an artistic device. So artists, when they're depicting
figures and pain and so on, they might not understand
that this is how the brain works, but they're wanting people
to actually be affected viscerally, to kind of empathise with what they see. And this is the actual
underlying mechanism of this. In collaboration with Daria Martin,
we've also looked at how these people with what we call
mirror-touch synaesthesia respond to artworks. This is a Giacometti sculpture - Giacometti produced
these distorted bodily images - and this is what
one of our participants says: 'I love to stand
in front of the Giacometti ... and it's a very good feeling. So I love to stand in front of them
and feel I am getting longer.' So what this person is reporting
is that their own body feels like it's being stretched or distorted
in some curious way. This image isn't depicting touch or pain,
it's just an unusual body, but nevertheless what these people have is a tendency for vision to override
their own internal bodily senses. In a way, they become
what it is that they're seeing, in a way that is really
interesting and profound. So what does this all mean with regards to the relationship
between art and science? For me, I think one of the most
profound things that synaesthesia tells us is that there really are multiple ways
of experiencing the world. And people have debated this
obviously for a long time - lots of other people
have drawn that conclusion - but science can give us
a real handle on that, and say, actually,
there are differences in the brain we can chase this
all the way back to genes and do the link right from genes
to unusual conscious experiences. But also that these different ways
of experiencing the world matter. They're not just epiphenomenal. Synaesthetes perform differently
on various measures. So seeing colours, they're good at;
they're better at memory, for instance. They gravitate towards the arts. These are real differences
that affect society. And synaesthetes aren't vanishingly rare. They're a few percent
of the population, we would say. I think the other thing
that we get from synaesthesia that tells us about the arts
is that it tells us the rulebook for how the different senses
play out and relate to each other, and this can be used
in various artistic ways. So, for instance, you can figure out what the best sound is
that goes with the colour red, and you can use this,
for instance, in art installations. One of the ways we're using this
is to work with people who are blind and have auditory depictions
of the visual world, so what we'd want to do is to sonify
this image that I'm seeing now so that they can understand
what colours are out there - the different spatial relationships
between objects. And that's not a trivial problem. How do you convert something
from vision into sound? But synaesthesia gives you a handle
on how you might go about doing that and have some real-world implications. So for me, synaesthesia is an absolutely
fascinating biological entity, but it really tells us something
about the human experience. Thank you for your time. (Applause)
