Sonification & The Problem with Making Music from Data

[Music] science what wonders you reveal regular discoveries of earth-like planets particle accelerators that reveal the building blocks of the universe and engineering feats that allow rockets to carry satellites into orbit we need to turn around and land themselves back on earth intact amazing however when it comes to music and sound you do sometimes disappoint and it seems that for every genius you provide you also impose a tax of one twice at watt with an unfortunate predilection for synthesizers and tall tales so what am I talking about well just listen to this so the process used to create this is called sonification and when I first heard it I just knew something fishy was up and it's not just the fact that you can't hear anything in space because there are no air molecules percent to travel through yeah when I heard this recording it wasn't the hand of God that I perceived but it was rather the scuff marks of a totally Chancellor I mean how is the sound of things is it really the thing itself but before I get into the bad aspects of sonification let's just take a look at its potential how it could be really cool so sonification is the method people use to turn data into sound two good examples of basics amplification or this and also this seven six that's inside a room so for both of these examples abstract audio was chosen to represent events in time and that's how that sound you heard in the intro was created - so right off the bat we can answer my first question no this isn't the sound of the thing itself in the same way that this isn't the sound of this so what's the benefit well let's take a step back for prospective data present the problem for us stupid humans it's often very detailed and tedious to sort through so over the years we've developed a myriad of visual representations to help us to quickly gain insight and recognize patterns at a glance just compare this to this however there's a wealth of data that for one reason or another can be meaningfully conveyed by graphs or heat maps which leads us to an interesting question is there a special type of implies that the sauna flight data can offer us which visual representation can't this is the type of question being researched by members of the International Conference for auditory display or ikat where researchers work towards the day where we might rely quite heavily on sound to understand things imagine wanting to know the latest on the footsie 100 and being places [Music] but more seriously another enticing aspect of sonification is accessibility for people who are partially excited or blind this is stronger quantity as fluster sight after a protracted illness and still what people have been able to do mainly visually for hundreds of years now I do it using using South sonification also excites researchers because there's a belief that our ears can better pick out patterns from noisy backgrounds than our eyes can and therefore perhaps it might lend itself better to more complicated data where the signal is picked up a lot of unwanted interference like that coming from particle collisions and that brings us nicely to the higgs boson which was measured at the Large Hadron Collider at CERN the data in question is collected when two proton beams are successfully smashed together producing an explosion of particle debris and a simultaneous explosion of dilatant trying to make a name for themselves here's a TED talk given in Geneva by a person from CERN who was involved in trying to sauna fight this data and at this time I became a bit interested I wondered what an electron would sound like so to achieve this they measure two things the particles themselves and the amount of energy belonging to each one but since a collision happens so incredibly quickly timers reenter standard isn't really a factor so to get around this they look to the position of each particle and read them in a clockwise motion to determine the order that we hear them in I mean you can already close the book on the idea that this is the sound of the particles at this point but let's carry on anyway they've been represented the amount of energy contained in each particle using pitch as its high energy high pitch this is the most common sonification technique there's more of something raise the pitch anyway the presenter goes on to assure us of the scientific rigor being used here the science is in these alloys we don't just create a vague relationship between the sonic properties and the physical properties the relationship has to be exact and rigorous it sounds very meticulous but hold on there's more the art is in where we place these areas and which parameters we decide to use all right so these are just arbitrary decisions you can map the position of each particle the time and the energy to paid sure the other way around it doesn't matter anyway they mapped a few more aspects of the data to sound so we wanted to try something a little bit more ambitious with the real data so we went to a four to six laughing it looks a bit messy and complicated yeah you don't say god you're presenting its head could you not tidy that off here look it's I'll do it anyway after all this behold the sound of the Higgs boson so now we've quite clearly divided those two things we don't know if you've done it right but it was fun trying so it is completely arbitrary the sounds you get out really some of it paid for by you some element of your tax okay so let me be fair I'm guessing this was actually part of an experiment to see if there was a new way inside could be derived from sonification particle collisions although the presenter didn't actually come out and say that which I find strange because that's the most interesting part right I mean without that word being led to believe that all we're doing is machine to the particles which is obviously 19 but then I found this statement on the CERN website about how they found similarities between the sound of the Higgs and well-known scores in classical music and have to admit my first thought was oh boy I got you now it became clear that the Higgs boson sonification contained [Music] Beethoven's fifth symphony oh right they were they were joking Jesus does anyone never do any work concern but enough about particles let's move on to planets here the problem of misrepresentation gets way worse listen to this now a fairly famous example of the sound of Saturn recorded by the Cassini spacecraft and published by NASA Wow sounds amazing no well unfortunately this is in no way the sound of Saturn because what's been measured are radio waves you know from the light spectrum which is then converted to the sound spectrum which as we all know is fundamentally a different thing even then the wavelengths are dramatically shortened so they fall within the range of hearing and you can add on top of that a liberal use of audio effects so the result sounds grandiose and all inspiring is completely arbitrary the fancy gets out there you see the act of sana firing data all too regularly ends up contaminated by a human bias for what we imagine things should sound like with the basic abstract sound you're allowing the listener to understand the trigger that provoked us in a way you're exposing the mechanics so that the truth behind the data is revealed when you begin manipulating tone sonority texture etc you begin adding a relevant human bias and unnecessary noise it is completely arbitrary the sounds you get out really here's a pretty well-known example by Robert Alexander who was tasked with sana find data from the Sun which was then presented to the public by various media outlets as the sun's music [Music] every single piece of that music is driven in some way shape or form by that data and all of the choices in terms of the rise and the fall all of them relate back to the underlying data and all of them are true to the data in some way shape or form [Applause] of course the Sun doesn't generate anything remotely like this and despite the sound designers assurance that the data is being reflected accurately it's still being ridiculously shortened so that the full rotation of the Sun is represented in a few seconds and the sounds that are chosen are well it's completely arbitrary the fantasy gets out ladies but on the plus side this experiment led to a discovery that sonification could track the solar winds more accurately than previously known methods so it's both really cool and really dumb where it represents the possibility of improved scientific understanding on the one hand it completely obscures that understanding from the general public on the other instead of selling us contrived myths about how the universe works literally the opposite of what science is supposed to do but as much as scientists can be to use this technique composers take it to a completely different level altogether you see writing music is sometimes soul-destroying spending days and days pursuing a creative idea it's nothing to show for it at the end it's understandable then that some composers want to find something to aid them in the process of creation and there's a certain allure to the idea of readily available data which can basically write your music for you now before I tee off I should mention that there's always a way to use a technique artistically take this short work by Jim Briggs and Michael Cory we're commenting on the orlando massacre in 2016 they used data based on the lifespan of each victim to form an ever-growing mixture of sound but despite the multiple different trajectories that you can hear the sounds all have one thing in common they all end at the same time in other words the data was used as a tool to accomplish a dramatic statement but all too often for other composers the data is the statement I mean how many times have I heard a piece based on climate change data it came to me with a set of data with the task of turning it into a piece of music and we wound up with a song of our warming [Music] my name is Dan Crawford I'm a student at the University of Minnesota I've been working with dr. Scott st. George to turn the climate data into climate youth so what's the comment here the temperature gets higher so the pitch gets higher yeah I get it this isn't a statement it's crap encryption no different from simply handing someone a piece of paper with the stats written on it except less accurate and more time-consuming to understand and this leads to my least favorite use of sonification latent emotional blackmail here's a tutorial just find statistics on a really emotive topic like domestic abuse inequality or child mortality hey presto instant concept now hook the data up to create some music taking care that it sounds appropriately sad and boom you have successfully passed from concepts to execution in time for tea with zero transformative insights and the best part is that no one with half a brain is going to criticize you for doing this because they don't want to be seen on the wrong side of an argument about sexism or racism or dead babies and I'm not going to make that mistake either there's no way I'm naming names here because we all know what the internet is like I've laid out my critical tools and it's up to the listeners to apply them as they see fit because there are tons of composers who do this it's really cheap and the results invariably sounds rubbish too so the next time a composer comes up to you and says hey I took the data from suicide bombings in Iraq over the last 10 years and hooked it up to a synthesizer your next question should be oh yeah and what so to finish up let's have a bit of fun here's a quick piece I created based on the analytics of my recent Eurovision video I took two days worth of data tracking the number of hourly views I got from four countries each hour gets a sound and higher the number of views in that hour the higher the pitch the well that's in real-time so we'd be waiting another hour for the next sound let's shrink it down so that an hour equals roughly a second [Music] okay well that doesn't sound very good let's use the better synthesizer and let's assign each country a nice sounding instrument let's also separate the notes out slightly and keep everything within the key of C ah what a transformation the overall effect gives the impression of something amazing and almost transcendent the underlying hum of human connection the sound of the internet scientists and composers need funding and therefore often find it very hard to refrain from cheapening their work by appealing to simplistic biases and assumptions with the media as they're ever willing accomplice in the case of scientific data it's way too commonly framed as music giving the impression of some unknowable cosmic unity and leading hippies all over the world to jump to vacant clusion about the oneness of everything the technique has the potential to be an extremely useful tool for learning but requires that the listener is allowed to know how the sound was generated in the way that I just did here otherwise what you're sending out is about as meaningful as an encrypted copy of 1984 and as for composers who use this technique to each other mode of topics I've created a sonification just for you it represents the scope of your artistic insight and prowess thank you very much for watching and not much to say about this one except if you liked it and you haven't subscribed please do otherwise if you could send it on to anyone who you think might get a kick out of it that would be fantastic - I'm not certain what my next video is going to be about and any suggestions would be welcome and if you are so inclined I have a patreon page and it would be lovely if you could contribute towards that but don't worry too much because it's sort of early days for this channel you may have also notice that the vocal quality of these recordings isn't quite as good as my previous videos that because I've moved to new house I have a new studio and I need to get it treated for acoustics and then we'll be up to a proper professional standard what was I going to say oh also like I've discovered her kind of important didn't think they were but when I got a number of likes in my Eurovision video it suddenly went into orbit orbit relative to my other videos I want to get sixteen thousand views but that's more than all the other ones put together and anyway to spread the word share the love say hello to your mum goodbye [Music] [Applause]

Channel: Tantacrul

Views: 168,174

Rating: 4.9365954 out of 5

Keywords: Sonification, Parameter Mapping, Audification, Science and Music, Tantacrul, Nasa, Space Music, Music of Planets, Sound of Jupiter, Sound of Saturn, Higgs Boson Sonification, CERN Sonification, sonification of data, sonification of the sun, data sonification, data sonification music, mars sonification, cassini sonification, the sounds of space, the sounds of space nasa

Id: Ocq3NeudsVk

Length: 14min 47sec (887 seconds)

Published: Thu Jul 13 2017