Teensy Audio Tutorial & Workshop

hi I'm Paul staff Regan I'm the creator of teensy and since creating teensy 3 I've had this dream of creating an awesome audio library so my name is Alicia dynamic and I am an educator and artist and maker and before we get started on the tutorial we want to show you a preview of one of the many awesome things you're going to learn to do with this library so right now we are playing two songs simultaneously from this SD card and we're able to pan between them using a software mixer so since starting this audio system there's been consistent feedback that there's been a strong need for more tutorial material that we have a documented and explained very well how to get started with it a couple months ago the hackaday folks asked me if I would put on a workshop so this seemed like a perfect opportunity to create this material and show how to use it so Paul asked me to help him teach this workshop and since it's been announced there's been a lot of feedback from people who aren't able to attend the conference that they would like access to the material as well so we decided to film this tutorial so that we can show you guys all the steps before we get started I should mention the workshop manual which has more detail than we can fit in this YouTube video PDF will be available on the hackaday i/o project page an HTML version which will update over time will be available from the teensy audio library page especially if you're going to do these tutorials in your own use these links to get more info at the convention we pre-built these bread boards to save time but if you're doing this at home you'll need to have a teensy three point to this audio shield the microphone and an SD card preloaded with four music files you'll need three buttons connected to digital pin zero one and two and you'll need two pots connected to analog pin a2 and a3 the software required is teensy we know one point two six which you can check with the help about menu all of the example files used in this tutorial and the workshop are in the examples menu audio tutorial and here they are all of the files so the first sketch that we're going to load is hardware test this will verify that your board is set up correctly and all your components are working once it uploads you should hear a beeping sound so there it is you want to open up your serial serial monitor next and as you press the buttons you should see a readout on your serial monitor so Digital zero didyou one digital to analog a to analog a three once you verify that everything's working correctly now we can go and play some music so now we're going to move on to the playing music sketch now this is just the equivalent of a HelloWorld program so you go ahead and get that loaded and you should start hearing music through your headphones now of course if all we wanted to do was listen music we would just use an mp3 player so the whole point of using microcontroller is being able to do other things while the music is playing so now let's upload our blink while playing sketch and you should see that the LED will be blinking while the music is playing one of the great features of this library is that you can use to lays in your sketch without causing a break in the music throughout these tutorials there's blocks of code that can be uncommented to change the function of the sketch and other things to try in this example we're going to uncomment this block which will allow us to use the knob connected to a2 to control the volume level so once it uploads you should start hearing the music again and you can also see that our LED is still blinking as we move the pot and you can hear that volume change you might also notice that it's really jumping around and that's because of our delay for the LED a function it's only reading this knob every half second so to get that delay you're going to get those jumps and you know returning the knob really smoothly so in the next section we're going to show you how to fix that problem let's take a moment to show how we're making this video we're using two of these little voice recording units or speaking into one and we're recording the output of the teensy with this cord on the other the teensy audio library is CD quality sound it's all 16-bit 44 kilohertz but after two of these recorders are mixed in video editing software then Rhian coded by youtube the sound you hear in this video won't be the full quality of the teen scene so in this section we're going to look at a better way to blanket the LED now the LED should be blinking because of this code here instead of waiting with the delay this code uses an elapsed Raley's variable to decide whether or not to turn on or off but it ever waits with the LED blinking we can do other things inside the boots and have rapid response for example here we eat the knob and now when I wish it turns the knobs or should hear the volume respond instantly because we're not delaying this code also uses the bounce library which we'll be using in all the rest of these tutorials to read the push buttons bounce library is very reliable waving with buttons and here of course we have some code that will read that will allow the push buttons to control jumping to all four of the files on the card that wraps up simply playing with these examples and in the next section we'll look at how to create your own audio systems that are not part of these examples you'll actually create and draw them with the design tool so the previous examples had audio systems that were already composed now we're going to use the design tool which you access with this URL and a web browser this design tool lets us create our own audio system in this case we're going to use the i2s output to send data so I'll drag this onto the canvas I to s is the communication protocol between the TC and the audio board that sends digital audio data we'll scroll down to the play section and find the SD card wave player now that we have these both on the canvas I'll draw connections between them and each of these connections will cause a stream of digital audio data at forty four kilohertz sixteen bits to flow between these devices so two data streams will flow from the SD card to the digital output we need one more object near the end of the list which has no connections but it allows us to control the audio board this one here provides control functions that we'll use to enable the audio so once we've composed our system I click the export button and it will create the code for Arduino and we just copy this code and from here on out and all of these tutorials we paste the audio system code into Arduino and then upload and when this finishes uploading we should hear it play the music that we heard in the first part so now we've recreated the system for the first part but we created the audio system ourselves using the design tool instead of having a given to us in the example and from here on out will create more sophisticated examples in the next part in this example we're going to be looking at mixers mixers are important because they allow us to combine more than one sound and to control how much of each sound is in the output so we're dragging two WAV files and two mixers two mixers because our sounds are in stereo so one will be mixing the rate Channel and one will be mixing the left channel so we're splitting these sounds into left and right and then into our output as you can see when I click on these objects it changes the right side that's our documentation panel and it explains what the object is and how it works so now that we have our objects arranged we can export the code copy and paste into our window now we can upload that now we know it's working because we can hear both of these songs playing at the same time but it sounds pretty terrible so if we're going to go ahead and turn it off let's scroll down to our mixer code and you'll see that the gain is 0.5 here and there's a really good reason for that our signals can only be between negative 1 and positive 1 they can't go above 1.0 and because we're mixing two together we need to multiply them by 0.5 so that the total of the to mix together is less than 1.0 if we try to go over 1 then it will cause clipping and that will create harmonic distortion so if you want to see what that's like go ahead and play around with those values the higher you go the worse it's going to sound so let's keep scrolling and find this block of code that's commented out we're going to uncomment it and this is going to allow us to use the a3 knob to pan between the two songs so now that the code is uploaded you can hear we're switching between both songs this uses two equations that you can see the top the first one assigns a value to gain 1 that number is always between 0 & 1 the second equation subtract that from 1.0 and that's how these shift as you turn that knob those two numbers go up and down but they never add up to more than 1.0 and that keeps us from having any problems with clipping this also shows the way that this library allows us to control the audio system with Arduino code while it's running for very short sounds we can play file we can play data directly from teensies memory using this memory player object in this example we'll try 4 of these these memory player objects have a lot of advantage in that they are very efficient they don't use hardly any of the processing power on the teensy and they don't require an SD card if we can fit the entire sample into the memory on the teensy so we can make a smaller lower-cost project if we put the mixer on here we'll just combine these so we can play 4 separate samples all simultaneously now in this portion we have one signal and in this design tool we can connect one output to two inputs this type of connection where there are two thing where one thing feeds into two is legal however you cannot connect two signals to the same input each input can only have one signal connected but each output can connect to many possible inputs so when we upload this code copy from the design tool paste it into orde we know and this Arduino sketch that we'll upload as these samples already placed inside of it and this is what the sample data looks like this is the actual data that we'll be playing into the output when Alicia hits these buttons over here so these samples are very very usable for short sounds like drums little sound effects and as we'll see moment of few moments later a little bit longer sounds as well near the end of this example you can see this commented out code which will allow us to play three other sounds when when the a3 knob is turned all the way up these are the three sounds are a kick drum a gong and a cash register you can see the uploading takes a little longer because we have these large samples as part of our code now let's see if we can hear them so which of those ways you can see something a little unusual here with the gains we can see that these four gains are set to point four these could potentially add up to one point six and we would get clipping beyond 1.0 and that really illustrates that the setting of gain in these types of audio applications is very subjective that you know we talked about a hard rule earlier of not going over 1.0 but in this case and in many cases you can break these rules or bend them with pretty good results in fact let's try doing something very extreme because these sounds are so short and because the probability of a clipping incident where the short sounds collide just right is very very low we can turn the sounds up and get a lot more dramatic impact and we'll see if Alicia is able to get all four of them at the same time and see if we can hear any destroyed so there we have sample claim and bending and breaking our rules about mixer gains so next we're going to test to make sure our microphone is working so we're just going to add another i2s object but an input this time to our canvas and connect the two objects together you might be wondering why they're called eye 2's objects it's just because that is a standard audio digital audio communication format that teensy uses to communicate with the audio shield now we're uploading our code test test so we can hear the microphones are going to have to test so sorry increases a gain on that we're going to raise the gain of 260 zfx our desk test so I guess you could say that's working so now we've been successful collecting live audio with our microphone we're going to try putting together a delay now this is going to look more complex than it needs to be because the eight tap delay line that we're using would be capable of transmitting back eight different copies on different delays of the same sound though in this example we're only going we're going to mix them all up and hear them as one one thing to pay attention to when you're putting this together is that your I to s input the microphone that you're splitting off one channel between the mixer three and the delay then you're going to be splitting that third mixer back out into two channels for the I to s output so now we're going to export this code and copy it into our doing a window now in this uploads it should start playing my voice back as a delay test test test test okay so Kay so you can play the way that you like delay my voice it's actually actually making it quite quite difficult to contribute a face to into it so I've turned off the delay so that you can hear me cuz I want to bring you down to a line in the code that we have been ignoring until now that's the audio memory you can see here that right now we're allocating 160 buffers each buffer uses 260 bytes of teensies memory so this is pretty significant in previous sketches we've just been allocating about ten or so but the delay uses a lot of memory due to the fact that it's storing that sound until it plays it back so you can see if you come down here global variables we're using 49 thousand six hundred and fifty six bytes of memory so it's a really cool effect but it doesn't come for free and you can disregard this error message if you get it as well it's a bug in this version of art do we know the one six six and it doesn't actually apply so in most audio systems we connect the signals in a forward direction like this we connect into a mixer but it's possible to use a delay object and connect in a backwards feedback path so we can take a delay and we can connect our signal into the delay and then we can connect the output of the delay and feed it back and allow the sound to travel in a loop and we'll hear what this does in just a moment this creates a repeating echo so when we export this code to Arduino come and paste it over here hello hello my love my love my love my love EPP so the sound creates a repeating echo that will flow around this lose so if we look at the code this line here sets the gain on the mixer channel 3 if we look back at the diagram as the sound flows into the delay and then flows along this path this gain on this channel of the mixer determines how much of the sound will make it around this loop each time we hear the echo this code has uncommented section where we can make the knob adjust that game and we can adjust how much echo we get each time we hear this so let me try this now we should be hearing the echo again hello so if Alicia turns the knob all the way up we should hear the echo repeat nearly forever keep repeating on and on on and as we talk more or we'll end up in the loop now if she turns the knob down we hear the echo daya quickly because less of it is feeding around the loop so one of the dangers with using feedback is it's possible to get too much gain and the end of this program is some code that will let us easily experiment with this let me upload this the the button the left button will increase the amount of gain feeding back into that mixer and allow us to have more than 1.0 around the loop so the sound will grow larger and larger each time it goes around so Leisha if you turn up the knob and we get an echo going hello we're hearing an echo now if you push the button we should hear terrible and so we're hearing the effect of too much gain around the loop causes the sound to become larger and larger each time it flows around in that feedback path so when you use feedback in your projects you need to be careful to always have the gain less than 1.0 so that sound doesn't get trapped and grow larger and larger each time it goes around the loop in this next example we're going to try using filters so first we're going to add the play as you wave back to our canvas because we're going to be playing music from the SD card again we're adding two filters one for each channel of our stereo audio now we're going to add two mixers and this is a different use of mixers than we've seen so far mixers can also be used as a switch or a router so here we're going to use them as a 3-way switch routing our three different filters back into our headphones so the three filters that we're working with here are low-pass bandpass and high-pass that's what these three channels represent just going to route all that back to our i2s one output export our code and paste it into the arduino window so when this uploads we should start hearing the music again so we'll start with the low-pass filter now as I turn the knob you can hear less filtering and then more filtering that says the frequency the filtering increases in the middle of the high notes are filtered out let's listen to the mid pass and the high pass now these filters don't necessarily sound great but there are a lot of applications for them and sound projects one that crossed our minds that relates back to the previous example is if you were trying to imitate in a natural echo you might want to include a filter so that the sound degrades in a more natural way as it passes back and forth through the loop so with this part we're going to look at oscillator x' the library the teensy audio library has many of these oscillators in the synthesis section this general-purpose waveform oscillator can create all types of can create several types of waveforms this is a frequency modulated oscillator this is a standard sine wave and we have noise sources we also have what's called an envelope which we'll get to in a moment we'll use a mixer to we use mixers like we used in the last example to allow us to select which of these things we want to listen to so in this example we can hear one of these four types of oscillators running and we can hear it either directly or we can hear it run through this envelope object which we'll talk about in a moment and then we're going to feed this to the headphones so when we export this system and copy it into the example code and upload it to the teensy we should be hearing the default which will be the sawtooth oscillator so we should be hearing we should be hearing sound as we should turns this up and down so this is what a sawtooth waveform sounds like now when pressing the left button will cycle through each of the types of waveforms so now this is what a pure sine wave sounds like and then let's try each waveform so now we can hear a square wave sounds different than a sine wave each waveform has a distinctive sound can you hear the triangle wave and finally let's go back to the sawtooth waveform that we started with now these waveforms in this example can flow into the the modulated sine wave so when Alicia presses the middle button now we'll be hearing the monthly sine wave and will be controlled now let's turn the control waveform all the way down to the lowest frequency an important point of these frequency generators are these oscillators is that they all are capable of being low frequency oscillators that are used to create control signals that control other oscillators so in this case we're trying very very slowly we should hear the sine wave increasing in frequency and then suddenly go back down because the sawtooth is rising up and then suddenly dropping so now let's change to each of the types of control waveforms and now we should hear the sine wave increase and decrease gradually in a sinusoidal fashion so the frequency is being changed by the slow waveform let's try the square and we can hear the each of these control waveforms is changing the tone of the sine wave there's the triangle we figure they're going up and down linearly and then the pulse will hear a change mostly one and then change to another for a brief moment and if we go back to the sawtooth where we started so we're hearing the sine wave go up and down the thing that happens with frequency modulation is when you increase the speed of the control frequency you get strange effects see if we chicken dial is not a funny sound with these two by changing these two knobs so if we can get something that sounds like a funny spaceship or if any sound of some sort so when you swing you settle on a sound let's leave it making a consistent sound now if you just listen to the sound forever it would be quite annoying but the the envelope object is the thing that will demo last year when you press the envelope button this will turn off the mixer and I use the envelope instead whoa and we can hear what the envelope does is it tries to emulate the intensity profile of a musical instrument when you for example blow into a flute moment you blow into the flute there's an increase in pressure and there's a louder sound for the very first moment and the envelope changes the intensity of the sound over time so it has an attack phase which is similar to you're building up the pressure in a tube or you've struck an instrument and the sound waves are increasing then there's a decay phase where the initial excitation blowing into the tube is now the pressure is equalizing and decreasing the sustain phase is like you're continuing to continue to blow into the tube of flu and then the release phase after she lets up the button is like an @ musical instrument decreases over time and we've stopped making the note they're still pressure in the air or they're still resonance in the in the string so the envelope is meant to transform these really quite annoying sounds into something that is like a musical note so that's a this just barely scratches the surface of oscillator synthesis there are all kinds of ways to use these things but at least we can see the basics here congratulations you've made it all the way to part three we're now going to be talking about peak detection so peak is the simplest analysis object that we have but up till now we've only shown you how to build systems that control the audio now we're moving into systems that actually analyze and interact with that audio data so we need to put two peak channels onto our canvas because we have stereo audio so one for each channel once again you can't run two outputs into one input now another important thing to note is even if you just wanted to analyze this data you need to have an input object or an output object for your sketch to for your code to function in this design tool the SD player doesn't count and Paul's made this a little easier for us by creating categories that are labeled input and output on your on your left side there so any of those objects will count but the SD player is not one of them so we're going to upload this and we should start hearing our music again and now if we open our serial monitor we should see an ASCII graphic that's showing us our peaks as well it has a numerical read now if we slide the window of our Arduino program over this so that we're hiding all but the last line you kind of get an idea of what this might look like if it was being used for a line of LEDs in the project there's a lot of things you can do with with this peak detection analysis object and creating interactive costuming or for DJ stayed light stage lights all kinds of projects so if we scroll down on our code to find our peak function you'll see some familiar functions available and read so traditionally these are about receiving data and they're designed to prevent data loss read gives the oldest data first and they don't want to lose anything in the audio library you use them the same way but they're designed to give you only the most recent data and they intentionally discard old info audio is more concerned with real-time information so the way each analysis object discards data is different so you're going to check the documentation in the case of peak each read is only receiving the largest peak since the last read all the smaller Peaks are being discarded so previously we were printing every 40 milliseconds and you can see that variable above our highlighted section let's turn that down to 10 and try printing every 10 milliseconds now MSX is a special variable because it is automatically increasing on its own a thousand times per second so it makes it really easy to time how often we do a thing so you can see how much faster that's scrolling than last time four times fast so let's try going even faster let's see if we can bring it down to zero now this means every time it reads it's going to print so you can see it that's just flying in this section we're going to look at Fourier transform the Fourier transform is totally awesome for audio projects that need to respond uniquely to different types of sounds what it does is it gives you an analysis of how much of each frequency or tone is present in your sound as it plays it's called FFT because the FFT is a fast optimization for the Fourier transform but even though it's even though it's fast it puts quite a load on teensy to do a 1024 point Fourier transform so we're going to connect both of the inputs of this stereo at WAV player to the mixer so that we can mix it down to mono and only need one FFT object when we put the code into the Arduino and upload we should hear this sketch automatically play music and if we bring the Fourier transform window the serial monitor you can see that the Fourier transform is producing a tremendous amount of data very very quickly so what does this data actually mean these these columns are what they're are called the bins the first bin is the amount of unchanging data or the DC signal which normally would be very very low in audio first bin the bins are 43 Hertz apart so the first bin beyond DC is the amount of audio at 43 Hertz and 86 Hertz then 129 Hertz so on and so forth you can see that there's a whole lot of bins this is actually only printing the first 30 out of 512 bins so there's a tremendous amount of data not even printing on the screen however those are very high frequencies that are only a small portion only a couple last few octaves of the audio band so when you look at these numbers let's let's turn off the auto scroll so we can see them still these numbers are all very small you can see that there they're numbers that are all less than one the reason why is remember from our discussion of mixers early on the signals in the audio library are always an amplitude of maximum of 1.0 so if the signal at any particular point is 1.0 all of the bins at that time have to add up to whatever the signal is if the signal happens to be less than one point out suppose the signal is only 0.6 all of the bins that compose that signal add up to whatever the signal is that's why we see fairly small numbers here in a moment we'll take a look at what these numbers mean so if we turn the auto scroll back on and if I press the middle button here we'll hear a guitar send now if we scroll back up in the data we can find some of the places where notes begin there's a good one right there so you can see what happens here in these notes is when the finger plucks the string of the guitar we see a lot of chaos the string is resonating at all kinds of different frequencies but then pretty quickly the string begins resonating at at a fundamental and some harmonics and as time goes on not very much time the string settles towards whatever the desired note is it resonates at a frequency that's primarily determined by where the player has his finger and the end of the guitar that's the note that actually plays and you can see this happening and this is the type of thing that a Fourier transform can do for us as we can see in this frequency data what actually occurred in real time as the as that note played so if we scroll down in the Fourier transform example code near the bottom is this print number function when we saw the music playing earlier there was a massive jumble of numbers and if we change this to point two four zero to four we can decrease the memory amount or raise the threshold and decrease the amount of data that prints now if we bring the serial monitor to the front after after it uploads so as this music plays we'll see that there's a lot less data being printed because we raised our threshold and as we see this woman sing C there we can see her voice appearing sometimes when she sings we'll see three or four columns as her voice has different tonal qualities they should see a guitar appear and rock the middle there's the guitar appearing about the middle of the screen so when you work with the Fourier transform data if you're going to use these numbers to control leds or motors or servos if you're going to use this data to make things happen a big part of the process involves coming up with thresholds or looking for patterns within the data to decide when to do things there's a kind of an art form to it that you get a tremendous amount of data from the Fourier transform and and just little things like this like playing with the thresholds can really make patterns apparent just with the scrolling data and there's all kinds of ways that you can try to analyze this so now let's take a quick look at some of the limitations of a fourier transform these have nothing to do with teensy in the teensy audio library these are just realities of how the Fourier transform works if we push this right-hand button here we can get tone we get a pure tone that's 220 Hertz when you look at the screen we see that we have numbers in five of the bins instead of just one or two and this is this is the thing that happens with Fourier transform because this this object in the teensy audio library is using what's called a window function it's using a handing window and the the windows prevent a problem called spectral leakage so before we before we look at that before we look at what the handing window does let's see what happens if we turn off the window so let's go back over to the Arduino code and scroll down to the setup function and we can configure which type of window Fourier transform object uses and if we set it to null and then upload this sketch I believe it will play music when it starts up but we can just play the same tone right away so now we'll see well now we'll see the Fourier transform run with no window at all try playing the tone look at that this is the same tone we saw just a moment ago and it looks really horrible but if we take a look at the code if we take a look at these numbers here you can see that in this column here which is the one closest to thee that's the bin for 215 Hertz almost all of the 220 Hertz actually ended up in this correct bin but if you look all over in here we see tons of what's called spectral leakage that is what the Hanning window seeks to solve the problem is is that you can't have your cake and eat it too the way the Fourier transform works is if your frequency is not exactly perfectly in one of the bins you either get smearing from a window function but no spectral leakage or you get spectral leakage small amounts of data all over the audio spectrum when you wanted it just in one little area normally the window function is used of course you can do either so to understand where spectral leakage comes from you need to consider the wave form that wasn't an exact multiple of the bins when when we put the two hundred and twenty Hertz wave form into the Fourier transform this is what we gave it this green rectangle is the portion of the waveform that the Fourier transform analyzed but the Fourier transform gives us the spectrum as if the data analyzed repeated over and over again so if we take that green chunk and repeat it this waveform here is the waveform that this Fourier transform actually analyzed you can see it's mostly like a sine wave but it has a bunch of parts on it that aren't anything like a sine wave these little spiky parts these little parts where two lobes occur too quickly these are the spectral leakage this is why we got data that wasn't like what we wanted the solution of course to this is called a window function what the window function does is you multiply the input waveform by basically a sine wave there are a variety of these different windows but what the windows do is they make the wave form zero at the ends where it would have joined together to the next Fourier transform and they make it one you know they pass the wave form in the center so they make it sensitive only in the center of the time analysis when you do this you're not analyzing the original wave form anymore what you're analyzing is something that looks like this you have waveforms that are sinusoidal in the middle or whatever you wanted and they are zero at the ends so what the teensy audio library does because this is only sensitive in the middle is it does twice as many Fourier transforms it applies the window shifted 50% and it performs two times as many Fourier transforms so you're getting the analysis so each Fourier transform is primarily sensitive in the center of its window and then you're getting overlapping windows so ever so even though we're forty four kilohertz sample rate and we're analyzing 1,000 points we get new data every 512 samples or approximately 86 times per second rather than 43 times per second so when you get unavailable when you get FFT available in your code you are getting the result of an analysis which is 1000 points primarily sensitive in the cent in the center of those 1000 points and you get a new one every 512 points or 86 times per second despite all of these issues with the Fourier transform then you either have window functions or spectral leakage it's still totally awesome for audio projects it's great for for responding to sounds but you do have to consider that the Fourier transform does have some of these limitations nothing to do with teensy these are just simply the mathematics of Fourier transform still great to use so for this last section we're just going to be adding this display and we're going to be running another peak analysis function so just going to get these wires hooked up in the workshop we have the ground wires and the the power wires already attached so if you're doing this at home you'll have to install your own wall hairs there so while Alysha hooks the wires up I'm going to just draw our peak system again in the design tool it's kind of hard to see where I'm putting these wires with the audio shield in the way but in the documentation you'll have the information about where they go right and these connections are also documented on the PG RC website where this display is sold okay now I can attach my display and I'm going to plug my teensy back in my USB cable see powers going to the board and now we're going to upload our sketch you can see how fast that have disclosed refreshing looks pretty awesome you

Channel: PaulStoffregen

Views: 113,152

Rating: 4.9759579 out of 5

Keywords:

Id: wqt55OAabVs

Length: 48min 38sec (2918 seconds)

Published: Sat Nov 14 2015