Introduction to the Web Audio API

you might not realize it but your web browser has a very advanced audio synthesizer built right into it in fact every modern web browser supports the web audio api which gives web developers tools to generate visualize and play audio of all varieties today we're going to dive into the basics of the web audio api and show how you can use it to synthesize your own sounds we'll create a soundboard program that lets us click buttons on a web page to play sounds from a drum kit or a melodic scale the web audio api is based on the concept of modular routing which has its roots in analog synthesizers we have audio sources such as a microphone an oscillator which creates a frequency signal and recorded audio and we can connect those two nodes like a gain node which adjusts the volume a filter node which lets us change the way that the signal sounds and an analyzer node which gives us a visualization of the signal then we can connect our nodes to an output node which plays the sound through our speakers analog synthesizers use wires and circuits to create the audio nodes and connections but with the web audio api we use code as demonstrated by this visualization by josh como sound works through continuous waves in the air which our ears are able to detect these waves move at varying frequencies which our brain interprets as different pitches higher frequencies means a higher pitch and lower frequencies mean a lower pitch the height of each wave indicates its amplitude or power so waves with a higher amplitude are interpreted by our brains as having a higher volume so let's actually build our audio sound board we're going to start with a vanilla javascript template in code sandbox but you could do this yourself even in your web browser's console it'll work just fine all of the work that we do in the web audio api starts with the audio context this is an object which gives us access to all of the other objects and constructors that we'll use to create audio it works the same way as the document or canvas context does for creating dom nodes or drawing on the canvas but for web audio so we create the new audio context by instantiating the audio context class that's built into the browser now we can use the audio context to create some audio nodes since the web audio api is used in a javascript environment audio has to be stored in a way that javascript can understand this is done using audio buffers an audio buffer represents a certain duration of sound and at any moment during that sound it has stored the amplitude value of the signal now computers can't store or process continuous data since computer processors can only handle one bit of data at any moment that means when working with audio in a computer we use discrete signals which splits a continuous signal into a sequence of samples each of them representing the amplitude of the signal at that particular moment in time so that's what we've done with this audio buffer the audio buffer is initialized using a specific sample rate which is the number of samples per second now the range of human hearing is 20 to 20 000 hertz and because of the way that sampling works we need to double the source frequency in order to get a sample rate that matches that source frequency so the web audio api if i were to come over here and console.log our audio context dot sample rate we can look at our console and see that it's 48 000 which is a little bit more than double the range of human hearing so that's what we'll use as the sample rate when we create our audio buffer we can create an audio buffer using the create buffer method the first argument of the createbuffer method is the number of channels which our buffer will contain so we could have a single channel which is mono audio or we could have two channels which is stereo audio or we could have six channels which represents 5.1 surround sound for right now though we'll just leave it at one channel the next parameter is the number of samples in the entire buffer so what we do is we multiply our sample rate by the number of seconds in our sample to get the complete number of samples and then finally we pass in the sample rate of our buffer so this gives us a mono buffer that holds one second worth of audio data if we wanted to read the data that's in the buffer it's currently empty but we could use the get channel data method so channel data equals buffer dot get channel data and we want to get the first channel which is indexed by the zero key this returns a float32 array where each item is a number representing the level of that sample and we can see if i were to console.log the length of that channel data that it's our same the same as our sample rate so it represents one second of audio then we can mutate the channel data directly to actually create a signal if i were to assign a random value to our signal between negative 1 and 1 we can create white noise so i can do that by doing a for loop over the entire buffer and then using math.random to create the random signal and then because of the way that code sandbox works it thinks that we have an infinite loop so i need to turn off the infinite loop protection now note if our buffer had more than one channel we would have to repeat this process for every channel but now we have a snippet of audio that contains one second of white noise there's still a little bit more that we need to do before we can play it we have to create a buffer source which is an audio node that takes our buffer and handles playing it for us we can create our white noise audio source by using the create buffer source method on the audio context object and then we just assign the buffer of the source to our white noise buffer we can now connect this audio node to another audio node including the audio context destination node the destination node is a special node that is attached directly to the audio context and it represents whatever speakers are configured to play audio on our user's computer before we connect our white noise to the destination we're going to want to turn the volume down to make sure we don't blow our eardrums out so we'll create a gain node which lets us control the volume then we'll connect all of our audio nodes to that game node and connect the gain node to the destination node so that all of our audio nodes are a little bit quieter now you might be wondering why we're using the strange set value at time method here the web audio api is designed to orchestrate complex changes to the parameters of the audio nodes the gain property on our gain node is an audio param which means it has a lot of other methods which we can use to automate changing the different parameters in controlled ways set value at time lets us set the value of the gain node at a specific time during the audio context existence so in this case we're setting the gain to 0.05 which is multiplied by the amplitude of whatever signal that we pass in so using a value less than 1 decreases the volume and greater than 1 would increase the volume now you might be expecting to hear some noise coming out of your speakers right now but just connecting the audio node to the destination node doesn't make it play you have to call the start method on the source node so let's create a button that we can click to play the sound using dom apis we'll give our button an event listener that is called whenever we click on it and we'll have that start our white noise source now when i click the button we should hear generated white noise but if we click the button again we get an error we can only play a source node once because the web audio api wants to make sure that we don't have any memory leaks fortunately source nodes are really cheap to create and we can still use the same audio buffer so let's create our white noise source node inside of the buttons event listener by rearranging our code just a little bit now when we click our white noise button it'll play a white noise sound every time let's create a snare drum sound effect using this white noise snare drums create their signature rattle using several springs which are held taut underneath the drum and if you've ever played an 8-bit video game you probably can recognize that the sound that they use for their snare drum is a white noise generator something like this white noise is nice but it doesn't sound much like any kind of musical instrument we can build on our white noise generator to create a snare sound effect using filters so in audio processing a filter is any process which changes the audio signal in some way we can use a high pass filter to cut off all of the noise that exists below a certain frequency leaving us with just the high pitched frequencies of noise we can create this filter using the bi-quad filter node in the web audio api so i'm going to create a new button called the snare button i'll give it an event listener for when we click on it and i'll use exactly the same code that we used for our white noise generator to create a white noise buffer but instead of just using white noise we'll also create a snare filter since the filter is going to be reused we can create it outside of our event listener we'll create a biquad filter and assign it the high pass type now there are several other types which you can use for biquad filter nodes and you can check the description or the accompanying post to see a link for all of the different filters that are built in we also need to give our bi-quad filter a frequency value which is the frequency that cuts off the noise and then we'll connect our snare filter directly to the primary gain control then instead of connecting our white noise source to the primary gain control we'll connect it to our snare filter the last thing we need to do is add our snare button to the dom now listen as i click the snare button versus the white noise button snare button has just a little bit less frequency at that lower register whereas white noise has the full frequency and we could adjust this frequency value to make it even more apparent now we're not quite done with our snare yet but we're going to take a little detour and create our kick drum sound the kick sound will be a low pitched hum we could create that ourselves using another audio buffer and assign function but the web audio api gives us tools to make this a lot easier once again we'll create a button for our kick drum inside of the buttons click event listener we'll create an oscillator node the oscillator node creates a signal that oscillates back and forth at a certain frequency and we can set the frequency using the set value at time method frequency that i've chosen corresponds to middle c on the piano then we can connect our kick oscillator to the primary gain control and then start the kick oscillator if we were to click our button our sound would play but it wouldn't stop and that's not what we want fortunately we can use the stop method to tell the oscillator to stop if we were naive we would call stop using set timeout but the audio context actually keeps precise time on its own we can pass a number to the stop method to tell it to stop at a certain time and using the audiocontext.currenttime property we can have it stop a certain period of time after when it started so i can say kick oscillator dot stop and say audio context dot current time plus 0.5 to have it stop half a second after it starts and then when i click this it plays the c note now if you remember the sound of a kick drum [Music] you can hear that the sound actually pitches down over time lower pitch means lower frequency so that means we need to adjust the frequency of our kick oscillator over time we can do that using the methods that are built into the kick oscillator so we'll still set our frequency to a certain value let's set it just a little bit lower at 150 hertz and then we'll use the exponential ramp to value at time function which is going to let us change the value of the frequency exponentially so here we're changing the frequency from 150 hertz all the way down to less than one hertz which is really really low and we'll do that over the course of half a second now when i click this you hear that it sounds like a kick drum but what's that pop that happens at the end well if you think for a moment about what our speakers are doing when we click the button they begin vibrating as the pitch drops they're still vibrating just at a much lower frequency so low that we can't hear it but the speaker is still being pushed and pulled away from its neutral position however when the oscillator stops the speaker suddenly snaps back causing a rapid change in pressure and our ears pick this up as a short click let's fix this by giving our oscillator its own gain node that fades out the volume with the pitch now we can play our kick and we don't hear that click anymore the oscillator node that we created here creates a sine wave by default but it can be configured to create square sawtooth or triangular waves as well each of these has a different acoustic quality with sine being the most smooth followed by triangle sawtooth and square waveforms have much more abrupt or jarring sounds to demonstrate this i'm going to take over my kick oscillator really quick and make it so that it just plays the sound and i'll set it back to that c note that's what a sine waveform sounds like if i were to change our kick oscillator to be a triangular waveform it sounds like this square sounds like this and sawtooth sounds like this [Music] all of them have very different sounds to them and we can use them in different ways based on the effect that we want to get so what we'll do is create an oscillator node for our snare drum and give it a triangular waveform then we'll create two gain nodes one for the oscillator and one for the white noise and we'll use the exponential ramp function to fade them out individually so with just adding the gain node to our white noise and making it so that it ends after one-fifth of a second our snare sounds like this so that gives us that rattling but we still need to have the tone of the drum itself and that's what we'll use the oscillator for i'll use a triangle type for my snare oscillator and make it so that its frequency is a low 100 hertz and then i'll create a gain node for the oscillator that fades out just like i did for the white noise now when i click the snare button you'll hear that tone in the background and we can play with this a little bit more increasing the initial gain to make it a little bit more apparent changing the frequency etc etc the last drum sound that we're going to use is a hi-hat and hi-hats are a little bit trickier to synthesize so we're going to use a recording of a hi-hat sound instead you can use whatever recording you want but i found this one which is hosted on unpackage and it's from freesound.org we'll use the fetch api to download the sound process it as an array buffer and then we'll use the audio context decode audio data method to turn it into an audio buffer much like the one we created for the white noise then we can just connect it to our master gain note and play it like any other sound now note that we could make this just a little bit more efficient by loading and decoding our hi-hat sound outside of the event handler but for our purposes this works just fine after the first click the audio file is loaded from the browser's disk cache so if i click on this it's pretty much instantaneous where i click and then it plays now there are other ways that we can modify the sound more without even needing to apply a filter these audio sources have a playback rate property which lets us change how fast or slow the sound plays since this either compresses or extends the frequency of the sound it also changes the pitch so if i were to come here to the hi-hat source and set the playback rate using the set value at time we'll set it to 0.5 that should make it so it plays half as fast now because it plays half as fast it's twice as long and it is half as high of a frequency so if i were to set this to an even higher frequency like double it then it's going to be higher pitched and even faster if we wanted to make the audio even slower but maintain the same pitch without pitching it down we could do that but it's outside the scope of this article and it involves a lot of complicated math so if you want to learn more about that there's a wikipedia article about audio time stretching and pitch shifting which you might find interesting now that we have the basis for a drum set let's try to make a simple piano synthesizer but this is going to require just a little bit more knowledge of music theory the chromatic scale that most western music is based on consists of 12 notes which are equal in distance between their pitches the ratio between two pitches is calculated as the twelfth root of two that means that we could technically calculate the frequency values for each tone by starting at a above middle c which is 440 hertz and then multiplying or dividing it by 2 to the 1 over 12 for each note however since other people have already done that for us i'm just going to copy and paste their note values in so we've got a complete scale from c all the way up to c uh starting at middle c and then going up to higher c so what we can do now is loop over each of these notes to create buttons for each note in the scale before i do that i'm going to create a break element so that we have a distinction between our drum buttons and our note buttons so i'll loop over each of the notes and i'll pull out the name and the frequency from the object i'll create a note button element and i'll give it an event listener which creates an oscillator sets it to the square oscillator type because why not we'll set it to the frequency of that note and then we'll connect it to our primary gain node and have it play for one second now we have all these buttons and i can click on them [Music] to play a scale our notes do sound a little forced though and not just because we're using the square wave we can improve this substantially by paying just a little bit more attention to the volume and frequency of our note we can use an envelope and not the kind the email letters in in audio and envelope describes how sound changes over time most of the time an envelope is defined in an adsr form that stands for attack time decay time sustained level and release time notice that all of these are time values except the sustain level the idea is if you were to press a key on a piano and then let it go abruptly it would create a very different sound profile than if you were to hold the button down for a long time the sustain time is determined by how long the key is held down so the sustained level just tells us at what volume level the sound is when it's being held down we can create an asdr envelope by adding a gain node to our oscillator and setting up some timed adjustments to the gain value we'll use this to model the attack delay and release time components we'll also store the sustained level in another variable to make it easy to adjust so i'll set the attack time to be 0.2 seconds the decay time will be 0.3 seconds the sustain time will be or the sustained level will be 0.7 so it'll start and it'll go up to 1 and then it'll come back down to 0.7 and then the release time will be 0.2 seconds so that means if our complete sound is 1 second long since the attack time is 2 and the decay time is 2 and the release time is 2 the sustained time will be three seconds so we'll create our gain node we'll set its value at the zero moment to be zero then we'll ramp it up to one as the attack then we'll decay it down to our sustained level and then after a period of time at our sustained level we'll ramp it back down to zero we'll connect our node oscillator to the note gain that we created and then connect note gain to the primary gain control now if we play our notes we'll note that it ramps up in volume and then stays sustained for a little while and then releases to zero volume if we want to take this to the next level we can add vibrato this is a very slight modulation to the frequency of a note that makes it sound more full and rich you'll often see violin and other string players shaking their hand up and down as they play a note to create vibrato we can do this by creating another oscillator however instead of producing a note this oscillator will modify our frequency value in the audio industry this is sometimes called an lfo or a low frequency oscillator we'll also need to create a gain node and connect that to our vibrato oscillator by default the oscillator produces values between negative one and one however it's difficult to notice a two hertz difference in frequency so our gain node will amplify the value making it so that our note has a wider oscillation by connecting the vibrato oscillator directly to the frequency parameter of our note oscillator we can rapidly adjust the frequency of our note so the frequency of our vibrato oscillator will determine the speed of the vibrato and the gain node will determine the intensity of the vibrato now if i play this you can hear that slight shifting of the frequency it doesn't sound like a piano but it's not a boring square wave anymore either in fact we can adjust this to make it so the frequency is higher or lower so here it is at 20 hertz and if i were to make this even louder that makes it so that the difference in frequency is higher so [Music] adjusting this up even more makes it so that kind of takes on an entirely different sound there so there you have it there's our sound board we made a couple of drum sounds we made some notes we added vibrato and there's a lot that you could do to improve this you could add some styling to make it a little bit prettier you could add keyboard events so that when you press buttons on the keyboard it plays the sounds you could randomize the frequencies of the snare and the kick just slightly to make it so that it varies a little bit each time you hit it you could play around with the different settings to find some that you prefer a little bit more you could learn about the visualizer node which is a special node that lets you create visualizations based on the sounds that are playing and you could figure out how to make it so the sustain portion of the adsr envelope is controlled by how long the button is pressed so if you hold the button down it continues to play and then once you release the button that's when it releases and fades out the sound now i hope you've discovered the incredible possibility that comes with the web audio api there's a lot that you can do with it so much that i didn't even mention so i hope you dive in and learn more about it and if you make something cool with the web audio api make sure you share it in the comments below thanks for watching

Channel: uidotdev

Views: 11,333

Keywords:

Id: laCjGMhASp8

Length: 25min 59sec (1559 seconds)

Published: Wed Dec 02 2020