FFmpeg Audio Decoding And Playback Tutorial

hello guys welcome to this video and in today's video we're going to be taking a look at how to decode and playback audio using ffmpeg library all right so today's video is going to be a gentle introduction to decoding audio using the ffmpeg library so the first thing we have to do is to get ourselves some audio files that we're going to be using for illustrations so i think i could search for free music i guess i think i think that's better where do i find free music alright okay what is this i don't know i really do not know the hell am i supposed to stay true um i'm just gonna search for it doesn't matter i'm just gonna go to the gym i'm gonna get myself um some rock music and here okay i got a couple of them i got blueway fury i think we could play that um that's pretty dope okay so we could take that um we could use that um so i'm just gonna download it what i agree i don't care really doesn't matter i agree okay so i'm gonna save that and see once the download is done um i'm just gonna copy the path all right um let's see nope i cannot so i'm just gonna open the folder i'm gonna copy the path to the song just like that right so we got ourselves uh an mp3 song we can actually play so the first thing we have to do is to install the ffmpeg all right and i already have mpeg installed you can do it with your package manager if you're running linux and if you're on windows then you're going to have to figure out how to do it because you know i don't really care about windows users because windows suck all right so if you if you want to be uh you know you want to get a feel of what real programming looks like i think you should come over to linux all right i'm just kidding anyways but windows users you're going to have to figure out how to get the ffmpeg libraries installed um and do not confuse the ffmpeg libraries with the ffmpeg binary all right so the fmpeg program is the one that you use for converting merging and splitting audio and video and stuff like that and uh so libraries that underpin the ffmpeg program itself is the libraries that we're going to be using in this video we're going to be coding in c plus plus using those libraries so um yeah so once you get them installed the there is one more thing we're gonna need to um add and we're gonna be using mini audio for the audio playback because after we decode the audio using ffmpeg we also have to play the audio back all right so i'm going to be using mini audio and it's a single header file um library that allows you to just include one header file and you can use it to play audio and it works for every um you know his cross platform works on pretty much every platform works on desktop mac linux and windows plus um android ios bsd and even web all right so yeah so the this library is pretty cool all right um so we're gonna be taking a look at how to do that so let's just hop over to here you can see i have this simple project set up in with visual studio code and i have just the mini audio header file that i've downloaded from the website so just click on this to download the mini audio.h and then include it somewhere in your project all right so once you've done that the next thing we want to do is to i'm going to create a simple cmake project here so that i could um you know so simmy can help me to manage that you can use whatever um whatever ide or you know whatever editor you care about but i'm gonna be using visual studio code with c make right um so i'm gonna be gonna find the quick start for c i don't care about the environment it's asking me for project name it's gonna be audio decoding and it's gonna be an executable okay so it creates a simple cmake project for me as you can see here now i don't care about the whole cpac thing so i'm just going to delete that all right um also let me see if i can actually increase the font size here i don't know um yep i think that's big enough yep all right i'm gonna close this out too and then i don't care about enabling testing or whatever so the next thing i'm going to add to this is just going to be target link libraries so i'm going to be linking to i'm going to be linking to the av format actually av format av codec and av 2 so those are the libraries that i'm going to be using in this video i'm also going to need the sw sample all right the right there will be sample library so that we can resample the audio frames and you see what i'm talking about when we get started right so i'm going to close this and open up the cpp file get out of my face what the hell is going on here so what i want to do first of all is include the ffmpeg libraries and because i'm using c plus plus um i need to wrap the includes inside of this extern c declaration and this simply means that i do not want um c plus plus to perform name mangling on you know whatever symbols that i impose from the ffmpeg libraries so i'm going to include the av format slash av format and h i'm also going to include the av codec slash av codec.h slash av2.h g all right i'm also going to include the the swear sample such as the beer is sample h just like that now once you have these libraries um imported the next thing we have to do is to start preparing to decode our audio all right we need to first of all open up the audio file so what i'm gonna do before i start writing any code is i like to think about um before i write any code i like to outline things that i like to do so that it could guide me when i'm coding so that's exactly what i'm going to do right now so the first thing i want to do is to open your audio file then i want to decode the audio and then i want to play back the audio so this means that i'm going to open the audio file first of all then i'm going to have to decode all of the audio samples that i want to play and then i'm going to have to store them inside of somewhere and then third i'm going to have to play back the audio so these are the three steps that i'm gonna take in this video and this is what i'm gonna do um right so in order to open the audio file this is actually uh simple the baby format as part of the the part of the ffmpeg library the av format library allows us to open files and read information about those files you know basically it's got a couple of modules in them where one module performs input output which is i o and that is cross platform by the way because um ffmpeg works on i think pretty much every platform all right mobile and desktop um so um the io is also cross platform so fmpeg will help you will assist you in opening up the media file and you can also read information about that media using this libavy format library all right so how do we do that first of all we're gonna need an av format context so we're gonna create a pointer to that i'm gonna call it format context now i'm gonna initialize this to no pointer to begin with and then i'm gonna say av format open impute all right now this is going to need a couple of parameters you can see the format context i'm going to pass in the address of the format context pointer secondly it's gonna need a url which is uh the pass to the media file we copied earlier and third is going to need a an input format that we do not have so we're just gonna pass in our pointer options we don't really know what options to pass in so we're just going to pass in our pointer too all right um i think i need to change the word wrap settings for vs code because i don't like i need this thing to actually wrap so i'm going to have to go to settings and i'm gonna search for i think it's like word wrap ah text editor what the i don't know [Music] let's see does he wrap now yep he does cool all right um i think i should also change the font size while i'm at it so i'm gonna do that right now [Music] fun size yep so i'm gonna change this to 18 then i'm gonna close this and then i'm gonna go back to this before yep okay i think that's cool all right so we're gonna open up the media file now this is going to return an integer which is more or less like the status result telling us whether the uh input file was successfully opened or not all right so we're gonna check the results to know whether the media was successfully opened and so i'm gonna say if rent is less than zero because if the media was successfully opened and there was no errors then it's going to return zero so we're going to make sure that we check to make sure that the result is zero if there is an error it's going to return a negative integer to tell you all right um so we're gonna see if red is less than zero then we're gonna print an error to stand that error okay and it's gonna say unable to open the media or something right and then you're probably gonna return minus one okay now once we're able to successfully open the media we have to tell the uh fm peg to hope to read information about this media because as soon as it opens the media file sometimes um it's not gonna have enough information to give you about the streams inside of the file and you know stuff like that so we actually have to tell it to go read information about the file and get as much as it can so that we can query those information from it so it's just going to be red equals to if we format find stream info okay this is going to be the format context so we're going to pass in from my context we don't know what options to pass in again so we're just going to pass that in again we check if it's less than zero the result okay and unable to find stream info all right and then we're gonna return -1 again all right so if we're able to find the stream information and we've successfully opened up the media then we're good okay we can start um you know we can start decoding this media file but before we do that we also need to know because a media file can contain multiple streams this is an mp3 media file um so it probably just contains an audio stream and an album art okay but we don't really care about the um we don't really care about the album art we're curiously about the audio stream um so we have to get the index of that stream from this format context and if you want to know the number of streams inside of a media we could do that with this number of streams inside of media it's going to be pressing d um let's see we're gonna put a new line right there uh and it's gonna be the format context nb strings this is this is going to contain a number of streams inside of this media file that we're actually reading all right so we're gonna know the number of streams we have inside there and uh yeah so let's move forward and uh the next thing we have to do is to get the audio streaming this like i said so i'm gonna say index equals to uh av find best stream okay so this means that i want to find the best stream inside of this media what what best stream do we want to find uh it's going to ask us for the format context which is just basically the format context and the media tabs was asking us what type of stream do we want to find all right so we want the av media type audio and then the one thing stream number we don't know so we're passing minus one the related stream number we don't care we don't even know so we're passing minus one now we get to an interesting uh point here it's asking for a decoder red okay we don't know what that means so we're just gonna pass in a null pointer right now oh you can type no pointer and then flags we also don't know so we pass in zero and if you if you read the documentation it says flags none are currently defined so we don't even care about it we just pass in zero all right now if index is less than zero then you know the media uh probably we don't have an audio stream inside of this media so yeah we're just gonna do std air again uh no audio stream inside of this file and yeah so we're gonna have to return -1 because we don't have any audio to play all right so if index is actually zero or you know greater than zero then we can start you know working with that um so the next thing we do is we need a decoder so remember here where it was asking us for decoder read but uh we actually just returned it now we should have just passed in a null pointer because we don't really know what that means uh now we got two options we can actually pass in a valid uh address of a pointer to a decoder so that ffmpeg will find the decoder as it's looking for the stream and this it's going to give us a decoder uh decoder that we can actually use to decode this stream or we could find the decoder manually now how do we find the decoder uh manually i'm going to show you both processes all right so first of all we could create an av codec structure here i'm going to call it a decoder and we have to initialize it to null pointer and then we're going to pass the address of the decoder here and what okay i'm going to spread the code correctly um so what is going to happen now is that ffmpeg is going to populate this decoder structure uh the pointer with an appropriate value once it uh finds the audio stream so this is the easy way to do it but if for some reason you can actually do it this way capacitor pointer again uh the other way to do it is to get the decode the codec id for the specific stream index and that's what we're going to do so i'm going to create an av codec again uh decoder set it to no pointer again and actually here i could initialize it here okay so what we're going to do is say that this is equal to av codec find decoder so we're asking it to find a decoder with this with the specific codec id now what codec id do we pass in well we know that this is an mp3 file so we could pass in av codec id mp3 alright and this is going to this this is going to give us an mp3 decoder all right but this is not good enough because sometimes we don't know what kind of audio file we want to play so we could get the codec id from the stream given to us by the format context all right and the way we do that is format context streams at the specific index okay so we we we just got a stream at the index that we got before remember the stream index that we got from if we find this stream and then we do code like par and then we do codec id all right so what is this line here this is simply saying i want to find the decoder well from where the format context stream has the index you know the audio stream at the specific index because for my contest like i said can contain multiple audio video subtitles or whatever streams so we want to get the stream at the specific index and the stream has some codec parameters which are basically information about the stream all right about the codec inside of the string because you know for an mp3 media file for example it's got an mp3 um it's gonna mp3 codec inside of them the the data encoded inside of that is mp3 so the information about that the data inside of there is gonna be stored inside these codec parameters and part of the information is the codec id which is what we need to pass to av codec find decoder to give us a valid decoder a valid mp3 decoder in this case because we're playing an mp3 media file all right now if the decoder doesn't exist it's going to give us a no pointer so we're going to check that uh decoder then we're just going to do fprintf stdr and we'll decoder file and then we're gonna return okay return minus one all right and once we find the decoder then we're good to go we're almost done here all right uh we're just about 44 lines so we're we're good we're good all right the next thing we need to do is to get ourselves a codec context because this decoder structure is just like uh something that describes a decoder okay it doesn't actually do anything you know the decoder the functions that we that we call to do something on the decoder or the function that we call to decode actually needs somewhere to store the decoder state all right and that is what an av codec context is all right so av code context uh codec context okay and we get it called av codec a unlock context and just like that we're going to allocate a coded context for that it looks like yeah i think it's hello context three yeah i don't know why this three at the end of it but that's what they called it all right so we have you can see you need to pass in a codec id here now you don't get a passenger credit card you could pass in a null pointer it doesn't matter but if you already have a decoder then it's just better to pass it in here all right and this is going to allocate the coded context for you and now you can start doing stuff with it but before you do that you have to actually give the decoder context some information because at this point all we did was find the decoder and now the decoder doesn't know what like what when we need to tell the decoder to decode something the decoder needs to know what am i decoding no for example if you want to decode a video you need to know the width and height that the video is decoding uh you know some other information on that in the case of audio you need to know the sample rate you know the sample format of what it's going to decode and some other things like that like you know uh i don't know basic you know the number of channels of the audio for example and you know several stuff and how do we get this information again we could get it from the streams all right and just to make this clear there's something called an av stream which is exactly where i'm getting this um streams information from so if i cut this off then this is the stream all right this is the stream the av stream is where information about each stream of the uh of the particular media is stored all right so here we need to give information about data stored instead of this stream to the decoder all right so the way you do that is called av codec parameters um to context all right and what this is just a minute so we're gonna pass in the codec context and then the streams collect part so what this does is that it's going to copy the information about this uh about the stream into this um decoder context all right so that's what it's gonna do so the next thing we need to do once we've copied is to now open the decoder and so we're gonna check again av codec open two and what this is gonna do is going to open the decoder and initialize it so that we can actually use it to start decoding stuff if you don't open the decoder before you start using it this is not going to work all right so we're going to pass in the codec context and then we're going to pass in the decoder again and options we don't have options so we're just passing all pointer now if the decoder wasn't uh oh you know sometimes when you try to open a decoder it doesn't work so you know you get a check for that fact refresh is less than zero um then you know couldn't open decoder uh so if the decoder wasn't able to be open then you can actually use it to decode anything so you're better off just returning and then you know creating the program all right um so if we couldn't open the decoder we failed and then we're just gonna return now that we've opened the decoder we're actually getting to the um sweet spot all right now we're going to start decoding so all of this the first step that i highlighted here um is actually where we open the audio file and initialize the decoder all right so the next thing we're going to do we now is to decode the audio and the way that works is that we're going to tell something called the mixer to actually take out the audio packets all right the compressed audio packet and give it to us so that we can give it to the decoder to decode and then uh we need to then do whatever we want with the decoded frames all right um so the next thing we got to do is to create something called an av packet all right so av packet i look now the av packet like i said represents a compressed um frame all right and the what a compressed frame simply uses like you know a number of samples of audio data that you can actually play back but because it is compressed the the um the sound card is not really going to understand how to translate the compressed data so you need to first of all decode it which is why we need a decoder in the first place all right um so a packet is going to store you know a chunk of that compressed data all right so which means that the mixer is what is going to be taking those chunks out bit by bit and then we're going to be decoding and probably playing it back or storing it somewhere before we then you know pursue the rest all right so we need a packet for that now once we decode the packet we're going to be decoding the packet into raw frames so we need a frame structure which is represented using this av frame okay so i'm gonna need to allocate a frame structure so this is going to store a decompressed packet which is a which is basically a frame a raw data that we can actually send to our sound card and get you know um audio output all right so that's cool yeah so the next thing i need to do now is to start decoding so i'm gonna have a while loop [Music] the format context and the packet now what am i doing here i'm simply saying while av read frame so this is gonna this is the av read frame is going to return zero if it was able to successfully read a frame all right but if if for some reason it wasn't able to read the packet actually this is called av read frame but it actually reads a compressed frame okay compressed packet and you can see here we're passing the packet you know for you to actually read so basically what this is is um when you call if you read frame it's gonna go to the media file you know you know read read a bunch of uh packets and then put it inside this uh av packet structure that we've um created and it's gonna return zero if it was successful all right so i'm inverting that to make sure that or we continue while this operation is successful now if it returns anything less than zero then this function is d no this the while up is going to translate it to true so i'm just inverting that to make sure that it's false so essentially what this line of code is doing is you know this uh block this while block is going to be like execute whatever is in this while loop as long as we have a frame to read another way of writing this to make this even clearer is to say well if you read frame is zero okay i think this looks uh you know this looks more readable for some people so i'm just gonna keep it like that all right so while we're able to read a frame the next thing we have to do is now to take the frame uh the the packet or the compressed frame whatever you want to call it we need to take that and send it to the decoder all right but first because ffmpeg like i told you before we have um this this could this probably doesn't have to be an mp3 file remember it could be an mp4 an mkv or whatever whatever media file we want to use in this case we're just testing with an audio file an mp3 to be precise but sometimes it's going to give you a stream that is not audio okay sometimes it's going to be video sometimes it's going to be subtitle so it's just going to read and then it's going to probably be like okay this is an audio packet this is a video packet this is a subtitle packet so it's gonna be like yeah i'm gonna do the um um so you know so that's basically how it is so the best way for you to um um do this is to be to check when you receive a packet if it corresponds to a specific stream index um so what this is is we're gonna just be like if packets stream index it was not equal to index continue now where did i get index from if you remember we were looking for the best stream remember the best audio stream so we were given some index all right good we could also use the streams in specific index and we could write it as a string index and it's still going to work because index here and the other index here is actually the same all right so we can keep it like that all right so basically like if it is not the audio stream whatever packet you return if it is not the audio packet we don't care about it so we're just going to continue cool now the next thing we have to do all right is to now send the data to the decoder and we do that we have right already send packet so we send the packets to the decoder create some text and they've got it okay so this is going to send the packet to the decoder for the code now if red is less than zero for some reason um this probably means that the um the decoder probably didn't like what we sent to it for some weird reasons possibly we sent the wrong packet and the decoder couldn't understand it or you know we got some corrupt data or whatever but sometimes this is going to return something called av error e again now if it returns av error e again this means that send the packet again after getting frames out you have to remember this okay this returns av error again what this is telling you is that hey make sure you call av code sync packet with this packet after you have you know retrieve all the frames inside of the decoder because sometimes the decoder will buffer frames for some reasons all right so you so if the internet buffer is full you have to read out all of the frames and that's what we're going to do right now all right so if red equals to av error e again all right so if it says try again then we're gonna actually read frames so to read frames we're gonna do av codec receive frame okay receive frame codec context and frame so in to red he goes to that right now so this is telling us that we are going to receive a raw frame now this is also as you can see this also has his own uh results so it's going to return whether the frame was successfully received or not and if the frame was not received it's going to also communicate that fact so this one this receipt frame can also return av error e again which means to tell you that hey go and send some packets um go and send some packets and uh how do i explain this yeah send me some packets and then come and try to read more frames right um so basically you know but the thing about the api is that you're never going to get av error e again for both av codex and packet and av code accuracy frame so if you get e again for send packets you're not going to get it for receive frame and if you get e again for receive frame you're not going to get it for cent package so the way i like to do this is uh actually um it doesn't matter the return type i always read frames after sending a package so what this means is that i only check if this is not e again then this is probably a legitimate error so so basically i'm like if whatever i get was not e again uh then something bad happened and probably need to quit the program or do something all right um now i'm just gonna log the error to communicate that fact and then i'm gonna actually have a while loop here [Music] okay and then i'm gonna have well what is this okay uh well this is greater than equal to zero actually you could just say while it is equal to zero right and so what this means is that when we when we send a packet then we try to receive frames so sometimes when you send the packet you can receive multiple frames of data all right uh what is this expression oh that's crazy i have too many all right so sometimes when you send a packet you are going to return you're going to need to receive multiple frames from the decoder so when i send a packet i want to receive all of the frames from the decoder before i go to send another packet again and that's what i'm doing here now we have to resample the frame why do we need to resample the frame uh most audio decoders in ffmpeg actually returns um the the uh frames in a format called av sample fmt fltp floating point planar so which means that it basically takes each uh channel and stores its samples in a non-interleaved manner you know um and we the most of the uh playback libraries don't actually work with planar data instead they want the samples to actually be interleaved and that is why we're going to need a resample to take care of that for us so we need to create ourselves every sampler and we do that using the swr context okay and swr a log set ups passing no pointer to the beginning did i did i did that do this correctly actually i don't think i did actually i did i did i don't know what's wrong here uh oh this was supposed to be a pointer even then it's not working i'm just gonna check the documentation all right so here we have the it needs an swr context first of all we don't have that so we're just going to pass in a pointer and next it needs an output channel layout output sample format output sample rate where do we get that from well according parameters is actually having those information so we're gonna do uh the output the output channel layout is gonna be stored in the stream codec power channel layout we don't want to change the channel layout so we're just going to keep that we're also going to need the sample format and the sample rate now the upper sample format we want to change so one we're gonna change that to av sample fmt flt okay so we're gonna be using float for our sample format and the sample rate we're gonna we're gonna leave that as is correct sample rate also we're gonna need to initialize what else the channel layout sample format sample rate again but this time it's for the impute so basically this is for the output this is for you know what what is going to be resampling to so now we we need to give you the data where of what it's going to be resampling from and it's largely the same except for the sample format which is going to be stored in today i can't really remember now next we also have to initialize the okay so we got log offset and log context as well logo set is zero love log context is gonna be no pointer i hope i was able to initialize this and actually i should actually i didn't have a name that was crazy uh i didn't actually name their sampler i think that was why the method actually didn't work oh geez anyways see this is format and we're gonna cast this to av sample format just like that now we should have our sampler initialized now if it if if our example is not initialized you probably check for that but most of the time i don't think i've ever seen uh so br context not being initialized from this but you know anything can happen in programming so you should check for that right so now we have to resample the frame and to do that we're going to create another frame called resampled frame av frame by look now once we've created that once we've allocated the resample frame we need to give it a couple of information uh shampoos frame sample rate is gonna be the frame sample rate sampled frames uh channel layout it's going to be the frames channel layout basically the information we passed in here right uh we're also gonna need these sample frames um uh we're also gonna need to resamples frames channels yeah we could copy that to frames channels and then the format so those channels that channel uh sampled frames format is actually gonna be av sample fmt floats flt all right so this is float this is basically a floating uh point uh you know for sample format but with the channels interleaved correctly interleaved then we can send that to the you know audio playback library that we're using all right now once we've sampled uh once we set our sample frames what we need to do is say swr um actually i don't need to yep okay i could use that swr convert frame okay so we're going to pass in the resumpla and the output frame is going to be the resampled frame and the input frame is gonna be free now here if there is sampled frame uh if this thing returns anything that is less than zero then you know that you were unable to successfully resample or convert this frame all right so yeah that is that uh but if we return zero then we're good all right but i'm not gonna check just make sure that anything negative that returns here is simply you know a failure and it was unable to resample the frame so the next thing we could do now is called av frame on ref and this is going to uh unreference the frame uh that we received from the decoder basically it's going to free the memory uh the data contained by the decoder but we're still going to keep the resample frame for a minute because we need to take out the raw frames now in this case there are a couple of things we could do with the resample frame at this point since we've actually gotten a usable frame what we could do is send it to the audio output but right now we don't actually have any other output so we need to keep this inside inside of a buffer all right we need to buffer this uh you know all these raw frames so that we can actually play them back and how do we do that well very simple we're just going to go back up and have another include and we're going to it's going to be in av u2 slash audio v4 now this audio fifa is simply a simple first and first out buffer that's going to help us store audio samples all right so it's basically used for storing out your samples so that's just what the other fifo is all right i want you to be keeping track of the objects we are creating and allocating because we're gonna need to free them you know when it's due you know the due time actually right so we're gonna create an av audio fifo now this needs a couple of information the sample format av sample fmt flt which this is asking us for hey what kind of samples do you want to store and so i'm just going to store i'm going to start float so that's what it sound performance uh it's actually asking for the channels well we want to store stream codec car channels that's what we want to store number of samples this is asking us for the initial size of you know what one how how big the sample is going to be doesn't really matter what you're passing here i always pass in one because i know that uh when you start filling the fifo buffer it's gonna grow automatically all right so just just pass doesn't even matter don't pass in the negative value because i don't know what's gonna happen if you do that but just pass in one or something bigger but not too big so you don't crash your program all right um so the next thing we have to do once we've actually converted our frame is to save the data inside of this v4 and we do that is called av audio fifo right actually i don't really like visual studio code this is actually extremely slow i don't know what's going on all right um so i pass in the fifo and the data is gonna be i have to pass in different uh there is sample frames data so that i can copy um the raw samples into the fifo and the number of samples is actually going to be in the resample frames and b underscore samples now where did i get the number of samples from well if you realize i didn't actually allocate any memory with the pre-sampled frame so the swr convert frame is going to allocate the memory set the number of samples field and uh yeah and that's how we get that value all right cool now once we write um the fifo we're done we're pretty much done that's all we need to do but we have to free uh we have to free the sampled frame so that we do not leak memory all right now once we're finished with all of this so by the time we're done with this while loop we would have decoded all of the audio and it's going to be stored inside of this fifo buffer and we can actually use it to play back the audio and that's pretty much it with um ffmpeg but we're not actually done yet we actually need to play back the audio so that the user can actually listen to it and how do we do that well let's go back so here i was telling you about mini audio and it's uh you know it's a simple library that allows you to play back audio and you can also use it to record audio but i haven't actually tried that but i've actually used it in my android application to you know um to test how it actually works i i think i tested it on linux and android and actually works great all right so it's cross platform as you can see it works on pretty much every platform um but how do we actually use the library well it's got a couple of examples let's take a look the first thing we have to do is define mini audio implementation and then include the header file so that's what i'm gonna do i'm gonna copy this and then i'm going to paste it here but then i don't actually um i have the mini audio included here so this should work all right cool cool cool cool uh so the next thing we're going to do is walk exactly we're actually going to need an m8 device okay so you can ignore all of this rubbish and we're gonna need an emmy device config and an m8 device i'm gonna copy those right now and [Music] the next thing we're going to do is initialize the device configuration i'm going to copy it with this and let's take a look so you can see here in order to initialize the device config we say m8 device config init and this is going to uh ask us for the type of configuration we want and you're gonna see that this is gonna say any device type playback so you have two types of uh devices you could create you could create one that is uh for recording audio or you could create one that is used for playing audio we don't care about recording in this video so we're just gonna use the device type playback now it's going to ask us for the format okay and this is an ama format so we could delete all of this we have m a format um f32 that's what we're going to use right then we is asking for the number of channels but we know where to get that from we have stream codec par channels we got that now the sample rate well we also know where to get that from stream code sample rate all right now the data callback well we get we have to create a function that it's going to call when it needs data so let's create that callback is actually here so we could copy that make sure you copy code as much as possible because you know especially example codes because you know you're gonna basically type the same thing they they give you anyway so why do why do you have to spend time typing it out i mean if you're just learning i think it's better to actually type out the uh type out the code because you know it helps you learn as you type but if you're already experienced that you know what you're doing then just copy the goddamn code right right um so the next thing we could do is go back to the button and set our data callback as you can see it's set now on the user data we don't have a decoder structure what do we want to pass in here instead well we want to pass in we want to pass in the fifo the fifa pointer okay because that's where our audio data is actually stored so the data callback is actually a function that will be called when the um when the audio device needs to render audio frames it's going to call it and ask us for the number of support or some number of samples that we can then give to it and then it's going to play back that audio frame okay all right um cool but what else do we need to do then well let's take a look we have any device start have we initialized the device yep no we actually haven't we just initialized the device config so we actually have to initialize the device and then we have to start the device so let's take a look now this is it so you see any device in it and may device start so this is going to initialize the device using the configuration that we passed in here and if it doesn't return any success then something bad happened and we should probably return something all right but we cannot return we cannot we don't have a decoder again so we're going to delete this all right uh so the next thing we got to do is um free some memory so basically after we've created the device configuration we need to delete all of the allocated memory for ffmpeg and we're gonna do it after here once we've uh so the only thing we're not gonna uh uh free is the fifo because it contains the buffer that we actually need all right so i'm gonna do av audio fifo um sorry not audio v4 if we format close impute uh the format context and that should close the format content i need to free the frame frame free crap frame av packet free packet and what else do we need to free actually let me check um so we're free okay so we need to free our resampler and that looks like it oh we also need to free our decoder because we're done using it right so av codec free context i think that's what it's called yep uh context and now we also need to free our soundcloud so sw our free resume and we're done we actually are good programming citizens because we have freed all the memory required that we need all right cool so now we've started the device and once we started device you can see it starts the device and begins playback which means it starts to call this method this data callback that we have okay so this data callback is going to request audio buffers from us and we need to give back and then it's going to play back the audio and if it fails to start then it fails to start we're going to free all the required memory at the end of the day though if we were on this map if if we were to run the program right now it's going to decode everything and it's going to start the device but the program is going to stop running because we reach the end of our main method and our program is going to terminate so we have to keep running this program onto something until we're done playing back the entire audio and how do we do that we have wow if the audio fifa um size all right and this is just the fifo so basically i'm simply saying here that while we have something to read from the fifo just stay there and don't do now this is pro this is actually bad because you probably want to wait for some time before you check again uh because this is going to use the 100 100 of the cpu so i'm actually not gonna do it because i hate bad practice um so what can we use to actually wait i don't know let me see actually i'm not gonna do this i'm just i'm just gonna do get char and so this means that it's gonna wait until it's time to um it's gonna wait until the user presses maybe the enter button and then once you press the enter button then it's gonna terminate the program all right i think that's cool i think that's good enough for this example and then we're going to free the audio fifa free the fifa just like that and lastly we have to free all this so any device rise and yes i think that's pretty much it uh yeah and so what we need to do now is no we can't test this yes because we haven't actually done anything in our data call back we don't care about an me decoder so we're just gonna delete this just like that so what we're gonna do now is call av audio fifa read passing in okay we don't actually have the fifa structure taken out yet so we're going to do um wait a minute pr p and p frame count oh yes sorry um so i'm gonna do ev audio fifa fifo and then we're gonna do uh we enter three cast to an av audio fifa uh p device user data all right so basically we're getting back our audio fifa from what we passed it in inside the other function uh at the bottom right there um so now we need to pass in the fifo uh it's asking us for data which is like where do we want to store the data well we're going to store it in p outputs i think and then the number of samples is just going to be the frame count and this is supposed to be the address of the p output uh let me just double check to make sure it's the p output or i think it is yes it actually is and this should actually do good but we don't actually know that yet because we haven't actually tested so uh let's run this and see what is this what is this what is this i don't think i have gtp what coupling what is this talking about i don't really know what this is talking about okay i actually need to use c make to do this i don't know yep she make wrong without the bugging so simic is going to help me do the compilation and try to run the program and see if we get any crashes or bugs so far um we actually did great we just have less than um one you know we have less than 200 lines of code so far so we're doing great uh but we don't know if our program is actually gonna work so yeah we gotta wait i don't know why this is this is taking like i don't know ages to build i think visual studio code is just too slow i i don't know what i'm supposed to do um i don't know i think it's better i should i think i'm just gonna go look for another ide to use for um c plus plus development while i'm recording videos because i don't know why it's taking this long what exactly is it doing oh crap um i think i should just probably execute this myself because i've never seen anything okay i'm gonna wait for five seconds four three two one this is crazy i swear to god i'm gonna run it myself and see what's going on all right uh cd2b um then i'm gonna do what is all this ninja do i have ninja yes i i do have ninja okay um she's compiling the main.cpp and i can't remember if i actually included the sw okay i did and just to warn you by the way the um what is the name of this library the emmy audio is actually about i think 2.5 megabytes or i'm i can't really remember the uh the mini audio file so it's actually pretty large so including it in the um in the in your program can actually take some time to compile but take a look it actually failed um it says dso missing from command line p3 join oh this is crazy i don't know what that means but i'm just going to go back and include um i think that's like the math library i don't but i don't know where that error is actually coming from um i'm gonna go back to the terminal and run ninja again and it fails again csds are missing from the command line what the hell is that p thread let's see if if we include p thread it's going to change anything i dot it i dot it let's just run it now what dl close are you kidding me this is getting interesting it's actually needing the dl i don't know i don't know this is not this is not good jesus christ so i had to i've never actually in my life had to oh sorry uh actually that was my fault i was using pthread on the deal before i actually included the mini audio library um so i didn't know that one of his dependencies was p thread and dl and that was probably why this wasn't building no it was kind of weird because ffmpeg actually never i think they use all the dependencies but they actually refer to them properly because you know i'm using actually this system builds library so he knows how to refer using the r path and oh i'm just saying a bunch of noises let's just run this vampy audio decoding let's see what do we got [Applause] okay that was good uh and then i press the enter key and i get a segmentation fault should we figure why should we try to figure out why that actually happened i think i might know you know probably we we we closed the whole thing um you know i probably should do this before i should do this before i you know maybe this is going to stop the callback from running before i actually free the fifo right so let's actually try to compile that again using ninja ninja and then we're gonna see uh again i don't know how long have i actually been recording uh american for jeez and how and that's crazy an hour i wasn't really planning for this to take that long but yeah it's cool all right um this is taking longer than i expected i think it's probably because this file is just huge i don't know i don't know it's just too huge uh but it's linking it now and i could run it so okay yeah so i press the enter key again as you can see i didn't get a segmentation for it because uh i'm actually cleaning up properly all right um this video has taken way too long longer than i actually planned for it to be but it's fine i hope this video helped you in some way and yeah i'll see you in hopefully the next video bye i

Channel: J Adebayo

Views: 534

Keywords: Java, Android, Game Development, AIDE, Programming, Mobile, Allegro, Allegro5, Android Development, Android tutorials, Reverse engineering

Id: -jugPJ_O8iM

Length: 63min 41sec (3821 seconds)

Published: Wed May 19 2021