Programming MIDI

hello and welcome to the first video of a sporadic series where I'm going to be messing about with MIDI now some of you may not be familiar with what maybe is it stands for musical instrument digital interface and you may be aware of it as some sort of music format from decades ago and you'd be right but what's interesting about MIDI is it doesn't transmit audio at all there is no sort of waveforms of samples sent via the MIDI protocol instead MIDI is a sequence of timing events that can be supplied to musical instruments and equipment as and when necessary I think the original creators of the MIDI specification were exceptionally clever and very forward-thinking and as a result the MIDI format has really stood the test of time so much so that it can be modified with ease to accommodate new developments in the industry and this has also become ubiquitous across the industry it fully satisfies the needs of making sure that musical instruments and equipment can talk to each other in a sensible way now if you're not interested in music that's fair enough that MIDI can also be used as a general-purpose timing protocol or maybe I'll talk about that a little bit later on but from a programming perspective I think that the MIDI file protocol is quite interesting and I wanted to explore it in a video so let's bust out our synthesizers and bags full of cables and let's get started and I thought we'd start with a very brief MIDI primer here I have some sort of digital instrument it could be a synthesizer but these days it could really be anything at all and in principle the user presses the keys on the keyboard and it emits some sort of sound that we can hear but unlike an actual instrument it's completely synthetic the keys are electronic switches which switch something on or off and accordingly a sound is produced now because it's digital we don't need to rely on the clumsiness of a human hands to press these switches we can of course send an electronic signal instead so we could send something like a an on-off event for a particular key has the key been pressed or has it been released we need to identify the key so that could be a particular note and there may be other parameters that we want to say - such as how hard did we press the key this is called velocity and these additional parameters allow all sorts of expression in the musical plane we can package you all of this information as some sort of event and send it to the instrument now as the name implies this event only does one thing but of course for a musical piece you want to do lots of things so it's likely we're going to want to send a sequence of events but it's no good just smashing out these events as fast as we possibly can well the entire song will be played well instantaneously it wouldn't sound very good and as music is a temporal medium it's important that we include timing information into these events so let's say at the beginning of the song I want to press this particular note but I only want to hold it down for a certain amount of time maybe a hundred milliseconds or so therefore I augment the packet with the information to say that this event is occurring 100 milliseconds after the previous event and subsequently we might want this event to happen 50 milliseconds after the previous event in effect we send the delta x between our events and sending the delta time is quite an important thing because what if we wanted two events to occur simultaneously well we just specify a delta time of zero so even though these events have been transmitted in series whatever is receiving these events at the other end can accumulate them based on their delta x and perform them simultaneously now on the actual hardware itself it doesn't work quite like that's very difficult to do variable numbers of things simultaneously in electronics hardware but the principle is that you run your electronics hard work far faster than you're sending these events so these are let's say we're operating in milliseconds here the processor on our digital instrument could be quite happily operating in nanoseconds and so the end result is that it feels like these are happening at precisely the same time it's imperceptible to us we've got the boring and limited human beings and that introduces two interesting properties of the MIDI format firstly it only has to be sufficient for the bounds of human perception but secondly it is undeniably real-time and will see as we're exploring the MIDI format that were possible keeping things synchronized and quick takes a priority now as well as being a programming nerd I'm also a bit of an audio nerd and I have quite a sophisticated setup for dabbling about with home audio I don't consider myself a skilled musician by any means but I like to think that I am and I try and play my best and whenever you start working with audio one of the primary concerns you have is latency in my experience when I'm clumsily hammering my keyboard with my fists I start to notice that I can become out of sync if there's a latency greater than 30 milliseconds and that doesn't seem like a lot but it does demonstrate how good the brain is at trying to synchronize audio to real-time actions within your body it actually becomes very difficult to play anything as the latency gets quite large and even though I'm not that good on a keyboard I am actually quite okay with a guitar when a guitar is typically quite a fast instrument to play and when I've dabbled with real-time audio effects in the past one of the things I noticed is that plucking a note on the guitar I could only tolerate a far lower latency and to be about 15 milliseconds before hearing the sound but where else my fingers would become out of sync with what my brain thinks they're doing the newer sources of latency in an audio setup but as you go towards the higher end that latency tries to reduce but where is this latency coming from well first of all physics you can't have zero latency and kind of system but originally when digital audio workstations were starting to be developed and started to be available to the consumer the latency was in the cables and the connections between the pieces of equipment you could only transmit digital signals at low speeds throw into the mix the latency of converting those signals from digital to analog and then at the time some rudimentary effects processing and eventually you end up by pressing a key and a little bit later on only hearing the final processed sound interestingly as we've progressed the technology the communications bandwidth has got much faster we've got things like USB 2 and USB 3 now but at the same time we're demanding a lot more from our CPUs to produce more interesting effects and more realistic sounds the one thing we have going for is in audio is that we're still talking about milliseconds here into a computer a millisecond is an eternity anyway I'm getting sidetracked let's get back to the MIDI video there are several layers to a MIDI sequence at the most bottom layer is the concept of a channel where each MIDI event is associated with issuing information to a certain channel and that channel typically reflects a specific instrument and it's usually down to your studio setup or dedicated MIDI processes to make sure that these events are sent to the appropriate destinations in a MIDI message you can have up to 16 channels which doesn't seem like very many but this is where the second layer comes in the concept of tracks tracks are sequences of MIDI events that are occurring in parallel and here in this new track some of these events may be mapped onto the same channels that we already have appear but other events can be mapped to well new channels so we can quickly build up quite a large library of virtual instruments before we go and dissect the MIDI file format for ourselves and I thought it would be useful to actually look at a MIDI file and I found this piece of software it's free to download but they encourage you a donation simply called MIDI editor by Markus Schwank it's very nice it's very simple but it allows us to analyze the tracks and the channels and the messages in some detail and it's typical of MIDI editors to look a bit like this in the y-axis of this frame we've got which particular note on the keyboard is being played now there are always keyboards even if the instrument doesn't have a keyboard say it's a drum kit of course the drums can't play notes but you could assign different types of drum to different keys in this editor the tracks are all overlaid on top of each other and they're all different colors here I've isolated track 1 and I'll click on this big event in the middle and you probably heard it just made a sound the nice thing about this editor is it allows us to analyze the event information that might be relevant to a programmer so we can see we've got this on tick and off tick that's the Delta time from the previous event it's actually visualized here in what is called wall time so this is real time but the event itself will only contain the Delta to the previous event it tells us that it's playing note 77 and it's using a velocity of a hundred so really hit that key heart and the channel is associated with is channel zero now don't forget MIDI doesn't contain any audio information it's simply a way of routing these event packets around the system so channel zero needs to be associated with something that produces sound and in this case channel zero has been mapped on to string ensemble one in fact we can have a listen [Music] these instruments on the site are virtual instruments provided by my Windows operating system they sound a little bit flat and old-fashioned typical MIDI sounds if you're familiar with those and that's because on Windows you get this Microsoft GS wavetable synth which is a basic selection of the virtual instruments and in fact I can change the channel to something else so let's emulate a piano sound instead one of the nice things about events in time and pitches is they can be visualized on a 2d graph like this and so I think that's what will aim for first we'll try reading in a MIDI file and visualizing it and I just love watching the complexity of things like this but when it all comes together it produces something rather special I like it a lot [Music] as with anything that is considered a standard somewhere there exists a specification for that standard and this is the middie specification official website and it contains lots of information about how we can interpret what's going on inside a file files are a little bit different to just regular MIDI streams between instruments as they contain additional information that the instrument simply don't care about regardless there's nothing we'll see in my code today that we can't find listed somewhere on the specification website now as usual I'll be using the OLC pixel game engine for the visualization quite a high definition one this time at 1280 by 960 in one to one screen pixels however interpreting the MIDI file we're going to be doing from scratch so it doesn't require the pixel game engine to understand how the MIDI file format works so we'll be coming back to all that later on because I want to develop a class that I can use in subsequent projects I'm going to create a simple MIDI file class I'll give it a constructor and all of the work is going to happen in a function called parse file which takes in a path to that particular MIDI file so just for convenience I'll add in an additional constructor which can take in the path and just call the pass file function now I'm going to treat the MIDI file as a stream because in effect MIDI messages are sent as a stream you don't really know what's going to happen next and so maybe by constraining ourselves to work with the file stream will produce better code for working with MIDI streams in the future so I'm just going to create a standard input file stream object open the file in binary mode and if there's a problem I'm going to return false the first curiosity when it comes to MIDI is understanding how to read values from the file MIDI was developed way back when literally billions of years ago when processes used a different byte order to store numbers than what we're familiar with on our modern desktop computers so every time we read say a 4 bytes integer we need to swap all of the bytes around so I'm going to create some little lambda functions to do just that I've got one here called swap 32 it takes in an unsigned integer now that will be a different byte order than what we actually want so the contents of the lambda function simply swap all of the bytes over I'm also going to create one for 16-bit numbers too as I mentioned before MIDI files contain lots of additional information that the instruments don't care about but perhaps the editing system for that file does so they can contain things like strings fortunately strings in the file are going to be stored sequentially and we'll know the length of the string in advance so I'll create a little lambda function read string which takes in that length and allows access to the input file stream and just successively reads the characters and append them to a standard string object now we get to the second curiosity with MIDI files most MIDI files will never contain information that requires more than seven bits to store and that might seem like an unusual design choice for platforms which routinely work with eight bits but some way it does make sense firstly you want to minimize the amount of bandwidth you're going to transmit as hopefully this will reduce the latency but also 127 in terms of musical equipment is quite a lot of different things for example if you had a keyboard with a hundred and twenty seven keys on it that's going to be quite some keyboard alternatively if you want some sort of expressionistic tool included in your midi event well a hundred and twenty seven levels of differentiation within that expression is quite finite and high resolution so this comes back to that human perception and real time thing we were talking about earlier it's good enough to get by but limiting yourself to seven bits does start to raise some interesting problems with the file format so what if we wanted to for example read in a 16-bit number of 32-bit number there are on several occasions situations one hundred twenty seven isn't enough specifically the timing the Delta time between the events that could be many thousands of ticks clock cycles milliseconds whatever we want to call it and so we can't store that in just seven bits alone and this is where I think the creators of MIDI is something rather clever they have a variable length numerical type and so when I know that I'm going to need to read one of these types I'm going to create yet another little utility helper lambda function as mentioned the bulk of midi information can be represented in this bottom seven bits but it's convenient for computer systems to read eight bits this means we have a bit here that doesn't really do anything well what if we used this bit to signal to the parsing system that the value are trying to read in is in fact greater than seven bits and that we need to read another byte in order to read the full value so when we're reading a value if that bit is set to one we now need to read another point again only the bottom seven bits are interesting to us but we can now construct a fourteen bit word out of these two successive seven bit reads now what do you think happens if the next byte that we read also had its most significant bit set to one well the process repeats itself we read in another bite take the bottom seven bits and form a twenty-one bit word and we can keep doing this until we see that the most significant bit of the last point we've just read isn't one and that tells us we've got our complete word now seven 14 and 21 bits seem like unusual numbers so I'm just going to stuff all of those into a 32-bit word now this may seem like a very rudimentary compression but it works very well when we've got low value numbers we only need to transmit a fewer number of bytes great for a system were transmitting things increases latency I'll store my accumulated value in this variable n value I'm going to need a little helper variable which represents one byte when instructed to read a value the first thing I'm going to do is read one byte from the stream now that byte could be completely sufficient to give us the final value so I'll read that into the N value variable but I need to check the most significant bit of this byte because of its set then I need to keep reading so I'll take the bottom seven bits of the byte that I've just read and then proceed to continuously read bytes until I read one where the most significant bit isn't set so in this loop I'll read the byte and I'll take my currently existing value and shift it all along by seven bits and or into that value the noobs bottom seven bits I've just read don't forget to return the final value from the lambda function now we're ready to parse the MIDI file for convenience I'm going to add in two temporary variables 32-bit and a 16-bit unsigned integer midi files begin with a MIDI header and this header contains information about how to read the rest of the file the first thing I'm going to read in is the file ID now if I open a MIDI file in a binary editor the file ID is always this empty HD it isn't actually a number that means anything is just for bytes that can be instantly recognized as a MIDI file once I've read it in I'm going to swap the bikes around and then immediately forget about it because I don't care the next four bytes represent the length of the header and today I don't care about that either the next two bytes tell us things about the format's of the MIDI file and mmm getting embarrassing again I don't really care about that what I do care about though is this next one which is the number of track chunks and we'll come back to that in a minute because we've almost finished reading the header there's finally another two bytes to read which ultimately I don't care about so out of all of that header all I'm really interested in is this value here track chunks is the number of MIDI tracks this file contains and those MIDI tracks will contain the MIDI events notice we've not mentioned anything about channels here that's included in the MIDI event itself but as we'll see there are ways to sort of augment what's happening in the MIDI ecosystem of whatever software is interpreting the MIDI file the MIDI header is a fixed size so as soon as we finished reading precisely that amount of information were into reading the data itself and we'll start by reading the track data I know how many tracks is going to be that's my track chunks so I'm going to go to loop which goes through all of the tracks and just for convenience I'm going to output to the console that this is a new track if we didn't know you can still output to the console in the pixel game engine that's why there's two windows that appear tracks themselves also have some track header information two 32-bit values one which identifies track ID and if we go back to our binary MIDI we can see that here again it's just a text string that identifies I am a MIDI track and the next value is how many bytes are contained within that track now if you pass your MIDI file properly you're not going to need that value either because from now on everything that we read is going to be a MIDI event and all MIDI events are deterministic in size and content I guess you could use the track length to check for corruption in your MIDI file but it also allows you to completely skip reading a track if you wanted to just skip the next n track length bytes you could get to the next track however we're not interested in doing that there is one particular MIDI event which signifies the end of a track so I'll create a boolean flag just to keep track of whether that's happened and then I will sit in a loop making sure that we don't prematurely reach the end of the file but also keeping an eye out for that event to occur where now we're going to read in and process the MIDI messages I'll start by assuming that all MIDI events contain a time delta value and a status value the status value will tell us what type of event it is that you'll notice however there's a little asterisk next to the status pipe because we'll see later on that not all MIDI events do include a status byte but let's keep it simple for now so the first information is always going to be the Delta time from the previous event for this particular track and that's going to be one of these variable length values so I'll call our lambda function to read that in and then I'm going to read in the status value of the MIDI message now there's lots of different types of MIDI message so it's time to start bringing in some information from the specification as part of my MIDI file class I'm going to add two e noms and I've primed the values of this enum with the words necessary to represent that event type so in this case we see things like voice note off and voice note on so that's pressing a key and not pressing a key we've got other controls which changed the expression involved with the note so pitch Bend will slide changed the note whilst it's playing we also have another type of event called system exclusive and we'll be using that to today system exclusive is not directly involved in producing sound it's more about configuring the environment or the instrument note that this is the most significant nibble of this byte that really contains the information the bottom four bits of the status event indicate which channel the MIDI event is being targeted at so we only need to compare the top four bits of our status byte to see what the event means and there's no real way around this we're just going to need to successively check each permutation so note off notes on aftertouch there we go I've handled all of the event types now fortunately for you guys today I'm not going to go into detail for each and every single one of these just the ones that I'm interested in for visualization but we can't fully ignore them either don't forget we're reading sequentially from a file stream so let's for example say we got a voice channel pressure event we need to read the data associated with that event or else our stream will become corrupted I guess what I'm saying is we still have to read in all of the information in a valid way but for my visualization I'm really only interested in capturing the voice note on and voicenote off events now it's time to talk about the third curiosity of the MIDI file format and this certainly caused some confusion for me when I was attempting to reverse-engineer all of this it's called MIDI running status and it's a way of compressing the data stream even further let's assume we wanted to send a sequence of MIDI events to an instrument we would send the event time Delta the event data type and the event data itself and we would send that repeatedly over and over again so if I wanted to press 8 key simultaneously on a keyboard I'd send Delta the event ID event data Delta event ID event data etc eight times but the event ID is the same for all of them so that seems like eight bytes I don't to send recall that most MIDI data is in the range 0 to 127 therefore we only need 7 bits and the most significant bit is superfluous again well you may have noticed already that all of our event IDs are greater than 127 so that most significant bit indicates that the byte would just read is an event ID so if we read some time Delta and then immediately read a byte that didn't have its most significant bit set we know that actually what we've got is some active MIDI data so which ID do we use well we use the previous valid ID that was sent so this means our stream of 8 events for the 8 keys being pressed would look more like the timecode Delta the event ID of the first one and then the event data and then simply becomes time Delta data time Delta data etc etc so this is just another way the creator's tried to optimize the bandwidth requirements and this caught me out at first because there are plenty of valid midi files out there that don't care about this optimization but in order to be compliant and make sure we can read any midi file we do need to care about it so as soon as i've read the status byte from the stream I'm going to check its value because if we know that the most significant bit isn't set then we've entered this MIDI running status mode this requires us to keep track of the previous status byte that we read and so if the bite doesn't have its most significant bit set I'm going to set our current status to the previous status but in order to get this far I've had to read the byte from the stream I've effectively just lost it and so when it comes to reading the data of the individual events later on I'm one byte out of place now I'm going to cheat a little bit here and simply instruct the stream to go backwards one byte just a recap on a standard MIDI message we would have read the time Delta and we read the status byte we then interrogate that status byte to determine how we're going to read the data for that event we've not done that bit yet however it's possible that the status byte simply isn't sent but in order to know that we've already needed to read as if it was a status byte where in fact it was the first byte of the data of the current event so this line here just make sure that everything gets back in sequence so how do we read the data well let's assume a key is being pressed voicenote on well rather lazily I'm going to cache the previous status and start parsing this data now the bottom four bits of the status byte reflected which channel the event was being targeted at so that's easy enough to extract and for a voice note on there are two more bytes to read the first byte indicates which note has been pressed and the second byte indicates how hard the note was pressed so the status byte told us that it was a voice note on event and the specification says the next two bytes represent the ID and the velocity and we'll see this is a common pattern throughout a lot of these events so if a voice note off that's the key being released we read the same two things now even though for this video I don't care about what these other packets actually do I do need to read the right amounts of information from them because we're reading them from a stream and it turns out the voice after touch is exactly the same voice control change is exactly the same so the voice control changes the other dials and buttons on your MIDI instrument however program change isn't the same it just requires us to read one additional byte as does reading the channel pressure pitch Bend is a little bit different again it does require us to read two bytes but this time they mean something different and we'll come back to system exclusive in a little bit I should also just add in here and else unrecognized status bytes just in case something has happened this corruption or the MIDI file hasn't been rendered properly so now we're reading in this stuff what am I going to do with it well my intention is to render the MIDI file on the screen in a way too similar to what the MIDI editor I showed earlier is doing so I don't really care too much about using this as some sort of MIDI processor but I do care about translating the events I'm reading from the file into information that I can use in my application and potentially I'm going to have multiple tracks so I'll create a structure called MIDI track which I'm going to have a string for the name a string for the instrument and I'm also going to have two vectors these vectors are going to store the events in two different ways the first one MIDI note it's just simply the key the velocity when it starts and how long it's going to be so that will effectively be what I can draw on the screen the other is going to be the MIDI event it's sort of a cruder form of storing the MIDI information and I'm only interested in notes being pressed notes being released and everything else I'm not bothered about so as we now read through the file all of the different MIDI events I'm going to add them to this MIDI track and I'll store all of these tracks in my MIDI file class in a vector of MIDI tracks at the start of our loop we know we've got a new track so I'm going to push a blank MIDI track into that vector let's take the voice nodes off event as an example I've read in the relevant information I'm now going to construct a MIDI event using that information in this case note off the note ID the velocity and the time Delta and I'm going to push it into my vector of MIDI events for that particular track I'll do something very similar for voice note on however here is another little quirk of the MIDI protocol it turns out that any voice note on event where the velocity is zero is actually going to be considered a note off event so I'll check what the note velocity is and then add to my vector events well the appropriate event now as I've mentioned already I don't particularly care about the other events but I'm just going to push something to my vector for now so far so good and reasonably quite simple it would be great if MIDI files just contained this basic event information but sadly they don't and this now adds some complexity to what we need to read in because we have to read in everything to make sure that all of the data were reading is valid and this complexity comes through the guise of the system exclusive events now in the source code that accompanies this video in the github I've gone through these events some detail I'm not going to do that for this video other than to look at one or two of them all we care about for now is knowing how to read them properly so I'm going to add another enum to my MIDI file class for all of the system event types meta events in this case and this contains all sorts of wealth meta information so generic text copyright what is the name of a particular track what is the name of a particular instrument are there any lyrics associated with this this is the kind of stuff that the editors will use in order to configure a project around a particular MIDI file instruments don't care about this sort of thing there are some critical ones however setting the tempo setting the time signature that sort of thing could be interpreted by the instrument to change its speed system exclusive is a way of making the file format flexibly enough to handle future changes and so in the future we may have messages which are not defined by the specification these messages will be guarded by these two status bytes meta messages however are defined in the specification and these always have F F specified as the status byte when we know that our MIDI event is a meta message we're going to read in what types is and we're going to read in the length that could be variable length of the message then it's simply a case of just processing the type now I've just cut and paste all of these in all of these different meta message types I'm not going to go through them in detail but we'll have a look at some of them so once we recognize the type for example is the copyright information of the MIDI and we've read in the length of the message the specification says that this is going to be a string of that particular length so we're just going to read in that string with our red string lambda function the same applies to things like the track name that's totally decided by the composer I think it's quite interesting to read the track name in because we can visualize it later with the MIDI events and believe it or not that's that read the header read how many track chunks there's going to be then read each track chunk individually read the track chunk header and start parsing the events the events all have a delta time they all have a state and they all have an event body if the event happens to be a system exclusive event then it could be anything that we want that needs to be defined particularly for a specific system or it's going to be one of these meta events which is defined by the standard across all MIDI files and that's it so we can start to test our parser by loading up a MIDI file and I'm going to do that in unusually create I'm just going to load the MIDI file that I was playing earlier you'll notice that's in the meta events I've outputted a lot of stuff to the console so if I run the application pixel game engine doesn't do anything but if I click in the console window behind it we can see what's been outputted and there's been a whole bunch of tracks there we've managed to determine the time signature and the tempo 185 beats per minute and we can see that each track that we read in has a name associated with it typically which instrument is going to be played there's also been some stuff in there that shouldn't be there and this is one of the final curiosities I've learnt about MIDI files is they contain all sorts of junk that doesn't necessarily adhere to any particularly known standard so that's why we have to be careful about making sure that we do read everything that we need to read well I'm satisfied that there is certainly MIDI data being loaded but right now our MIDI file class contains vectors per track and those vectors contain MIDI events I'm now going to convert these MIDI events into something I can visualize don't forget the MIDI event works with Delta time and Delta time is a little bit tricky when you want to draw things in the future so I'm going to convert the relevant MIDI events into discrete MIDI things that have happened so I'm going to turn them into these MIDI notes I know the key I know the velocity I know when it starts in real time and how long that note is held for ie my MIDI note doesn't care about on or off once I finished passing the MIDI file I'm now going to do this conversion from events into something which is more useful to me so I'm going to create a little auto for loop that goes through all of the tracks for this MIDI file tracks can be treated in isolation they run in parallel to each other so timing information doesn't happen crossed tracks I'll create a variable called wall time which is my real time value and it's my intention here to look at all of the MIDI events as we go through time and reduce them in to my MIDI note events and this has a little bit of complexity because the MIDI event is far simpler it's on or it's off but my note is on with a duration so I need to keep track of which notes are effectively being held down at any given time and when they are released and I'll do this by creating just a quick temporary list of MIDI notes of my list of notes being processed and now it's time to iterate through all of the events in my vector of events we know where we are up to in real time simply by accumulating the Delta of every event now the deltas could be zero that's okay real time hasn't progressed that just means two things are happening at the same time if the event is a note on I'm going to add one of my MIDI notes to my list of notes being processed the problem is I don't know what the duration of this note is until I find the corresponding MIDI note off event for that particular note and I'm really sort of hacking this together I'm sure there's far more effective ways if the event happens to be a MIDI note off I'm going to search my list of currently processed notes to see if it's the same key and if that search yields something that means this is a note off event for a key that's already being processed and now know its duration and so I can work out its duration by looking at the current wall time - the note that we've found start time and since this note is no longer being processed it graduates to being put into my tracks vector of notes and I can erase the note being processed from the list of notes being processed those of you really paying attention will have noticed in my MIDI type structure I've got two additional variables max note and min notes and I've set them to 64 potentially for a MIDI event through a hundred and twenty seven notes this is very rare to see something going from 0 to 127 and if I was to visualize this I would have a lot of blank screen with just a small line of things happening in the middle of it I'm going to record what the maximum note pressed was and what the minimum note pressed was and that way I can scale the height of my track visualization and I'll make a check for that here and this will allow me that when I'm visualizing the track I don't need to draw lots of blank empty space above and below were really all the action is happening in the MIDI file let's quickly construct the visualization first I'm going to clear the whole screen to black and I'll add to the class a 32-bit unsigned variable and track offset and what I'm going to do is use track offset to move those forward and backwards through time I'm also going to need some variables that tell me how to actually visualize what I'm seeing so per column so this is one pixel across is going to represent 50 tics in the MIDI domain now ticks we've not really talked about but fundamentally it is the clock that is operating behind the scenes for the MIDI system so when we talk about these delta x i've sort of roughly mentioned milliseconds they're actually in midi clock ticks and these ticks can be converted into real time by looking at some of the meta messages that have come through the midi file fortunately most systems have the same definition for ticks it is important that the frequency of the midi clocks used across your system is the same not only does it keep your instrument synchronized but it means that one instruments is playing stuff at the same speed as the other i'm avoiding going into timing control directly by maintaining the idea that a tick is an arbitrary time duration specified by something in this system so I'm just going to make sure everything is relative to the tick so per pixel on the screen I've got 50 ticks and each note I'm going to display as being 2 pixels high for each track in my MIDI system I'm going to draw its contents but I'm only going to do that if the track actually contains something you'll quite often see in MIDI files that tracks contain additional information just as composers and orchestras and musical notes and that sort of thing stuff that we can't visualize I'm only interested in notes on and notes off because we know the minimum and maximum note of a particular track I know what the range is for that track and so I'm going to draw a gray rectangle across the screen using the fill rect function the same height as the range of that particular track I'm also going to draw the name of that track - I've just introduced this n offset Y variable we're going to need to change this as we draw the tracks because each track is going to have a different height and we're going to have several tracks going down the screen now it's time to draw the notes and I'm not going to optimize this in the slightest I'm just going to draw them all even if we can't see the majority of them each note is going to be drawn as a filled rectangle the x coordinate for the starting point of that rectangle we already know because we've extracted it from the event sequence and I'm going to offset it by our track offset value and don't forget that one column of pixels represents 50 ticks so I need to divide this number by than 50 to effectively scale it in the X direction the Y position of the note is determined by where we're starting to draw the track on the screen and the key value of the note that affects its height so as the notes get higher we want to see them moving up the screen the width of the note we've already determined because it's the duration of the note divided by the time per column and the note height is just a constant that we've specified and I'll draw that as a little white rectangle I'll just add in a couple of user controls so we can change the track offset manually with the left and right keys and that's our complete on user update function as you can see there's not much to it and that's because we took the time to change our MIDI events into notes something that we can tangibly work with in our own framework so let's take a look so it's loaded up the MIDI file from before and we can see we've got tracks going down the screen each gray rectangle is a track and they're labeled with the name of that particular track and inside the tracks we can see there's lots of notes and if I press the left and right keys we can scroll forwards and backwards in time so in a way it's very similar to the MIDI editor software we saw at the beginning I'm really pleased that we've accurately visualized the contents of a MIDI file but I'm feeling a little bit empty and underwhelmed it's MIDI we're meant to be hearing things were meant to be making lots of strange noises so I'm going to now just really quickly hack in something and I'm sorry it's window specific it's a total budget is a complete hack what I'm going to do is take our notes convert them back into MIDI events and send them to one of my synthesizers so we can hear it and this time I really do mean we're gonna do this quickly because I'm not going to go into any details what I'm going to do is load the windows multimedia library which will allow me to access the midi drivers built into windows i've no doubt people out there will be able to come up with a linux alternative i've just added some additional variables to the main class that will maintain timing the windows multimedia library is considered a bit old hat now for real-time audio but it's still fantastic for midi in fact very simple to just quickly open an instrument in the on user update function after we've drawn everything I'm just gonna throw in a quick test where we're going to be sensitive to the spacebar being pressed and released the windows library easily allows us to send a MIDI message to an instrument and it's the same format that we've been working with already here we've got the status byte here we've got the note and here we've got the velocity so when I press the spacebar I'm setting a note to be on and when I release the spacebar I'm turning it off so here you can see one of my synthesizers if I press a key beautiful and I've connected my synthesizer to my computer via USB and it will transfer MIDI information via the USB so let me start my application and we can see the MIDI track that's been drawn but I press a key on my synthesizer but I can also press my spacebar [Music] so the spacebar is linked to my synthesizer via MIDI now I'm not kidding when I say this is bodge code as its best it violates all sorts of rules and principles I do not recommend that you do anything this way but it's just to allow us to prove a point so what I'm effectively going to do is take our note sequence convert it back into MIDI and issue those events to my synthesizer so let's take a listen now my synthesizer can only play one track at a time so I've chosen track 1 which is sort of the lead string ensemble from before and I'm going to play [Music] and hopefully can hear that I can real-time manipulate the sound from my synthesizer the MIDI file I've been working with is quite a simple one so I want to try slightly more complicated one so let me just choose an appropriate patch on the synth and that one I'll do and let's play this file at this time now I'm going to play both tracks simultaneously I have a small loop here instead of just one track let's have a listen [Music] [Music] and as we can see it's quite happily loaded in absolutely every single MIDI note was an additional track at the top there that can take to tempo change information which I'm not using in my playback but this is a very sophisticated lots of different notes it seems to blow disease all and handles a coin one so that's that a quick and simple look at the MIDI file format now I like MIDI I like sound I like programming and I think I'm going to merge it all together you don't have to just use MIDI for instruments it can be used as general-purpose timing control - and I know for a fact that there are certain firework displays and lighting shows which are completely coordinated by MIDI also that actually gives me an idea for a game so hmm i'ma think about that anyway if you've enjoyed this video give me a big thumbs up please have a think about subscribing the source code will be avail the source code will be available on the github and I'll see you next time take care

Channel: javidx9

Views: 66,924

Keywords: one lone coder, onelonecoder, learning, programming, tutorial, c++, beginner, olcconsolegameengine, command prompt, ascii, game, game engine, pixelgameengine, olc::pixelgameengine, MIDI, Synthesizers, Audio programming, Tracks, Channels, DAW

Id: 040BKtnDdg0

Length: 47min 28sec (2848 seconds)

Published: Sun Mar 08 2020