Jules Storer & Cesare Ferrari - Build a Synth with SOUL (Workshop)

hey everybody I'd like to welcome you to the official Seoul workshop with the creators of Seoul jewels store who of course it's creative juice and Cesare Ferrari it's got the best last name in the world and they're going to give us a master class on how to build a simple synthesizer using salt so before we get started why don't we just talk a little bit about why are we using school in the first place so what makes it what are the advantages of trying to use this to build our synthesizer versus C++ as we would normally do well it's a little easier I mean we saw we announce I didn't know Spencer AEC 2018 where we laid out all the great great reasons to use Sol the main bits about it are in the long run it'll give us better performance on more interesting hardware than C++ spoil and in the short term it cuts out the ability to do anything stupid which is kind of a stumbling block for a lot of C++ learners and experts too because I make a lot of mistakes as well I've been doing this a long time so with Sol we kind of it's this kind of sweet spot where the language stops you doing race conditions memory allocations all the bad things that you have hours of footage on your channel telling people not to do but they're saying that doesn't stop the performance side of things and we've tried to make it as familiar as possible for people who have used anything like C++ or Java or JavaScript or all the other things that look like that before we Sol kind of looks the same it just doesn't let you do some of the dangerous stuff okay so the if you're a beginner Sol will be much easier to get you started because you know rather than crashing your program you'll get a syntax error and if you're an expert in the long run we want Sol to be running on really interesting embedded platforms we want it's gonna have latency advantages when we can move Sol programs and have been running on external devices and I mean this is a huge topic we could do an hour about that but yeah it's it's got a lot going for it we think in terms of the architecture of the language amazing so so where should we start Cesare right well um we did we gave a talk at the printable audio workshop in Leon in December last year and I think we're going to work through that material okay which hopefully would turn out to be just over an hour or something like we also do the workshop at AUC which we failed to record hopefully hopefully the camera is actually and the batteries don't just keep asking is like okay wait this time this time we're gonna do it so yeah this is the kind of more or less going to be the talk he gave it P aww which is why it says P aww on the slides itself first don't be confused yeah okay so the material going through is is available on github but there's a link at the bottom so in Solvang is a poor 20-19 project okay I think it's a project so um let's go through this so I think just covered what Sol is yes so guess what we were what we wanted to do was in this in this tutorials is get people actually writing some salt now as this was a live coding workshop people were able to follow along by actually downloading the tools and people listening to this will be able to do the same so there's a coal command here to pull down the latest version don't think we can work out quickly how to make curl pull down what is the latest version when you actually do this but this is as this is being recorded to the latest version if you go to github you you look into the releases latest directory you thought it'll show you what is the current latest version yeah so this is for OS 10 this is how you pull down the binaries for OS 10 there's Linux release and the windows release there sure available if you go to the releases directory Linux we've got arm 32 and 64 and also an x64 build if you went there pull that down and unset the binder is you'll end up with some stuff in your directory and there's an alias here for the sole command basically with unpacked the OS 10 x64 soul binary so if I switch to a command line this is this is basically what that would look like if you've followed those commands the alias has been set up so if I were on the sole Val I get the sole command as you can see here this soul toolkit version tells you which version we're running this one is not be eight seven five this is a slightly older one eight eight thirty so there we are so we talked about how you can run sole oh yeah because this is one way of running some soul code you know the command-line tools that you can download other ways are on the salt dev website wave the same code can slap it into a webpage no installation needed and it'll run using Web Audio and web assembly and the other way at the moment is you can download tracks on waveform and the latest version is objection waveform come with soul patch support so you can also take some code and actually have it natively compiled and running in the in a reelin door which is a really really if you're getting a bit more advanced that's kind of probably how you want to do this because then you can record what's coming out of the code and put it through other meters and do you know complicated routing with it actually get a bit more in depth with with what's going on and also have more control over automation and MIDI in and that's us but I think from main mainly for this we're gonna use the command line which just opens a soul patch runs it using the default audio devices is very simple kind of quick tool to get going might be worth mentioning one of soul patches a soul patch is one of my favorite pun names that we've come up with so far a soul patch is soul itself is quite powerful and quite flexible you can but a soul patch is a kind of a plug-in like way of using soul that will fit into the kind of workflows that we have with existing VST a you a Xcode plugin architectures so soul patch is a little bundle of files on disk which was one of which is a little manifest saying what's in what the patch is you'll have some soul code in there you might have some wav files or other resource files you could have some GUI UI files in there as well and the traction or the there's the command-line tools can load that JIT compile it and and play it back as if it's a plug-in is that enough details so with that you saw would that be pretty much like a max patch sort of I mean yes yeah I mean in the saying that in the same way it's not compiled it's not natively compiled so you can for example if you're running if you have a soul patch open in a track just like you would use a normal plug in in wave form you can have a code editor open in the other screen and you can be editing that code you hit save on the soul file and travel recompile and like the lot you know it'll won't stop playing it'll just recompile the code in that patch will change on the fly so in the same way as a match back max patch to other live coding works you can you can do the same kind of interactive tinkering with the code while it's still running yeah but this is actually very fast it compiled with the same clang Gallivan sort chain that you'd use for c++ okay so the the performance is really good Wow so so what you're saying is there and from people that may not know waveform is a D aw everybody knows almost every but yes by protection oh you speak collecting we called the double use no wait for so so so what you're saying is that you could have waveform open and that you could have a soul patch that's doing some sort of processing and that you could actually have a code editor that on the fly absolutely yes the DSP yeah in the in the plug-in so if you had if you were writing a synth you could have you could record yourself some MIDI loop it have that playing through the synthesizer and be tinkering around with the synthesizer listening to the loop MIDI and and record the resultant pier at the by Foreman and have lots of them going if you want so so theoretically just a use case scenario you could have a let's say a guitarist distortion mmm that was running as software in a kind of traditional plug-in format and then you could actually take the same code and you could put it on to embedded hardware absolutely yeah yeah yeah I mean we the demos we've been doing we've got we've got solve running on the Bella board which is a very low latency embedded kind of Raspberry Pi style board with audio i/o and this thing can do like four eight sample latency so it's basically latency free but it's the same code will run on it so you can run the soul patch on there quite happily so the soul binary that we're running here there is a Abela version of it included in the linux repo so which is very low latency it's 16 sample latency by default yeah so if you're if you're adventurous you could be following along with the tutorial on the Bella board if you want that's amazing so so this could be a game changer not only for C++ plug-in developers but for embedded audio developers as well yeah yeah I mean I hope so that's that's really what we're going with it yeah yeah and for security as well I mean that's the other big win here is that you can't do anything dangerous in the Sol code mm-hmm so if you're what if you're building them embedded box and you want people to be able to run any old plug-in on your on your embedded hardware you don't want to do that in C++ because people can just put arbitrary code on there and it's a security backup yeah whereas we sold you our safe so you can safely run code that can't do any harm which is kind of what you need in order to get to safe embedded systems that can do though legacy you need something like this amazing ok let's see some right so the other aspect of this is if we generally code with vs code with the others as a code editor and there are some extensions which allow you to get syntax highlighting with salt so again if you're following this and you want to use vs code it's worth following the instructions on there to get that co extension installed if you're all using the web we can follow these links and this will show you the various stages that we're going to go through so basically we can use the soul dev playground and run these in the browser so what we want to do is create a synth so using a command-line tool we're going to create the most basic synth that comes out and the software by default and then edit it and enhance it okay so as far this so basically well first we can use it so to create a mono synth and see if we can run it so from here I've run the sole command as you can see there's a little bit of usage help I think we're going to do is we're going to run sole creates we're going to create a mono synths which is a synth and put it in a mono synth directories and envious code you can see there's this new mono synth directory has turned up mm-hmm and we look at the contents and as jules mentioned there's a small extra file that comes along which is this soul patch file which is this manifest telling you a little bit about what this patch is main thing here is it tells you what source code is included mm-hmm there's a soul code can be the actual source can be split across soul files and there's no dependency order problems you can you can put processes and graphs in different files are all resolved into one compilation unit so you can split things as you as convenient for you the is instrument is just a little flag here which helps the command line tool notes it's a synth it's gonna need MIDI and hence stick up the keyboard and things like that mm-hmm so from the command line tool I'm going to play our newly created patch a small little window appears and there's an on-screen keyboard and MIDI works so this is where this is the starting point this is the the most simple synthesizer that we could we could think of writing as just a starting point to show people how to get going soul itself is composed of graphs processes and namespaces namespaces being like C++ a collection of other components or the graphs and processes and namespaces and in this case we've got a top-level graph we can tell this as a top-level because of this annotation saying its main mm-hm it takes some MIDI messages in and the outputs a stream of floats mm-hmm and it uses a built-in library function this message MIDI message parser and it uses an oscillator here as a voice the connection section tells you how the plumbing works so this is connecting inputs to outputs they have to be the same type and in this case the midian from our top level graph is fed to the parser the events coming out of the parser are fit to the voice and the output from the voice is the audio that comes out as a stream of audio so we can look at this this simple processor it has got two events that it handles note on a note on note off events and it produces a stream of audio these little functions here are for handlers for each of the event so when a note on event is sent into the system this this event handler will be fired it's a sample accurate system so yeah it's it's it's very performant and it's it's no it's its sample accurate so the notes on our note off handlers are implemented here and then this run function is is the meat of this processor does so the the processing model is that every processor has a run loop think of it as a thread okay so these your run function is is for each of the processes in the graph is continuously running it is executing transforming inputs to outputs and when it is finished doing its bit of work it moves time full to the next sample and that happens with this advance call mm-hmm so if you look at the run if you look at the run function there's this infinite loop and forever it is running around here mm-hmm it's in this case determining an amplitude based on whether or note is being played or not this is a very very simple envelope and it's implementing a phase based on the phase increments which is determined from the the note that is being played and then it's outputting the stream of sign in say sine functions which is why it might looks like a signer mm-hmm MF if I were to take out the amplitude there and we save it we might hear some nice so yeah it's a it's certain I suppose simplicity and get really precise it's bad in many ways oh don't do this at home yes don't write it like this um as I said the the way that the envelope has been sort of mashed into this processor isn't the right model the fact it's only a sine wave this is quite boring mm-hmm so I think let's take this and go a little bit further with it oh and also the the notes on note off handling is terrible so if I were to play two notes and release it's not going to return that I mean it's got no it hasn't it's not acting like a sophisticated Mollison fit so it's a very naive monitoring so it is up and running so we wants to do is with some make it more interesting so the first thing that comes to mind is the fact that the voice is currently used a sine wave if we could do something better than a sine wave mm-hmm let's start first okay just have a quick question yeah so these events mean to me so I'm just trying to kind of associate this with what I would normally see in C++ this looks like a function to me but they both have the same name so they have different types it's like an overloaded function okay okay so the overloaded function okay great so this this the name comes from the so the input event has these types the unnies called event in yeah and there are these two handlers for venting okay great great okay and then run is basically maned for people that are Oh No yeah sort of main but there's multiple yeah because you might build a graph with a hundred different processes are all connected together with audio flowing through them and each of these little processes has a run function and each one is spinning in their own little loop okay forever and each one goes round and round and each one has the call every time each one calls advance it moves time on for that that node by one sample okay magically that whole thing will run on one thread or two threads or three threads no matter how many there are but each individual processor thinks it's just on its own looping forever running without anything having sever interrupts it nicely so so one question so if run is a looping function mm-hmm then what if you just wanted something so phase so so this is more like a audio audio callback no it's not like an audio call bank okay it's it's generating audio it's more like a it's the opposite of a callback it's kind of pushing with you each time around your loop you're probably going to read some samples from your input so if you have input streams you might not but if you do you'll read some you might say you're just doing again two streams together you might read one read to add them up push them into your output and then advance by sample and then do that again okay but your is active you're not being called so here's a sample you're saying give me your samples push a sample okay I've done that done with that simple now and that's that that's the model I mean we took the compiler sends everything around secretly behind the scenes to make it into a callback ah-ha but you write it as a push ash and all of these different you can build a graph with any number of these things in it they will kind of magically cooperate I say hmm okay in ways that it's so complicated the thing is that that's our problem yeah so we try and do that right and then nobody else has to worry about it yeah right so that's that's that stage we've got a basic synth up and running and then the question is what can we do with it so I thought it'd be good fun to add a wave shaper okay so let's use a very simple function and we're going to choose tan H to modify the sine wave and make make me up a little bit more exciting and enjoyable okay we'll also add a parameter because obviously the world is full of parameters where people want to be able to tweak how how how in this case the altering how severe the the wave shaping is how much of those extra harmonics we see hmm so we're gonna create a wave shaper with the parameter using tan H so we come back to our our sort of working copy of the of the synth so we're going to do is create a new processor called wave shaper mmm it's going to look very similar to the other one in fact it spots a stream of audio coming in so first of all there's all rightit's that it doesn't really do anything you swore whatsoever does it all hopefully prove the point I'm going to create a loop that's just going to so for now that wave shaper is just parsing the audio through we can we can plumb it into our graph so let me come back up here this is our synth and we connect the the output of the voice to audio out directory directly what I'm actually gonna do is I'm going to add a wave shaper slap nearly typed so hopefully I should sound the same sound exactly the same okay now I'm going to just break it I'll outputs nothing every other sample oh it's um bats and dogs will be able to hear but we can see there's lots of mutant I see right but clearly if it's having an effect so good I've got my plumbing right this is on the this is on the audio path so what we're going to do is use the tonnage function here we're going to transform the input list NH and we'll have some multiplier times the audio in waveform visible so as we change this this parameter here we get more and less I'm on resetting this so we can we can see some control here which is giving us moving from the sign more through to something than square now clearly we don't want to read keep recompiling that each time you want to change this drive this drive parameter so what we're going to do is we're going to create some we're going we need to add this parameter as a separate input to the wave shaper mm-hmm it's gonna have a name it's gonna have a minimum value probably good and some initial value really good so yeah that would be nice unfortunately we're not saying how we how this value reaches the the wave shape or what we need to do is put this declare this at the top and then connect it through we have a nuisance expert we do have new syntax so the drive parameter goes to the wave shapers I've control no now all of a sudden on myself I have it a parameter [Music] hmm okay so does that make sense yeah so so just starting to kind of put this together now so the graph is your overall graph containing you can have other graphs in the graph okay a graph from a graph can contain graphs or processes okay processor is kind of the leaf nodes and graphs of the the other nodes okay so so then in this situation we have graphs that contain a processor in each process contains a bit of DSP essentially yeah okay and and so you have to declare your drive parameter in the graph and then you have to declare it again within the process no not anymore you can actually type input you can actually have promote one tweest one up from an internal something's buried in the graph you can just declare it say hey at the top level I want this one to appear and they will come up with all the parameters yes I'm not totally happy with the syntax yet but we're we've got that we've got something we've got we're thinking that way about it I mean because clearly it's going to be the case that if if you are using a component provided by somebody else and they are over time changing adding and removing parameters you want those just to bubble up to the top all you might want to say actually I didn't want this one parameter mm-hmm so there's gonna be work done in that area to remove some boilerplate because in the moment that it's it's not it's not beautiful but it's pretty understand you understand okay well I mean isn't great yeah and when you're creating a stream so you've created this and you've created this with very specific arguments so how do you know what that well is there a VI where we can see this is all documents in we've got some some documents about what all these parameters are but okay everything that's in the double square brackets yeah is an annotation that's not that's not really part of the language the compiler ignores that okay and what where that goes all those the drive and the min and Max they all get sent they've got passed up and you can have anything in there it names been any kind of names any kind of values you want they end up in the host in this case a host is loading this patch okay and the host gets told hey this has got some premises called drive and your legal name and min and Max and in it and then it decides how to show this on the screen so that's not a part of the sole language okay so that's what the that's what the kind of double brackets mean so in in different host doors that these might be different and you might have a host which actually has its own special named parameter in there for something that it does that nothing else does I say oh yeah I mean this is um they are just strings so if I were to change this and type in wrong I have a drive parameter that can only have the value 1 mm-hmm so that there's it would be it would be nice to have better support for warnings if things look a little bit weird but yeah okay so yes no there's my premise I mean it's yeah there's this other there's other stuff here I think there's a step for ammeter yes q as well rescue so now it can only take integer values yeah and there's also an interesting slew rates so if you think of it in terms of you can hear a step mm-hmm it's not smooth and we can I make this very slow it becomes really obvious what's going on here mm-hmm safe but it doesn't so which means I've called it the wrong thing you call it slew rate rather than skew rate and there's a skew excuse for skiing the like the distribution of the when points are uh-huh and slew is kind of where we introduce it like a filter on the parameter so it won't it all kind of seemed a little smooth issues it's and it's not working which is absolutely turning on all the names of all these noises are in it's embarrassing well anyway we'll look at that looking at it yeah we'll find that and so where's this documentation where it's all on github okay I'm get hub icon the readme it at the top it has like lots of stuff you could write the language I think there's a language document which has all of this all the detail in a moment yeah so so in in that just to ask so would that would that apply to input so now I have input and I can use all of these different arguments or parameters is input like the class and no inputs not the class input at the top of your graphic processor ah you declare your graph and then the first thing you declare our our is inputs and outputs yeah you say whether they're a stream in their event or just the value mm-hmm and you give them a name and you give them a type yep so you know what kind of type of sample or what kind of event objects is going to come in and out mmm and then you put these annotations on the end okay and interesting so if I were to remove those yeah how the wrong time is going to see this is that there's an input of of floats called drive and it's gonna say I've got I've got a microphone I can I can do that so if we were to save this weird parameter but it's actually being modulated by the microphone yeah so the drive parameters being is being you know the audio in its it's unaware of X it's interesting so it's um the annotations are being used to modify the intention of the stream so it's now otherwise gonna know like it's it's telling that this is a MIDI stream because it knows about this built-in message yeah by the fact I've called it median it doesn't matter to people than to anything so it's it's the runtime that the runtime that we're using here happens to have some capabilities that we're using soul itself is totally agnostic to all of this it doesn't know or care doesn't yeah it doesn't really know what audio or MIDI are it's just processing streams of data streams of events of series of events objects which may be MIDI like mm-hmm or they may be something else so so the annotations are how a host that's loading this code knows what to do with it and how it should connect it to the outside world okay yeah back to something that works as we were expecting okay so here we are we've got our we've got our synth it's it's got a drive parameter the envelopes still terrible so I think the next thing if we go to our stage is Verde I think we've and we've done that so I think it's the next thing to stare up is the envelope and the fact that the envelope is really not very good at the moment so let's write a very simple attack release style envelope so fairly traditional we use a linear rate for the attack and an exponential release we'll need to remove the existing optically good envelope from the oscillator and then stand plum on your one in so we're going to put the envelope after the wave shapers think about at the moment the oscillate the envelope is in the wrong place so as the envelope decays the wave shapes could have less effect I mean it might be what you want but probably isn't what you want mm-hmm so back to our code we're gonna add a envelope and it's managed to change this it's about this point that joules tells me why don't you have the syntax highlighting Erick's or the error reports yes because I haven't and I guess YouTube's every workshop yes because yeah we do have like a like a visual code plug-in nervously okay plug-in which will highlight the errors in your code but chess doesn't make errors so doesn't matter what happens for me is if I if I put in error is it's actually appears in the ah and this window here as it would do if it's running in a door I think it'll give you the same I generally use that because I'm a little old-fashioned about these things so I'm not getting eventually and we'll get there so what we're gonna do here is our envelope is going to take the same events that we use for the notes so he's gonna take note on note off events and use those to trigger the umbrella behavior yeah so again I'm just going to copy the input events from our envelope spacing my others Jules gets cross they're also going to have two parameters if you think about it we didn't even attack in a release so in the same way that we did for the wave shape we're going to create two G parameters we decide what these mean then we'll do the same for the release will add oh we haven't got any audio coming out but you've already been useful and outputs detail watch we call this audio and our envelope but let's call it audio out so I'm going to just so that this this rod Lapeer isn't ever writing to anything to the audio out so what will come out is zeros mm-hmm so hopefully that's now still compiles jolly good we've got these again the same year we had before where we want these extra parameters visible at the top level so I'll include them here I'm going to declare envelope and danger and what's that you're going to happen here if you think about it is that the the we want some gain component that's going to take the envelope and the audio stream and multiply one by the other an additional processor is very easy to write well the second lease I should mention that we do have library classes that do again and an envelope and like we have not complicated envelope classes we're writing this here just to sort of show you how easy it is so you know we have lots of staged envelopes we have the set of gains that can I fix you know mixed things fixed levels or or levels are regulated by other parameters so I've just made a classic mistake which is to not put them in a loop that would what would happen then is run would output one sample advance drop off the end of run and effectively that thread finishes so it's zeros from then on that's a good gotcha that's that's one I've spent time so okay so what have you got now as before we've got the parameters we've got another should say they should get our two extra attack and release naught to 100 mm-hmm they're the envelopes are plumbed through to are the parameters of a plum tree to our envelope but we if you remember if we come back down to the envelope we've also we need the events hmm so the events are currently only going to the voice but I've corded a voice is actually a oscillator really but let's feed these to the envelope event in the wave shape output is going to go to the the gain components the envelope audio out into the game in and the game does audio out is what we're going to hear now that should still compiled a little hopefully produce absolutely nothing the reason for that is our envelope is currently only producing zeros as output they look quite and see if it's really is working yes there we go okay so that's a very very simple envelope for us to fill in mm-hmm the other thing we all need to do is take out the existing envelope from our sign oscillator so really make it as a sign a sign later without any envelope so this stuff here is generating our amplitude amplitude is a member we don't need that and it will come out of there if you run that actually if I play that what will happen this will get stock note we save it and the reason for this is there's now no envelope effects of video envelope is always output the signal okay so now we want to do is work out what we're going to do in here we've got these two events that we care about a note on a note off and what we what we're really going to be doing is saying we need to be able to increment when we're attacking and also when we're releasing mm-hm so we need event handlers for [Music] so decide what we're gonna do in here mm-hm so this is now hang second time I'm talking nonsence I'm actually very good we need to know that we need to know that there's a looser note so now what we need to do here is we want to say okay we we're currently playing a note so I'm gonna I'm gonna data to say that we're sort of active in some in some way this sort of this fusion here should thus be about turn around yes it should be on yes I'm just testing you their job amount of diverging from how I've previously written this just and we're gonna have some boolean which starts so forever what are we going to do maybe this is before a note is being played while we're not make sense waiting for a note starts our starts and stops back yes and it's a good illustration of the fact you can have loops within loops and so you don't just write a single loop within advanced call you can write lots of loops you can you can have different bits of code there on or off and different times and so now clearly the game that we've got Norton 1 we want something new on Stinnett so whilst we're not that's because that's actually although it comes first it's actually us the note isn't active so we're heading down towards 1 1 0 and then the note is active we would we really wants to ramp up to 2 our talk to one based on our attack rate so let's have a go at writing that first we're going to again output the level the minimum of and thus change it throw in a small ramp to make it come up and I'm going to the max some factor further way the way back down mmm see that it's working it's not these clearly we don't want these numbers in here we would like something based on these these attack and release values now the problem that's happening here is these attack of and these valleys are actually changing every single sample we effectively I could be dragging this all over the place maybe we don't want to do that because there's going to be a little maths in yeah we could use events for this instead of a stream as there's a lot of choices don't know what the right one is it depends on on the situation it's worth of that point noting the different kinds of streams you can have for mm-hmm if you've got if you're writing a sub pattern you have a parameter then the hosts will pick up either a stream like they serve just floats just a continuous stream or they or they can use that as a sparse stream which basically is it looks like a stream of floats in the soul code look feels like you've got a full full race audio stream but actually what's actually coming in there is only occasional bits of data so it's much easier for when you have sparsely changing values or events and if you if you declare it's an event you'll get call backs every time the user moves that it'll be some black you it so you can but with events you can then do your own ramping if you want to sort of smooth out in your own way exactly so I'm going to leave it as streams for the moment we can change it right ramp step I'm gonna calculate this so basically this value here I want something to map this float between values not 100 into something meaningful so we want a linear rate based on say a 10 second to take 10 seconds to reach one if it's at 100 and to be basically instantaneous if it's if it's low so decide a bit of maths here the other thing we've we've got if you think about it is when we call advanced time moves forward by a sample that we need to know what the sample rate is at the moment nothing in here as mentioned in this envelope class as mentioned some parade there is precedent frequency is this machine would be 44.1 believe that's in 44100 it's right okay it's also a process adult period period which is one over so maybe what we want to do is say how many cycles should the attack take given the current value just way of thinking about it what I'm gonna do is I'm going to say it's the first C correctly multiplied by this is how many sausage to take the value of our attack mm-hmm that would be between Norton 100 mm-hmm I'm going to reduce it down to an order 10 then the step would be one over that and that's the value we want to use in taking one over attack samples yes now this could actually be infinity because our attack value can be zero mm-hm so it's a nice little hack there we go at least one sample I bet space either side to keep Jules happy tell me I can't type yes we're quite strict about float and int conversions and narrowing of values mm-hmm so process it up frequency is actually a float64 not a float 32 float and float 52 on Sheamus they're not really and it's really dependent on the platform you're running on so we're using float and on this machine float 32 it's the preferred floating point type okay you can imagine a future where in the same way that int might be 32 or 64 and young platforms float could be 64 or 16 maybe I mean who knows ah it's still builds this is good it seems very slow well it's up to ten seconds let's bury it right amuse you something similar for the release now instead of this calculation to do a linear step I mean I think the number of samples we want might be the same so novice let's go up to ten seconds for our release and it's the same same calculation but release not attack what's ugly confusing otherwise question yes why do you call it processor dot frequency rather than processor dot sample rate it's a sample we could have done that you know I think we thought about it I think one of the difficulties is that different processors can be running at different rates sample rate sort of feels like there is a sampling rates and moeka you can over and under sample processors within the graph so it is a it is a per process specific property okay yeah it's a good question if if it confuses people we could always add as a synonym I mean we went with frequency it sounds a bit more I have been a bit more grown-up and we have frequency where we have period as well so we have frequency in period which is kind of a bit more physics in naming if we had sample rate in period that you could have sample rate and length of one sample in seconds would be what period is mm-hm it's yeah we've spent a lot of argument these are these are the sorts of things that happen down the nicely think Layton used to come people say that was a really good decision there's also a possibility people say anyway right so let's let's come up with a multiply here now and this is the sort of stuff that I remember very very badly how to convert one into the other so I've got a little crib sheet here which i think is getting to tell me that I need to do something like this I believe that power function will give me something useful if I can type that suit is yours it solicits again you need the cast seems like because you've gotten up weak of course oh no issue nope we go on he's an f1 it does indeed [Music] interesting thing you do here I'm just looking at the code that might be quite fun illustration is you could move the release samples values their variables out of that scope but it outside the while loop and that would mean that you would and it would only up update your changes on the pulling the premise around it every time it finishes or slots on those so it wouldn't be doing that calculation every sample they will be doing it every time a note starts or stops yes because there's this system cost here but it's like you would still get so effect well oh you wouldn't hear it changing as you do it yes that's not speeding so good it's quite nice one of the nice things we found about this kind of infinitely style push encoding rather than pull coding no no call back coding is it you can change the scope of something and actually change is moving a bit of code up or down actually changes the behavior in a really interesting way it is some quite powerful ways whereas if you were trying to do the same effect in a call back yeah then that having to like write if statements and have a remember variables to remember where you're up to and you know it like using local variables and scope it's quite you can do some really quite powerful things with them so I think what jaws are saying is we could do things like if we pull that up here for example what would happen is that the release multiplier would be latched when the note is first releasing mm-hmm and then we could if we alter the parameter it's not going it's not going to notice the change with the parameter but this calculation is happening outside that loop so it's more efficient there's different different trade-offs that doesn't feel natural to me I don't think I'd like it to do so if I did want to do that what I can instead do is use this event mechanism mm-hmm so what I could do is rather having these as being streams I could have these as events and as before I could have say the attack on tight mu this logic into here so basically it's saying I can update the release multiplier which is going to become state member mm so when when release when released changes the event handler is triggered it recalculates the release multiplier and then it can use it within this loop mm-hmm this feels I think better to me it's certainly more efficient and it's also could tighten though the two types of variables we have you can have like these ones now are we call them state variables because they're always there any function within the processor I can refer to these things being set to read and set them can they're kind of like a global variable or a member variable in that for that for that processor that instance of the processor those are part of its State mm-hmm and then there are local variables which just like any other language you have local voters so yeah we call them stay because they're not quite members and they're not quite Global's mm-hmm so this now will not compile because I haven't changed the top-level event being the streams at the moment so we will get some it's actually complaining about something else I've moved the logic and I haven't changed that should be called attack that should be called release and now it's telling me I've got a stream tried to connect to an event and that doesn't doesn't make sense the top level here when I change these also to be events that's back in operational yeah so now we've got the that's potentially complex calculation being happen only when the parameter changes so that's more that's more efficient so that is a very very basic Manas it okay ever have another question I have two questions ah so you said run as a loop but then you have but then you have the forever loop this inside run yeah so what's the difference between the run which is a loop no no runners motion okay so run as a function but you're expected to have a loop in your room function you're expected to have if you just write an empty run function yeah compilers gonna say that you've got it you've got the call advanced in there okay actually you could just call it Vanson quit and then that would just admit one zero and then do nothing maybe actually pull up the I say the other thing is if you're if you write a minute if you think about it what Ron is doing is he's transforming streams into other streams or adverting streams okay if you have a processor that doesn't have any streams you can omit the Ron function of that only however their handlers okay so it's it's really that the entry point for the sample by sample behavior of the processor yeah the if you look at the classic ringtone example now the latest version of it it's actually just it's quite a good example of it's a run look that isn't infinite the run just placed some notes and then stops and it's kind of your rights it just declares if you have one note player notes playing and just appear in here it'll be in there some are not being updated but by the time you watch this video this will have that will be a lovely little function it just has a loop of pictures to play in this cause play note play note playing outline eggs and each one has blocks plays a note box plays looks note and that's something that would be really difficult to write in a callback based mm-hmm style yeah I'm just pasting our code into the website yep that's our that's our example of the web the website has some nice little thing allowing you to show the graph that's going on behind the scenes this is a view of the graph the command-line tool allows you to output this as well it's just not as pretty now you can ask me when can I edit this and plumb it all together by hand yes yes one day soon yes looking for this right yeah and it comes back to that idea about what what's the advantage of Sol versus traditional development isn't tooling mm-hmm but we don't have it yet so really we're building the groundwork that allow some really lovely tooling here mm-hm and I think to be as an entry point into DSP if you had a library of beautiful pre-existing wave shapers and envelopes and things and you compose those existing components and plumb your parameters through and build since high performance and run across platforms and embedded stuff I think that's very compelling yeah and the moment you start double-clicking on that envelope and seeing the source code it's then a very easy route into the world of DSP development as in programming rather than five sound design or patch creation yeah yeah this is this is really interesting here and it makes it very easy to see where the parameters are yeah yeah this can get pretty pretty complex yes so I mean that was that was a really really simple and manually written synth example you go to the website we've got examples of like much more complicated since that use more library functions so they've kind of they'll have what we have voice allocator library classes and the enveloping envelopes are in library classes now but we have like examples of how those are put together to make some really quite and complicated since all right yes so there's talking about other things that are interesting here yes the slew rate stuff I don't know why the slew rate didn't work before I've got a feeling after Otakon on the website at maybe that it's not on the commandment or can't remember remember no as I said it's to do with the wrongful environment which of these parameters are interpreted and understood aliasing this is an interesting one that the wave shaper is very aggressive and will produce massive number of harmonics so if we become back here that's why I [Music] think it's saying on the oboe playing on the other on the other one yeah I just use it on the screen keyboard no I know Bogg yes I'm Franco if you recompile the parameters get lost sometimes inserts feature so if we see this with the drive set low we can see that there's not many harmonics here if I like the drive up to maximum so as is very typical of wave shapers that this is a problem for them and I mentioned before that processors don't will have to run at the same sample rate mm-hmm the way we do that's in Seoul is this declaration of a wave shaper I can put a multiplier on with an over sampling factor so this will simply insert the wave shape her into the graph with up and down sampling either side of it I've got to hopefully reduce these aliases it was before the comics that version feeling I had to put it so it's quite easy to experiment with things I hadn't because we can do individual processors you've got a lot of flexibility to tune mm-hmm tune exactly where the over something needs to be done Wow so they are that is our that is our very very simple mana synth now that the the actual behavior for note or note offs is still very naive so as I said if you if you play multiple notes and release them you'd expect maybe the pitch to move back to the still held notes and things like that there's a lot of things you could do which is one of those areas to explore I don't think we've got those components in the library to allow you to do sophisticated allocator might do that I can't remember no yeah and now I have two more questions yep different what's what's the difference between a stream and an event so you have a stream which is this continuous yep sort of floating point numbers so an event meaning that you're waiting there both streams okay these are all streams in the sense that there are continuous flow of things coming in mm-hmm what we're calling a stream is the thing like an audio stream where you have a sample per a number per sample mm-hmm but if you're only having occasional objects coming in which could be flow so they could be big structures with lots of complicated things in like a MIDI event or a you know we have a whole set of notes event classes inside those are like time-stamped things that come in but they're all in sync you imagined imagine audio streams event streams promess of streams look all these things are all just a big bunch of parallel streams coming in it's just they have kind of different qualities some are okay occasional objects mmm particular times and some are just continuous streams of data I know this may be running at different rates and so as an event like a listener then I think if I were thinking about here they keep the key thing is that you could add a billion events and there's no overhead unless you definitely farm yeah okay so it's really turns into that calling into that function you know streams there's obviously the more streams you have there is overhead because there is real data every single sample there's a value from the stream and events are discrete in time but we also have sparse streams so if have strings where you do want it to be continuous but you're only okay expecting to be bursty or or just linearly interpolating between things subprime misses are a good example then we do that in order to keep the data rate low because the idea is that this should scale up to like thousands of parameters but we don't want to have thousands of streams of every sample going the whole time because it won't work so you have sparse streams you have events or wave thinning down the data that can be thinned down mmm but while but where you the code you're writing in soul kind of looks the same mmm you know you can't tell a sparse stream from a real stream even though the numbers you're getting from it have been kind of interpolated and they've not been sent so so I could declare a namespace with a struct in it with whatever data I want I create an event handling that and passing those around so I can have complex data yeah I can generates events internally event handlers can generate events okay I think a good example of that is what we do with MIDI because we kind of try to leave MIDI as the MIDI at the door or as far away as we can from the code you're writing so we have in the in the example here we use the MIDI an MP pass class which gets events coming in and those events are just an integer which is a MIDI raw MIDI numbers but the MP parts are spit out a different server events which are nodes on events and those off events no pitch change events and some other classes of objects and then in your code you can handle those individually as different types of events so you'll get a note on object which has a float for the pitch and a float for the pressure and not the actual MIT wrong MIDI numbers are long gone okay it's been split up into new types so then you could generate other types from that you might want to turn your note on event into you know something as specific to your app as a structure and pass that around okay I haven't have another question so so the the need to cast the need to cast some of your numbers some of your parameters as floats or is that something that you doing away with because no no no it's all the rage all your rust people and although although all the trendy new languages it's all strict about typecasting yeah and like when it's certainly I can C++ for ages we've been we've had all the compiler flags on for about warning as when we do narrowing conversions because it's a it's any when you're when you're new to it and you kind of do you type in like a double and you kind of try to use it as a float even then the compiler goes oh no no yeah it's kind of annoying but when you're a bit more experienced and you've seen a lot of bugs that way mm-hmm you kind of come to realise and actually just just do it properly yeah so we thought I mean yeah we're going to go stripped from from day one and it seems to be working out well for all the other modern languages that do that okay okay so it sounds so it sounds like anybody that's really planning on using this really it's not a big deal I mean it's no different to using C++ with the warning flags up mm-hmm in terms of what's what you can I can't do mm-hmm there's there's so many things we haven't touched on yeah that the the sort of things that I was kind of expecting you to ask is well let's say I've written this generic game components mm-hmm what happens if I don't want to take floats maybe I want float64 or some other type well we have parameterised processes so it would it would basically you declare a type okay and then you and then you use it within within here or there's this there's a lot of other ways of making things more reusable mm-hmm and that's that's kind of the intention here so if you take away some of those type safety questions about costs and things then if you were some use to instantiate a process you've written in their own context you'd like them to get told hang on that that's the wrong type oh yeah I I can convert that to that but I'll lose some precision is that really what you want you want those sorts of warnings so there are warnings out the system as well as compile errors trying to think of a really good example if only whew we only have one warning of them I said only one it's a performance warning oh that's John yes okay yes has to do with the race which you've not got anywhere near no not your nose it's lodged in right I mean there's the other thing that's kind of interesting here is there's no memory allocation in this as it stands in a sense of dynamic allocation mm-hmm and so there's a lot of things that you can do which are which submit restrictions are the language that feel em as a C C++ developer unnatural and feel like they're going to be overly restrictive but once you start thinking these things through you find that it's much better to know upfront how many how many voices do I need how many notes can be played at the same time rather than saying I'm going to keep it all flexible and open and have open-ended behavior the moment that you start thinking that way about it you'll find that it's it's actually quite useful it doesn't mean you have to make decisions about what happens when limits a hit what happens when the ninth note is played if I've got a an 8 voice polyphonic instrument hmm but I think that's that's a powerful and useful thing mm-hmm but you know that what you might do as a C++ programmer if you're in that situation you think oh if they press a ninth note I'm just gonna go I'm just gonna do a heap allocation it'll be fun it'll be fine and it's not fine it is final yeah 99% of time there's not five what what and what we we have the flexibility to do here though is say I'm playing my I'm playing my soul patch synth and I've got my polyphony sets at eight and I find my ninth notes of I'm you know I'm using all 10 fingers and I won that ninth notes mm-hmm I can add that add to that soul patch a parameter which changes which which changes it to 10 or 9 or 10 or some other number but which actually is not you content it dynamically you'll change it and you'll have a second pause while the compiler reg it's and recompiles your code with a new constant in there for the number of voices and then it's going to use a bit more CPU now because you've got a little bit more overhead you've been what the matrix is a little bigger but I can use my I needed my 10 so I'm happy with that but that doesn't have to be a runtime dynamic change from that kind of person for things like that I think olefin II is a really good example yeah it's like you know upfront you know how many you gonna need for a song why have that a doughnut why why should that be dynamic it's all already know your limits so you might as well compile code that's absolutely optimized for the least number you need because that's going to go the fastest and also the fact that it's sidesteps all of those horrid problems all of those function calls you shouldn't make you know methods you shouldn't do do we just are hopefully reducing the surface of problems that you have to cope with and instead you there these other limitations that you then have to decide how you're going to deal with mm-hmm but I think it's a good trade-off yeah those were all limitations that were there anyway but at the moment people learn those limitations by watching your channel about like watching a whole hour and why you shouldn't do this thing uh-huh and how not to accidentally do it whereas we're just not like and not let you do that it can't be done I said another thing I can do here that we've not mentioned is I can add the bug yeah there's an incident I've decided an infinite loop yeah there should be only one runtime error there's possible which is an infinite loop how do I add one easily do that yeah that would work no it's still catching it sorry so yeah the only thing you should be able to do that can't be detected before as bad isn't it's impossible mathematically to detect that during proof that so we're not we're not going to pretend to we're going to fix that one but that's something that the runtime that's actually playing this we'll have to kind of detect hey this audio cool best things you long and then yeah then you'll get but it should be about that have a go well it hasn't spotted I say yeah that one's going wrong but um but then so hopefully it'll probably tank out in a minute but um we said I know it's you're bright that should be the only thing you can do that can eight won't crash it'll just solve everything will block up for a second depending on what host yeah that's gonna be stuck for a while so that so that's you that's you hopefully what you're left with and if you think of all of the all of the problems you could have yeah we do try and stop you hurting yourself but that's gonna be hard what's impossible and other other things to mention are that we're obviously ruining this in the JIT compiler and then the web has a JIT compiler using web assembly but we're also in the short term not everyone is gonna have a door that hasn't sold JIT compiler in it for a while so there will also have a salt or C++ generator so you'll be able to write something like a patch like this and run the command line tool and you'll spit out basically some juice code which is which is a VST au that you think and parlin it is this in C++ and we'll also have like a plug-in that can load these things dynamically then you can using normal hosts and yeah lots of lots of different ways to run it I thought I just mentioned the sort of where to go next with this litany would be nice yeah this is the same code rewritten which I can show you here we written to be polyphonic mm-hmm so what we had before is the top-level - in these basically what this voice is mm-hmm so we've we've renamed it but it's got the same parameters the same inputs same outputs about now that represents one one of the possible many voices and we've basically put a top-level graph on which has an array or voices yep and it's got of voice allocator and what happens is if you look at the plumbing the past MIDI events go to the voice allocator and the voice allocators voice events get sent to the array of voices because it's been instantiated knowing how many voices there are in this case 8mm it knows that it's got it's got an array of eight output event handlers which are plumbed one-to-one with the voices so this ends up basically wrapping the monophonic instruments and producing a polyphonic one yeah so if I were on the command line before now if I were to look at it in the so that should the California condor if I look at the the graph here we see that graph structure with it with the sub graph Wow visible The Voice allocates his names a bit chunky yeah that's a feature yeah I again here before we see this array of eight voices yeah so this is a good example that because I clearly pull if it is is important and as we mentioned the number of voices ends up being effectively hard-coded into this mm-hmm now if I were to say as Jill said maybe this could be a dynamic thing I can imagine this top level graph allowing you if I put space of necks all those jewels across you could imagine the top level graph looking a bit like that mm-hmm and there being some other parameter to the to the runtime saying how many voices of politically I want that doesn't work at the moment but um we could see if going in that sort of direction that's exciting or it might be it might be it's a parameter it might be a annotation it's gonna be down to the house you've got postcode that's loading this soul code it's looking at what who's got it Scott you're telling it hey build this build this graph this is the top level and you're gonna look the host is gonna look at that and say okay what can I do this might say oh look it's got a thing called int voices what does that and decide what to do with that but yeah it leaves a lot of having the JIT compiler and being able to just dynamically change this stuff on the fly means that you can do some really quite clever things in the hosts eventually mm-hmm when they get more sophisticated will it be cause to have it where you can have the graph dynamically change yeah yes that's easy it's the other way of making the graph change this is you but but but because we have the code because the code actually is the architecture of the code is graph based that we can have tools that move graphs around yeah and which modify the code yeah which you can't do with any other language there isn't that didn't have like these graph objects as part of as a first class citizen in the in the language itself mm-hmm so there's there's there's lots of there's lots of things we need to do we're very much heading towards something which we believe is complete enough to be useful mm-hmm so again feedback on what what we're seeing all yet what we want to know is it's great but how would we do this or this is very hard and we've got a long long list of those things mm-hmm it's all time to mean we haven't got frequency domain in here yet we've got plans for that mist that we don't feel it's a block for making it useful mm-hmm as it as it stands to over you know a proportion of people that would want to use this yeah so so that's the question really is helping us prioritize what other what are the missing features and also if anybody thinks this is a terrible idea and there's absolutely no way they could ever use it because of something rather than yeah we and I'm short kV all over the place yeah let's keep all the negativity on KBR it's ya know do feedback is great me yes I need feedback I need I need to go down the back I want to know everything yeah great feedback would be hey I'm a professional plug-in developer and I'd love to use this but I can't because it cuz I'll do this and that's because that's stuff that we maybe can easily fix so what about what about you I know you are not even talked about you are there okay I mean this is gonna be this is the other thing I was gonna just quickly do is fire up and wait for oh yeah so yeah waveform we've uses a little testbed for one of our first bits of UI I think is that we've got on a reverb haven't we well that what I've done I think here if I go to plugins no if I go to this okay if I put my workshop directory on here so so basically I've said soul patches are found here if I scan there and click on format' nothing sure they were in there somewhere are they in the middle oh there they are so these are the before example some them on a synth the one that I was just editing yeah I've turned up here so now if I go into a song and drag plug-in I can take that poly synth version of it's [Music] varrick's working in the Wow well again and I think no okay the other thing I mentioned this is actually live coded so if I was to say change the parameter name its recompiled on-the-fly so you can be developing your plug-in whilst running it in the door whilst playing with the keyboard it's it's quite a flexible system it gives you a lot of things I think previously have been quite a slow development cycle of builds drop in try convinced the host that the plug-in has been updated or whenever it is so there's there's some good stuff there and there's also talking about gooey stuff I would have just removed the yeah well what what what waveform has which has been quite fun to experiment with because it fits quite nicely this is we've always had we've had this always it always pissed that we have this thing called face plates and these were originally put in for a rack for plug-in racks but it means you can click the lock then I can add I can build my own GUI in in wave form and this gets saved as some XML and you I mean unison we really quite complicated stuff in here we've forgotten one thing one of the examples in the sole repository has a faceplate and so you have a soul patch in there you've got your manifest your soul patch file and some soul code and attraction faceplate an XML file and when interaction sees that patch it loads it loads the faceplate with all the GUI controls and you know you have some images on there I think you put SVG in PNG s and stuff in there and colors and gradients and the text mm-hm and that's a really simple declarative way of adding a UI to a soul patch we also I think probably the main way as this goes on people will build guru soul patches will be as JavaScript mm-hm and we'll have like an embedded chrome window in the posts so you be using web technology be using like react or just HTML Javascript CSS in a window which is then communicating with the soul code via the parameters and then event streams in and out and I think really nicely because that's also completely dynamically loadable yes so you could be messing with your GUI in real time you can be messing with you DSP code in real time and nothing ever gets compiled mhm so I can I can ship that patch over to you and you can load it and and you can also even them that patch in a project yeah so at the moment attraction is loading this from where we've got that on disk mm-hm but the catalyst all code could easily be just squeezed into that project so I could give you the project and it's got my synth in it and you don't have to have that plug-in you just get the cross-platform right on platforms that didn't even exist when the when the patch was originally written Wow with even the Giri in there and I think what we can end up with is you can have say sound designers releasing 100 sounds mmm they might have a hundred patches which reuse a lot of the same GUI but maybe that's slightly different in different patches for different uses and they could reuse a lot of the same synthesis bits and bobs behind the scenes but each one kind of looks the same and each one gets compiled individually when you need it mm-hmm but you can distribute any one of them individually in it and it's just a more it's a more flexible way of building sounds that just do what their sound needs rather than being a sound for a fixed plugin yeah they definitely blurs that idea between what's a plugin what's a patch mm-hmm is is very fluid with this system with with a jet system and so I think we could see some really interesting ways of using that technology and it'll work on the web as well so ideally the same patch that you load in the chrome and Betty thing in the door you'd run that in the web the web site yeah live coding as well yeah yeah it has a lot of potential for live coding as well and I think for you as there's a lot of interest I think it'd be a lot of interest in live coding DSP workshop type online material yeah where you can talk something through a concept and there can be an editing window and they can be altering some soul code and it's and it's playing in in the browser yeah that could be very interesting Wow that's very interesting so so how how far do you think we are away from having being able to take a soul patch and put it with like a real act UI we think in years months oh yeah I think with if we don't get guess that project started this year then would be a bit far behind because we really need to see that something else like we can do it in parallel to all the DSP stuff we do yeah we top of our priority list no we have this this thing which we're trying to make sure we haven't there aren't gotchas in what we've done mmm that is a good idea because there's so many parallel streams that can be it can be taken forward things like that mmm until we get some of the processing model right so the way that's the code would interact with the DSP mm-hm you've got a lot of ideas there and we're still tweaking and fiddling when we get that absolutely nailed down then we'll be able to provide a stable API for that interface we need so yeah so it's waiting on us to convince ourselves that we've done some other things right but that's good that's great Wow well yeah that's amazing a lot of lot of extendibility I think for audio developers yeah great no more issues we've done yet I know I'm stuck yeah I've got I've got loads more questions but Allah it's probably best if I just actually go into Solon oh yeah yeah and yeah people have questions then we they know where to find us yeah absolutely more than happy to chat yeah leave a leave a comment in the in the comments below or also on the discord we have a dedicated soul room where Cesare has been in there every now and then and people have been experimenting so yeah be sure to to ask questions and complain and yeah I'm looking forward to seeing what people create with this yeah see right great well uh thanks again for here thanks so much for having us on hopefully view actually recorded this yeah yeah see it's in practice

Channel: The Audio Programmer

Views: 4,188

Keywords: audio programming, creative coding, audio coding, creative programming, digital signal processing, dsp, plugins, vst, software development, ableton, max msp, c++, sample rate, bit depth, nyquist theorem, juce framework, tutorial, beginner, easy, games development, games programming, basics, openFrameworks, open Frameworks, ofx, Maxim, Maximilian, synthesizer, synth, soul, sound language, audio developement, audio dev

Id: ONN8v4UhNIg

Length: 90min 20sec (5420 seconds)

Published: Tue Jan 14 2020