Audio Programmer Virtual Meetup - 8 Dec at 1830 UTC

and we're live all right hey um yeah hello everyone uh welcome to the monthly audio programmer um virtual meetup my name is timur um i'll be your host tonight together with josh hodge hello josh hey how's it going and we have two wonderful guests uh today as well um and um yeah um so um really excited to be here thank you so much for tuning in um before we get to our guests we have a couple announcements a couple kind of things to share one of them is obviously last month the audio developer conference took place um so really really happy with how that went um thank you all so much for everyone who has participated either as a speaker or as an attendee or as a volunteer we had a great time it was awesome thank you so much um and i think josh wants to talk about abc a little bit more as well yeah absolutely and uh once again like teamer said we'd like to thank everybody who uh participated in the adc so we had a lot of people who submitted talks we had volunteers from the community who also gave their time selflessly and thank you also to all the people who actually attended the conference as well and it was a really successful conference we got a lot of positive feedback uh of course we always want these events to be welcoming and inclusive as possible as well and we uh i think that we made a lot of positive movement forward on that and um so thank you to everybody who who uh who came out the adc the videos are uh most of them i think i think nearly all of them are now on uh the juice youtube page so you can actually go view them for free it's on the if you go to youtube and you search juice it's on their youtube page and they have uh i think nearly all of the videos there now so you can view them and a lot of really amazing information speaking of juice we wanted to thank them for being a sponsor of the audio programmer meetup along with sonix and focusrite so thank you to our sponsors for for making this meet up possible and for helping us to do this every month also uh wanted to let you know to that we encourage people to continue these discussions in our audio programmer community you can join the audio programmer community on discord by going to the audioprogrammer.com forward slash community and we also have a number of roles that are out there from some of our clients so some of our clients approach us and ask us to help them recruit for roles and we have a number of c plus roles uh and dsp roles as well and you can actually find that on the audioprogrammer.com as well so uh if you're looking for a new opportunity come talk to us so uh i think we're ready to get our first talk started so what do we have coming up teamer uh we have something really interesting coming up let me just uh say one more thing before we do that um so this is actually the last uh meetup of the year i really hope that we're gonna have more awesome content next year and so for that to happen i really encourage all of you to actually come here be part of the show uh give a talk or do a q a with us uh participate no matter whether you're a student can you start it or whether you're a seasoned uh industry expert i really encourage you all to go to the audioprogrammer.com submit and submit your idea for a talk or a presentation here at this meetup and to come and join us here in the new year so really really excited for new content there yeah and with that being said uh let me introduce um our first speaker of tonight or morning or day depending on what it is in your time zone um who is um we have with us um chris moan um hello chris hello chris uh works at nvidia he's a programmer uh from nvidia but actually uh tonight is not representing a video but as instead he's here to talk about a personal project that he's been working on uh for a while and so i'm really excited to hear more about that um chris over to you oh and sorry one more thing if you have questions for chris or any other speaker here actually please feel free to ask your questions on the uh youtube like chat thingy um and please do us a favor and if you ask a question please prefix it with question and capital so it's easier for us to spot that in the stream of comments and so that we can pick out your question and ask it um at the end of the talk all right with that thanks and over to chris leaving let me share my screen hi my name is chris morn i work at nvidia doing graphics but today i'm here to talk about a personal project that i've been working on for a few years now it's a live coding ide so who am i i started my career at 3d labs i read the opengl manual on the train on the way to the interview and luckily got the job and ended up spending five years writing directx device drivers and working on gpu architecture after that i ended up at nvidia for about 23 years now in counting i do tools for developers all sorts of things plugins particularly over the last few years i've worked on the insight tools gputrace i think i've got a screenshot of that here you can see uh this is gputrace on the top left which is a tool that's looking at the gpu and figuring out what's going on inside a frame there's also a screenshot of n site here which is showing i think some of the rare tracing stuff that's gone in recently which i didn't work on but um the uh the the trace tool at the top here the scrubber here and the um and the traces of the graphics chip in the gpu trace application are things that i've tended to specialize in it nvidia uh the tools at the bottom effects composer is a shader editing tool which i worked on for a few years uh early on and then an even earlier example there of shading i do like to do other things over the last few years i've done many experiments i've done a couple of simple games played around with various visual things i like to do visual programming i like things that draw pretty pictures on the screen my website there if you want to look these are just images from that so what is live coding it has very many definitions one of the examples here is a an algorithm where you've got people sat in front of laptops composing music and graphics presumably there's an audience there as well dancing to the music it's it can be done in many different ways it's quite typical as you can see here to have one programmer doing the graphics and one programmer doing the sound i've put a couple of links there there's a really nice article in the ft about what algorithm is and the awesome creative coding website has hundreds of links for all sorts of interesting tools so i'm just going to highlight a few that are kind of important to me that have influenced me i'm going to talk about sonic pi which is a a music composition tool uh written with a ruby dsl and i'm going to talk about title cycles which is based on haskell i'm not going to show title cycles actually because i'm having trouble running it on my machine which isn't uncommon uh it's it's quite picky to install sometimes i do have it running on my macbook but not on this machine i'm going to show shader toy i'm going to show super collider which is often the foundation of some of these tools is because it's the back end synth that is generating the sound and i'm going to show chibo which is a javascript uh audio visual web application so i'll just show you a little bit of each of these so this tool is super collider uh probably familiar to many uh it uses a built-in language called sc lang which you can use to generate quite complicated uh synthesizers i've got a simple example here i can hit ctrl enter we'll hear that as two sine waves being played at different frequencies four five six and four four two i've got a more interesting example here of the sdk try this one you can see that that's uh well you can hear that that one's a much more interesting sound and you can look at the chord and it's quite obvious what it's doing uh it's generating some envelopes um it's doing some low pass filtering white noise here to generate this snare drum so a powerful system this is the windows build it's it's a little bit um difficult to use the windows are very tiny on my display it didn't work for about a year on windows it seems to be quite linux centric at least from the ui's point of view um but it works the editor is is is sufficient it has uh pop-up tooltips and things like that so it does help you write the code but it's a very as you can see very minimal user interface this application is sonic pi a very popular live coding tool for music it was created by a guy called samaran and i really recommend you go over to his channel and listen to some of his music created with it he has a very professional setup which enables him to do this great av performance that he does all over the world there's an example here a simple uh a more simple piece of code if i run it you can see it's uh it plays the music and it's it's generating the the music at the back end using super collider actually which uh is being used as a server to to supply the synthesizer facilities so that's sonic pi this is a live coding environment for pixel shaders for graphics let's have a look at this one here you get the shader chord here and the visual effect there i can actually spin it with a mouse very pretty have a look at it full screen so this shader works by generating rays which are fired into the scene and stepping along the rays to try and figure out what the rays hit and what the color of each pixel should be the shaders here so you can read through it and tweak the visual effects see what happens by changing things let's make that a nine see what that does it moves the sphere further away because we changed the camera the last one i want to show is jibber this one is a javascript live coding environment that runs inside a web browser [Music] as you can see this program is generating some music there's a drum pattern here a synthesizer and then it's doing an fft of the audio in order to convert it into a frequency bins and then drawing the color bars so it's a tool that you can use to make music and visuals it's very nicely done very easy to use and very accessible because it's right inside the browser it's a nice tool so if you look at all those tools what's interesting about them they often have very basic editing they're not built as powerful ideas usually they're usually the editor is just the thing that they need to get to the next thing and it's not something that's been not all the time being really thought about so there are some exceptions the sonic pi editor is pretty good um pops up tooltips helps you with the chord as you write it but but in general a lot of these tools you see have very basic editing facilities um there isn't usually any connection at all between the code you write and the music you hear and what i mean by that is it's really difficult to see you know why is that why is that visual happening now and how does it relate to the sound i'm hearing and how does the sound i'm hearing relate to the chord i wrote which note is being played from the chord where i declared it if you if you see what i mean so that's an interesting uh aspect of live coding and something that i want to do more research into if you go to one of these algorithms there's these what i've called here transient moments of greatness where you might hear 10 minutes of confusion while the program is trying to figure out why their program isn't sounding nice or they're experimenting with various um pieces and and they kind of haven't quite hit the groove yet there's not the music isn't there um that doesn't always happen but it it can happen um but there are often these moments of greatness where everything comes together and there'll be a minute or two of great music and great visuals and and i always think it's a shame that that goes away because the next stroke of the keyboard the shader has changed the music has changed and you've lost that you know that serendipity of discovery that often occurs so i've been thinking a lot about that i would love to make my tool uh capture snapshots of where it's at and allow people to play back a performance and i think that that's somewhere i really want to get to eventually as i said some of the tools are complex to install i couldn't get super colliders to run on this machine for about a year i couldn't i can't get tidal cycles to run at all sonic pi did install although i had to remove a virtual drive which took me an hour or two to figure out um and i actually helped them improve that software at some point in the last year or two to make it easier to install um and they don't usually have our reporting debugging uh there isn't really an easy way to see you know why is why is the machine suddenly dying what's wrong where is the you know is the machine have i created too much music have i created too much that is a visual too expensive um what's wrong with my shader is it really reporting the right line in the shader where i wrote the code incorrectly that's something that happens quite a lot and it can be very frustrating and you know as i said what's my performance i might want to be sure that when i give this performance on my laptop in a club it's going to run it's going to be fine and it's not going to glitch so this is the tool i've come up with this is where about so far uh it's called resinality um it's an ide for for audio and visual um it's kind of a toolbox really there is a layout mechanism so you can be in this mode you could be using it just to research shader development but there's also a mode where you can swap the layout and it becomes more like a live coding environment where all the ui gets out of the way and you just see the shader code overlaid over the visuals that's not something i'm going to demo today because it's currently a little broken but uh i'll get that fixed eventually um but yeah so it's it's it's what you make it you can configure it for how you how you want to use it so i'm going to give a brief demo of where resinalty is at at the moment obviously i'll show more specific parts of it later but just to give you a general feel for how the tool fits together let's start in the editor here we'll make a synthesizer this is the output graph which is the synthesizer that sits at the back end and does the mastering and and a bit of audio analysis it's needed in all graphs let's create something really simple this is an oscillator see it was created here i can play with the ui [Music] let's go and make something more interesting down here i've got an analog synth set setup i've got another synthesizer you see they're all playing nicely the 3d renderer here is sampling the audio information we have anything on the timeline line yet but we haven't scheduled any notes we'll do that in a minute uh okay so let's make some music using the xe language just something simple we can make a drum paddle button here let's just stick with that we can fire that off oh i need to initialize all the synthesizers let's do that uh okay so we've got that um let's go back into the music and try again we'll make the drum sound [Music] you can see in the timeline now we've got some patterns appearing and you can see the visuals reacting to the music the pixel shader and the background let's uh try some claps and we can go in here and start adding more synthesizer sounds let's just make it [Music] somewhere [Music] so [Music] we'll just briefly turn off the music so you can hear me speak so as you can see the timeline here the those patterns are being generated and and scheduled uh we have the visuals jumping around to the music um the uh synthesizers uh the orange lights here indicate which synthesizers are active and running at the current time there's a debug here which is also telling me kind of how many notes are in each synth i'm going to show you next how to change the output of the visuals [Music] [Applause] [Music] [Music] so [Music] so there i went in and changed the vertex shader which is describing the shape there and i've changed the background to a different shader and the final thing i just wanted to show briefly uh is the profiler which has been running away in the background here uh this thing can uh let's actually run it again with some audio so you get a more interesting pattern we'll start it again [Music] and you can see here um these are i actually have a profiler that can look at the audio thread or the graphics thread so the line along the top is what's going on in the graphics the line along the bottom is what's going on in the uh audio and then i can analyze what's happening in the threads in the engine what's happening in the ui and the graphics here uh and that's it for an introductory demo i'll i'll cover some more parts of it later on [Music] uh yeah so the basic technology in the tool it's c plus plus 17 it would be 20 if i could get away with it um i like to use the latest tools why shouldn't i on a project that's a pet project of mine it means i can learn new things i'm using the janet scripting language which looks a little bit like lisp but has some mutable pieces to it i think it's a very nice language to embed very easy to use i've been really pleasantly surprised by it i used to use a built-in scheme chippy scheme i think it was but i've found janet much more user-friendly in many different ways uh it uses i am gui most game developers use iron gui for their overlays for their tool kits when they're building games it's a hugely popular toolkit i've only just noticed that that appears to be mirrored on on the slides which is interesting and yeah the graphics library at the moment i'm using opengl just because opengl runs everywhere it's not really recommended on mac anymore but it does just run i also have built a directx 12 version of it that kind of works but needs some tlc and i even started a vulcan version because i am interested in the graphics aspect as well as the sound and i'm really looking forward to to building those out so um the c plus plus bits uh because of the threading i thought i'd just mention how i built these things i i'm just using stood thread no libraries no big layers of anything i i most of it is stood threads atomics condition variables there's a single header thread pool class that i've put a link to there which is fantastic you just drop it in and you can schedule work by passing in a lambda and it just does the job and one of the foundations of my tool is this moody camel concurrent cue which allows you to stick things into a queue and then pull them out to the other end without having to do any locking on mutexes and it's incredibly useful and incredibly easy to use a node project like this gets built without a lot of help so this is my package list from vc package which i've said here is the package manager of champions i was using various other approaches but when i discovered vc package i didn't look back it's unfortunately named it isn't really visual c only it's a cross-platform package manager which you kind of build yourself and and just has so many libraries that just automatically install and work first time and that's the full list i think if the ones that i'm currently using uh the text editor so as i mentioned earlier on uh i wanted something a bit out of the ordinary i wanted something where i could really go in and change how the text is displayed depending on what's going on it also had to run in an opengl volcano dx window so it had to be able to draw itself in a graphics window which is not as easy as you might think there are open source versions of basic editors that do that but typically you're looking at the client server architecture if you wanted to use something like neovim which is complicated also i wanted vim support uh if it's a tool that i'm using to do a live code performance i don't want to be stuck using like notepad style editing i just find that very frustrating so that was something that i definitely needed but of course with that comes the problem that i want people to be able to use it to so i can't force them to use vim so the editor had to be able to switch between a no pad mode and a vim mode as i said i wanted to have text adornments places over the text where it could show interesting things again to show this link between what you've coded and what you're hearing and i wanted it to be able to show great error reports and warnings there's a couple of screenshots of it here i'm going to demo it in a moment you can see the screenshot on the left is the minimum mode that i mentioned earlier actually the uh it's drawing uh the visuals in the background and the live code is just typing chord and notice that he's misspelled uniform there at the top and the editor has highlighted the line popped up a tooltip and told him exactly what's wrong and that's something that i'm really picky about i want to make sure that it works well and it's easy to use screenshot on the left is showing some ixilang music and the orange highlights are showing which notes are currently being played and it's showing a split window and some syntax highlighting so i'm going to go ahead and give a little demo of zap now so here is zap the integral editor you can see it looks very much like a bim tool got the status information at the bottom here it adds syntax highlighting you can see one of the interesting features here it can highlight the text based on what's playing so this is actually feeding back from what the music is generating and telling me which notes have actively been playing and i'll demo that later on i can search for things i can search for mandolin for example hop between buffers search for connect [Music] i can do all the usual vim stuff let's delete three words i can also switch to notepad mode and then it just behaves like notepad it's not model anymore which is obviously required for users who don't like vim or don't use it and there are adornments built in all sorts of different ways to mark text for example add an underline here let's put another one in and the tooltip here of course also a demo in this case but enabling me to underline and highlight parts of the shader that are wrong so that the end user knows what's going on and that's up the uh integrated editor so let's talk a little bit about audio which is why we're here um most of you will know that the typical starting point is this callback function that you get from your sound card driver or api you're asked to fill in a buffer and uh given a number of frames and you return that to the sound card driver and it sends it to the hardware and so hopefully you hear something at some point in the near future um and it on the face of it it looks pretty simple right you can uh start making sound waves and hear nice nice tones coming out of the speaker the first thing i found which was interesting was i was making combining a few sine waves and suddenly there was too much latency and the sound was broken and i was like well this isn't i'm not going to get very far here um it turned out of course that calling sign on at least on windows in debug is really quite expensive and by the time you've called that a few thousand times you're starting to use up your cpu so uh yeah that was an interesting revelation early on uh you can't use mutex uh in a in the sound thread this is something i learned recently from teamer's talk last uh a couple of months ago i was using mutex i was kind of getting away with it but it certainly helped when i removed all of my locks and put in a spin lock for the very the small case where i needed it you can't dynamically allocate memory i am doing a little bit of that at the moment but not intentionally i need to finish sorting that out um and you can uh you have the problem of synchronizing what you're doing on the audio thread with what you're doing in the rest of the system if you want to show the notes that are being played um or or anything else that that you figure out on the audio thread you've got that synchronization problem where you can't use locking so that's that's a hard problem too the other interesting thing i immediately found was that using a stud vector was very slow and that's because there's a lot of debug wrappers around that in visual studio on on windows the easy work around is to sacrifice the range checking and just make a pointer from the beginning of the container and then you can walk around in memory regardless and it won't do all the checks that it normally does so just to think about this a little bit more you have the along the top here this is typically what's going on the sound card is reading the previous frame you sent and playing the music while you're filling up the next frame there might be multiple of these there's probably more than one deep stack of frames but that's essentially what's happening you don't really have much control over it last month's audio programmers talk did cover the new linux audio api which has some interesting features but generally you've got a buffer you have to fill it in reasonable time uh and that's more or less it because it's sound there's not you know you're making a bunch of sound samples to be played that's all you're doing um but if we look at the bottom here the interesting thing about this is that on a graphics chip this problem is massively multiplied you can have multiple buffers they can contain different things you can have multiple queues you can assemble them on different threads the gpu you can pass the gpu barriers and semaphores to tell it when things can be accessed and when they can't and to schedule things um and there are multiple hardware cues as well a gpu can be copying memory from one part of its one region to another while receiving geometry data so on the one hand graphics is much more complicated on the other hand there are more options more more solutions to problems um and graphics chips are you know obviously much more complicated than than sound chips that's obvious that's why for example when a new uh game console comes out it usually takes a few years before the game developers really start to push it to the extreme and get the best out of it um but yeah i thought that was interesting that um there's a bit of a parallel layer obviously i'm used to this problem of feeding gpus so when i started looking at the audio side i kind of had a feel for what i needed to do and how to make it work i've put a little link there for uh nvidia's insight talk on gpu trace which covers all sorts of things about how to optimize using the gpu if you're interested in looking that up so this is uh the next problem that i came to which was uh generating just simple sign waves as i said i couldn't just call sign you just don't do that because it's too slow so then i came across this thing called wave tables this is in from the ear level website which is a fantastic resource for this thing and you can see he's got a little image there of all the wave tables he's generated and when they are switched between but essentially the problem is that you can't just sample one lookup table for your sine waves because then if you are sampling it at different rates you will get aliasing the solution is to make multiple wave tables and pick the parts of the tables depending on how on the frequency that you want to generate now in graphics this is very similar to something called mipmaps we generate multiple resolution versions of the image and then in runtime the graphics chip is sampling them depending on the coverage on the screen of the texture map to pixel ratio so if if your if the texture is a tiny little is on a rock at the back of the scene and it only occupies three pixels on the screen then you're going to pick one of those tiny little images but if the if the rocks right in front of the player you're probably going to use the high resolution origin and they are filtered and switched between using mipmaps in on the graphics hardware and the reason for that is to stop aliasing it's the same problem essentially except it's visual aliasing versus audio aliasing so that was an interesting comparison i came across and there are some links there if you're interested in following up on that so once you've generated sound waves and you've got that running nice and fast uh it gets a bit boring they're not very interesting to listen to um and so the next thing was where do i go next how do i learn something uh new about audio to make some more the kind of sounds that i wanted to make and i discovered after a lot of searching these great resources uh there was somebody from on the last meeting talked about audio kit and that uses internally a dsp library in c called sound pipe and these two things combined really gave me all the knowledge i needed in order to generate pretty sounds audio kit has this great sample code where it calls sound pipe and it builds a pipeline the audio kit synth one application is you can get all the source you can just read through it and you can see the example i've posted here um they're doing some oscillations some panning some phaser and crossfade and it just shows you you know what parameters to use how how these things work and there's all sorts of little tricks and tips in there that i would never have known without a reference um just different and interesting ways to do things so yes once you've got to that point um well at least once i got to that point and i was generating nice sounds i wanted something that was configurable i want the live coder to be able to say today i'm going to have this you know weird and wonderful synthesizer with uh four oscillators feeding into this low pass filter and doing this and that and the other um and i wanted them to be able to configure that perhaps before the performance or maybe even during if they're brave so i wanted to build a graph essentially um if you're familiar with this kind of um of approach this is a bit wig for example um where they have uh a tool inside a bit wick i'm not sure i can remember the name of it but it allows you to drop these blocks in and connect things together so this is a triangle wave and it's fed into a some kind of envelope and then fed to the output and you can have a lot of fun playing with this i actually love bit wick i think it's such a beautifully designed app and the ui aesthetic pleases me greatly um and i actually copied the way the buttons my my control knobs work from big wig and mine are similar enough that i'm almost convinced that they're using the same graphics library as me but anyway so that's a that's one use of graphs in the audio industry this image on the right is actually an output from my graph i have a script command which can just drop a created graph onto a website and display um the connection information of how the graph is built which is great for debugging so how does a dag typically work this directed ace click graph at least in my system the way this works is i start at the output and i walk back into the graph evaluating all the inputs so if i'm at the mixer here i might think okay can i run the computation of this mixer and then i look at my inputs and realize hang on they're not updated yet they haven't been updated for the current frame so then i walk back into the parent node the oscillator and i have the oscillator doesn't have any inputs so it generates its data and validates this blue arrow worth of data the adsr does the same thing and then eventually the mixer can then run so in a directed acyclic graph you want to run every node once and that's the basic approach to to generating a graph it's it's powerful because you can the user can lay out a graph of nodes how they want and you just figure out what the output node is and walk back into the graph and as you evaluate it once you've finished you have your sound samples that's the theory anyway graphs are interesting because they're used in graphics all the time this is quite an old image i think from a few years back so you can imagine where they're at today but this is uh the rendering graph from the battlefield 4 game and you can see all of the the notes here that are building up this scene and i don't know what they're doing perhaps they are generating lighting effects working up depth information they come together eventually to form the final frame of the game which is probably this right hand node right at the far end and they had built a debugger to uh there's a talk about this at gdc i think it was that you can go watch if you want but it sure talks about the complexity of building this big complex game using a graph um so yeah graphs are something i've worked on before i'm very familiar with how they work and surprisingly similar concepts another tool in uh graphics is a is called maya which is a cad application and internally that uses a scripted graph which i have used a bit in the past to write plugins so this is uh a bit of a closer look at my graph you can see uh here i'm showing the same thing a slope and an oscillator being fed into a mixer i have some mechanism to get the data into the graph uh from the timeline or from a lock free queue and it sits in this pool of notes in the graph and there's a this is basically keeping track of what notes are currently active and playing there are inputs to some of these units uh using portamento so that if the user twiddles a knob um it doesn't immediately switch it kind of it catches up slowly in order to make the user interface to the synthesizer clean and not sound jerky when you when you're adjusting things um so yeah i've showed some inputs here and then once the uh once this bit of graph is evaluated it can output two different types of information the adsr is outputting control information which is uh control slopes for um for controlling the volume of the uh the sound that's what this one's doing and i have flow information which is actually a polyphonic bundle of audio so what happens is that at each tick the graph has to sample all the notes at once and make sure that it generates channels for every active playing note so each compute step in each node in my graph has to do all the work for all the notes at the same time and essentially this mixer here is taking two types of input it's got the control and the flow input and the control is going to use the slope to modify the audio channel and out of the mixer at the back end you'll still have those polyphonic channels but they'll all have been mixed using the control slope this is an example of well pseudocode of how it works inside that oscillator it looks at each note it creates a channel for each note which is fast because it uses a memory pool to to cache channels inside the graph and then it walks the frame count of the audio samples the wave table for example and stores the output data and it's doing as you can see for each node for each sample in the frame so it's doing all of that work at once and outputting those channels sometimes the there are two channels and they represent stereo but often there might be many all bundled together so how do i handle modifying the graph this is a problem that i became aware was fundamental in audio is if you have your graph running how do you change it i looked at super collider supercollider does it i believe by sending messages from the ui thread into the audio thread which are effectively instructions for how to modify the graph i didn't choose to do it that way i modify the graph outside of the audio thread essentially making it dormant for the time i need in order to edit it and i'll come up with quite a clean approach to this i think what happens is the graphs are all running in the audio thread here on the right the first thing the audio thread does is spin locks it does a try lock which doesn't actually take the lock it just spins and waits weights it doesn't do a mutex lock i should say but it will spin until it can get a lock on this mutex here and then it'll process all the active graphs and then it will remove it'll free the lock but this is engineered so that this trilok is very minimal it almost never happens in fact the reason it never happens is because on the ui thread on the other thread it's doing this thing where it does a lock marks a graph inactive and then freeze the lock immediately so what the threads outside of the audio thread are doing effectively are saying okay i need to change this graph the users just rerun the script what i'll do is quickly mark the graph as being inactive using this lock and i'll only take the lock for a moment um and then once it's done that the audio thread side will no longer play that graph if it comes in for another audio cycle this means that really the audio side is never waiting uh but there may be occasions where the audio side does not have a graph to play when asked for frames now that will only happen if the user is rebuilding the graph essentially tearing apart and rebuilding the synthesizer so that's not a not a problem it just that's how it works uh you could there might be interesting ways in which i could make this delay until the synthesizer is quiet and all that sort of thing but i haven't thought about that too greatly but this does the job really because typically people are building synthesizers before they start playing the music but yeah this is an interesting way to to ensure that the graph is never modified while it's running inside the audio thread and the audio thread is never really waiting to be able to use the graphs yeah so an active graph is on the audio thread if it's inactive it's on other threads being manipulated if a graph is awake it requires compute but if it's asleep it can save all of that work there's no point running a graph just to get a bunch of zeros out of on the audio channel so when is a graph asleep this is something that i've wrestled with a bit i think i've got a reasonably good solution now you could say that the graph is complete when the slope the adsr slope is finished that would be one approach uh you could say that it's finished when there's no more sound coming out of it there's a danger there that some effect unit is about to cycle round and play something again though um you could say when an effect is done for example if you've got reverb and you know the reverb is adding another half second or something then you could say okay i just need to know i need to finish in a half second because no more notes are coming in or yeah when the user says there's no reason why script code couldn't tell these things to go on and off at some point in practice i use a kind of a combination of all four effectively all my notes in my graph are allowed to say please keep this note active when they play and effects are also allowed to say i want time to run for another three seconds after all notes are finished and there is also a back end check to make sure that that synthesizer isn't playing there's a danger of course that with all of these mechanisms you might find a graph that is continuing to play and consuming resources when it doesn't need to but i consider that really a bug there should be some reason why the music slows down or finishes unless it's a graph that is designed to just keep playing continuously something that has it delayed so yes janet scripting this is how i build the graph as i said before it's a lisp-like scripting language very simple and easy to use very easy to embed it has a great foreign function interface if you know what one of those is uh and it's easy uh it was an easy step along the way to building graphs eventually i want to get to the point where like bit wig where i can drop nodes using the mouse and drag lines between them and build synthesizers that way i think that's that's always something i've wanted to do i never quite got there um i have a lot i've got a long way along that route now and i'm pretty close to being able to do that but this was a very simple way to get the graph building without having to worry about building the visual tool first so in this example i'm creating a simple synth and then i'm adding an oscillator this fm up oscillator is basically two oscillators where the frequency of one is being modulated by the other and then at the bottom here i define that synthesizer and i just play it and i'm going to give you a demo of that let's go a little deeper here and just play a little with the janet scripting language you can see here it's it's quite lisp-like but very easy to follow this particular part of the chord here is building an output graph i have the concept of a graph which runs after all the other graphs and that's where i do the mastering and add the audio analysis the audio analysis can be put anywhere in any graph but i always have one on the output so that i can see what's going on so if i just evaluate this there's our little output graph we see the the mastering node and the audio analyzer which isn't getting any audio but let's go and make uh another simple script this is a very simple one here this defines a simple synth which is made up of an fm oscillator which has a one oscillator controlling um another so let's define that and let's play it [Music] i can also just hit keys on my keyboard here and yeah so two oscillators i can play with the attack curve in the ui of course and i can also send commands from the script let's do something a bit more interesting there's a function here which adds a special effect path so we'll define that first and then here's an analog synth and this is a specialization of that analog synth [Music] which creates this what i've called dark sci-fi synthesizer i can also hit keys on a keyboard [Music] and of course i can still tweak things i can turn the sub oscillator up [Music] and change things as i go i've got the delay on there the reverb you can hear turn that off here maybe add some fm synthesis into that as well [Music] so yeah that's the janet scripting um there are lots of examples in here and i need to build more interesting things um i have some things where i build a dx7 style of synthesis just because that growing up that was something i knew um here's a a function that builds a three on three operators let's try that these are various functions that set it up or here's a flute let's do uh tympani drums and there is the synthesizer that was built you can see all these different oscillators feeding in a different way it's quite effective you could probably also notice um the notes flowing along the filters here being fed back and it's i can't zoom in if i can just find my mouse wheel eventually this will be a full graph editor where i can drag and drop things and connect things together but at the moment it's showing the control surface view which allows the artist to tweak the synthesizer while it's playing but that's the janet language one of the challenges of course of ui is how do you communicate with the audio thread this is one approach that i've took with i used i called it lazy memory originally i'm not sure if i'm still calling it that but um effectively i on the audio thread when it want this is the this is the case for these little blue dots which are representing the notes where they are currently are on the adsl slope what happens is in the audio thread it reads um it has a buffer a pair of buffers that it's working on the audio thread has access to one of them at any one time so it gets the right buffer it writes to it without locking it knows that it can always access it and then it does a try lock if the try lock fails it just throws it away and forgets about it if the trilok succeeds it swaps the pointers and now the right buffer becomes the uh the read buffer so what this is doing is essentially ensuring that the audio thread runs in constant time and never waits the disadvantage of course is that it might throw away information about where those notes are but that's okay on the ui side it's locking the read buffer it does a proper lock it reads the notes and processes them they may be old because the audio thread may not have swapped them but that doesn't matter it's just going to show the previous position this is happening so fast that as long as those notes update you know even 60 times a second or whatever or less than you're never going to see the difference so this is a an approach i've used in order to make sure that the ui can always run and show something and the audio thread is never blocked because i don't care about perfect accuracy of showing this note information i use the same approach for the fft it's slightly modified in that the audio thread now puts the audio frames onto a lock free queue this cue is fed until it's full if it's too full it won't even bother doing that um the fft processing thread wakes up periodically samples the pulls things off the queue again because it's the lock free queue updates the audio processing and then the same approach as on the previous slide happens where it tries to write memory but it will discard it if it can't excuse me um and so on the ui thread this sim uh the same thing as before happens it always locks and reads and processes it might well be processing old audio data in practice it almost never is it's very hard to see this ever happen on the profile so there are just a couple of approaches i've taken to connecting the ui to the audio yeah i have a regular tick in the system this is how i manage patterns obviously the user might be creating musical scripts which contain lists of notes to be played in a pattern and and i need to schedule these things so i need i need a reliable timer and this is the best one i found so far um essentially there's a thread running which sleeps for a given amount of time and then wakes up and then takes all the consumers and then it sleeps until the calculated time for the next for where it should wake up it doesn't sleep um it it wants to wake up at a regular interval it doesn't really care how long the the middle part of this takes but it should always wake up at the same time interval and in my measurements i've had microsecond drift when doing this certainly no more than a millisecond i think 700 microseconds rings a bill but it doesn't drift very much and it's the best i've come up with so far obviously this is only scheduling information doesn't have to be perfectly in time because the notes can be are also picked off at the right time in the audio thread when they're being played this is an example of pattern scheduling this is still somewhat a work in progress it works you'll see in the demos it works but i still am not happy with the design but essentially what i have is external maybe a midi keyboard i might have orca which i'll mention briefly later um i have ixilang they're all doing timeline updates and what they're doing is they're they're putting notes on the end of this ring of data which is uh contains all of the notes that are scheduled to be played you see the blue line in the middle of the the visualization here is is time zero when they have to be played so these are all in the past these are the ones that are just about to be played and what's happening is the ui thread is consuming the back while the other threads the timer threads are producing notes and the audio thread is doing exactly the same thing now interestingly the they might end up not quite scheduled at the same time and the ui thread might be behind for whatever reason but that doesn't matter they're pretty much the same uh but there's no lock between them there's no link between what the ui thread and the audio thread are doing the audio thread is basically sampling the notes that are upcoming and figuring out when it needs to play them and as long as these two this consumer and producer in the ring buffer never collide we're all good this is kind of the design i've gone for it's not quite right yet it does work which you know tends to make i mean i just leave it alone and work on the next complicated problem but that's where i'm at with that [Music] yes i mentioned dixie lang it's a powerful music language i've wrote my own parser for it because it's fairly trivial it's great as a beginner synthesizer sequencer for the synth and and jibber actually used it as their inspiration for a music language i think it's it's underused and it's definitely something worth looking into so janet isn't the only thing that um i can use to sequence music with uh orca is something else that i've been looking at this is uh an esoteric programming language it allows you to build little these little or cellular automata programs that can run and schedule notes and fire off um events to the sequencer i haven't uh got this running at the moment i have had it running in the past it's a lot of fun to play with and it's potentially going to be another way in which you can schedule notes i just thought i'd show it because it sits inside the editor it's already implemented in there uh music and graphics yes how do we connect the two together um there are several ways but essentially you need to get the information about what the music is doing to the gpu in some way so the way that i approach this is i convert the music to frequency bins using a fast fourier transform that happens on the thread as i mentioned earlier on and those frequency bins are then converted into a four row textures texture which you can see here expanded out the top two rows are audio data the bottom two rows are the frequency analysis data and because they're in a texture the graphics chip can sample those and use them in any way it sees fit which is great at the pixel level per pixel on the screen the image on the right is actually a pixel shader which is sampling them and drawing a more traditional um equalizer which is which obviously runs very fast in a single frame it's just a single well it's a pair of triangles drawn as a quad are very very easy to very easy to accelerate for for a gpu it also converts the the frequency pins into a four component vector and that's used in the vertex shader in order to modify geometry which is useful if you want to have things bouncing around on screen to the music for example there's a problem here though which is how about all about latency really how do you get that data in reasonable time the the issue is if the microphone hears a sound but the visual only updates um a fraction of a second later than that then it's really easy from a user's point of view to see to see that disconnect to hear the audio and see something it's like battling lip syncing on a on a tv movie and there are all sorts of reasons for that there's whatever the audio hardware is doing the size of the frame that's being captured the size of the binning that you're doing on the fourier analysis how fast that is how long it takes to upload the texture to the gpu how long it takes the gpu to render the frame and then the v blank sync which is uh if the screen is locked to a refresh or whether or if you've got a frame lock a frame per second cap because you don't want to run too fast and starve the system of resources so there's a lot of things going on on that can mean that the distance between the microphone and the and what you see aren't always the same and this the worst case for this is a microphone which is sitting in a room listening to music and then which some live coders have and then you see uh you can see that the visuals aren't quite synced to the audio that's why often when you go to these things people will do visual demos that are more ambient which pick up the general feel of the music but don't tend to bounce along to the beat not often anyway um because it's a harder problem to solve it's not unsolvable but the nice thing about this system that i've built is it's an integrated system where i have first class knowledge of all the pieces which makes this a little bit easier to solve one thing you can do as well which i intend to do but haven't done yet is you can insert a delay between generating the sound and sending it to the driver to give the graphics chance to catch up and it sounds funny to say that the graphics needs to catch up but typically that might be the case if you have a really slow shader running you know at only 10 frames a second your actual limit on those visuals is is what's causing the visible delay that tends to be the critical path in my experience if i whack up the frame rate of the graphics to a few hundred frames per second i can pretty much get the audio and the visuals to sync up exactly and look just um look as if they're happening at the same time so that's an interesting problem one way in which i can look at problems like that is with the built-in profiler um graphics tools often have a scribble like this as i showed right at the beginning and tools that i've written uh in my day job um and so it's something that i've written actually many times before so this wasn't very difficult for me to do because i knew all the ins and outs of it this is an example just shown here of what might happen if the audio uh takes too long so for some reason there's a blockage here the audio has taken too long this this this frame of the ui is 35 milliseconds so the audio is probably just over half that and you can see what's happened the the hardware the audio hardware has gone oh oh i need more data so it's started to ask for more and more frames and it keeps asking until it catches up and then you get a gap and so this is the um the sound card driver doing the right thing and making sure that it's going to have enough buffer so that it's not starved and the vertical um distance here is represents what i think should be the longest the audio should take in order not to starve staff the hardware this is all the audio catch-up i just showed this is a closer look at a graph you can see all the sub nodes inside the graph being evaluated in parallel on different threads this is an example of something that i found when i first built the profiler that was incorrect the the spectrum graph was waiting uh and the ui was blocked and that shouldn't have happened and it wasn't how i designed the system as i showed earlier on with the um the lazy memory approach but there was a bug there was something that wasn't doing the right thing and it only became apparent on one particular frame this is actually a zoom in but it only showed up by grabbing a few thousand frames and looking and seeing ah why is that one taking longer so that's more or less it um i just wanted to cover what i'm thinking about for the future the first priority is obviously to get this into people's hands so they can play with it i love building tools i love to see what people do with them i think i will consider it a success when i see somebody live code using it um i'd like to implement the session recording that i mentioned i i think that's really nice i would love to be able people to be able to say hey let's check out this live coding performance and just send you a project that you can load and watch on your local machine um or record something that they did the night before and figure out why it sounded so good at that point visual graph editing as i said i'd like to be able to drag and drop nodes and hook them together i think that's a fun thing to do anyway both for graphics and for um audio actually uh geometry generation yeah the at the moment i'm just drawing spheres and tauruses and things i can use a geometry shader i haven't tried that for a while but i'm sure it still works but i would like some way to script the graphics more so that i can control generate all sorts of interesting primitives i'm interested in music languages i think there's a lot of interesting things that can be done there especially once you've got to the point where you've built an ide that has a pattern runtime can can efficiently schedule things i hope that people might take the tool and just say hey i'm going to go use it to experiment making a new language i think that would be awesome um i'd like to get dx12 in vulcan that's kind of more in my wheelhouse and i'm really interested in shared editing and shader authoring uh and ray tracing with vr i thought would it be cool if you could have you know a vr environment where you were live coding and that's that's pretty cool and and because of the way that i've built all these pieces it it would lend itself to that it could be dropped in the editor obviously can sit inside a 3d world and all the pieces i suspect i could get there with that if i had the energy we'll see and that's the end of my talk thank you [Music] um thank you chris wow this was really really impressive stuff um yeah i mean obviously you've done this it's a huge project but thank you so much for like giving us a glimpse um under the hood like how all of this works and i think it's like a mixture of like you know i guess anyone who's built like an audio engine before has bumped into some of those problems before but like seeing like your solutions to it and like just like the whole like the whole picture and also like your particular solutions that's that's super interesting and just like it's a massive project it's like so impressive you know how you just uh put this together so um yeah definitely very impressed thank you for sharing all that thanks for saying that it's a fun i could have talked all night about it i think i suspected and i had about twice as many slides i think um but uh yeah i i do have a question how long did it take you to put all of this together it's many years actually it's not um a short project because i only do it part time so um as i was saying to josh before we started i you know i get up in the morning i go out early for breakfast i take my laptop and i and most of it's written in a coffee shop before i start work in the mornings um and i usually get the day on saturday or some big part of the day on saturday while my children are at school for them um but yeah so it's uh it's taken a few years um the editor was a sideline that a tangent that i went off on which took a full year of part-time work just to build that piece um the graphics stuff is some of it is all ancient you know little experiments and old things i did some of it needs to be working by now um and yeah the audio probably took it's probably a year of part-time work to get the audio stuff done the ui looks complicated but i've done that kind of thing throughout my career so for me that isn't the hard part the hard part is the generating the audio and figuring out how to play the audio graph and that's why i concentrated on that but yeah it's multi-year project several years in fact and if there's no clear defined end right it might be finished next year or the year after um and it you know it depends on the energy i have and how well things go um yeah that would have been my next question like when is this actually going to be you know out and like when can you have your hands on me but i guess yeah you just answered it like yeah yeah i mean my my preliminary well my work in progress is that i i want to have something shipped by early next year um there's a different difficult problem here right because it's all going to be open source it's all free parts of it already are shipped the editor is already out there and people have dropped it into their own engines and done things with it um but yeah it's it the the the idea is to get it in people's hands because once it's open sourced i might get some help which would also be great but i don't want to open source it and then have it not be finished well enough that people can't get something out of it the worst thing to do would be to ship it and then have the bugs get in the way of everybody and everybody kind of walk away from it and say oh you know i can't use this because it's not right yet um but it is modular so the audio project is an entirely separate application that i can just run without the graphics so i'm thinking that i'm going to just ship the audio first and you'll be able to play with the synthesizer stuff and the note sequencing without generating visuals if i do that it'll give me you know another few months to to make the graphics really great so yeah that's that's what i'm thinking but i i just there will be some of it shipped next year for sure especially the audio part but i'm not putting that pressure on myself because obviously i have a day job i have a lot of work that i have to do in that um and so i can't um you know that has to come first obviously well yeah thanks i'm really looking forward to this coming out at some point i definitely learned a lot from uh you know just you talking through these problems that you solved it's it's really really so interesting um we do have uh quite a few questions from the youtube chat sure um so igor abdul aguila asks what are your thoughts about pure data vanilla for live coding it's really cool um i'm on a slack uh group called the future of code where they talk about all of these kinds of tools building software using dragging and dropping blocks and all that sort of stuff and i think your trader's great and i did kind of allude earlier on to the fact that a lot of these tools are sometimes not easy because they're all open source and they're not finished and pure data is one of those that really wouldn't run very well on my machine it it isn't high density aware so i've got a 4k monitor i run pure data i can't see anything and i'm forever tweaking fonts and stuff so i'd love to play more with pure data but uh until they you know sort it out and make it work well on my machine then i'll then i i won't um i've got a macbook so i i have dabbled with it um and i think it's a fascinating way to do it and i've seen live coders use that and tools like it so there's a one there's a paid version as well called max msp which i've never seen but i hear is very good um obviously because they have to pay for it i haven't i haven't purchased it right uh so joe black is asking have you considered doing audio and dsp on the gpu yeah that's a fun thing to do right i haven't done any research in that area um it might need bigger brains than mine i don't know um i mean i've done graphics programming and i'm i'm capable of looking into that but if i went down that rat hole i would never come back i suspect and i'd never get the thing finished so right thank you so um george contours is asking how does one get into a complicated api like vulcan if you only have some cheap opengl experience i would start on the tutorials there's a fantastic tutorial i think it's called vulcan tutorial if you google it it'll be the first hit you get and they've got a really great one um vulcan's hard right directx 12 is hard it's not for the faint-hearted even if you've done opengl it takes like a thousand lines of code to draw a triangle on the screen to get your first thing on the screen you actually have to type a thousand lines of very complicated code and you really have to understand what they're doing and what the hardware is doing in order to set it up um so yeah i'm not to discourage anyone i think it's graphics programming is fascinating i hope i convert that in the talk but yeah i would start with a vulcan tutorial that's that's definitely the place i would go first and just you know just do it just get on with it all right um okay we have a couple more questions so john rush is asking it seems like your understanding of audio coding really benefited from truly understanding graphics first so would you say that understanding audio is beneficial for graphics like the other way around um are you you're asking if audio has learned taught me anything about graphics i think i'm not sure i understand the question 100 right i think i think they're saying that obviously like your understanding of what has benefited from doing graphics first and i guess they're asking whether that's the case and whether you think that's also the case the other way around yeah i i did actually see that question as it went past on the chat um and i think that was probably before i had demoed some of the some more of the parallel cases between graphics and sound so i'm in that direction certainly you know when i've encountered wave tables i was like well this is just bit mopping it's just you know audio mapping and not visual mapping uh and and also my knowledge of program because i used to write drivers for gpus so i've got quite intimate knowledge of what the hardware is doing so when i see an audio api that says tick give me some frames i know in my head that what's happening here is that this buffer is some kind of shared device memory or something that's going to get dma to the hardware and how it's going to consume it and how it's going to manage the timing of that i've got the knowledge of of those things so that that helps with my understanding for sure um did i learn anything in the other direction um probably yes one of the main things i learned is it's hard to do better threading code i'm still learning that i mean i've talked with youtuber on discord about problems i've had in the past and but i've i've become a much better c plus plus um multi-threaded programmer now i i think that i can write stable threading code now that i couldn't do before because although we do a little bit of that in my day job nowhere near what i had to do to make this work so so yeah there was a bit of in terms of a learning experience i would say it's been great yeah that that makes a lot of sense actually i think that's something interesting maybe about the human brain i don't know i find that often when you like read a book about audio or i do that sometimes in my own talks when i just explain like basic audio concepts that you kind of um use like an analogy and like this is the analogous thing in the graphics world and that tends to like help a lot of people understand it and you actually had quite a few of those in your talk uh which were completely new to me like the whole mid-maps things like wow i've never heard about this but this is really this is a really cool way to explain what's going on you know you had quite a few moments in your talk like that so yeah and maps have been around since uh you know texturing was invented maps have been around forever and these days we do all sorts of even more complicated and interesting things along those lines but yeah yeah it's good stuff um so finally we have one more question by aaron smalkis who's asking what do you do full-time like what do you actually do in your day job if you can share anything about that at all well i i pointed at the beginning the slide i think slide number two which you can go back and look at um i i've worked on tools like n site recently um which is a graphics debugger so you you running your game and something looks wrong in a particular frame of your game or there's a particular frame that's running slowly and you can fire up this tool grab a frame of the game and then look at everything that's happening on the graphics chip and in the in the api so in gpu trace you can actually see what units in the graphics chip are busy and so then you can figure out why the code you've written is running slowly in insight you can look at the contents of the frame and figure out you know that that triangle you drew was bright red and it shouldn't have been you can get down down to that kind of level so i write tools like that i actually was the first um developer tools person to work for and video when i first arrived i remember somebody asked me what you're just going to talk to developers and that's all you're going to do here um so yeah and i well actually you're gonna write tools because nobody there was nobody hired just to write tools when i arrived so that's what i've done for my whole career i love to write tools and i hope that's obvious from the presentation i like i like to put tools in people's hands that make their life easier and i just find that a really interesting thing to do so that's what i do yeah all right that's awesome and and uh so you said that you had some stuff that was already open source so what was the what part of this is open source at the moment just the text editor the resonating bit um so if there's a sorry there's a result there's a github um organization that i created called resonality and underneath that i can put a link in the chat i guess um is um is a shared cpus library which is just my grab bag of bits called mutils and it used to be a library that built the world it used to build all of these projects like port audio and open ssl and these horrible to compile c plus plus libraries um and it used to be mine central source and then vc package came along and just removed that whole problem um but yeah so there's a shared library there's the text editor called zep and if you just google zep github you'll find it um and it has pictures on there of various things that it does and that you know these are all works in progress don't go away and think that that's even that is finished because it isn't um so yeah and then the audio stuff um and the big ones will come out the next year as i said yeah one one thing that i was thinking during your discussion was and and is is something that i always think about because i come from kind of a high level using juice as a framework to create uh my plugins and audio apps and i've always wondered what happens under the hood or how you work from the bottom up and i think this talk has been really helpful for me because it really let me know some of the work and some of the thought processes that go on a couple levels below what happens at the framework level and another question that i have was just around working with the different apis because you said that this this is a cross-platform system so you must be working with core audio and whatsapp separately and what do you find are the biggest challenges of working with those uh different apis you know setting up setting this up from windows versus setting this up for uh for mac platforms and what are the differences in dealing with those apis yeah so the specifically the audio case um i don't actually talk to individual apis i use a thin layer called port audio which basically gives me a list of audio devices and audio apis and lets me pick and i can say okay on this system use wasapi and give me you know this many frames and this this and then and it just gives you back that callback that i showed at the beginning and asks me to fill in the sound buffer so so i don't go that deep for those things because it would just be a nightmare right if i had to write all these little uh use all those different apis um to be honest the biggest cross-platform channel challenge is build systems as i was saying earlier on i probably spend quarter of my time uh editing cmake files making git of actions work properly making continuous integration work properly um all of that stuff just takes a lot of time and effort and maintenance but you have to do it if you want to build a piece of software that everybody uses you have to be building you have to be compiling it for all platforms on every check-in and once you get to that realization you are pretty much going to have to be a cmec expert and you're going to have to be a github expert you're going to have to be a scripting expert and so that stuff takes a lot of time and effort that's the big effort we cross platform once you've got to the point where you understand the basics of cross-platform coding that's not such a big deal right occasionally there'll be some compiler difference some small thing but i if you keep it generic usually that's not the problem it's the systems around it that are hard i would say yeah i mean this completely echoes my uh my experience as well you know i'm working on a project right now which is also cross-platform we also use cmake as the build system and like the actual differences between the platforms if you have a framework that kind of wraps them that's not very often like the the cause of the problem it's like cmake ci like all the kinds of like scripts you need around that to make things work and like all of the stuff um so it's interesting to me like on a high level like you know it it seems that many projects that kind of like build something like an audio engine they bump into against like many of the same problems like what you talked about with like threat synchronization and then obviously now that we talked about the build system and um you know the audio graph like it seems that a lot of people are bumping against the same problems but then you kind of find slightly different solutions to it and yeah it's really really interesting because you know uh every every time someone talks about like how they actually implement like a whole big system which contains an audio engine um like there are always these like differences and like okay there's a slightly different use case here and then you just take this other approach here and it's just this is really quite interesting it's just maybe like a meta observation rather than a question really but yeah no it's true and part of the fun of all of this is discovering how to solve those problems for yourself right and that's why i don't use juice right not the juice isn't awesome but for me that takes away part of the fun and this is like apart from like just being amazed at how much work you've done and how impressive this all is like the other takeaway for me is like obviously you know i'm working on an audio engine and like i'm like oh we have maybe we should consider this you know and like i guess that's kind of uh why it's like so valuable to have these thoughts where someone talks about like how they actually implemented stuff because you you get to like uh see these different solutions and see if they apply to your problem and learn from each other really i find that really cool so yeah very much for just sharing all this yeah sure and as time goes by you think i think things do get better that's the interesting thing about this when i started off trying to build all these complex libraries and then vc package came along and that really reduced the amount of work i had to do and then another uh rare flight was when travis started charging for their continuous integration and i had to learn how to use github actions and then suddenly i'm like oh github actions they just work and they're much easier to use and so you know you you live and learn well a friend of mine says you live and learn and sometimes you just live but it's true and also i'm very relieved to hear that um other people spend quarter of their time on maintaining build systems yeah common problem yeah i i have one more question before we moved on i mean it seems like that you you're dedicating quite a serious amount of time to actually doing this project and it's the type of time and the type of effort that a lot i think a lot of people would charge for so i'd like to know your motivation for making this an open source project where does that where does that motivation stem from where you say okay i'm going to put it in all of this hard work but now i'm going to open source it and i'm just going to give it a wave to people to use i think part of it is as i said i just like to build tools another part of it is that i've been at nvidia a long time and they've been very good to me um so i'm not looking to you know make a fast book on on some side project that's kind of not where i'm coming from i'm just i'm doing this because it's something that i i really love to do and i if you look at my website which i showed a screenshot earlier on i'm always building something you know my biggest problem honestly is that is to tell myself to stop coding so that i can go eat and live the rest of my life so you know i'm always fascinated by something new whether it's advent of code which i've i started doing this month before i realized it was insane to do that and this talk and keep on on top of everything else um you know so there's always something new to do but i i enjoy it so there's no reason not to give it away for free um what would satisfy me would be if people used it you know if it became part of the scene where i went to an algorithm and a couple of the guys on stage were using it to make music and i think it would be very hard if i said i'm going to charge for this application probably no one would use it then but if i make it open source and free it might actually become something that people use and then i can you know smile proudly and uh and watch it go out into the world which would be great it's to me it sounds like it would be a great educational tool for people that are trying to get more a level lower than than juice and really learning how the nuts and bolts of audio programming really work yeah i think that's true i hope that's true anyway um you know the fft for example there's a page of code in there that shows you how to do an fft in c plus plus when you've got the audio buffer to from one end to the frequency bins and at the beginning that was really hard to find i could i find little fragments on github little pieces of information from people but you know working through those problems figuring out the decibel conversion the hamming window the way that you convolve the data all that stuff is hard but once it's done and there's a page record that shows you how to do it then it's never going to be hard again so i hope that stuff will go get out there and people will look at it it's not all good but but i i don't tend to release it until it is so hopefully it'll be okay amazing great i think it's a good time for us to uh move on thank you very much chris for that talk that was that was so educational for me and uh and i think for a lot of people that that are joining us as well thank you it was great to have an opportunity to show it off so i appreciate it thank you fantastic okay so now we're going to move on to our uh our speaker session on which every month we do one presentation and then we also do an interview where we try to go beyond uh just technical aspect of what people are talking about uh in code and trying to discuss a little bit more behind the scenes i've actually named it this afternoon i came up with a name for it which is beyond the code so uh so what we're going to do is uh we have our guest here kevin uh mulcard who is the vp of software uh no you're the director now uh director of software development yeah at arturia uh welcome and thank you thank you for joining us today thank you for inviting me yeah so uh so it's great to have you here and i think that a great place for us to start would maybe to be to talk about a little bit about who you are uh how you got into audio and how long you've been in the auditorium yeah um i i haven't been always been passionate about music and uh when i was a teenager i started to to make music with my friend and to to make computer music so i i started in maybe in in the end of the 90s uh playing with uh the first version of appleton live fruity loops and stuff like that and making a techno music with my friends and uh and then uh after the high school i wanted to become a sound engineer unfortunately i didn't manage to to go to the sun engineering schools because i haven't that much experience in some recording anything like that but i was good at math and uh so i i talked to a bunch of people and i said okay i like to to play and to to make music with this kind of software why why not trying to to become a software programmer and uh and making this kind of software that would be a great uh great things to do yeah and so i i did a i go i went to the university and do a computer classroom and i went up to the my master's degree and then i um i i looked at a very famous website in france called audio franzine which is a place where all people passionated about music can look at and you know you have all the companies and all the products available and i i took all the companies you know that were doing software at the moment and i i sent my resume and applied for my uh last uh last last year of internship and uh fortunately for me i thought yeah actually answered to me and uh and i i comment come up at the company as an intern um sorry what what year did you join as an intern sorry again what what year did you start at arturia uh yeah yeah we i started to work on the origin project which was the the first arturia hardware synthesizer it was back in the 2000 and and seven and um [Music] and yeah i started as an intern working on the software uh on the updates of the firmware of the of the hardware and also the software that manage the presets so you can send back from the computer to the hardware and the presets i worked on that for maybe two or three years then i started to work on the spark project which was the drum machine a hybrid drum machine a software and a dedicated controller yes to to do a drum machine so i was involved in the software development of that i did most of the yeah of the software architecture and all the sequencing [Music] pretty much everything about the audio dsp from the software um and then um at that moment i started also to manage the project because this project like quite a bit long it took i worked on that for maybe four four to six years because there was first a first hardware controller called spark and then we worked on a second smaller hardware controller called sparkly [Music] and the spark software went up to version two so yes it took me about five six years working on that at the end i was also managing the work project so yeah i grew up like this and then i started to work on the on the recollection project it was maybe yeah some eight years ago something like that on the v collection 4 and yeah so i i was both developing and managing the projects and then the company continued to to grow and i and the team i was managing continue to grow and step by step yeah version after version so yes version five six seven today we're probably releasing the eighth version of the v collection yeah so yeah it's a long journey yeah yeah and you missed that i was gonna say you must have seen a lot of changes because you joined you joined arturia yeah definitely definitely i i when i when i come at arturia we were about 15 people one yeah and now the company is about a bit more than 100 people yeah amazing yeah and and the uh one thing i was thinking about was how much the landscape of plug-in and audio development has changed a lot over the past 20 years you have a lot more independent developers you have a lot more people who are releasing plug-ins probably early on i imagine that you didn't have a whole lot of competition yeah that's true yeah yeah at the beginning we were among the the first uh working on software instruments there was not that much at that time there was uh of course native instrument uh timur's uh no hit a bit and and uh there was time big waves of course um yes geforce versus geforce yeah there was some history about gfrost and arthur yeah i won't comment that because i i i wasn't there at that time and and yeah and no yes the landscape is much much larger but uh i think that's that's very good to have so many so many actors so many competitions and it's it forces us to to even reinvent and think about what uh yes what we have to do and not falling asleep and uh just uh yes so yeah one one thing that um one thing that we think about with companies that have been active for so long is working with these larger legacy code bases and you don't have to deal with maybe with um like a big daw like ableton does that has probably a code base that spans over 20 years but in your products i imagine that you have uh code bases that are quite old and yeah yeah yeah how do you how do you handle that how do you deal with that it's uh it's a nightmare as you can imagine we we try to yes along the years to to fix that by constantly removing some old code and replacing it with some new codes uh what we we started i think it it was five years five years ago maybe six years ago we we started to work on um yes uh russell having all the same code base at least for the what is not the audio engine but everything else to have the same code base for all the software because at that time we we were having maybe three different graphical frameworks at that time for the jupiter 8 the first version we were starting working on juice but before we were working on vst gui and we also add some old code base on and stuff on mac os and and some specific windows stuff on mfc and it was yes a nightmare so we decided to get rift to from all of that and uh put everything on on juice and we we developed uh our own framework on top of juice to have more flexibility you know on the on the widget and things like that so what we are using for juice is mainly the component system uh photographical rendering and all the user events uh keyboards uh maus handling a thing like that and yes for everything else we have developed our own custom widgets and and we also uh we started to not use the it was also a long time ago but to starting to use more standard libraries code rather than juice code for instance for you know the strings the files things like that thanks to modern c plus it's much more easier to add i know that in the meantime juice is becoming much more standard compliant on all the containers and things like that but yeah that's uh that's what we we've done it was a lot of work uh it has been released on the i think on the recollection five so at that time it was much more comfortable to maintain and develop new products but we still had legacy code for the dsp stuff and that's that's why from now we are starting also to get rid of that because you know it's most of these code have been written by people that are not anymore in the company there is not that much documentation and anything like that so we have developed our own dsp framework and all the new products are used using this are developed using this dsp framework and for instance we have released just today a new update of the jupiter 8 which is completely rewritten from scratch on the dsp uh legal code wow that's amazing yeah i imagine i imagine when you're dealing with that older code base as well that it must be tough to introduce new staff into into your company because they may be used to maybe working with juice as a framework and used to the way that it's working and then now they're coming into uh an environment where there's a lot of legacy code that may be working with a completely different paradigm that maybe might not even have the same mighty might not even have great documentation for it uh that how do you so there there must be a big question of you know how much time do you allocate towards updating the code versus working on new products and exactly yeah that's that's uh a big deal and yes we we are at uh at um at a step where we think now it's the time to get rid of this legacy code so it will take lots of time because we cannot do it for all the product at the same time of course so i think in the yeah in the upcoming years we continue to to update maybe one or two or two maybe three if we are lucky uh products uh per year and continue to develop also new products because that's uh yeah you know you you have all these user base all the the collection user that love these scenes and that want them to be upgraded updated and up to the standards of 2020 and not the standards of the 20s or the 2010s but on the other hand uh only putting the resources on updating existing stuff is not it cannot work on a financial basis so we have to balance that of course yeah yeah um so so you started as a software engineer and then as time moved on you moved more and more into management and one question that i have was how uh do you do any coding now or is your are your responsibilities mainly management now how do you uh do you miss coding yeah i to be honest no i i don't code anymore i i still can have a look at some and subcon bases and especially for the very old code base which i i am the one even if i didn't work on discord base i am since i'm the oldest guy in the in the software team i know more than the other discord bases because i worked on the on them and so and even today we we've had before the release a new bug coming from on the modular v and this is one of the instruments which has the oldest codebase in in the v collection and yes we looked at that and we found we found a bug uh i looked at it and i helped the team to to fix it uh and it was you know just a um a array um uh overflow and uh but because it was written using old sea array uh things that are not uh and uh it was not uh catched by our unit test and like that and this was a bug that we we knew it was here because we already we've seen it a couple of times in the last years but it was very very difficult to track it because we were not able to reproduce it on a you know on a regular basis and uh for some reason with the latest version it was highly reputable on a particular presets and so we fixed it in a couple of hours but um it's you know it's that's also why we want to to get rid of this old codebay that is not not easy at all to maintain and yeah so i'm i'm not coding new feature of course and it's sure that i'm missing it um but i'm also enjoying a lot uh the managing stuff it's of course completely different it's more you know human interaction uh with the team and trying to [Music] to know the people very well and to to fit their strengths with what is key for the company that's really how i like to work to to make sure the people are really enjoying and liking what they are working on uh and it's uh it's working well uh so far yeah and how how big is your team uh the software team is about 12 13 people there is the dsp team which is about i think now eight people they have a dedicated manager but i'm still managing the world the bose team so yeah the the software in itself is around 20 people wow uh one question that i have that that i think about a lot because i work with quite a few companies uh helping them find helping them recruit for roles and things like that and a lot of companies they really only hire for kind of senior or very experienced developers and uh and there's this challenge where things seem to be very kind of top-heavy in terms of the hiring at arturia do you how do you handle bringing junior developers on board uh what what are some of the things that how can you how can i guess my question is how do you keep yourself from falling into a trap where you say i only want the most experienced developers um to be honest at arturia we have a pretty young team we we hire a lot of people uh after then their internship from from school and it it works quite well it's [Music] it's you have to balance it with a more experienced developers but we are not that much in in the trap to trying to to get the most experienced developer we were giving their chance to the the new the newcomers and uh yes as as i uh as i was when i started at the tutorial and uh yeah i think for instance in the software team um i think most of the people are between 20 and 30. yeah and we we have a couple of very expensive very experienced guys which are helping the younger to to grow but [Music] yeah we [Music] you know it's the the more experienced people can also be a bit more difficult to [Music] to manage and to it's yeah it depends those are kind of set in their ways yeah exactly yeah sure uh how do you how do you actually hire for your for your so you have somebody that is just coming out of university maybe they know a little bit of juice maybe oh maybe they've done plugins how do you how do you spot a person that you feel is right for arturia what what is it that you look for that's a that's a good question uh i think uh being um passionate in music and especially in computer music is a very good point uh to be to to come to to be hired but it's not it's not required and there are a lot of guys that were not at the beginning um you know knowing computer music and things like that and that came to it with us so it's it's um i think it's a trade-off between that and uh being yeah you know curious open-minded and uh and of course uh very skilled in c plus plus i would say that's that's the the first thing uh you you have to know your c plus plus uh and um when you say know your c plus plus what does that what does that mean to you what do you what would you say is like the bar the bottom the bottom level when you're hiring a junior developer and you say you need to know c plus plus what does that what are some things that you would it's maybe i want it a bit too fast maybe it's not you know you're surprised but it's more how you how you reason about code and how you you are also open minding to [Applause] discussion and um and feedback about your code and if you if you are able to to really understand easily what's what the code is doing uh what what is missing from it and what could be the improvement on it uh i i would say that these are the most important uh skills i'm i'm looking at uh so it's how you attack problems yeah exactly yeah exactly and um it's not you know it's not uh knowing uh all the different uh sorting algorithm or things like that it's i mean this is uh this is not needed for for us at least on the software department on the dsp side they are much more demanding on the you know the health skills the mathematics and things like that but uh on the software departments we are more uh yes on the under understanding of how a program works uh what uh how you can manage your memory and uh what are the consequences when you have a multi-straight in multitude applications uh thing like that yeah yeah because a lot it's interesting to me because a lot of the things that you seem to mention seem to be things that are away from coding in a way things like discussion and problem solving and that so it seems like the actual raw coding skills uh are part of the equation but not the whole thing so are there are there other things such as uh you know the persons how how they integrate into their into the team how how do you how do you work with that how do you kind of calculate i think uh what's what i really think is important and that's something i uh i was thinking a lot uh for myself for what i learned from my um uh university school uh i felt when i arrived at australia that uh um my school didn't give me that much uh you know hard skills uh in computer programming and in how computer works inside things like that compared to other people from other schools but what my school gave me was the ability to to learn and to learn quickly and that's that's what i think is important uh it's because you you know if if your brain is well connected i would say uh even if you do not know at the moment uh you know how to code a lambda with this algorithm in c plus plus 11 or plus 14. um [Music] you you can learn it and that's for me the most important thing yeah because your knowledge at the moment at a given time will be outdated in a couple of years or even in in one year depending on what you are working on so it's better that you know that you have to learn that you will have to learn continuously during your career and that's also [Music] better for you and for your wellness to to continue to learn and to not fall in the all the same thing and random yeah yeah very interesting now um shifting gears a bit so of course arturia uh when i think arturia i think analog uh analog emulations of synthesizers uh and one question that people ask about all the time is about analog modeling how do you model something after an old piece of gear i know that you probably or you might not be able to really tell us like the inner workings of how you do that i'm wondering if you could maybe give us a little peek into yeah how how you approach that i i would say there is no magic first you need to get the unit you need to if you can you need to get the schematics of the electronics you need a very good knowledge in [Music] electronics engineering and a very good knowledge also in discrete discretizing things and um and then it's uh if you have access to the to the schematics you first you put these into equations uh and then uh you run it for instance in you you start with splice uh spice thing like that uh software and and then you put it in in code and you compare and you have to measure measure measure measure on the on the hardware and you compare your results with the measurements you've made and usually you are close but not close enough to to what you to what you are you wanted to get and then it's it's uh yes we we have the [Music] we are lucky at atoya because we have some very skilled people in electronic engineering for all the hardware um synthesizer we are doing so they are helping our dsp team a lot to better understanding what could be the not the yes the problem in the in their modeling because they are not taking into account for instance the ground uh coming in the electronics that could making uh some spikes in the in the cpr yeah not not in the cpu and uh in the frequencies in the 50 hertz frequency or anything like that um and uh yeah and and then it's uh once you have a model you you like uh then you have to make some compromise in terms of cpu optimization anything like that because the component level modelling is very very cpu angry and if you want to have several voices of polyphony you you have to do some compromise so you have to simplify the models where you think it can be simplified and keep the very best uh modernization where you think it's very very very important amazing and how much has computing power really helped with that with that with the need for simpler yeah a lot it's um yeah of course it's a lot um but at some point we add a you know in the i think it's in the early 2010s uh where the processor start to become more uh with several cores rather than increasing the the frequency it's it's start to be a problem for us because we are always running on one cpu thread and so the the processing the increase in processing power were not that much helping for us so but it's still very good i mean uh even the you know the even if the processor are not getting a higher superior frequency uh it's still a crazy the amount of computation you can do with uh cpu nowadays it's uh yeah can i ask a quick question just yet of course um i'm just wondering do you are you saying then that you don't do threading inside of the audio thread you don't use multiple threads when you uh attach not that much a bit it depends on the not in all uh instrument and uh we were working on that to to make it possible in every instrument but at the moment it's not uh yeah it's it's a bit complicated also because if all the we are working our software are run in dw and it can also conflicts if several software are using multiple cores at the same time so it's it's a trade-off i would say it's uh but i think this is still the way to go even the new the new computer coming from the the for instance apple with the new m1 and things like that but yeah i i still don't think it's a good way to go to have all the vst plugin manufacturer making multi-core processing without knowing what are doing the other plugins yeah yeah and i remember i think it was maybe in 2010 or something like that uh when uh steinberg was presenting vst3 at musicmaster and there was these guys from yuffie which were asking why they didn't [Music] had multi-multi-thread processing in the vst3 sdk and at that time i was already thinking it was a very good idea and i still don't understand why there is no moving that way in the plug-in [Music] development but if we start to talk about uh plugin format and like that i think we can talk about the world all night on that topic so amazing uh so one one thing that i thought about was about how arturia is so known for analog vintage synthesizers but then you came out a couple years ago with uh with pigments which isn't uh which is a whole new type of synthesizer one question that i have was how do you when you're creating something that isn't modeling analog analog gear how do you make a synth unique how do you make it sound unique because people start off with your typical waveform sine wave and saw wave square wave what what are some things that you do to try to get a unique sound out of a synthesizer that's that's a very good question um i think we with pigment we started on a on a blank page so it was a very good opportunity but it was also very you know scary because you can do whatever you want [Music] and i think what what makes pigments unique is uh it's it's not that much in the sound uh it's it's more all the integration of the what's we try to make the workflow of the software very uh easy to to grasp and uh and also it's the yes the combination of the different feature um you know how what what you can modulate and what you cannot and what you can access directly from a knob or with other with a more click something like that it's i think it uh it really uh defines how how the instrument will sound also at the end uh what you what you what feature you decided to add and what future decided to not add and yeah i think that's that's really hit for pigments uh so yeah for sure when we when we started to work on on that project uh we wanted to to make our our very own scenes uh but yeah it was not that much the we didn't want to to make it have a particular sound we wanted to make it as versatile as possible also i see yeah wow and one of the things that struck me about it was the graphics the graphics aspect of it was that was that with opengl or uh yeah it's uh yeah it's not a it's a it's a combination of opengl rendering and uh standard juice rendering yeah nice yeah um that's great and uh and as you were saying before you um you have the you you have a new release today can you tell can you tell us a little bit about your uh yeah so we released the version 8 of v collection so it it contains uh lots of new software so there is a vocoder so this one is very particular for us because this is um our first instrument where there is a mic input where people can really sing in very very very particular and it's it's very very funny because you can also put your own sample in it so you can play it just like a scene so if you want it um [Music] there is an emulator too so this one is very funny and yes the characteristic is very on the factory library which replicating the famous sound of the emulators and but there are some very very nice features in it uh it's it's it's surprisingly very powerful uh despite the original instruments well not that noise it was one of the first samplers uh at that time but uh yeah compared to other scenes from that same era the the sonic palettes yes it's mainly on the possibility to sample and so you have some sounds that are not less synthetic i would say then there is a um elimination of uh juno uh called the june 6v um so this one is we wanted to have um a synth that is less intimidating and easier to to start with when you do not know that much in synthesis so this one is very oriented to that because it does not have that much parameters and a feature and it's still very powerful and the sound is very fat and then we have yes the obxa which was previously released but was not included in the in the recollection we have a very big update as i told you on the jupiter hv uh which we completely we wrote the dsp code from scratch so it's um yeah i really um engage people who knew the first one to compare and you see that uh yeah the state of the art in dsp modeling has changed compared to what we but what was possible in 2007 the date of the first release of the jupiter 8 v amazing um now there is sorry no go ahead please yeah there is uh the last there is an update of the stage 73 so we've uh updated the physical modeling engine so it's much much better we have a very big update on analog lab v which analog lab is the is the combination of all the presets and the audio engine of the world v connection so in it you can have access to all the sounds from the v collection uh and we we completely redesigned the interface and uh and the workflow uh so again i invite everybody to try it it's very very nice to use when you are just looking for sound and not that much into creating your own presets and sound design it's very very nice and yes we also completely redesigned the sun browser for all the existing instruments and we've added a new dedicated site panel with some features like in-app tutorials so you can learn how to how to play with the instrument and how to sound design with it for all the instruments wow that sounds like a big that sounds like a huge huge project yeah it's it's crazy i think this was one of the biggest one uh we did since since i'm managing the project so yeah i as i told you i'm managing the vigoration since version four so it's uh yeah i would say maybe between six to eight years so yeah and uh and that one uh yeah is internally it's uh i would say it's around the 30 people working on it and with all the external sound designers anything like that i think it's more than 50 people working on the project and uh on the on the last year we are very very proud and happy to finally uh make it public and uh yeah yeah congratulations how long does that have i'm curious how long that cycle takes if you could tell us like where do you start where you say okay we're going to we're going to do this i mean are we talking yeah a year two years it's when we are on a cycle on every collection we're on a cycle to every 18 months we release a big update of tv collection so once recollection 7 was released 18 months ago we started to work on the recollection 8. but first you know it's looking for and defining what would be the next instrument so it's always a complicated uh we need to to find what would be interesting uh uh and to discuss this what what is feasible what are the the team available or not what are the the skills required um and then once we know that it takes between i would say between six to eight months and then we start the project first we start with the the modeling of course or you're going to say something else yeah and uh and this takes uh it really depends but it takes up to up to one year to to model the the instrument yeah [Music] and uh and once we start to have a good model we start to also to work on the software uh thanks to our internal framework it's now easier and easier to make prototypes and uh so to test what he's doing the dsp we put in it and we put into our qa department to to do some testing during during the developments and and then once we we think we we are good in terms of modeling we start to work on the presets uh it's always a tricky part because if you start too early you you can find issues that can change the sound of the preset so it will require the sound designer to redo the presets but if you start it too late uh you won't have enough time to do the proper sound design so it's always uh complicated wow and when you have uh four instruments in parallel uh plus uh a complete redesign of the uh for instance on the browser the preset browser we changing we have changed the the preset types and styles and how the tags are managed so so we had to retag and uh all the existing presets so it's uh yeah it's a big project it's lots of things to to manage important great i got i have one more question and then i think we could turn it over to the audience for their questions uh where do you where do you feel that plug-in is plug-in development is going so of course you see that with apple there seems to be this move towards uh taking the mobile the mobile ecosystem and maybe merging it or mixing it with the desktop uh yeah what we're doing on the desktop where where do you see plug-in development heading in the future in terms of yeah what do you see that's that's a good question um on the mobile thing uh i really it's it's very complicated you know because there are the the tech companies who want to go somewhere but there is also the the market and the people uh and we cannot force people to to go wherever we want to and you know the mobile platforms are here for maybe i don't know but it's it's almost 10 years now that we have the ipad and anything like that and at the beginning we also it was the future but not 10 years after it's still it's a bit there but it's not where the majority of the market is so where it will come that's that's a good question it's uh i think it's um it's a very mature market the the plug-in uh and i really would like to see some breaking new technology that make things change but it's you know it's it's very difficult to to make this kind of move take for instance the midi the midi is here for 40 years now and we we now have the midi 2.0 coming finally how much time will take the manufacturer to really embrace it and um that's a good question i don't know but i i really would like because we we are off fighting with all you know all this retro compatibility for instance you know vst2 ust2 is we are still forced to support it because of sub companies some important companies are still on it it's it's changing maybe in a couple of years we'll be able to remove the st2 support but it's not yet there so i won't be too much you know i won't do some [Music] hypothesis that things will change a lot in the coming years i i see more in the hardware and software integration and and maybe some you know some some products that are more hybrid and let the the user play with or without the computer thing like that because it's now easier and easier to [Music] to have an embedded platform that can run software that were designed for the computer yeah you can look at your ikea with the force and the ipc no native instruments with the machine plus i think this is more what things will come in the future yeah i do i do have one more question before we before we move to audience questions um what are your thoughts on of course there are new audio development technologies like soul and other languages that are trying to lower lower the barrier for audio programming and what what do you think it would take in order to for a company like arturia to really take a serious look at these other languages these other workflows and say okay this could save us a lot of time a lot of a lot of heartache with c plus plus or do you feel that you're stuck or do you feel that you're kind of married to c plus plus now yeah you know it's it's uh it's a bit like the the assembly then the seed and the c plus plus and uh now our embedded guys are starting to get more and more into c plus plus i i don't think there is any more people at arturia doing c but going to even more modern language the language is like you know yes the soul the soul is very very interested but when when it came up i i thought if a very very big company you know like apple or google or microsoft is uh supporting it and uh it's available in their platform it i think it i thought it was it would be more um [Music] adopted uh you know like uh like uh the cuda for instance in the in the graphic platform um but from smaller company like even the bigger company in the music industry it's we are still in a niche market and it's it's very very difficult for a new language to to come and to to be a de facto standard so yeah do you think it's do you think it's because of the knowledge base of your existing software team or do you think that it would just be the time to the time that it would take to adopt it or do you think that there are things that are missing from the language that you that you see i think it's more on especially for things uh toward an embedded platform it's more on the industrial and the risk that's you know the the long term uh maintainability of the of the thing um what are your thoughts on rust i really would like to to have my team working on rust i think it's uh technically speaking it's great and i will try to push my team to work on it but it's again it's uh it's very difficult to to make the the jump uh once you uh once you know your your tools uh it's i think what it was it's related to what i i told you about uh learning things for for the people for the other single people it's very interesting to to learn these new languages and things like that but for a company it's a much more difficult step to change these kind of things yeah great i think we're ready to move to audience questions thank you very much kevin thank you yeah uh thank you kevin uh yeah we do have a few um audience questions uh i would say quite a few of them are kind of about uh something that you discussed earlier like kind of getting a job hiring this kind of stuff so maybe i'll just i'll just ask those first so uh the summax 80 is asking uh how to get experience to get a job in audio programming i think this is an interesting interesting question because you said earlier you were actually hiring a lot of people who just come out of school yeah but uh or maybe you have like relatively junior junior people but i'm actually wondering this as well like uh how what would you tell someone uh how would they get even acquire the skills to get a job like that like where do you what would you recommend um [Music] i would see first it's uh if you if you are passionate with that i think juice is a very very good uh framework for that because you can learn a lot of things with it and uh from all the aspects of audio plugin development i mean you have the dsp you have all the graphic managing things and you have all the problems uh yes uh ready to multi-thread it and it's like that uh so i i think i i would say start to work with juice it's it's uh it's the best thing to start with uh and and then uh look look um look at the resources if you if you like to to make this uh this kind of things uh look at the resources on the on the internet like the music dsp.org and try try to [Music] to really take the end on it and uh and and then uh make you make your own projects i think when you when you receive a resume uh if you if you see that uh you you have a a girl or a guy that is um doing some open source project uh even if it's not a big things but uh just some some some some some some small things uh it's it's always better to to to sell you to to the recruiter i would say all right yeah uh thanks uh i think that's really good advice um there's an a couple other questions around this kind of area so john rupture's asking i know your role is in software development and i'm really interested in dsp jobs is there someone arturia that would be best to ask questions about dsp specific roles um [Music] yeah definitely uh if you want to uh i um i won't i won't be able to to give you uh on public uh his email address but i if you if you drop me a message or maybe you can we can find a way to um we i think we'll be happy to to give you some insight on on it definitely uh and it's it's it it's uh it's kind of funny um i remember maybe five or six years ago i received an email of a young guy a young student i didn't even know how he got my email address but he he asked me just that that question uh what should i learn to to come to work at tattooia and i answered him so i give i gave him pretty much the same advices i i told you um and so he did that and and a couple and i yes i also gave him um a couple of uh universities and a school that i like i know are good for learning that and a couple of years after after that he he applied as an intern and he gets hired and uh and at the end he he finished being higher at the company wow great so you're saying that a lot of it is about determination definitely definitely that yeah yeah that's what uh that's what i've seen as i mean that's i mean now we know thousands of audio developers and it seems that right away i can tell the ones that are going to get hired that are going to be successful in the career field just from the sheer determination and the work the work ethic and like you were saying the curiosity uh i think you underestimate that so much and this also this is kind of more important than like experience right you can have never worked on like a piece of real audio software before but if you know how to program if you have like an understanding of like roughly how stuff works you can like dig in and figure it out yeah just try it just try it yeah try it and you you will have a lot of fun i mean it's it's it's crazy when you start to do audio programming and you you have your first piece of code working it's such a pleasure to to to get this and um and i mean i think we are all the same uh when we see young people passionated and we we all want to to help them and you know find so giving them advices and uh so yes the advice is the advice i would tell them is just don't hesitate to to contact uh to contact us and uh i think uh you you can find me on the internet somewhere in different forums and things like that and if you ask me directly i will i will uh answer you and give you advices so and i i don't think i'm alone on that there are lots of people like me in this industry yeah um right so very related to that jackson abel asking um whether you currently have internships in engineering uh yes um it's seeing we have done because you know there is some kind of seasoning internships and so it will start in early next year we have done pretty much all our recruitment for the period of february to the summer but again if you if you are motivated and interested don't hesitate to to drop an email at jobsatara.com or and we'll we still look at it and even if it's not that year maybe it will be next year or yeah so um there is a question by elliot bernard asking what is actually your educational background my educational background so yes as i as i said i i went to university in uh so i started with uh i graduated in computer science and then i i went in masters again in the university masters degree in bordeaux university of bordeaux there is a specialty in audio and image processing but again it was very so i have a very generic background uh and the audio and image processing was only in the very last year of my of my um [Music] student so yeah yeah right yeah we have a few more questions so michael kaufman is asking um actually somewhat related to the educational topic so at university we learn a lot about software development process analysis design etc but in real life how do you do this at arturia and i've heard that people for example rarely use uml these days so i guess the question would be how much how far removed is what you learn at university from by about like building software from what's actually going on at a place like arturia yeah yeah it's true it's true that we are not using uml uh as much as what we should i would say but i think it's it's good uh to learn that because it helps you to to reason about a program and to to think about what's what are the yes what are the aim of each uh individual part of your program and to not couple much that much the things and to really to you know the separation of consenting and all that i think that that's why it's important to learn that even if it's true that when you are at your day-to-day job you are not that much doing uml even if it's still very very important to try to document as much as possible what you're doing yeah all right thank you um so jonda is asking can ai be a tool to emulate analog analog sounds so i'm shifting gears a bit we have a couple questions about analog modeling and analog yeah exactly what do you think about ai in that context it uh it can definitely help at the moment at arturia we do not have a running code in the software that are using ai to model stuff but we have some tools some in-house tools that using ai to help us model stuff and i think uh in the upcoming years it will continue to be more and more used wow in in all the aspects even not not only uh modeling electronic modeling cool so that's really interesting um looking forward to like that you know becoming more more widespread because i think it's a technology that a lot of people kind of have maybe heard of but i think in traditional plug-in development isn't necessarily mainstream yet but they're definitely really looking forward to that yeah but there are more and more company uh using it and marketing it and very important company like isotope uh even recently native instruments uh had this ai stuff on the on the gita rate so and uh yeah it's uh i know that there are more and more people more and more research field if you look at the difficult stuff there are more and more papers related to that aesm so yeah it will continue to grow and i think [Music] you were asking what will come in the future i think definitely this is coming and this will continue continue to grow in the coming years all right so um final question from the youtube chat is by george contours and it's a bit of a more general question about analog modeling can you expand on the state of the art for analog modeling in 2020 to be honest i'm not the best person to to talk about that but again if you want to to have some ideas on the state of the art look at the academic papers uh coming from the day fix from the well-known universities like alto university like some very good also in in a uk in italy and uh yeah yeah you got queen mary queen mary in london goldsmiths in london uh ercam of course so yeah quite a few quite a few resources yeah right so then i don't think we have any more questions at the moment so thank you again so much kevin for uh patiently answering all these questions i definitely learned a lot yeah you're welcome thank you thank you again for inviting me and uh yeah that was a first time for me and uh very very interesting and uh yeah very cool thank you yeah thank you very much for your time and and chris as well uh for your time uh with that amazing ide i mean absolutely phenomenal uh this is fantastic meet up and once again thank you thank you to our sponsors uh juice sonics and focusrite for their support and for helping to make the audio programmer meet up possible and we'll be back next tuesday so it's the second tuesday of the month i don't have the date right in front of me 12th of january the 12th of january fantastic um i believe we are still looking for some speakers for that next meet up and uh one of the most important things is that you don't have to have the most experience uh to actually present at the meetup even if you have one of your first real projects that you would like to show us and share with the audio programmer community we'd love to see it uh and it's a great way of course to show people your skill set and show people what you're up to you can actually submit a proposal over at theaudioprogrammer.com forward slash submit also be sure to join our audio programmer community on the audioprogrammer.com forward slash community and uh yeah is there anything else that we need to mention yeah this last one for the before the holidays so yeah we wish everyone a merry christmas and a happy new year and i hope to see you again in in yeah and yeah it's been uh obviously a challenging year for for many of us uh obviously because of what's going on in the world right now but i think i'm actually really hopeful that next year is going to be better and um yeah really looking forward to uh yeah carry on with this meetup and have exciting new talks and q a sessions and meet exciting new people here so um yeah i think it's going to be awesome so thank you again so much for joining us especially those of you who tune in every month i saw on the youtube chat you know that some names they just keep you know keep repeating so people are actually coming back so i'm super excited about that i'm just thank you so much for uh for you know listening to what you have to say and to our guests and it's it's really a huge pleasure and huge honor every month so yeah yeah it's definitely a huge honor and uh we wish everybody a safe holiday season and we of course will be continuing even through the holidays to be talking about all things audio programming on the discord and the tutorials so uh be sure to keep up with us and yes we will uh with that we'll sign off and thank you and see you next time new year thank you guys bye bye

Channel: The Audio Programmer

Views: 1,088

Keywords: audio programming, audio programmer, audio programmer meetup, c++, arturia, plugin, vst plug-in, coding, audio developer, audio software developer, digital signal processing, dsp

Id: l8rJGFW68tk

Length: 156min 15sec (9375 seconds)

Published: Tue Dec 08 2020