ElixirConf 2021 - John Palgut - Printer Perfect: A Case Study in Building a 3D printer UI

[Music] [Applause] [Music] [Applause] [Music] so welcome to my talk uh this stock is called printer perfect a case study in building a 3d printer ui with nerves in live view uh and so a little bit before we get started we're going to talk about who i am my name is john palgut i'm a software developer with about 10 years of professional experience and luckily for me in my last four or so years i've been able to work in elixir um one thing i'm always kind of tinkering with stuff so i go home and i write software and i do these side projects and i build things with raspberry pi's and uh you know just all sorts of tinkering and how does this work and can i take it apart and so for me like 3d printing is kind of a tinker stream um you know printer 3d printers there's open source hardware there's open source firmware there's open source control software you can sort of print new pieces for your printer to kind of customize it in ways that you know other people haven't thought of it's just a super cool like tinkering um hobby and so one of the pieces of software i use with my 3d printer is called octoprint and octoprint is a printer host which means it's kind of responsible for talking to a printer and you know sending print jobs and keeping track of the status of those print shops and you can move the printer around or send commands or do various debugging things and octoprint's like super super good it's a tons of features and an extensible plug-in system and it does way more than i'd ever needed to that little tinker part of the back of my brain kind of says hey could i write a 3d printer host with elixir and so i set out to do that for this talk and before i got started i sat down and i came up with some goals um things that you know i felt like a 3d printer host would need to have in order to me feel good about you know something that i would um and i've ordered these goals on this slide and kind of uh you know like the ones i consider most necessary to the ones that are sort of nice to have being at the bottom and you may be seeing john uh it looks like printing is like second to last like shouldn't that one be up at the top like printing is pretty important for 3d printer um and so for me i've kind of thought of some of these things as sort of you know they're the core pieces and it's like learning to walk before you crawl i wasn't necessarily sure that i could accomplish you know printing with the 3d printer and i wanted to make sure i got the crawling steps done before i started stumbling around everywhere so what tools did i choose when choosing my tools i sort of took a look at how octoprinted stuff because octoprint is open source and very popular so octoprint is written in python um and i'm a little bit biased but i think that anything python can do elixir can probably do just as well if not better so we're covered there the hardware i wanted to run this on a raspberry pi which is connected to my printer by a usb cable and so luckily for me nerves is really great at getting your elixir projects up and running on the raspberry pi similarly octoprint uses a library called pi serial it turns out the s and usb actually stands for serial and so this pi serial library allows octoprint to talk to the 3d printer and for us there's a library called circuits uart which is part of the nerves family of libraries um and so we can use that to talk to the printer over here and then finally um octoprint has kind of a real-time web ui built with flask and some client-side javascript and we can sort of throw out the javascript part and just rely on phoenix and live view to get things done so where did i start um if you've never written a nerves application before one of the things that they recommend you do is structuring your pot showing what they call a project in what they call a poncho project and so essentially a poncho project is a bunch of standalone elixir applications in a single directory and then there's a firmware application that is your nerves application and it pulls those other applications as dependencies and it starts them up and it bootstraps the hardware and gets you going and the way they kind of recommend that you do this is it allows you to develop and test these projects separately so for example i could develop and test and iterate on the ui all the way on my desktop and then when it's time to put it on the printer i don't really have to do anything because i've already tested it on my desktop and so speaking a little bit more about the firmware application um this was the part of the application i touched the least um i basically ran you know mixednerves.new i told it a little bit about my wi-fi information i told it how to pull in my other applications and that was it it just kind of worked um it was a lot more straightforward and really easy than i thought it would be so that was pretty cool and then the next application is the printer application and this is just a normal elixir application and created with you know the mix new supervisor flag and the way i like to think about this application is it's like a library for all things dealing with my 3d printer you know if i want to move the extruder there's an api call for that if i want to heat it up there's an api call for that if i want to get what's the status of my product there's an api call for that and so this one's much more complex and most of the development actually went into this application and so i went over a couple different uh possible application architectures and the one i settled on is this i've got a named singular gen server called the printer server and the printer server is one responsible for sort of handling api requests and keeping track of the state of the printer and then it also spawns these connection servers which are gen servers that manage a connection and in this definition a connection is the piece of code that's going to actually do the sending of commands to the printer over usb and so one nervous tip i've sort of come across as i've done nerf's project is mark you may be saying john isn't that kind of a whole point of a nerves project is working with hardware why would we want to mock that out um and the answer's a little complicated but basically developing against hardware is slow if i want to you know move a new element on my web page to the right a little bit and i'm developing on the hardware i have to change the code i have to burn it to an sd card i have to plug that into my raspberry pi i have to wait for the raspberry pi to turn on i might have to reset my printer and then oh it turns out i moved it too far i've got to move it back um so if you can mark that out and test it on your desktop things will go much better additionally testing is almost impossible like let's say i wanted to write a test that says you know move the extruder 10 centimeters to the left um if i'm running tests against the hardware there's no way to really assert that the extruder moved in real life i'd have to kind of like stand there and look at my printer every time we're in the test screen and it's just not realistic um and so most of the time when we talk about mocking an elixir uh people often use a behavior but in this case uh the connections that we're marking out i thought they would have state they'd have a bit of state to each of them and so you'd be passing that connection to behavior function calls every single time you're calling them and to me a protocol seems a much better fit for sort of this mocking out of the connections so why mock this connection exactly um and when i like to think about mocking things out in an application i like to think about what what's at the edge of my application what's the part right at the edge of system that's going to be talking to those external systems so you can kind of think about you know if you've ever written tests before and you're making remote calls to an api in production you don't necessarily want to make those api calls in tests because what if the server goes down then all your tests are going to fail or if they update something and you're not ready for it you know your tests are going to fail so it's all about getting that sort of reliability at the edge of your application um and again in different environments they kind of have different needs right so in production we're going to want to talk to a real 3d printer that's what our connection is going to do but in test we maybe want to replicate error scenarios or check that correct messages are sent and development you know it'd be ideal is if we could mimic a real 3d printer as closely as possible so we're going to talk about a little bit um a little bit about the different connections that i built to work with this project and the first up is the serial connection so this is our production connection this is the one that actually uses the circuits uart library to send messages to the printer over usb and when i was designing the connection protocol i took a look at the secret ur api because i knew that that at the end of the day that was the one that was the most important and that was the one we wanted to mimic closely um and so a lot of the serial connection implementations of the protocol are really really boring because they're just calls to the underlying library and so you can see here this is the send implementation for the serial connection and it's almost an exact direct pass-through to the uart library we're just writing a message to the serial connection and again you know there's other functions like open and close for this protocol and they're all very straightforward just pretty much pass-through calls to the uart library so one of the other connections that's a little more interesting i think is the in-memory connection and so this is one of the connections i used for tests um and it keeps kind of a history of function calls so if you call send it's going to record that if you called open it's going to record that and so taking a look at the implementation of send for the in-memory connection um we can see that instead of sending a message by the circuits uart library we're actually updating an agent at a pid right and all we're doing you know in this function is we're saying hey um somebody called send with this message and we're adding it to the list of messages that we already have received and that allows us to write tests like this where i can say hey you know printer i want you to heat up to 60 degrees celsius right and kind of like you get you know connections in your phoenix controller test you can say hey what was the last message sent on you know my in-memory connection and it'll go into that agent and i'll pull out the last message and says oh you know it was send and it matched this value and so it makes writing those tests where you know like i don't want to look at my primary when i'm running the test it makes those pretty straightforward because the edge of the application is kind of marked up and so the last connection i want to talk about and the one that i think is the most interesting is the virtual connection and so this is the connection um i used for development and so uh when i was talking before about like the needs of the different environments you know in development i mentioned it'd be pretty cool if we could like talk to a virtual printer without talking to a real printer because the real printer is kind of a little bit cumbersome um and it turns out octoprint already thought of this so they have a virtual printer python class which is essentially just a bunch of python code that mimics a 3d printer firmware it's hundreds of line longs and super complex but you can send it messages and it'll send back responses that are basically how a printer would respond and so while that's written in python elixir and airline has this feature called ports which allows you to spawn an external process and sort of manage that process and send messages to it via standard in and out and so i went one level deeper and found this library called erl port which is a wrapper around you know the elixir airling port library um and it allows you to spawn python processes directly in a much more friendlier way you can say hey i have all my scripts in this folder can you load those into the python runtime and on the python side they've written a bunch of python code that allows you to pass messages back to your elixir code or your ailing code as though they were you know you were in elixir itself and so we'll take a look at that so here's the send implementation for the virtual connection again we've got a pid and so you see we've got this python atom and that's actually the erl port library and what this is saying is we're saying hey there's a python process at that pid um and that python process has a virtual script and in that python virtual script there's a function called write and i want you to pass these arguments to that function this is very similar to like if you've ever seen the module function arguments calls that are sort of all sprinkled around elixir and early and then on the python side it's going to get that message it's going to send it to the virtual printer it's gonna process it it's gonna get the response and then we want it to send it back to us and ideally it would be sent back to our printer server process you know like we would do an elixir like this top line you know process.send you've got a pid we've got a tag two pool with some data it's great and so fortunately for us that url port library they've written python code that does exactly that so we have this function called cast it takes a pid we're passing the python version of a tuple and you can build atoms in in python with the aeroport library and we're just sending back some data so these two lines are functionally equivalent and but it makes working on the python side super interesting because you can just treat it like it was elixir and you're sending messages to processes and so it made for a really nice development environment so that's the connection so now we're going to talk a little bit more about the printer server and so the printer server one of the main things that it does is it sort of powers um the api and it keeps track of the state of what is the printer actually doing are we are we connected to somebody are there messages a message to you are we printing you know what's the temperature things like that um it's also responsible for sort of parsing messages from the printer and figuring out what to do next and so going in um i'm sort of daunted by this printing so i put it towards the bottom and i was like printing is gonna be really difficult i i have that feeling and one of the reasons i thought this was um sort of every 3d printer firmware is just different there's hundreds of g-code commands um and there's a wiki that sort of lists out what the commands are and what parameters they take and and and it has tables like this listing you know does the former support this command does it not support this command who knows if it supports this command so i saw when i started digging into g code this kind of scared me i was like oh man it's going to be really hard to send commands while we're on the subject of g-code g-code is the sort of command format that you use to send messages to a 3d printer um so this is an example of the g-code command the bit on the left in gray is the line number and essentially when you send a command to a printer it expects the next command to be one line number above it so for example if we sent this n1 command the next command i would expect to be n2 n3 and so on and if you kind of send commands out of order it's going to complain and ask you to resend a specific line number the green bit in the middle is the actual command telling the printer what to do so in this case a g0 command is a move command and it's saying hey printer move the extruder on the x-axis 12 units and then that orange bit at the end is a checksum so when you build this command we take the bits from the line number and the actual command but itself and we run some bitwise operations on them and we come up with this checksum and we send that with the command and the printer firmware is going to do the same thing and if the checksums don't match it knows that there was some data corruption on the line and so it's going to ask you to maybe resend the command over again so it turns out once i got down to actually sending the commands um parsing the responses was difficult even if sending the commands wasn't and so let's talk about why that is so one of the most common responses that can come back from the printer is just the string okay and okay means i received the last commands you sent um sometimes okay comes with extra data which is kind of weird for the person you can't just parse for okay you have to check if there's also extra data and maybe parse that too um and then helpfully it doesn't actually include the line number that it's acknowledging it just says hey i got that last command so you have to keep track of not only the commands you sent but what order you sent them in in order to kind of acknowledge them off the stack another message that you can get back from the printer is a resending of a specific line um and so this would be like hey you know please reset line 100. but unfortunately while okay is just like a singular command uh resend a command is actually three different lines so it's like an error line a line with the number to reset and then the ok command again so now not only do we have to you know keep track of these okays you have to keep track of when um sorry excuse me you have to keep track of you know the last couple of responses you received from the printer just in case they actually form one like big meta response and so here's a couple examples of like some response lines you could receive from a printer the top two lines are temperature status lines that contain information about like what the temperature the printer's at um and that first line is from the virtual printer um and so that's kind of the one that i had built the parser against and tested against and that second line um is from my actual printer and for some reason it has like some white space at the beginning um which is not part of the spec and not listed anywhere and it completely blew up my parser when i plugged in the printer i was like why isn't the temperature data coming through uh so it's just an example of like some of these weird formatting errors that you can run to and then below that we've got two okay lines and one of those has just some temperature data that's kind of similar to the stuff up top so it became apparent that like parsing was kind of the real problem um i tried to come up with some good solutions and i came up with a few and the first two that sort of popped in my head were regular expressions and sort of pattern matching and i played around with these a bit and basically i found that they were kind of either too inflexible or a little too hard to read or debug and they just kind of weren't what i was looking for so i dug a little bit deeper and i found this library called nimble parsec and so nimble parsec is a library that allows you to build text parsers um that are easy to combine with other text parsers so it's easier we're going to take a look at an example i think i'll make it a little more clear so this is a parser for extruder temperature and we're saying hey extrude temperature it starts with the string t colon and we don't actually care about that part we care about the floating point value that comes after that and similarly here's a bed temperature parser you know it starts with b colon and we don't actually care about that part so you can ignore it but we do care about the float that comes after that so we're going to return that and you can build small parsers like this and combine them into a bigger parser that matches sort of that okay temperature string that we saw earlier where you can say hey you know the string starts with okay and a space and we don't care about that but we do care about the extruder temperature you know and we don't care about it in the middle and we do care about the bed temperature at the end so i found this approach to be the right combination of easy to read and understand and debug and it's also fairly performant because it actually compiles down to pattern matching um when when you build the library so that was pretty cool and it turns out i was kind of wrong about the printing thing printing was actually pretty simple um and that comes down to the way that the printer application kind of sends messages so the way messages are sent is somebody will call an api or make an api call and say hey you should a printer i want you to move or i want you to heat up um and if we're not in the middle of sending a command that command gets sent right away otherwise it gets put into a message queue and so the printer you know sends a command it waits for a response it processes that response and then if there's more messages in the queue it just repeats it all over again until the key was empty and then we're just kind of waiting for someone to send an api call and this actually kind of unintentionally mimics printing except instead of a queue uh when you're printing a file it's just a file full of g-code commands so instead of reading from the queue you can say send a message from the file wait for the response process that response and if everything went well you just read the next message from the file um and so that was kind of cool it's like oh this just kind of works that's great and so now we're on to the last application and that's the ui application and this is our phoenix live view where you know we've got our ui and it depends on our printer application and sort of uses that like an api um i expected this one to be a little bit challenging too and it was honestly like the nerves one where it was really really straightforward and most of the work was kind of in the actual printer application so here we have um a mount for that component that we saw on the previous screen you know and if we're connected we want to subscribe to updates from the printer it's going to send us a status of what's going on you know if it's heating up or moving or something and then additionally we're going to grab some printer we're going to grab the current status and we're going to assign it to a printer the printer assign and we're going to grab the currently available connections you know just in case we want to connect to one of those and then messages come in just like you would expect you know normal phoenix pub sub messages to come in because that's what the printer application is using under the hood you know there's just a status dock that's broadcast to anybody who's listening you get it on a handle info and we just update the assigns and and that causes the ui to re-render like your normal live view so again that status component like it's just pulling that status out and just displaying it on stream it was very very straightforward you know just reading values from the thing um a little bit trickier was sort of the move buttons but again that wound up being also almost pretty straightforward because you define a component like this where we're passing you know the amount we want to move and what access we want to move on and it's sort of rendering it and when somebody clicks on the button it fires a move event and getting down to implementing that move event um i was just calling these apis i was like you know this this is really simple like the printer api exists we just kind of pass the parameters on through to the printer api i was like is this going to work is this really going to work um and the answer is yeah it kind of just worked i plugged it into my printer um and it did its thing so revisiting those goals i sent for myself at the beginning of the project um the ones in green i would say success completely done uh great unfortunately um i discovered a bug in the printing code the other day where the firmware thinks that we are one command line behind where the application thinks we are and it's re it's asking the firm or it's asking the printer application to resend lines over and over and over again um and it causes like the stuttering and back and forth and it took me a while to figure out what's going on but i'm confident with time that um it's a pretty simple fix i think it's just an off by one or a race condition somewhere and unfortunately i just didn't get up to file uploading in time so what went well writing this project um honestly it felt good it felt like building a normal elixir application the pancho structure just sort of let me not worry too much about nerves and not worry too much about live view or phoenix and just like do the thing i was normally doing i was just writing elixir code additionally um i had done some nerves projects before and it feels like nerves has really grown up before um when you did like mixed nerves that new a lot of things were like kind of piecemeal you'd have to say oh i want to connect to a wi-fi network so i have to pull in this library or i want to do a firmware upload so i have to pull in this library and configure this thing um but all of those libraries they're still there but now they're included with the mixed nerves that new uh generator and it's much more seamless and pleasant experience um so i wasn't happily surprised about that and i feel different connections were the right way to go um when i started this project right after i started this project my printer broke um so i spent 90 of the time developing on this project against the virtual printer plugin and i was able to write code without having a real 3d printer while i fixed my 3d printer and then was able to plug it in and start running commands um and it pretty much just worked outside of a couple parsing errors and that was an excellent feeling and i think it's a testament to sort of that mocking strategy and nerves and you know being able to you know you don't always have to have access to the hardware that you're developing against if you have a good sort of foundation what didn't go so well um i was honestly surprised by the lack of like a singular 3d printing spec um it's wild to me that like you know every firmware just kind of does whatever it's going to do and sends its own responses um and like i said you know i was developing it's a virtual printer plugin and then i plugged it into my real printer and you know all of a sudden there's just random spaces here and there and so there's a bit of guess and check um and as i'm kind of digging around into this on the internet i find a lot of other developers who are very um not pleased with the same sort of thing also i feel like response parsing could really be improved um currently the connection is sort of just passing back the raw data that it gets from the printer and i feel like it might be better if the connection actually parsed the data itself and invited it excuse me emitted events like okay or please resend this line and that would simplify you know the printer server area and maybe make areas and maybe make bugs like this you know printing off by one not so much of a thing and then also it's just a bit of a humbling experience you're kind of never too old for off by one errors and so that's my talk um if you want to check out the code it's up on github at gw sonic cliffswall um if you're into twitter you can follow me on twitter um and then i'll answer questions while i play this video of my printer just kind of moving around assuming that it works do we have any questions oh there we go you muted richie hey congratulations on the talk i'm looking at the chat um it looks like we don't have questions yet hopefully we can have some of them soon and you got a lot of circles yeah but you got really nice comments people enjoying the talk nice work glad to hear that thank you very nice talk and bold statements you made durian talk so so let's see if we get a question and if not i'll thank you in advance for the talk enjoy doing um will will you be printing the elixir droplet once your final issues are resolved yeah so that was what i was um that was the first thing i thought of to be quite honest i wanted to print a lexicon thing and have it cut in the background unfortunately didn't get there um and when i looked into it there's actually no i couldn't find a good 3d model of like the elixir logo or a droplet um so if somebody has one of those or finds one of those or makes one of those feel free to shoot it to me on twitter and i'll print it once once i can there's another how does this project compare to the other ones you've done in nerves um this so the in terms of the like one of the other projects i did was uh i had a cat feeder that had some motors and it turned and it broke um and i pulled out the control board and replaced it with raspberry pi and hooked it up to the motors and so that one was very similar in that it's sort of the poncho architecture with a web ui um controlling some hardware um but the actual content was kind of different right it's it's not as complex as a 3d printer you press the button or at a certain time it moved the motors and stuff so similar like that poncho application is pretty um i find it pretty flexible and pretty pleasant to work with actually over like an umbrella application but um yeah in terms of the content they're pretty different do you have phoenix in the picture to have live view um [Music] i'm not sure i understand that question i think you're saying do i have phoenix so i can use live view and if you're asking yeah i think it doesn't plan yeah the dead views or whatever as i guess they're called um nothing really in the day it was just all kind of live view there's a singular sort of live route and um no real plans for kind of dead view stuff because um this is i think one of the more ideal environments for live view because it's running on a raspberry pi in my house on my own network um and so i'm always going to be connected right like you're always going to have a connection that server it's a very fast connection so um there's a new one do you plan to integrate it with a webcam with a webcam for time lapse um so that would be like i think a stretch stretch goal um that's something octoprint does and um i think they have issues with the raspberry pi webcam and throughput um and so it'd be interesting to tackle that uh in elixir but when you get into like dealing with the raspberry pi webcam it's a little weird it's like a bunch of series of images and so um i've thought about it but i i don't have a concrete plan on how to move forward there yet cool i think that's all the questions we have right now all right thank you so much for hosting thank you so much john bye you

Channel: ElixirConf

Views: 271

Keywords: elixir

Id: jxiZ_K39-dw

Length: 28min 25sec (1705 seconds)

Published: Sun Oct 24 2021