ElixirConf 2021 - Desmond Bowe - We Wrote An Entire Single-Page App in LiveView

[Music] [Music] hi everyone my name is desmond bowie we built a single page app in live view i'm here to tell you about it a little bit about me uh that's me in uniform with my hair and mustache uh sophie said i work as a cto pay it off i founded a conference called mpex which happens in new york la and maybe somewhere else soon i'm also a coach to the elixir talk podcast with one and only british chris bell he's right there and i've been getting a lot of questions from podcast listeners we haven't done an episode in a while and i've got a lot of concerned questions people who've been following the show regularly uh i know are just dying for updates on what's going on so i'm here to answer all of your questions about my truck i know you've all been worried about what's going on with this thing well there's been a lot of progress i've torn it apart painted the frame you can see the dashboard is coming together wiring is coming in i'm going to start a conference called truck pex conf change my handle doesn't monster truck and then maybe start a new podcast the truck talk podcast all right all right enough for the jokes back to the talk so um we're here to look at what we did at my job uh building a single page app in live view there's four pieces to the talk a background in both live view and our company and what was our use case why did we choose live view we're also going to look at the architecture that we went with there's clients art architecture and server side architecture issues faced and lessons learned uh one interesting use case about this is that it's not just a single page app that we built and deploy on our site this is the thing that goes on to other people's websites so we don't actually run this it adds an extra layer of complexity and interesting challenges which we'll see a little later so part one there's two pieces of background i want to cover one is in live view itself and the second is in our use case you can understand a bit about our product and and how we got here so what's live view live view is an elixir library that lets us build easy dynamic web experiences via websockets without having to write any javascript who here does not know what live view is all right well cool spend any more time on this slide we're all jazzed about live view i think we've seen a lot of the cool things that it can do games dynamic tables whatever twitter clone but really the question i think we're all asking ourselves is how far can this library take us what's the boundary of it a couple years ago chris mccord did a very cool demo where he piped live video through a live view and showed that off and it was cool it worked but even at the time he said this is not really a good use case for it so we're still left wondering what is the final frontier get it william shatner now just went in outer space final frontier i was worried someone's gonna go wrong with that but if we can show that live view works effectively in a single page app then i think that breaks down the last barrier for using this library everywhere single page apps have been the vanguard of front end or client side or just user experiences for the last several years they've been dominated by very well-known frameworks um and in comparison live view has been seen as a toy so if we can show up to that party and hang with the rest of them then we'll have earned our seat at the table and there's no reason why we cannot use live view for everything else so in my personal view i think live view is the future um as chris mccord said in a podcast recently dead views are dead i think we're only going to be seeing more and more of this i think the work that's going into it is going to enable a lot of cool stuff and in three years everything's going to be live view but today people still ask us well why not just use react uh react has been around for several years very mature huge community backed by a very large company uh can do all this stuff so why like why go in for this new thing similar tools already exist we're not exactly breaking ground with live view but there's a couple of reasons why i'm not interested in using react the first is listen to the podcast know my feelings about javascript there's the additional overhead of now i have to deploy two apps you have your front end app going out what if you change the api now you have to coordinate your deploys with the back end app make sure things are compatible this gets to be quite a big headache which a lot of us are familiar with another issue that i face as a manager is okay do i want to introduce a new uh a new technology into my tool chain our company started off as an api company i hired a bunch of backend engineers and we don't really know how to write javascript we sort of know how to write javascript but react is a specialty people have spent a lot of time working on this and we didn't have those skill sets it's a wholly new tool chain there's a new build process how do you get these things working uh in development so that they're talking to each other that takes a bit of work to say nothing of the deployment that i mentioned and it's a different environment to deploy in now i have to instrument a different type of application now i have to deal with exceptions out of a wholly different type of application so from a strategic point of view it's not as simple as oh yeah we'll just use this common commonly used off-the-shelf thing this has huge ramifications for my company and what we what kind of products we're able to deliver um but apparently react is coming for us on the server so maybe i i should be concerned about that watch my back but um i also want to talk a bit about our use case so what what brought us to using this i mean of course we're all elixir enthusiasts at our company ground up elixir app i've been using elixir for six years other people have been using it for longer i think at the company so we were kind of biased in this direction but where did we come from so my company pay it off where i'm the cto we build student loan solutions and our customers are fintechs and financial institutions we don't work with borrowers directly we're b to b to c so we sell an api as i mentioned that takes intelligence and guidance and payments and loan aggregated loan information services it through an api so other companies can build their student loan experiences into their app and meet their customers where they're at so what we saw was that this api was sort of challenging to work with i mean it's an api but you have to know a lot about student loans in order to build an effective experience so there was a lot of pressure to make integration easier and what we ended up landing on was this idea of a drop-in widget with a couple lines of javascript you could be up and running with a hosted uh solution from us but it's still inside the client's experience and that's going to be key to like this whole it's sort of a single page app um with some nuance to it so the widget we call nexus that's how i refer to it uh the user experience is a series of workflows it's not quite a single page app in terms of you just freely navigate to wherever um like a regular website uh it was originally built as a series of the borrower wants to do something so we're going to step them through this thing it's kind of a it's a wizard basically and there's a couple of these things which we call workflows uh the next iteration of the product breaks that down the reason it came up this way was just knitter and iterative this is what we want to do oh well now we want to do this other thing and now we want to do this other thing so that's how we ended up where we did but i said as i said the key difference is that this app does not live on our website it lives on our customers websites we call our customers partners i might refer to them that way so a little bit about one of these workflows we're going to look at an example use case to see how this plays out in our system the first thing we want to do is link servicers these are loan servicers that hold the borrower's loan information before we can do anything we need to figure out what kind of loans a borrower has and so we built an aggregation infrastructure to go out log into these loan servicers websites we reverse engineer the apis and we scrape the loan data from them ingest it into our system run it through our recommendation engine cool as part of this linking process we need to gather username and password we also need to answer challenge questions so now we have to go back and forth with the uh the servicer to see what are they asking us in order for us to get in what does this look like here's our user experience so um simplified a little bit but we proceed from left to right the borrower sees a list of all possible loan servicers they choose their ones from the list they put in their username and password they wait for a minute and then they uh put in their challenge questions in the next screen challenge questions can be dynamic we can't just drop a form set of challenge questions in here and be dumb about it because not only can challenge questions be different from servicer to servicer they'll be different on the servicer if i come back tomorrow they're rotating through my challenge questions to make sure it's me so now we have our scraping infrastructure that has to communicate with our widget so you can see where this is going with message sending and and otp uh and then while it's uh thinking doing its work on the the back end on this loan servicer we see a waiting screen okay cool so we're going to look at how this ends up getting built out on the back end all right time for architecture so as i promised in the description we're going to look at some code we're going to start with the client-side integration we're going to look at some of the server-side architecture and then various sundries things that just like didn't fall into one category the other that i wanted to talk and we call it client-side integration because it's not as simple as just spinning up a live view we have to deploy some javascript somewhere that creates a creates an iframe in their dom that then connects to our server and begins the live view experience so let's look at how this works from the client's point of view the partner client partner customer whatever they have to choose which workflow they want the borrower to go through we will be looking at the link workflow there's a couple of others for assessing their current situation maybe enrolling them in a plan making payments and so forth they also have to provide their authorization token as well as a borrower id who's the person we're looking at before they spin this up they have to create the borrower on our server via the api so we know about we know who the borrower is and then they're going to want to provide some callback functions so that they can deal with the data that comes back did it work cool what's the loan data did it not work all right what's the error message i want to show to the user and then as i mentioned that's those those are the things that the partner has to do but then we pay it off has to have to deal with the mechanics of creating an iframe in their dom so here's what this looks like at the top of the page the partner has to source this widget from our cdn this is not the live view javascript these are not javascript hooks this is our custom little javascript library that's going to be used to bootstrap the live view it makes available this nexus object with several functions each function corresponds to one of the workflows we're in the link workflow so we call nexus.link we pass in nexus key which is their auth token a borrower id so we know who it is we're dealing with and as i mentioned these two callbacks for on success and on error this uh this link function is generally triggered when a bar clicks a button but it could happen on page load that's cool too inside of link so this is our code our nexus widget js we have our link function we set some um we do some setup doing our current workflow we store the borrower id and that and that crap and then we attach ourselves to the dom so we create a couple of elements on the shadow room the first is style a style tag so that we can style our next tag this div in order to put this modal i say modal it's not an actual modal but it's modal s angle center of the screen there's a nice gray background on the rest of the screen it's centered depending on the viewport size it fits in quite nicely and it's got a certain uh set of dimensions to it so we do all that in our style tag then we have our div inside the div is an iframe the iframe load source which you can see here is widget url this widget url is a url on our server so this is when it hits payout.io whatever which loads the live view experience widget url also encodes um the auth token and the borrower id and the um domain that this thing is being running on into the params so that we can uh extract that and figure out what's going on and we we want to know what domain this is running on because all this stuff is in the page so if someone were to just view source they could pull out our js bootstrapper they could look at the tokens and i mean the tokens expire but someone could try to run this on their own so we white list the domains that this is allowed to run on again not your typical single page app so this iframe is now connecting to pay it off server and it's received by our router we have a pipeline for our widget called nexus where we run a csp plug which i'll talk about in a second and it's received in the scope nexus inside of a live session with this live route slash encoded slash workflow so what it ends up looking like is slash nexus base64 encoded data slash link link is our workflow that we're talking about back to this plug we have to set uh csp headers because we're running third party javascript on somebody else's page and browsers do not like when you do this so this was a big pain in the butt to have to work around which you wouldn't have to do if you were running a normal single page app on your site but we did in our use case so the solution ended up being not that difficult once you figured out what it was we set a we set a content security policy on the response headers um and then that's fine thus concludes our client-side integration piece of this and now we will look at how is the server-side architecture set up so we left um no i won't start with that let's look at how this is set up it's generally architected as a single live view that re-renders templates so we saw the different screens before each of those screens is a different template the overall box is a single live view this confers some advantages and some disadvantages advantages include we have one receiver for all events clicking the submit button on each of these forms sends events to just this one live view so we all know what's going to receive it it also receives all the events from that scraping infrastructure for hey i have a challenge question hey i had an error getting data hey i successfully got it stop showing the waiting screen it's also one place to handle authentication the live view talk yesterday talked about what happens if i forget to add my auth code into all my different live views then i'm out of luck and yes you can be we don't have that issue because there's just one place where this goes and this this one live view it has help it's got a couple other files one for coordinating the routing which we will look at and then some workflow specific helper files for validations and building these response objects as well here's a loose diagram of it um remember from our router we get transported into this widget live live view this is the thing that's going to do all the work it's not a small file it handles state management it's got the mount function it sets up everything it has to handle state for all the different workflows so the state object is a little hairy i thought about putting it on a slide here and then i decided let's keep it pg-13 it's also responsible for rendering it's got the render function um and as i mentioned it handles all the events that come in it coordinates with this steps module uh which is sort of the router it's essentially a state machine it looks at what's the screen that i'm currently on what happens if i go if i want to advance more challengingly what happens if i want to go back there are some screens that we skip over if i have been over a waiting screen and i hit back i don't want to see that waiting screen again so there's a lot of exceptions there and then there are workflow files you'll notice this doesn't say workflow.ex the actual file would be say link.ex link is our workflow and those are helpers form validations and as i said they generate responses for callbacks uh the question came up yesterday about how do we okay so how do we maintain state what happens if someone comes back into this how do we pick up where we left off uh our widget doesn't have urls and one it doesn't have urls two it doesn't have urls that are exposed to the user again this is inside somebody else's experience and so what we the approach that we took is upon each page transition we save the data and then update it in our thing what is also nice about this is if a borrower gets halfway through they x out of it and they may not just x out of it they may close the window and come back later so we can't rely on you know fun jiggery pokery to like serialize a change set or i suppose we could store it in uh like a change session in redis or something but it was just easier to put all this stuff in the database and then load it again when we came back so then we can advance along back to where we left off so what does this widget live file look like it looks like a lot we're going to just look at a couple of chunks of it starting with initialization so i left off the giant state object but we do do a number of things with this to get the thing up and running we have to decode the params okay which bar were we talking about what's our auth token what's the workflow that was requested we set partner we set the borrower is the domain okay is it on our white list preconditions is this person allowed to go through this workflow what services have they already been through are we doing a demo and then workflow set is it calls into one of those workflow files for additional like workflow specific setup how about routing let's see what this looks like so on the left we have a snippet from our steps.ex file this is also pretty large because this handles all possible screens across all possible workflows so basic state machine we can see this we're pattern matching on the state that comes in again there's just one state for the whole session we look at the workflow and we say okay if we're in the link workflow and we are on the step called login and someone hits back take us to the list servicers page um if we're on the challenge screen also take us back to the list services page don't take us there but set this is where we are supposed to go and that is handled by the code on the right our render function now we're back in the original widget live the central live view which holds the render here we have a heax template our widget view render step we pass it assigns i don't show that code here but you can figure out how it looks at what this atom is for the step find the split and then renders it we've just started migrating all of our templates to well not all of our templates but we just started integrating surface which has made things a lot easier and trying to use more live components actually not really but i'll talk about that a bit more later and then the other main piece that this widget live handles is um events so we have two events on the screen one is a submit event from a form someone clicks the answer challenge button so they've put in their challenge answers they hit okay send it in it receives that and then fires the thing off to this challenge module to send it off to the back end and start waiting and then the second handle info uh this matches on a tuple coming in from pegasus pegasus is this uh our subsystem that goes out to the servicer's website so if it receives a message from that system and says hey the job that i wanted to do is done then live you say all right well how'd it go no errors then we're successful toast party time if there are errors then do something else so it is nice to have one live view that gets everything sure the file gets kind of long but if you want to know what's listening this one thing is listening if we were to transition between live views suppose a message comes in a little late i mean yeah you can drop it and that's probably what we'll do when we re-architect this but it's still nice to have everything in one place and just kind of scan through all of our handle info functions i have that getting there so what's the last piece of code that we're going to look at very sundry so this falls into the category of miscellaneous stuff that i wanted to show you all or things that i wanted to talk about and i'm not good at having like structured trees in my talks so this is my bag of tricks the first thing is about deployment what's cool about setting up your app this way without using reactor view or something is that deployment is super simple because we have nothing to deploy we make changes to our live view changes to our experience if we add a new screen if we reorganize the forms if we do any of this stuff there's nothing to deploy i mean there's just our one app to deploy so the screens go out along with the new back-end code that supports it everything happens at once very easy very simple we will deploy the js wrapper separately to our cdn that has its own life cycle but we only need to update that if we're adding a new public function any workflow if we're changing the way we do authentication or something but changing it in a destructive way that happens pretty rarely and so it's pretty easy to it's it's easy enough to manage deploying of those two separate things so i got to give a big thumbs up for uh writing your front-end experience with live view um and then there's some other miscellaneous javascripts so we did write a number of js hooks a number there's like two um the main one is for masking a sensitive input so we receive borrowers social security numbers and we basically had to write a custom input where we showed the digits until someone blurred the input they clicked out of it and then it would mask them because if you're trying to type in a social and you just see a bunch of dots you're like did i type in seven or eight i mean i think i got all of them what if i missed it and then how do i know i typed in the right thing you don't have that visual confirmation of it but we also didn't want to just show people socials on the screen because they get a little worried about is this safe so a little custom javascript there it also puts in the hyphens um and our talented engineer bob argued a lot with okay what if i advance further then put in a hyphen bubble mode if i'm deleting it then do i take it out do i leave it in i know that was a big piece of work that he uh hacked on to get working so that's that was a chunky piece of code but again this is a custom thing that we wanted to do hopefully someone will write a clever component for social security number inputs that could handle this for someone else in the future we also had to write our own file uploader because that didn't exist at the time that we were working on this about a year ago which is another theme of our experience was that we started this a year ago already so much has changed with file uploaders with a live redirect that didn't exist which is another reason that we went the the path that we did down this but it wasn't all smooth sailing um we did have some challenges that we had to overcome but you know like what engineering problem doesn't there's always stuff that you have to fix and personally when people are like yeah but what if i what if i ran into a thing that i just can't do well when have you ever run into a thing that you just can't do it could be tricky though i mentioned we had to add these csp headers to our response that was a bit challenging uh to figure out but we did um there was also an issue where users would come in in incognito mode and this messed with liveu's use of local storage and sessions handle debugging information or i don't know some other stuff but it ended up breaking our implementation and again this is happening at a distance we didn't have a lot of control over it so we submitted patch to live view um to be able to bring along our own storage bag into um into the widget um this p most people probably didn't notice this change um but it let us get around our workflow uh to handle incognito mode so thanks to james edward gray ii uh one of our engineers who was able to knock that out and of course chris mccord for being receptive to prs i already talked about our own file uploader that was like eight lines of code that was a lot simpler than i thought it would be but now it's even easier than that apparently which brings us to okay what's going to happen next with this so we we built this app it's sort of a single page app it's a series of wizards series of workflows not everyone's app is going to be like that parts of our app will continue to function that way but the next iteration is going to be more single page which basically means it's a normal app that you navigate around arbitrarily we will probably still not show urls uh because again this is embedded somewhere else um but the future iterations of this thing are going to be more what do you want to do and and here's what we think you should do as opposed to hop on the slide at the beginning and hop off the slide at the end at the top of the list is using separate live views i've been extolling the benefits of having a single live view to handle everything it ends up being pretty unwieldy to work with and the workarounds now are workarounds the strategies now for dealing with passing state around among different live views that's improved quite a bit navigating from one live view to another with uh live redirect that didn't exist a year ago and we were working on this and so we could also combine we could get rid of uh a lot of that steps module i mean again we're still going to have the concept of workflows but i think that's going to reduce quite a bit and could just be a link from one screen to another but being able to use live redirect makes our thing look just like ordinary web pages and then we remove the concept of a router or at least we can leverage the existing phoenix router instead of having our own our own deal we're also going to expand our templating with heats and using surface components we started to do that with a couple of these workflows and it's cleaned up a lot i'm very bullish on service in general as a way forward for our community both in itself and what's going to get built on top of it which cue something from chris's keynote i hope you all watched chris's keynote earlier um yeah well i will pull some in something in with slots and the js library which looked pretty cool but i'm also very excited about what's going to come on top of that uh as chris mccord mentioned there's it's less about like some of these things that are about to get released in live view and more about what are people going to do to take these building blocks to build like off-the-shelf sweet ui components i know a famous british person really wants to be able to move quickly in building new features and he can reach off the shelf now for existing sweet tables sweet drop downs like other things that people have already been working on and i think in a year we will have that as well and i guess in a year we'll re rewrite this widget again to use that stuff one of the main takeaways is that all problems are solvable um either by us or by the community like it's all moving so quickly that things that were difficult or impossible a year ago are now trivial so um which brings us to the end um i guess what i want to say about the experience of writing a single page app in live view is it's just like writing a normal app except it's dynamic it's pretty easy to do and i don't think there's any reason to reach for a non-elixir solution for this these days because building a single page app is no different from building another app like there's some mystique around it like oh you have to learn this this new language and figure out what a jsx template is and all this other stuff and i do have to admit that the things with the slots starts to make tempering look pretty abstract like at this point we're getting kind of far away from some of the html but i think once we get used to it and you internalize it it it starts to feel pretty natural so now this uh this rarefied world of a single page app that has these smooth experiences um that is like on the cutting edge of user stuff is now available to all of us and so in a way like all of our apps should be single page apps and even the notion of like a single page i don't even know what that means anymore like you have your one screen like your browser viewport but you're still moving between pages like you want to look at different stuff you want to see things updating i don't think a single page is the right way to capture what's going on i don't have a cool new name for it yet but people that know me know i love cool names so i'm gonna think of something but really there's no reason not to use live view for everything i mean start with your dynamic table start with your like auto updating twitter feed but you could also have the whole page be a live view that then transitions to the next page which is a live view as chris said earlier like dead views are dead there's no reason to use a dead view and there's no reason not to use a live view so i'm very excited about um everyone's ability to build single page apps in live views uh with live view we did it it worked great the experience is really nice it performs beyond just doing it on ourselves like we were able to make this work at a distance in someone else's app so if we can do that you can do it too and if we all do it then we can essentially take over the world so uh i'm looking forward to that future thanks a lot [Applause] we have time if we want to take questions or i can dismiss you all early and get to lunch well i know how it is to wait for lunch so um if anyone does have questions i'll be around and i will probably be at an after party later that we're hosting so come up and ask me about that as well thanks a lot [Applause] thank you i didn't mention we're hiring we're hiring you want to work on live view

Channel: ElixirConf

Views: 2,282

Keywords: elixir, liveview

Id: DA32q8kd9pA

Length: 33min 56sec (2036 seconds)

Published: Sat Oct 23 2021