Lonestar ElixirConf 2017- KEYNOTE: Phoenix 1.3 by Chris McCord

[Music] but we'll talk we're talking about Phoenix 1 3 today show a hand to has Phoenix in production here serious Wow awesome so nothing's going to break so for those of you in production but today we're talking about some of the kind of design decisions behind the new generators and some little bit of a course correction and he saw the Orlando talk the first few slides would be review but the rest of the talk is really gonna be all about the thinking behind that and how to think about structuring applications and why we're making these changes so I work a Dockyard if you haven't heard we do a Luxor and Phoenix so if your company needs help building out a product we'd be happy to help you so check us out and a little personal level up in my own life I've been mean so so thank you Luke and Jason this photo was taken yesterday and I'm now I've been meme so it was my own kind of personal achievement for for turning 30 this year I became an internet meme so pretty cool but on a on another note so a couple things are going away with Phoenix 1 3 you may have seen this slide we are losing the root level web directory and we are kind of killing the term model anywhere in the code base and we'll talk about why this is happening but I think we this term has kind of just been more confusion more frustration than it's worth but with Phoenix 1 2 or prior to 1 2 we had a directory structure like this that Jose Ana I thought was a great idea because we wanted to send the message that ok this is an elixir application glad you have a Lib and just when you run mix Phoenix new you get a no it's crap you don't get a Phoenix app and we would create this root level web folder to send this message that the web part of your application isn't the domain of your application right so we chose this decision to send a good message but people saw this and they were just like you're a framework and you're prescribing cost rum application you're generating something special this isn't an elixir app so this ended up being like a backlash or people thinking we were trying to be extra special taking over your application so what we're doing now is something really creative is well this is we wanted to send this message right you have your apps you have a web interface but people got mad so we hit the drawing board did some computer science research and now we're going to take web and we're moving it inside Lib and now everyone's happy right yeah this is this is like the last six months of research yeah that's it and oh talk so so it's funny because there's actually there are other reasons behind a lecture compiler why we had a root level web directory but it originated good message but did need to be changes to the elixir compiler to move web inside of live because of code reloading so those changes were made so a little more complicated than that but we still want to send this message and I think we can send the same message by having web inside of live because live is where you write your application code and the web is part of your application and you write your web code in Lib web just like you would structure anything else so we can still continue to send this message and generate a normal list or application and the only special thing is we're going to give you a root level assets directory so now we don't pollute your root level project with like a package.json or node modules folder all of that regardless of what you want to use if using brunch using ember web pack whatever you want to use will just live inside assets and your 30 million node files and in modules will live inside assets so as long as those as long as your asset builder can build your assets from the assets directory into pre attics and then it's fine so this I think is a big win and we're also introducing a web namespace so a little bit more typing but again we want to send this message of you know you'd naturally namespace your modules based on your folder hierarchy anyway but when you see this this is kind of reinforcing like okay this module of my application is the web layer right this is the the web interface to my greater domain it's a little bit more typing but with alias name this is pretty much a non-issue and kind of the bigger changes around the code that we generate with mixing exchange so we've always said generators or learning tools and they're supposed to be like not the endpoint of your design but the starting point to try to kind of give you get your feet wet get up and running quickly and teach you best practices but I think we haven't done enough teaching in the right direction so let's walk through maybe why that is so with Phoenix 1.2 we generated code like this with echo so you could say repo or email comment like there's two lines of code it reads nicely the database access like you know I'm hitting a repo but that repo could be a gen server right that reco could be a net stable it could be posters database it doesn't really matter so there is some some nicely here but it gets worse when we kind of look at more complex code so when we also generated a persistence function in the controller this is where we would kind of be hand wavey we would say like well by default we're just going to say you're going to have a repo you can insert some changes in the database and if you need like something more complex we just kind of hand wave and say well you can then extract that code and put it in a module but the problem is what newcomers see is like okay I run this command and Phoenix put my code here so this is where I write my code and so it's not their fault it's our fault because we're kind of sending that message and then it gets worse when there's no clear there's no clear point where you feel like okay should I extract this is this too much is this too complex so we can do a better job because if we walk through what would happen is people would say okay I want to add this simple feature I want to be able to comment on on posts so they say well I only have to add two lines of code in this controller so if I need to associate a post in a comment i'll just add depth Oh put a sock at suppose and I'm good to go but the problem is we're now sprinkling these details of like how do I associate these data structures how are they persisted how do I fetch code all in the controller so if I want to reuse this any anywhere else from application I have to know that I need to extract it so most likely people are just going to duplicate this code so we can already see like as soon as people start trying to extend the code that we generate we've sent the message like this is where you write it so then they just add a couple lines and this is like the the ball of mud scenario where you think like oh I'm only adding two lines of code it can't be bad and then you do that for for five years and now your controller create is underlined code so we can do a better job here so with Phoenix 1.3 we're going to generate code like this where you used to say mix Phoenix Gen HTML or JSON comment where you have a module we're not going to require you to give a namespace module what we're calling a context so you're gonna have to say mixing extend JSON blog comment so you're going to take a little bit ahead of your design look so we're making you think okay like well what is this comment system where does it live what's the purpose of this and then we'll generate code in the controller like this it just says some generic blog list comments like I don't care how they're being listed or fetched maybe it's from Postgres maybe it's karma nectar repo maybe it's directly out of an agent running some process and then likewise with persistence we're going to generate some generic Craig comment function on this way we're going to generate code by default that doesn't live in the controllers the controller literally just an interface web interface into your greater domain and then likewise when a user comes here for create they have kind of a nice rubric that they're saying okay I'm going to add this feature Phoenix has kind of pushed me in to say okay I have this function already that can create a comment now I want to associate the post in a comment there's a place in my application where that live so they can say okay I can fetch the post because I have a public function an interface to do that and then I can just extend the code that Phoenix has generated for me and you may notice here that I'm using the lickers with expression so if you haven't used it much it's amazing but if you notice here we're not matching on like what happens if I a blog create comment has some other return values in something that was okay like what if we get an error back with some invalid data or if there's a you know unique and straight violation and we don't have to handle that here so one thing with especially with Jason controllers like a doctor I rebuild mostly Jason api's we had a lot of repetition in our code where it's basically like if you get an error back or error chain said like okay convert the chain said errors to adjacent message return the right spot and you're done and then you do that in every controller and every other actions over and over and over so we're introducing this new feature called action fallbacks it's a macro on the controller that says if your so your controller is a plug so if your controller action returns something other than a valid plug connection we're going to invoke this other plug whose job is you take a connection before the action and the value that was returned and then convert that into a valid response so this gets rid of a ton of duplication and gives you some nice features so imagine here where I say okay create this comment and then if it was accessible I can do this Kessel path but if there was some invalidation or any other air condition I can define this fallback controller that's just a plug so I can just define some functions that pattern match on the data types of my domain so if I can say you know in my domain if I'm using echo I'm going to have a set of changes that could be returned and if it's like a error change that I know that I'm going to put the unprocessed bowl entity status and I'm going to render the change that view and I can just do this in one place and then I can continue in just to find more clauses for other data types in my domain so if I want to standardize you know if my API boundaries defined like error not found as a return type I just want to handle that in one place instead of 100 places in my controller I can just use a width expression use action fall back and say okay anytime I call into something in my domain that returns error not found I know that that's going to be a 4/4 render the correct view so the right response so for us this is especially a doctor oh that's definitely removed a ton of boilerplate for HTML a controllers it's not quite as useful you may be able use in some cases but there's often times where you have to put a plasma sage renderer specific templates but if you're building Jason API is I think people will really like this feature so let's talk a little bit about why web models is going away other than this root level folder on so I think part of the problem is just the term model I think is wrong so when we took this decision it was like okay web models we've used on other backgrounds a lot of us come from Ruby but if we look here I was client code was coming in where people would say hey we need this new feature you need to help us out or I would be talking to other people in the community to have larger Phoenix projects and as we were like pairing going through their code we'd open the web models directory and it would just feel like 50 files and the problem here is like you have no view in the application like if I look at this directory structure it's just a dumping your own like I can't really tell what this application does right like I can say okay you have users you have comments somewhere like we don't know how these are associated we don't know what depends on what so we're not sending the right message here like I want to be able to look at your app just open the directory and see like oh I can kind of see what this application does understand the feature says so we want to get rid of this dumping your own and also the word what the word model it just doesn't make sense because the entire point of your application is to model your domain right there's no reason to call some individual file a model it's like the entire point of the application is the model your domain and you do that with books or files that have modules and functions right so the generators are going to generate code like this where we can say when you say mix mix in HTML or JSON blog comment blog post we're going to generate a blog IX file and then within that it's going to have its own data structures right it's go say I'm going to have this boundary here and I'm going to manage my own data structures and you can only talk to me through this public boundaries I'm going to define and we're calling that a context so we ship with EXO by default so by default within a blog directory we're going to generate a post and a comment schema and these are just Jerry data structures that feeds the blog system uses but if I had some sale system right I can just look at this directory structure and say ok instead of web models blog sales post comment payment order I can have a clear look just by seeing no code at all I can say ok I know this app Blagh system I know it can have post in common and I can see okay there are sales system with orders and payments and I know just by looking at this that I can only talk from the sales in a blog system through their API boundaries so the sales system can't just suddenly call in to repo insert comment right these things are totally opaque to each other so we're trying to send this message of isolation and I'm going to get a little bit into design patterns here but no one freaked out about this but the word context might be a little unusual but as Josie and I we're thinking about okay it's like we you know we clearly need to do better about teaching but we have to you know there's no one-size-fits-all so when we generate code as the best best practice we can't we don't know what you're trying to build so the fact that you are passing blog comment there's a ton of information there that like you know what do we name the function so it's it's a balance of we can't assume too much about your ass but we want to teach you kind of the best practice to take that code and grow it into a successful application so we started looking around at no prior art and the mounted context pattern for a martin fowler's I think kind of resonated because it's not some grand pattern right it's pretty much just a set of ideas so this is a little bit of opaque here like you know found in contact fields with domain is and driven design where different contexts are being explicit about their interrelationships but I like to say basically bounding context makes the boundaries of your API is clear and that's the goal so it's not that we're adopting a design pattern from Fowler but don't start tweeting that like all phoenix is now chosen the design pattern it's just we took this we took this concept and we took inspiration from it and it fits well with functional programming because well one of the things that happened when I really got into functional programming and elixir was I started like looking like googling for like functional design pattern books like you know I needed a like how do I write my code properly how do i structure it and I couldn't like I thought I was doing it wrong but it turns out like like spoiler alert like view of what you do and punctual programming is like you write modules and they have functions and they're well named and that's it Sears that's it so so just this term bounded context isn't a design pattern it's just you need to think a little bit about the boundaries of your modules and functions and that's that's the extent of the design that you need if you think properly about that I think you're going to end up with a successful application so we're taking this idea and like I saw I mentioned with like these sales in the B blog system there's another diagram from Fowler where he's just showing that like you have these different boundaries of your abs and the sales and support example here like these things can only talk across well-defined api's so you're not going to end up in this case of web models where you can just see okay it's like I have a post here and I have an order so they're not going to just call repo insert and I can kind of cross this boundary I end up with like all these interrelationships there's no clear separation and again I want to be able to look at a directory structure of anyone writing a a motor or Phoenix project and be able to see exactly what the app is that apps doing and I can say like okay I can just visually form a boundary here and I know that I want to talk to these these inner modules I need to do it through their public API and then one thing I want to push that the generators don't do because I think it's it's not a one size fits all but I want people to start thinking about data centric schemas if they're using echo because XO is almost two libraries at this point one is like a a data casting and validation library and one's a persistence library so one thing I do in my applications is I model my like I think for promotable you want to keep your view layer concerns at your core data structures so especially for me like coming from rails where I'd open up a user model and then I would just have like 30 fields related to like whether the user check remember me all these like anytime you need a new feature on some form somewhere it would become a new field it's like that these are core data structures of your application let's keep them core to what they're designed for so if I want a registration form an SEO I can use the embedded schema feature to say okay I can model this this form as a data structure pure data structure with a name email password and a memory field and then I can have some registration module that has some well-defined function like regice your users so instead of like repo insert user right it's going to be you know it's like we have a boundary here that says I have a registration that handles user registration and it's going to register a user right there's like intent in that naming and it takes in Sam a raw map of data it casts it as an embedded schema which just returns a accounts registration data structure and an internal a kind of my last step is I convert that to my core data structure and I persist it and whatever means possible so this is what this looks like and again we don't generate this by default but this is what I want people to start thinking about is you know using echos multi features I can take in a registration chain set which is just an embedded schema so not connected to the database and I can apply any changes to it to see if it's valid so I'm just casting user input into a raw data structure if that data sure should happen to be valid the last step I'm going to do is take that data convert it into a user and then insert that user into the database so a little bit more code than you have to write but in this way I'm not polluting my account user with whatever feature I need on some form somewhere right and keeping it isolated two parts the application that required so this is just the kind of things I want people to start thinking about when they're designing any kind of application that touches the database especially with that go so part of this isolation message is getting people to focus on okay it's like we're isolating our modules we're isolating the boundaries of our API so maybe we should isolate our application now so a Phoenix will ship within - - an umbrella flag to generate an umbrella project like mix new umbrella and it's just not I don't think every project needs to be an umbrella but I do think umbrella projects are kind of underserved in the community so we're trying to walk through the merits of them why you wouldn't wouldn't use them but if you pass the umbrella tagging in Phoenix you notice we want to continue to send the same message so instead of live and then a web directory you pass - - umbrella to make Phoenix new we're going to generate an app that's your application domain that's where your apple is right and then we're going to generate a web app look like whatever if you type in my to make it my eff web and that's the web interface and then leave these are literally isolated like the only way they can talk to each other is across the boundary of the applications so your app would not actually depend on the the the web application but the web application and its own mix dependencies in the umbrella would say okay I'm going to depend on this app so we can continue to send this message of literally you have a web interface here to some greater application domain so the whole point of Phoenix 1:3 is to get you to think about boundaries so it's not the that we can design your whole application but we're going to make you think just a little bit upfront about your design so instead of like you know when you're scaffolding applications you want to generate something some crud base it's easy just to say run a few commands and persist the data and it mostly works how you want it but then you're down the road two years down the road now you've tried to grow that code and it's an abomination so we want to think we're going to think a little bit upfront a little bit of effort to think okay like where do things live how should this be named and we think that this hopefully is going to pay dividends for you and the rest of community in the future so I'm going to take you on a little bit of a thought experiment based on adding a feature to an application and we can kind of see where where we're going with these ideas so let's say that we want to add some reaction feature for liking posts right I'm going to like a pose I want to dislike a post or you know Facebook has like the rageface whatever prior to feedings want to be kind of mode of operation would be it's like well I have a web models directory and I know I want to put this in post RIS so I'm just going to add a reaction TX file and now I have three you know three models here right but the problem is we're not you're not thinking about the boundaries here you're not thinking how this relates to the application and there's no we're not giving you indication that you should be thinking any other way right yes you'd have to have some intuition on the design here so we know that you know this isn't necessarily where we want to push people and it's worse because in your controller with the way we were pushing you is you say okay I have this code here and this is where Phoenix were on my toes so this must be where I write my code on that you would start having to put a sock here right have to associate multiple things and have to control or be concerned with not only how the data is inserted but how do I actually associate it so we know that this is not necessarily what we want but now imagine you're using Phoenix 1.3 and you've used the generators your directory structure looks like this so now you come to this directory structure and you're like okay I'm going to add a reaction feature by default even if you're using the generators to generate the reaction feature you have to think like um where does this live there's no dumping ground anymore right we can't we won't give you the easy out so you have to like okay well where do i where do I save this file so you can think by default like well I want to be able to like post in comments so maybe I add it here right and maybe this is the right answer but you have to think a little bit upfront on so at least you're putting it here so let's put one through this let's say okay this is part of the blog system I can only have posts and comments that I can like it should live here and then I can compose something really nicely like this this Phoenix is going to generate a function for me to kind of give me this hint I can say okay get the post and then pass the post into the blog system looks like it nice and pretty the controller doesn't have to know anything about it and then within the blog module since we've this is where we've written your code for you before if you're not even really thinking about reader design right you're a newcomer you just want to get started you at least see like Oh unique incinerated this blog module this is where it's doing this persistence and this is must be where around my code so even if this is like the minimal jump that you can make keep that cause this is where I write my code there's a module here in function names then we've won now so at minimum we wanted people to get here where they're like ok I'm gonna add a function like post and this is where I'm going to do the Association and would do the the database persistence or whatever I need to do maybe just store them in an agent or s but if we get people here we actually have a a well-defined API that has reusable code that you can grow in the future right if like liking a post and now something we want it to cash into it we can do it right we don't have to change our web interface or any other any of the other qualified but my hope is you get people to think a little bit upfront I'm not saying the previous weight was bad but if you're thinking about boundaries you're gonna say okay I'm going to add this new feature you open up live and you're like hmm I've got a blog system I've got some comments and you're like you know what a reaction system itself just as a boundary there right what if I I'm going to later add new resources into my system you know I could add user profiles that people could like videos anything like maybe I should have this as its own boundary with its own data structures an own well-defined API so our hope is people think okay I can create a new module and functions to do this thing here and then in my controller I can actually have these things completely isolated the controller could just say okay if you want to like a post and we can actually call it we have a model for that get the post and in reaction add like right and these things don't even have to know about each other necessarily so this could be nice because we don't have to depend on the blog and reaction modules don't depend on one another but you may also say like well I don't want to have to call into this explicitly to get the light count every time maybe every time I attach a post or elissa post maybe I want the light well then the blog module can depend on the reaction module and they could just call a function right but the whole goal is these things can only communicate across their well-defined boundaries so the fact that they could both be using echo and pursuing that data is transparent to the modules right there's only well-defined boundaries and if I want to add a like the blog module can just talk to the reaction module through that boundary and then I can put the likes into the post and now I have a nice attracted reacting system right and I know Phoenix was pushed me towards making this boundary so I can think okay my eyes would look something like this I'd like remove like get the light down this is nicer succinct but now the jump that we think people might make is okay I went from adding this file into a dumping ground to thinking about maybe it can live in the blog system to thinking about oh maybe it's its own boundary as a module but now you're like well if it manages its own storage right it has its own boundary that can like any kind of resource what if I had an umbrella and I could take the leap and say you know what if this thing is totally isolated instead of being isolated in module maybe it's an entire isolated application so this is just kind of the way we want people to think so I'm not saying go home and rip every feature out as a module or just make everything in application so don't don't read too much into this but at least think a little bit about your intent right we want people to start being able to make these leaps earlier on and if this is an isolated system so if this reaction system manages its own storage it can be starting starting to stop deployed in isolation and then it could be put on hex and shared right so if someone makes a reaction engine right they could put that on X and now anyone that wants to be able to have resources in their app like hated dislike I can just drop that as in as a dependency maybe configure it with a couple lines of code and now we've shared this feature across the entire community so these are the kind of things I want people to start thinking about so don't go home and start you know ripping out all your augur code but just start thinking a little bit about the kind of intentions of your api's and the boundaries of your modules but naturally it comes up like you know should or shouldn't you use an umbrella and I was thinking like really hard about this because like a Dockyard I'd say probably 50% of our projects are umbrellas and I think there's a push that everyone thinks like ol umbrellas it's a new secret sauce so everything should be in an umbrella and I don't think that's correct so I started out like okay I'm gonna make this flowchart it's gonna be like it's gonna be really impressive but it turns out like for me personally like this is pretty much my thought process so this didn't maybe a flowchart but I was excited about making a floater and this is what I ended up with but it boil down to this this is a hard problem right so you don't want to over optimize too early on but it came down for me that you know it's my context so I'm I'm I'm I'm I'm module manages its own storage if the answer is it doesn't I'm using some shared database in my application some shared storage engine then I keep in the same app because for me those things are necessarily coupled so I see some people that like split their application a bunch of different umbrella applications that all depend same repo and some other applications that's depending on Postgres like those things to now have like implicit dependencies across each other that like if you deploy one app and you need to run a migration like it's just you lose sight operationally what's going on so for me unless the management is its own storage I don't even think about splitting out but if it can be isolated and it is using its own at the table it is separate from my SEO Perez database and I at least consider an umbrella so I don't say like immediately extract it but at least it's worth considering like this thing is totally isolated if it manages how its data is stored whether it's in a process whether it's in another database anything then it can be deployed in isolation starting to stop in isolation and it's worth considering extracting it out so I've kind of you know I've been hyping like extractive check isolate isolate because this is what I want people to think but I also want to in put the brakes on a little bit that like if everyone goes home and gets too carried away that you know everything everything taking and like taking step by steps to extract like it feels like you're making your code better but it can actually increase the complexity of your application so anytime you decouple your data model so like you know databases get a lot get a bad rap but any time you try to decoupled or data so like you know what we don't need to put this in go stress let's just put it in a process right maybe it's ephemeral ish data that doesn't necessarily need to stay around forever but every time you see a couple your data model you didn't have to increase the complexity of your app because anytime you want to join or a great aggregate data you're now have to stitch that together by hand so it's something that you have to think about it may be worth it maybe not so here's one example so this blog application so a trivial feature right I could store this data in ecto and then I want to get to light count from my post I could say repo preload aggregate state data in one line of code with echo and I have all the light count but if I decided to write this reaction engine now that is isolated manages its own storage has a API boundary there I have to write code like this where I have to say okay well now if I want the blog every time I get a blog post if I want that light count value I need to call into action I need to hat and need to have an API that takes a list the post IDs and then returns a all the likes for any of those IDs and then go through each of those and if there wasn't like four then put the likes into that data structure like good it's not terrible right this is a small amount of code but every time you make one of these decisions to isolate components it's now on you to aggregate and stitch today to get together so just think about isolation design with intent but then be carefully about the trade-offs right because it's not always the best idea so that's the end of my talk phoenix 1.3 is making you think a little bit but I think in the future we go one years down the road or I inherit a plication zazz a consultancy and what I want to hope that we open up a directory structure and see some some kind of like intentional intentional design instead of web models 100 files who want to see the feature set of the app and I think as a community if we do that we'll be able to generate new features that are shareable that are taking the best of ecosystem has to offer right so like I said don't get too carried away put the caution on but design with intent so that's all I have so I just got net and DDD for only 10 years and looking at this bounded context concept and the curious like how far down that DDD rabbit hole you went and if you did go fairly deep did you look at the aggregate and aggregate root concept yeah so I looked at quite a bit but the problem is like I was also interested in like securest design me and I I've done a lot of looking for this Phoenix 1 3 was actually it was like I promised it was out by the end of the year in Orlando last year and it's taken a lot longer like the code hasn't been hard but trying to make like training like this is these are best practices so it's like how much do we put on people that is the right design for a broad use case right because it's like anytime you apply any kind of decision in your application like it's going to vary on your guevara in use case so for us this is just getting people to think about isolated modules and functions with well named meanings right and then if you want to use aggregates right if you want to apply that then that's great but we don't want to tell people like the last thing I wanted was to have like okay Phoenix now uses like I look in the sea GRS because like personally it was exciting but then I was like okay Phoenix alone through your ship now reads its 400 page book so I think that I'm very interested in those ideas and how we can solve like how we can better aggregate data so if people do have those questions now it's like okay I followed this idea I have this isolated component now how can I make this code better that's definitely a great idea but for the default someone two weeks into the ecosystem it'd be too much hey Chris um just interested to know if you were studying Phoenix again today thinking about these concepts how differently would it be structure how will it be structured so I think how we're doing it now but so yeah it's tricky cuz like there's always there's a back story there because initially a web there was no web directory at the root level so this is where like that was actually Jose Jose a pushing in that direction which is kind of funny to you because people that complain about it in the past alcohol phoenix is special cuz its web directory I'm like Jose day himself he was the one I mean I agree but yeah so I think a lot of this is a learning process for Jose Jose and I as well about these pushing people in these right directions and it's always a balance of like you know how much do you you push and how much like because every time we push we potentially put up a barrier at the end of the day someone is coming into the ecosystem and they're like I just want to I just want to insert a post in the database like I don't like you know enemy it's a balance I think we're at a happy balance here you know obviously this is how I would design it you know if I can go back that's how I would design it but talk to me in in three years and maybe an answer be different I was going to talk to you next to me now I'm sort of saying you don't hear about how you design user experience and partition metal I'm talking about how you designed the framework itself like the code of the framework itself so I don't think the code of the framework itself would change much it depends enactus phoenix the the stateful bits the interesting bits for me or in phoenix pub/sub now so phoenix core is mostly a plug extraction and you know about about transforming a connection and building a feature set around that rendering you know taking my data and rendering it in the phoenix cue layer so i think like our our neat features in phoenix core are are mostly transformational features so like taking the data pre compiling the template phoenix pub/sub definitely was more of a learning process where my initial take at phoenix pub/sub was a disaster was horrible that's where like josée was like wow this you know basically nicely said this this could be a lot better but the high level API that I got was was what I wanted so like our channel layer worked exactly I wanted it but I didn't know enough of ot be in how I should be structuring the underlying bit so that would be totally different but I don't think it'd be it's not something that I think I could have done any better at the time something I had to kind of level up into so as Phoenix evolved then we're making these changes on how you think about how to organize the structures we've seen - over the past three or four versions has several distinct changes do we have a web directory now we don't have a website now we're having these contexts are you worried at all about sort of this semi whiplash hurting adoption and getting people confused as they're either looking at documentation and the new version comes out how the synapses which their mental thinking or inferring like that yeah I mean it's definitely a big concern because like one and then like the guys have to be rewritten so I had planned on releasing one three with the guides done and I'm gonna release one three this month this month it's happening it's almost done but then the guides are going to be my next effort and the Phoenix book has to be you know has to have a new edition l so it's just personally it's a big undertaking but then also like that's why it's taken so long to fill out these ideas like the data centric schemas I mentioned that was I wrote the code generators to actually do that EXO multi embedded schemas by default and then ended up back traffic because it was just too much like it wasn't a ton of code it's code that I would write personally but you have to balance that getting started experience as far as the changes in whiplash i think this i think this change will not be this is not going to be whiplashes I'll give you a reason because all we're doing is regenerating modules and functions right we're not there's no when you generate a context there's no you scenic context it's literally death whatever and a function definition since we use echo by default it's good to have your repo insert code so I think there's no backlash because that's how I think that's how if you're in the ecosystem and you're an advanced programmer you're going to say okay I have this web interface if I'm not thinking about bounded context I have some feature I'm going to write a module and my to write functions in it so I don't see some giant course correction from here like I don't think it's going to be like oh I should have we have to go back and write all our code in the controller's now so I don't think that there'll be a big course correction but it is something I have to wait heavily with with new ideas right like if we did introduce like I want to I'm interested in service discovery right so I want to introduce these discoverable services and processes every time I do that I have to think like this has got to be a stepping stone to kind of my future goals so Chris I really want to commend you on how much you are thinking about bringing the new users along I think it's a big deal that you're up here saying that scaffolding isn't the way we're going to be writing applications it's the way that we teach and we need a mature book we need the documents all be in line and that patient I think does a great justification because those great things for the community thank you so your flowchart where you're exciting whether tis right into an umbrella application is pretty damn similar to our flowchart of what we might split something into its own github repo and micro service so I'm wondering like do you have any opinions on when you might choose one approach or the other well might go into that decision sure yes so if it's if it's an elixir so if we're strictly speaking as like all those micro services our elixir applications then and they're all part of the same greater platform that you're writing I would just put it all in one umbrella lab and granite is like depending on your team size and the commits and complex with merging but for me umbrellas give us the microservice architecture like we can define so here's one example that I didn't mention like a Dockyard we we made it like by default I chose an umbrella for this application and it was a pretty simple application but it had its main purpose was to connect to Amazon s qsq that was powered by other parts of their domain or other applications not in elixir and it was going to serve some channel interface on this on the browser so ahead of time I said okay we've got this staple thing that's going to be listening on a cue and this thing can be that thing could be deployed in isolation so if you think about micro services like I can have pin web front-ends cuz I'm on a load balance web front-end but I want to consume this cue I don't want to have every web front-end I deploy shouldn't be running consumer I only need one of these things processing messages and deciding how to you know broadcast them so for me it's going to depend on your deployment so the microscopes have been all the benefits of micro services through deployments starting and stopping in isolation you get front of it from an umbrella lab so the only reason I would even think about splitting as a separate get repo is if I had like a huge team who couldn't manage a big codebase I think isn't like Facebook I think Facebook's whole codebase is like one repo but anyway yeah so I think there's no benefit in splitting I think there's a huge benefit and keeping everything underneath one if it's all an elixir because then if you can deploy those applications together and you can cluster them what used to be a you know rabbitmq or HTTP Jason API call is now just a Djinn server calls internally right I have some name process from one of my apps I can just sage and server call whatever and now I'm talking you just app across the cluster so it's a huge win for me to keep them together I think the million-dollar question is you have a pulpotomy TA and then also when you do have an existing model folder that has positive models and/or in the dumping ground is it possible right now to basically take those and start organizing or not possible is it recommended approach to start organizing those at a more fabric context to remove those later to when one for three years oh yeah I mean you can move them now so like 1.3 as far as like we've got some minor features so action fallback is college of the bigger future there's just you know a handful of API is like being able to get your current URL in the controller so mostly it's about quarter code organization and the majority of the new code from from the commits is just into the generators so if you're not using generators you can just put your code into the new format I would recommend that maybe I think it would be interesting I think for me to talk to people who have gone down that path because I think you're probably gonna find like me like we were right in the Phoenix book that we ran into these cases where we were like wow if we if we would have thought about isolation just a little bit upfront we wouldn't have had to end up refactor our code later so I think as you start trying to isolate increases there just got to be spots in the code that you're like wow there's really no reason I should have coupled there but since you didn't have to think about it not even as a beginner it's like just as a expert programmer like if not having they think a little bit you know it actually it's gonna change your design so I think yeah start splitting now and let me know how it goes Oh et a so I said this month there's so the generators are all in place now so I like on the flight here I polished the documentation at this point I just need to add a few checks like if you try to run the generators in an umbrella now like if you try to run mixed finish in blog comment from an umbrella route it generates the files like inside lived in the umbrella route so there's just a few like validation checks and then I need to walk through Jose and I and you kind of go through with a fine-tooth comb and just make sure like the conventions that we want and in the generated code are like fully base so this month good happen in response to a question my talk at 3:30 is exact tweeting about you've had a monolith you broken up and then we broken up and do an umbrella yeah shatter name children fellow yes take a look and look very for you you guys to redo a bunch of slides or we good but but no major scramble to okay yeah like I said at the end of the day these ideas aren't like let's go down this uncanny valley of like new design patterns new ideas like the big idea is like oh we can have modules and functions to have wall to find a clear api's and I think like no matter what everyone can agree that that's a good thing and that's kind of the starting point that we're pushing people towards and how you grow from there is go depend on on your you know your application the I really like what you just talked about I think you make sense I think you should always try try to do the right thing you know even though it could have some collateral damage and so avoid I think you know can be avoided but you keep doing the right thing I think the only good circuit one thing that I I see you know that patience is a lot of times in the bottle level there's a lot of business logic you know are used today I gotta make sure that I'm going to be wearing and that's where the kitchen sink you know syndrome - right what are you talking about you know how about just treating storage just for what it is you know I get inserts you know press that stuff violations but whether you put the business logic you knows is that good a separate I realize you know the universal rule about how people are actually going to use the app leaving their own context someone or what are your thoughts on that yeah so for me there necessarily coupled together not always but I think like if I call into let's see if I have I go back here like the whole goal with like blogs list comments right by default I don't if I generate that or not but by default all this comments is going to do is call reso all common right but the caller doesn't have doesn't know that we're forming the ecto repo and we're hitting the post first database right so to me that's not necessarily the details of that boundary and you can split like it's not that we need some like God module there in blog like that blog module can split out its logic but for me personally storage the storage of my app usually lives alongside the Business Objects lives alongside how that storage is handled but the collars don't know necessarily the details of that right yeah and I'll have one more thing though so one more thing so part of this is like if we talk about like peer functions so I'd like if I'm writing some authorization layer in my ass where's Niko in here from dr. Niko's in the back I think Niko wrote a blog post for Dockyard like how do you to write some authorization layer like can a user like a policy type system like do I have that authorization to take certain actions on certain resources you know that isn't for me so that's that's quite a bit of business logic there but it's all pure functions so for me that would be one case where I we're going to define a module with function definitions and they just operate they operate on data structures you don't have to fetch any data they get a user they get a pose and they get the users you know they think they have the data that they need to make a decision so I think in general it's best to structure your code your functions were possible but I wouldn't then put that out in a separate app I don't see the value of splitting that out in a separate I mean I think II care of goes a long way in terms of still try to figure what is in terms of the application you know when I define application actually since you have the store umbrella size project and also business logic which is kind of the common ground where people can go and look at what is this application actually doing so maybe another you know photo reference that we think in the application yes another commerce as you know I wasn't the I'm not sure what it's gone now but when you have the action action callback or action fallback rather so there are sort of it'll come to a lady you know it's my Google background swing go you know when you make a call you know you have to ok comma your signature right here I CQ 0 bit of magic you know I think if if there is you agree this repetition about every time having to check for the return signature but I think he clarifies what the intentional these things can spell you know what I think being kind of maybe less magic but yeah maybe a little bit of redundancy you know terms of checking still clarifies what the instead of the call is my just my opinion yes so in this case so we thought like I think if you watch my Orlando talk I call this a responder which is kind of a troll on the rails community but it's a similar so we didn't want introduces like new concepts right like the whole the goal was that we had these data structures that we handled in the same way every time yeah and it's like what do we call this thing like you know a whole job is to send a response like so George is a and I spent like this is where like this white stuff just takes so long it's like you know a data source calm up like what do I call this thing like we were like aw we just caught like a handler it's like you know handlers like it there's like anytime you name something a handler it's like a cop out there's like there's a cowboy handler in it and there's like a few handlers in the Phoenix codebase so like like we can't call it a handlers League so anyway so this like this may look like magic but the goal is like you have this plug pipeline and the plug contract is you take a connection in you return a connection and anytime you don't do that it's going to fail sorry they raise an exception rightly so so we said okay like the whole point of this thing is really just to take a value and turn into a connection that's all it does and actually fall back while I doesn't matter all it is is a plug so it's like we were it was actually beautiful at Joe's in our like you know let's introduce this responder idea and then that can you could say use Phoenix responder backing to find some functions and we pass them in and Joe's a was like you know this is almost like just like a to Kunming function like you know takes a value and Carissa to a connection and we were like oh wait we have a contract for that like that's the plug contract it's a def call in X connection and a value as like an argument so like well we have the connection has the value passed in before the action and we need that data we don't get the connection back so we actually had this beautiful contract already in there which is plug so there's the only magic here is that we wrap your controller functions and say if you return something that's not a connection then we just call this plug and that's it so it's very little magic you [Applause]

Channel: Confreaks

Views: 37,910

Rating: 4.9659863 out of 5

Keywords:

Id: tMO28ar0lW8

Length: 51min 44sec (3104 seconds)

Published: Tue Feb 28 2017

Reddit Comments

[u/rtrevisan]

Really happy to see these changes in the default Phoenix structure.

[u/[deleted]]

Gotta join the new-structure-loving train. Really looking forward to it, hopefully it'll be the paradigm shift that'll defeat struggles in long-running projects, adding feature after feature.

[u/Teifion]

I'm loving the separation part, I'd been doing it since I started using Phoenix as a carry-over from the way I'd laid out a previous project (though not quite the same way as Phoenix 1.3 is doing it). Really looking forwards to changing everything over.

Aside from the new structure what are the headline new features/changes?

[u/Kritnc]

Thanks for the link, I have watched his talk from Elixir Conf a few times but the sound quality was horrible on it.