Paul Biggar - Dark Lang and Tablecloth powered by Reason

[Music] all right so I'm here to talk a little bit about complexity because complexity is the thing that that that I make my brand these days or trying to remove complexity and a lot of this what what I plan to talk about is is how we build dark which is a tool for removing complexity and all of the tools that we use to build dark which which is a camel buckle scripture reason or whatever we're calling it these days back-end and front-end so it's end-to-end in in this ml thing that we're all working in and we're talking about complexity there as well there we go so roughly gonna run through what dark is gonna talk about dark stack stack and how we built it and then we're going to talk about the two tools that that I used to make or that we use to make dark which may be useful to you one is called Philippe the second which is a tool for compiling from elm to to a camel I make compilers for a living that's what I like to do and then the second one is a tool called tablecloth which is a library that sits over standard libraries within reason ml and back-end Ocala which which I guess we're calling the reason native I wasn't aware of this this is news to me and then finally I'm going to talk about our experience with using reason a camel I'm just going to say our camel please note that I mean all three whenever I say that that will simplify things for all of us so the there we go okay I'll just stand in this small box here and this worked better for everyone so I am the CTO of dark which is a dark man calm you might have seen it on the on the slide that was setting up there for an uncomfortably long time I also founded circle CI which is another dev tools sort of startup so this is this is sort of an area spend a lot of time in my eye I joked that my career is convincing people to give me a lot of money to build my phone compiler projects which is all both of these these kind of companies are and then before that I did a PhD in static in compilers and static analysis nearly went to work for Facebook but remembered I hated PHP i-i-i don't really hate PHP but when you're in the moment you just sort of feel like you do you know what I mean and if you've spent a lot of time writing static analysis for PHP then then it becomes a little clearer of what what I mean so I'm going to start right by talking a little bit about dark and dark is is the tool for solving accidental complexity and one of the realizations that that a lot of people are having in this world is that accidental complexity is is absolutely everywhere and accidental complexity meaning complexity that we shouldn't really have to think of it and so dark was the thing that that that we started with the intention of removing all the accidental complexity from building backends and the the main kind of accidental complexities that we're talking about our infrastructure and deployment so the infrastructure we mean that whenever we go to build a back-end we need to know so many tools so we need to know darker and kubernetes and tons of AWS services that we need to know SQL and NoSQL graph QL thrift and G RPC and like nginx and memcached and security and observability and monitoring and optimization and just kind of like an endless list of tools that we need to do where as compared to the front-end while there's kind of a lot of like to do that there's not there's not so many moving parts and the assertive sort of a visual illustration of this complication this this is the clave native computing foundation landscape and it's all the tools that you use to build cloud native clinton ativ is a word that means roughly kubernetes and so to build to build backends you need to know possibly if there's four hundred and fifty five technologies in in this slide and at the very top of the slider you might not be able to see it everywhere there where the light is it says overwhelmed question mark it's like are you overwhelmed course I'm overwhelmed is 455 technologies in this slide so dark is is about removing all of the Excel complexity from coding and the way we do it is we have a holistic system we have an editor a programming language and infrastructure all rolled into one service so you write code in our editor you write code in the dark programming language and it deploys to the dark infrastructure and by doing all this we managed to remove a massive amount of the complexity so remove remove the infrastructure complexity so you don't need to think about any of the infrastructure and we remove the deployment complexity so as you'll see in a second you actually write code live in production basically we call it deploy lafe's sort of like serverless it's not a term that we coined out I'd be far too sheepish to coin that but since someone famous kinda I'm going to exploit it so the idea is that you write code live in deployments sorry live in production and we have tools that make that really safe so things like feature flags database migrations so it really simplifies a lot of that complexity and I'll give you this is the hello world application in dark so it starts just after the a bed to start in about one second so to create a HTTP handler which is called hello and responds to a get you just create it and write your code there and then if we open that in production we'll see that that is live on the internet immediately no intermediate step there's no deploying something to service there's no CITV pipeline there's none none of the sort of complicated things of building backends and API is it's just immediately done and that's the kind of thing that we're doing so I'm not gonna not gonna talk too much about dark just to give you a rough shape of what it is that we're building and the rough ideas is that you can create basically handlers that do the standard things that you want to do in back-end so HTTP responses background workers schedule jobs and data stores and it all gets put into a nice simple package all right if you want to know more than those videos on dark line calm and if you're at the workshop yesterday you might already have seen this so that's roughly what we're making with with all this Ocala stuff so let's talk about how it's actually made so our back-end is a camel and our front end is a camera and that's that's nice and simple right everything's everything is a gamble we get to share code that's absolutely not true I'll talk about that at the end but on the back end we have the dark language and its implementation is written in a camel something that you might be familiar with is that our camel is exceptionally good at building compilers and interpreters and language implementations it's like the thing it's the best in the world of building so it's a was a natural fit building the entire languages like it's like 500 lines it's really easy to build interpreters the we use on the backend we use Jane Street core as our sort of standard library and then we also compile that whole thing into G SFO camel which is a reason ml alternative a different way of compiling a camel and it's the advantage of jessa4 camel over reason ml is that it is it compiles with the exact same semantics it works exactly the same and the advantage of it working exactly the same is if you're building a language or something you want it to work exactly the same so while we build our front-end in all reason ml are our back-end is compiled into JavaScript by by jessica camel has exactly the same semantics as on the backend and we we run that in like a web worker in our editor so that we can show you these really cool things like as you type we show you the actual value of the code that that is under your your key as you're typing in it ends up being really cool so that's that's our back-end it's it's about it's about twenty thousand lines of code and it's it's largely it's pretty stable the we run it in in the infrastructure so there's we run our gue credit uses Postgres uses kubernetes has a lot of the same lot of the same properties as a shaun groves infrastructure that he was talking about earlier and a lot of the same problems like the the talk he gave is basically basically the exact same experience that that happened to us which which is no surprise since i basically follow six months to a year behind all of Shawn's technological progress and just copy what he does so the the front end is is the really interesting thing if you're if you're building a programming language and an editor and infrastructure you think oh you know we're going to spend a lot of time on infrastructure infrastructure is gonna be really hard or you know a programming languages are hard but actually the thing that's really hard is the editor we spend maybe 80% of our time building the editor that's the thing that's built in buckle script or reason ml it's it's about thirty thousand lines of code almost I don't think any of that is generated and it started life as an L map so we started building this this front ending helm back way and when reason ml was was a lot less mature so that two years ago and at some point we reached the end of life for the L map and we we converted things over to tobacco script and the tool that we that we used to do that is a tool that that I wrote called Felipe's ii explained that a second but the so elm Elm is a beautiful language it's it's really really wonderful in lots of ways but it they have a very strong belief in removing things that do not make the language better and one of the things that they removed is the ability to directly call javascript code which unfortunately is something that we used absolutely everywhere and in the last talk Ricky talked about about escape hatches escape hatches are thing I fully believe in I believe that but every language every implementation every every tool needs escape hatches of some sort and when Elm removed their escape hatch it actually removed our ability to stay working with them we could have stayed in the old version of Elm and the one that that's still supported it but staying in old versions of web technologies tends to get you in a lot of trouble as I'm sure you're all familiar with it I see some nodding so we knew that we couldn't do that and there was a tool called a library called buckle script T T stands for the elm architecture so we switched over to that and the way we use it we did so is using a tool called Philippe in a second Philippe the second was that was a French King in the 12th century he is one of the stories for which he is famous is that he had a meeting with the English and the English stood in the shade of an elm tree in a field in tank Alcazar's in in in France and he stood at with his entourage in the in the blazing heat for several hours while they had their their tete-a-tete and the next day he was so pissed off that he got down the elm tree and that that possibly is not the most becoming name to choose but but I was in a moment when I chose it so that is why it is called that so Oh camel and elm are are basically the same language you'll see here these are two versions of same code it looks exactly the same they have the same types they have the same type system and so switching from one to the other is is not is not that big of a challenge it's it's a direct syntactic change and that was the challenge that we had ahead of us if we have I think we probably had fifteen thousand lines of code at the time fifteen thousand lines of code is hard to port by hand so what can you do to write a tool that that converts it from one to the other and so what we did was we looked at the Oh camel compiler so this is a definition in the India camel compiler and this this is an expression so the interesting thing about programming language implementations is that they're implemented in exactly the way that you would expect if you were implementing them yourself every every feature in the language has some type as a variant so an expression can be it can be an identifier like X or it can be a constant or it can be a function or can be a Latin it just you know the if it's the left then it has a recursive flag and it has the things on the left-hand side and then it has an expression on the right-hand side it's basically represented in memory exactly as you would model it yourself and then in in El surprised exactly the same thing that this is the the Elm expression and it's got a literal it's got it this is sort of equivalent to a let I think not really sure I sort of copied and pasted to the last night but it's close enough and so that the two languages are basically idaite identical so we what we did was we took the Elm compiler we actually took Elm format which is the Elm for matter and then we parsed it using that and then put it into Felipe's a second and did some translation and put that into the Oh camel compiler and then spotted at the other side so it looks roughly like this so we started with with Elm we Elm format it wasn't wasn't perfect it has its own built in JSON format which which tries to be really pretty and altogether too clever so we we used a deriving one instead to just get every value about out of it then came JSON we parse that JSON in in Philippe's a second and we add put a camel IR IR it just means intermediate representation it doesn't it's just a fancy word for the memory that we use to represent your program and then everything else was done by by other stuff so I guess about it we put the IR into the atomic compiler which then printed out some some uglier camel and then we use the camel format to make pretty and you can use reformat to put it out as reason ml on and to make that pretty as well and so very very simple project it took about two weeks and to end and then it took about a month of of engineer time at the end of it to make it actually work because if you try to compile fifteen thousand lines of code from one language to another that isn't exactly the same you're still going to get you're still going to get some problems so yeah we're we're in we're in Ocampo wonderful but of course there's there's a bunch of things that that don't go well when when you're dealing with her camel and one of these we built we built a tool for servi one of the big problems we had once we switched over is now now we have back-end or camel and we have front-end or camel and they don't share any code at all so how do we had we deal with this it's a relatively small team at the time we were we had three engineers before I think maybe with four engineers at the time now we have air we have six engineers so it's relatively small team throughout the life of this project I'm so had to stretch it over so I made this thing called tablecloth and tablecloth is a standard library which is wraps other standard libraries on the existing platforms so it wraps belt on the front end and it wraps the Jane Street core on the back end and has the same interface to these two libraries and if you the interface is something roughly like you you know it looks exactly like you might expect it to it's you know got list mapping you know string to list and and all this sort of fancy things and you install it or you implement it by just in your calling open table path in it then it overrides all the all the existing types and the reason that we created this was because we had we had a bunch of problem so we had one one issue that we have and if you if you've ever written any back-end or camel that they use underscores for everything and on the front end we use we use camel case for everything and snake case and camel case are different and you keep forgetting which one you're supposed to write and so on so if you've got a standard library that works on both which which one is it supposed to be is it supposed to integrate with the standard become a way of using snake case and underscores or is it supposed to work with the with the buckle script recently way of doing camel case and honestly bigger problem than I thought similar that they have different libraries and functions the thing was going on at the time I don't really know what happened with its that button front end was going towards pipe first whereas the back end used pipe last and belt is is written to support one sort of on the other and that was that is quite a big difference between the two and then core does not work in the front end belts does not work on the back end belts was sort of incomplete certainly relative to court Korres is huge and mature and then belts it doesn't really have support for PPX which or perhaps it didn't at the time but certainly that the PPX is that we were using on the back end weren't really supported with in Belton we wanted to reuse those pbxs because they were kind of useful so our solutions all this we took we took Elms standard libraries and we basically wrapped them we I say wrap them we reimplemented them with exactly the same interface so the tablecloth has the same API as as all of elm and the reason for that partially you know we we used the elm libraries so being able to slot something in it was really useful but also because Elm has really really well-thought-out standard libraries actually Elm has really really well-thought-out everything it's it's sort of a master class in in beautiful API and library design and so the simplest thing is like let's just let's just copy them they've put all the work into it already we solve the camel case and snake case thing by having both a so any function that you want if it's called you know to string there will be two capital s string and then B to underscore string and it'll just work the same exact copies of each other and hopefully the the optimizers world will remove them at various points the front end one was backed by Belton and the back end one was backed by core so we just we just call the functions and manipulated the the parameters to make him do exactly the same thing and occasionally had to have to implement something but but not not too commonly and then the solution to pipe first and pipe last was that we used labeled arguments so every function basically has one positional arguments and so it doesn't matter if you pipe if you prefer using the pipe first operator or you prefer using the pipe last operator all be anything the thing that you would pipe into it is going to be a positional arguments and all other arguments are going to be labeled and that that has worked really well for us actually it's it's super it has made this distinction really nice and it's sort of what core does on the backend anyway they use labeled arguments for almost everything in there and that's really nice as well so things that we intend but have not gotten ran to our support for PBX drivers not throwing exceptions all that kind of thing so I want to talk a little bit as well about the language support so the difference between our experience in L versus our experience in in reason ml is the we spent a year in each they are they're both these sort of preeminence statically typed functional what's the the ml derivatives of the front end there you know there's this other thing called they might have heard of that I guess is is statically typed on the front end so I'm not going to talk about that at all but can't really claim that that these are the only ones but these are these are the the ML variants that that are on the front end and they they chose a lot of a lot of different things and one of them is is that the language is kind of different so Oh camel is this is this old language and it doesn't really have a whole lot of user experience that has gone into it it's not it's not a beautiful language that's one of the reasons that reason it always was created and the it's not a beautiful language syntactically but it's also not a beautiful language semantically on the other hand Elm is is an absolutely gorgeous language if you haven't tried it it's it's it's it's just it is hard to describe how a programming language can be can be as beautiful as Elm is but it is but unfortunately there's things that they don't do in Elm that buckle script does add that I think that that Elm could learn and and vice versa there's that there's a couple of things that that a camel could could really improve the experience of coding but that Elm does really nicely so the first of those is that constructors in in Elm are callable directly so if you want to you know wrap something in an okay that that's a function that you can apply if you want to pass a function that wraps something in a constructor it's it's you know it's just an argument you just call it with just or whatever it is and some I guess but that's that's not how it works at all in in our camel instead you know camel you need to you need to create a a lambda function you need to create a I'm blanking the word I'm looking for yeah you need to create a lambda function and you need to you need to wrap that and we end up doing that in tons and tons of cases you see it in JavaScript a lot as well and and Elm has a really nice experience around that it does it with tuples as well and then standard libraries is built around using pipes really well and it ends up being this really beautiful experience to code in elm which is not which is not what we got when we were when we had stretched vocal script and we sort of tolerated but the reminisce to the better days in the past on the other hand our camel has has lots of lots of cool that we really like like the ability to have multiple match statements or search statements that are different types into sort of unify them and to add the when in statements in I forgot that what that's called the reason ml maybe it's when maybe it's where but it's it's super nice and once you once you start using those you realize that it simplifies a great deal of your code and we really enjoy using that on the other hand there's also this this object thing and it's like this class system does anyone know what that does I really don't know what it does it has these weird things like like functors and GE T's and I was in a conversation last night that the speaker dinner where people started talking about functors and JDT Zenaida immediately zoned i had no idea what those things are and honestly no desire to learn I don't think this should be a thing that someone should be required to understand in order to build you know web applications that that doesn't seem like the right trade-off so gonna go back to the standard library standard library in Elm is its glorious this is this is something we tried to help with with tablecloth trying to make something as glorious for the Ocala ecosystem I think we sort of got there um I think it's I think it's really nice I think you'll enjoy using it looking at what we lack that that belt has belt has all these functions ending in you ending in a capital u I don't know what they do we don't use them I I put him in the category with JD G's and sort of sweep them under the rug and hope they never come up but I think it might be something to performance and if someone could tell me what what they do and should I actually be using them or ever know what they do that would be helpful documentation documentation in local script and reason ml is bad does everyone know this am I alone in okay okay I thought I was going crazy with this and that's somehow everyone had to figure today I was trying to use the the documentation to understand how to do the foreign function interface things and I wanted to wrap an object that had a little bit of a complicated thing and there's two pages in the buckle scripting and then there's one very long page of the reason ml Docs and I have used a lot of foreign function interface --is and I've worked on JavaScript compilers and I have a PhD in in compilers and programming languages and I could not figure out what are supposed to do from the documentation and I think I think I think that's not me I I believe it might still be me but I think it is probably the fault of somebody else and if you are that person um well I think you should apologize so so the the coming from Elm where the language Docs and that the standard library Doc's are absolutely amazing it's it's really a hassle to come to comical script try to understand how this language works and this is known your camel already and also we hire you know how our junior developers we hire people who do not have a year of other camel to try and understand how to write stuff in this and you know we we kind of need better tutorials we need better documentation I suspect somewhere in facebook people have written this documentation because presumably there's hundreds possibly thousands of developers in Facebook you who presumably have some sort of like good onboarding but it certainly doesn't exist in the community at large so ecosystem final thing is there everything aren't that many users as I know notice this is it's not it's not a very thriving ecosystem Oh camel is worse like a camel is the back end of camel I guess what we call reason native these days is sort of flatlining Elm is is doing very well buckle script is doing middling I would say I I could be corrected on that we're not no we're not in a wonderful place and I think that there's a there's a bunch of things that we can do to address that the in terms of interacting with the rest of the ecosystem so Elm is incredibly bad at talking to the rest of JavaScript you you have to wrap them and if you wrap them and they don't think of the world in exactly the same way as elm which is a very prescriptive way if you've ever used redux sort of sort of that model the where is in Bacau script while all the documentation around it might be horrendous you can actually directly call JavaScript and that that is a super useful thing and I can go both ways so the the escape hatches are working live and well and and that this is the thing that enabled us just that caused us to switch rimmel and cause the search debacle script so that that's a thing where there's not a whole lot of libraries within buccal script but we can sort of if we use the JavaScript although it's it's still a huge amount of work to to wrap a single a single javascript library so our experience report how how things are going with dark so we've been we've been working on this base about two years first year in an elm second year and vocal script on the backend both years in you know camel on the backend we are having a lot of operational issues if you saw Shawn groves talk it's kind of the same where this is this is not a good place to be it's not it's not good there's not a lot of movements everything sort of works but sort of dozens and I I kind of thought that that reason ml was leaving the the native world behind but I saw a ton of talks on using it today so there's there's clearly was clearly some desire and perhaps the the killer feature although camel is is that we can write both but right now right now we can't really write back and suffer so it's it's it's horrible we we we have a bunch of we basically pay Google Clyde twice as much as we should to get maybe a quarter of the performance that we could if if a camel multi-core came along which it will any day now promise so the on the front end I would describe our happiness on the front end as a middling it's it's not bad we're we're not looking to move off but we're not we're also not like super ecstatic with it I'm part of this is is that part of the happiness actually comes from the fact that we have a fairly customer we're not really trying to reuse a lot of the community stuff and most of the time that we that we had trouble is when we is when we hit the community stuff and part of the part of the problem from us is that we're using this buckle scripty thing so we're using a thing that's sort of on the side of the main path that seems everyone here is possibly using reason react and perhaps that's a much nicer experience I I don't know because we don't use it but we're sorted we're sort of off to the side so it may have done this to ourselves and the as we as we start hiring more and more people were started we're starting to think like you know is is it a good place for for us to be is this the right thing since none of the engineers who are coming in are going to know how to use it are we going to be able to teach them or is it better for us to for us to switch off it and we ran through this exercise recently and we basically feel that we've nowhere to go this there's nowhere that's better so things things aren't terrible here they're fine and and we're probably going to stay here but all things are fine but but from that we we do get a little bit of sort of suggestions perhaps for the community and this is this is meant from a from a place of love from a place of wanting that the community to succeed I'm a little bit of an ED Sardar I'd slide around this I did get to go the speaker dinner but I didn't really know anyone there so the so the things that I would say about this community is that it is expanding rapidly and at some point when communities expand rapidly they have to contract and this is this is a necessary precondition to making things nice so if you think about this as the JavaScript world we had this huge explosion back in the back in the 2000s where there is like mootools and Yui and there's a bunch of other things in the world you know it exploded for a while then the world eventually settled jQuery and then we had backbone and angular and ember and the world eventually settled on a reactant and the yeah there's another explosion view in Felton and that sort of thing you know that double settled somewhere and each time Louisville will get happier but the important thing is that is that things is that after things explode for a while things need to contract and we're definitely in the exploding phase like this there's lots of things being built and none of them work together there's we have three names we can't even talk about the same things like we're talking about a camel and we're talking about Paco script and we're talking about reason ml and do we mean something specific by this when we say reason ml do we just mean do we just mean a syntax where they're camel toes just mean a syntax I I don't really know what people are saying necessarily and and I think I have a fairly good sense of the ecosystem and and this this plays out like if you're if you're a beginner this plays out if if you're searching for things on Stack Overflow it even plays that in in the communities like this very definitely a split community this the people who are during o camel and then there's people who are during reason ml and that there's some people who are doing buccal script which some reason includes me and they're sort of within the reason ml thing it's very confusing and the the Ocampo people have started their own discussion forum so there's reasoner there's reason chatter is name elder chat and then there's like discussed other camel and they no one's really talking to each other it seems thank you and it seems like y'all know each other and you know Jane Street sponsored so like presumably presumably there's some cross but yeah I don't know I don't know what's happening and that's that's really problem and I mean even if you install reason then all you do npm install' BS - platform and lots of libraries that you start with bs instead of like re ants and what what is happening and I can't I know that like there's a new name and there's no name and then and there's this other thing but like in order to make this accessible to people you kind of have to make it one thing and as it expands and it explodes great but yeah let's make it one thing again so I think that there's there's a lot of really cool things that are going on it seems that a great deal of them don't work together and that it's hard to tell what works and what doesn't and if you're using one thing I find a load of P P X's that don't work if you use into your camel syntax which is what we're using and maybe the thing here is to ban using their camel syntax maybe we're doing it wrong and maybe the actual thing we want to do is like everyone has to do it this way and everyone having to do it one ways is often better because there's less things to learn there's less things to less documentation have less less paths less things to integrate so I think there's there's still a lot of complexity in this but honestly I think it's still the best thing out there and I'm super psyched for for the direction that it is going and for things getting simply thank you

Channel: ReasonConf

Views: 3,449

Rating: 4.9619045 out of 5

Keywords: reason, ocaml, react, bucklescript, reasonml, reason-react, Reason Association, reason-conf US

Id: EnPodoPzimE

Length: 34min 39sec (2079 seconds)

Published: Mon Dec 16 2019