Confusion in the Land of the Serverless • Sam Newman • GOTO 2018

[Music] thanks Vinnie so thank you so much for coming out this morning such a great lineup I would rather be in all the other talks other than mine I mean that's a normal state of affairs but there was a really great lineup here so do make sure you check out the videos of the other presenters for the sessions that you couldn't see so my name is Sam Newman I'm here to talk about micro services some of you may have read my book on building micro services which contained nothing about service because we didn't really use the word back then very widely so now I've got two a second edition so thank you progress I also as a penny mentioned I work for my own company now I do training advising a consultancy work in the area of clouds micro services and continuous delivery but you can find out more information at my website as well we also have the slides of this presentation so this year's last couple of years have been a failure momentous years for me I don't mean in terms of politics I mean quite selfishly for me I mean I created a company with my own name which I think just highlights the inherent narcissism and ego that I have so rather sell rather self-obsessed generation so I figured I'm I should be getting in on that gravy train of being also self-obsessed and I think a lot of this came about because I'm now starting to feel very very old I turned 40 fairly recently I know you wouldn't tell to look at me but I am I am now 41 I was tell it I was explaining to my to my son the other day that I the first programming language I ever used was written in 1977 this is that was back in an era when you used a programming language they put the year it was written in right there in the name as you really knew how old the technology was you were using I kind of miss those days so anyway but I also remember when the internet came on a on a floppy disk I do remember the five and of course I even remember that eight inch floppy disks just to really date me we used those at school and I remember the internet used to arrive on often on a magazine on the front of a magazine and you could connect it and the thrill I got when I got my first 14.4 came modem and I still call Facebook the Facebook because that's its name again progress isn't always good I don't use it for that understand it equally I do not understand chat I don't understand what's going on with snapchat I watched people sort of making grinning faces and take photographs I don't understand how that business is worth billions and billions of dollars but then again I've come out of two really failed startups so you really shouldn't talk to me whatsoever about those sorts of things I'm not a complete and utter Luddite though I have been using cloud services for a very long time I sort of by acted and ended up helping create the first ever training courses for Amazon what many years ago was ten years ago now so I stumbled into it very early days and so I've always considered myself to be something of a person who actually does come to new technology can sort of get my head around things fairly easily I like trying to distill down the essence of the thing and try and understand what actually makes it important I was a consultant for many years but an odd kind of consultant because I like smite I like spending my time avoiding talking about hype and buzzwords which again you'll think is doubly ironic for a person who wrote the book on micro services but nonetheless you know we are we are people of cognitive dissonance all of us but nonetheless serverless has absolutely left me a bit confused it's taking me about a while to sort of get myself and dig myself out of self out of that confusion and that confusion starts with the name like any name you start with what is that name about you start trying to dissect it and think well what the hell is this service thing now obviously is somebody who is independent and it's basically trading on my former glory I'm always worried about some new hype or buzzword about to destroy my business you see right now in the guy and the hype cycle my own personal subject matter area of expertise micro-services is clearly running hind as we all know any consultant can turn a Gartner hype cycle into profit but I'm concerned here about surveillance coming up on the left and figure I better find out what the hell's going on just in case I need to shift my allegiances for one bandwagon to the other you can serve to bandwagons at once don't try into a third that's what I'm saying so when you start digging into service of course the first thing you start seeing is the near you know the sort of cynics are saying well service there's no such thing you know there is no cloud it's just other people's computers and this is sort of where I still have started from as somebody who's a half an OPS person I fell into operations accidentally I was on a project and I was the only person that knew UNIX and therefore I ended up doing all of the sysadmin work because how these things normally happen so I know there are computers there I've had to look after them really badly but I've had to look after them in the past so I sort of shared this cynical view and I sort of interested in sort of the the origin of the word as well because often when you look back to the origin of the word there's some truth or there's some reason or rationale behind it and subsequently gets lost the same thing I think happens with microservices it happens with DevOps it happens with agile if you go back to the origin of these things you can often understand well what was the idea behind it and serving us just as with microservices was nothing more than a banner term slapped on some things that are already happening I think the first citation I can find for the word service comes back to this article by came from an RW Labs and it's an interesting art of course is back in 2012 and Ken was reflecting on the different cloud technologies out there and saying you know this is this is sort of the description of the obvious and I'm sure the word was around before this is the earliest citation I can find but the freight service doesn't mean we don't know we don't have servers it just means that developers no longer have to worry about the underlying computers we're not worried necessarily about managing around physical capacities or units there's something here about abstraction it's not saying there are no computers that those are abstracted from us we'll come back to the idea of abstraction later on now that we don't want to pull back to this reference is a 2012 is because one of the first signs of confusion I saw was a conflation of terms when people say service a lot of people say more service is just functions which is kind of incorrect that's sort of like saying that agile is just continuous integration or agile just stories function as a service is maybe part of what we call service it might be a product with classic which is classified in the term service but it's definitely not what service is and the entirety of it and we can see this very simply by looking at when lambda was launched which is in 2014 2 years after term was first created so lambda might be one of the examples you know best as a cloud product that gives you service capabilities but it's far from the first thing that was being talked about as being a service product there is also this other issue that comes on the pit certainly people like me operations people think well I've got to manage the computers but we come back to that definition from from Cain which is the developers no longer have to worry about the underlying resources as developers we work with some platform that we find fantastic and amazing it's great look there's no detail here but of course underneath it all there's an awful lot of heavy lifting going on which confuses operations people especially you talk to them in traditional organizations you start talking about servers and it's like where my job's going the key thing here is not only are these somebody else are looking after these computers they're often people in a separate company that we don't even have to worry about but like all abstractions service is very much about you know the eyes that beholder it's all great for my point of view but I know somebody somewhere is having a really really bad time now I started wanting okay so we've got service so I'm getting closer so service is about abstractions I'm being hidden to an extent from the computers but hiding me from computers there's lots of software that hides me from a computer there has to be a more accurate definition or encapsulation of what what service is about that would help me understand why is it useful what products are and aren't serving us does that trade into how I think about using technology Mike Roberts who's an old colleague of mine wrote up a big piece on martin fowler's website on service architectures and so he was trying to explore this space look at different ways of using service technology and so it's really a great introductory article if when I take a look at he's also with his colleague John sharp in Danna a short book for O'Reilly which you could download for free which goes through this in a bit more detail Mike's come up with these definitions of what service is he sort of five-point definition which I think starts to get a little bit closer so we've got so Mike's five definitions of services that we don't manage server hosts or server processes again it's just from the point of view of the user the developer Oh using these server systems we be self auto-scale and auto provision based on load so we don't have to think about those things they'll just scale up or scale down as we need the cost is based on precise usage this is a real pay-as-you-go model so you're not paying for a process to be running 24/7 if you only use it for a few minutes a day you're paying hopefully for the few minutes per day and the performance capabilities are defined in terms other than host size or count this can often be bewildering to us actually as developers if you start using especially some of the database options things like cosmos we start talking about allocation units it gets incredibly confusing as to what the hell's going on in fact when sometimes it's quite nicer to deal with a good old fashioned I want a big machine and finally we've got implicit high availability that the system itself is just going to handle resiliency for it for us we're gonna come back to that idea of resiliency cuz I'm not sure it's entirely true now I think Mike's getting a lot closer here with the definitions although it's kind of interesting when you start whenever you make a explicit definition like this it starts becoming fun to see what things and fit what exactly things fit into this definition of service so obviously we've got the function as a service type offerings from people like Google my Microsoft and Amazon yeah I think Amazon lambda gets a lot of the attention I actually think Microsoft as you as cloud functions are actually better executed products personally so you know props to Microsoft for that one but we also have to look as well at the various different back-end to the service that we have things like cosmos Dynamo as its database actually get a lot more confused we sort of assume that all these products are going to be complete turnkey that there are simple just turn it off and often turn it on and off you go it turns out that when you start getting into the database plain then things get a lot more complicated than you might think with a function you tend to talk about how much memory you're going to give the function that it then when you start looking in a different database options from the vendors you start realizing all this idea that there's no operations work and this stuff just magically works out how to scale turns out to be a little bit far from the truth now it's interesting for me because I still hold up the gold standard of a developer experience for deploying and managing applications as Heroku I still think is by far the best execution of a platform as a service and it's no surprise why multiple companies have basically copied that developer experience you know looking organizations like Deus and you're looking at the re riot of the cloud foundry interface to mimic Heroku like deployment and that's for good reason it was excellently executed and developers love working with Heroku kind of interesting thing here is that doesn't match Mike's definition of service it's because we're still managing containers you talk about a certain number of dynamos our costs are based on the number of dynamos we're running not necessarily on whether those dynamos are being used or not we certainly don't get implicit high availability with those mechanisms either it's still fantastic so hearing is the first lesson you can use fantastic stuff if it's not serviced don't let that worry you the kids might bully you in the playground but you'll still be a happier developer turns out there are other things that sort of fit into Mike's definition of service when you get down to it let's start with sales forces force comm platform there you go can deploy my apex functions which is heard a horrific thing but you know I can use effectively sales force like a platform as a service I'm not managing containers things are just run and scaled on my behalf really fit what I think of a service though there's something a bit missing there for me you could also argue things like fastly or CloudFlare are serving as platforms I'm being priced charged based on my bandwidth use CloudFlare for example and they even allow you to execute code on the edge now that is a service product as a service cloud feature it's not a function as a service platform but it's a service platform it's highly available it scales up or down without me being to need to be involved I'm charged based on what I use I think some of the earliest products from Amazon actually fall under this bucket of service Amazon's s3 which was probably in second or third product they released or their sqs service sqs was the first-ever product released by AWS in 2006 which is a very very simple q I'm charged based on the amount of items I put into or out of that cube that is a service offering we've had that with us since 2006 this is just the story really when you get down to it of abstractions we have a history of creating higher and higher order abstractions we think about the cloud journey we started off with infrastructure as a service virtualized products virtualized primitives sitting on top of operational physical equivalents storage compute networking and then we realized that actually working even with these great virtualized low-level primitives was useful but we wanted and we urge for higher order things and so now you're looking at things like maybe container orchestration as a service things like eks for example or the very cool Panetta's platforms you can argue a Clow family to an extent but certainly the platforms that allow you to manage and orchestrate containers is a higher-order abstraction on top of these underlying physical virtual abstractions and then of course we always had the glory we always thought that platforms of services where it was going to be Heroku was a very early shining light and we thought this is fantastic clearly everybody is going to execute along these lines and you know five six seven years on we're still sort of waiting for this in a way when I think about the different products that fit under the banner of serving us what I'm almost seeing is like in the same way that infrastructure as a service had these virtualized primitives of storage network and compute under the service banner we have a whole load of platform primitives we have things like messaging databases function execution that also sit underneath sort of banner up here and the server stuff for me is really the primitives is the lower level primitives we always wanted underneath the big platforms as a service the issue always was with the early platforms of service executions it was sort of a take-it-or-leave-it you had your all in or your all out and now we can kind of mix and match these building blocks now of course you end up you've also got a lot of confusion in a space like kubernetes is wandeling around you know where am I in all this thing and arm are going up stack sto is confusing all this and you try explaining what you do to people especially you talk to non tech execs and take them through a diagram like this and you become overwhelmed with acronyms and it all becomes quite confusing and you end up feeling like a little bit of a it gets really confusing sometimes and you end up do feeling like a little bit of an ass now you know it took that even Kelsey Hightower who is an amazing sort of campaigner for kubernetes which you would you know you could are I think we should increase it he will become an implementation detail for most of us you know he he gets it right it took it you see he was like I love this because I was like I saw this on Twitter he was coming to terms of what service was and then coming out with a really nice distinction of what for him makes service so important no service is about I don't have to worry about a lot of detail I'm now just going to let these things get sorted out for me I'm going to focus on building my product right and this is somebody that is you know championing kubernetes which is all about implementation detail I think you could argue probably increasingly will be hidden from us underneath I suspect more service type offerings I do come back those that came from definition you know the phrase service doesn't mean that servers are no longer involved it simply means that developers no longer have to think that much about them it's as freeing us up we don't have to worry I think there's some lies we tell each other I mean we don't have to worry I think that what means is we don't have to worry as much it turns out we can we worry about a lot less but the things we worry about are different for me it feels like when I moved from to being a really bad C programmer to being a mediocre Java programmer part of the reason that happened was that a whole chunk of my brain when I was a C programmer was worrying about memory management and it turns out I'm really bad at memory management I would leak memory all over the place when I moved to Java I had more of my brain power to focus on other things because Java did the memory management for me this is what happens right we build higher-order abstractions this tweet recently I think really also summed up another element of service I think it's overlooked and to an extent is a continuation of the journey that we've had when we've moved to the public cloud functions is a programming monarch a subset a service but service itself is as much in anything a different change in billing it's a billing model I don't think it's a new billing model though right I mean pay-as-you-go has or being a challenging thing for organizations as they move to the cloud for those of us who are developers this idea that we only pay for what we use has been really attractive I like the idea that if it's an architect I come up with a stupid design I see the impact in my bottom dollar the next day because I've spun up way more machines that I need and I get to address that and change that whereas before I would to come up with a stupid design I would then spend three months going through a procurement cycle to buy too many machines I would then have too many machines in my physical infrastructure structure for like the next three years and that mistake would be looking at me for years and years to come and it's very little I can do about it but it's okay cuz normally most was left by that point and we can get you know someone else to blame get the blame for it my lines as an architect my first move for the trial about that feedback cycle I could make a decision I could see what happens I could understand the cost impact of that and it becomes like a little bit of a game you know it's fun when you sell to teams right as a challenge let's see if we can reduce how much we're spending the cloud that's fantastic the service in a way is just a continuation of that idea it's giving developers primitives to build applications and those primitives allow us to have this sort of pay-as-you-go mentality this is a continuation of the original drive behind the creation of Amazon's internal services to reduce this amount of undifferentiated heavy lifting those Amazon teams that are spending all of their time racking up boxes and connecting cables realizing well actually we're spending all of our time racking up boxes and connecting cables and we should be shipping software so how do we reduce those things that don't make us special my ability to deploy a function shouldn't make me special my ability to write a function should make me special and again this is just a no-no an ongoing journey that we're on so Brian Merrick said to me years ago that developers turn caffeine into abstractions it's almost what we put on this earth to do we do it naturally like doing it we fetishize over doing it every single new framework that gets released every seam library that gets released is this a continuation of this we don't you know start off a machine code and then a long cane assembly code and all the machine code people said that the assembly poke code people weren't proper programmers and then the assembly probe code programmers showed that yes we are look we can put entire operating systems in assembly code and then an Along Came application code and programming languages like you know C and Fortran and and all the assembly code people said you're not a real programmer and we just keep creating higher and higher order abstractions and waiting for the people from the previous generation to die or leave the industry and that which is great because then we get to create all that we get to recreate all of their mistakes in our new level of abstraction so nonetheless there are still some questions that come around service and they come out quite a lot so I thought it might be nice to explore some of them one of the first ones that comes up is this idea of resiliency and this idea that these platforms are inherently highly available and that's sort of true ish so many years ago I was working for a company pricing something called collateralized debt obligations I won't explain what they are but if you want to watch the film the big short about the global financial crisis you'll get some inkling as to how I sort of helped it along by accident we were pricing we're Kenya's calculations on a big grid of surprising notes and so the idea was some requests would come in saying do this calculation and we've tried those calculations faster than anybody else in the City of London that was basically our job it turns out that just made us lose money faster but we found that out years later so it was fine now and you're working users great okay we'll go fast and fast so how do we go faster well we've got to add more machines so we had you know all these upstream services that we're extracting data from to do our calculations getting information about different risk levels and market data sort of invents we even had data on the weather patterns believe it or not coming into our system the number of kidnappings in Columbia was one of the inputs to some of these financial products you do not want to know why and then you know we've put all these calculations into a database and the world was we felt fantastic stuff and so you know one day we thought well let's go how do we go faster and friend of ours and the operations team said Sam right we've got a data center sitting there idle is basically a whole disaster recovery zone that's turned on all the time there's got to be ready to be used for trading but it's empty right now we can just run all the workers for your big pricing cluster on these idle machines we thought this sounds like a fantastic lunchtime activity and so we went from about 20 machines to having about 200 machines over basically in space of two hours it was fantastic like it's completely elastic scaling grid that we just wrapped it all up and it was great so this scout out beautifully and then some really interesting things started to happen that we couldn't quite explain at the time what's going on miss teams are getting a lot of errors it turned out what had happened is we had scaled up our pricing grid and promptly blown up every single thing that wasn't actually in our pricing grid and these sorts of things were like oh this is mismatched capacity problem right and so you know we think about we have mismatch capacity problems all the time and we have mechanisms to deal with these sorts of things you think about something like a classic application is talking to a database we have a connection pool that taught that we use to constrain how many requests can go through to a database at once one of this is one of the ways in which we balance the load on a database you restrict a number of connections that can come in and hopefully if those connections have fairly even load which isn't always a case of course we can hopefully make sure the database remains up and we don't stop you know we don't sort of bombard it with too many requests so as a requests come in there's a certain number of workers and that becomes a throttling mechanism right so we throttle a load and we can even allow those things to timeout if the database underneath us is is acting slow it's not responding correctly we'd get that traffic and so I started thinking about this because I thought well hang on a minute connection pools are a throttling mechanism to defend other services or other dependent subsystems from being overloaded I remember I blew up everything that wasn't in this elastic pricing to it because we had no throttling mechanism is this prot a problem for the service platforms and I started think you know we think about say a function of so it's platform like cloud functions or lamda now what happens is we have these functions running and as the load comes in as a request comes in you launch an instance of that function to handle that request it's one of the nice things about the programming model now as those requests come in if I'm talking to a traditional database that's just going to relate one to want or more load coming as the database and there's no throttling mechanism here because a connection pool requires shared state across function invocations which with functions you don't have because when that request is finished that function goes away so you have no strata lling state and I was thinking is this a theoretical problem am I worried about okay nuking the database in this way because I've lost the ability to throttle connections via cross request state the answer of course here is that people and certainly proponents and vendors will say well the way you solve that Sam is you don't use a traditional database that is constrained you have your functions talk to sort of you know cloud based databases functional server space databases like cosmos or dynamo that are going to dynamically scale as that load increases and the my answer to this is always been well that's great but it sort of implies I'm all-in on service so I've got to avoid this mismatch capacity problem all of the things on that critical path have to be able to scale elastically and the issue is you know I work with lots of most of the clients I work with already have software it's a surprise right and if you want them to embrace service products you can't say rewrite everything over to these new products you want to give them a path to migrate piecemeal over to this new product these platforms and therefore most people I was chatting to they're interested in making use of things like cloud functions needed some sort of migration path meeting they've had sort of a hybrid app status when they wanted to mix some of their products being serviced and some of the some of the features they're using being implemented using service technology and some of it being more traditional now at this point I'll starting tomorrow I'm sort of tilting of theoretical windmills until I saw a really interesting talk by Steve Faulkner at bustled and so bustled are often cited as sort of being one of the great the children for being all serviced so bustle if you don't know or a media company in a very new media company in the US they run about 60 65 million uniques per month so they've got decent load in fact they're not all service advisor this interesting talk from Steve talking about the issues they've had with exactly this problem so their back-end data tier is already snowed z-- we can talk separately about the rationale of using Redis as a database and but they had exactly this problem they were finding that their Redis instances were being completely and utterly swamped by function invocations and that was really interesting to me the sort of the really puzzling thing in that situation was the sheer number of requests that they were seeing their Redis nodes were hitting the 10,000 connection limit but at the time you can only have a thousand function invocations at any one point in time on lambda these requests are coming in the functions are being spun up but those connections weren't actually dying when the functions went away and so you had these Redis nodes that had over 10,000 inbound connections and the whole system fell apart they actually had to do things at the Redis level itself down at the Ethernet network level to expire those sort of left collector connections that are being left behind by the function invocations you don't you sort of having to deal with the implementation detail of the underlying servers platform to avoid those problems so gonna find that now if you search for Steve Faulkner's talk on bustled he talks all about these issues that they had we also have other throttling mechanisms we use like the circuit breaker a circuit breaker again is that it is a mechanism to allow us to more sort of gracefully handle downstream failure you know I'm talking to some downstream service like the risk service that we had in our pricing example and if that risk as the load comes in all the requests go through that connection pool and also go through the circuit breaker if my risk service has an issue it's not behaving correctly I can open that circuit breaker state and allow my connections to basically be expired so I move to failing fast which can then also lead to load shedding and load shedding is an important way of relieving back pressure on a system and keeping systems running but again circuit breakers things that we assume we're going to have to have in a micro service environment again rely on maintaining per client state across requests where is per client state in a service environment now my my theoretical idea here is that we basically need some sort of magical throttling and load shedding middleware something that can give us those that that cross state view of our resources that are outside of our function invocation something that gives us the ability to do things like circuit breakers and connection pooling that aren't necessary applications now I think theoretically this is what we could be out use a service mesh for I haven't yet seen anyone do it which is the interesting thing I have asked some of the vendors they say well theoretically you could do this I am wondering how long it's going to be in fact before the cloud vendors themselves start providing their own effectively service meshes for this exact use case because I'm far from the only person seeing these types of issues of course there is still an issue and potential problem I might be tilting at imaginary windmills worrying about my massively massively scalable application but of course one of the promises of this technology is you do not have to worry about highly available how about high availability there is a big difference between high availability of a specific product from a vendor and high availability of your actual system on your solution and that requires a much more holistic view another issue that got cited very early on a certainly around function as a service was around security and part of the issue was that first when lambda first got launched is that people very very quickly busted outside of the function and that were able to run command prompts inside the function and you could see very easily that under the hood it was all just a container right and so under the hood function invocations are being done inside a container instead of contain a runtime that's obviously you and you think about it from an operational point of view makes perfect sense containers spin up very very fast never quite a good quick response time which is coarse what we want for function invocations they're not launching a virtual machine every time you launch one of your HelloWorld functions because beyond the fact it would be honestly expensive it would also be horribly slow so obviously there's some sort of container technology and the issue of course here is that is ongoing mantra you know containers are great for isolation at some level but there is this issue about having untrusted code all running on the same container engine because the levels of isolation that we get from containerization are better than no containers but they're not as good as virtual machine isolation and so you start worrying about this multi-tenancy exotic problem if I'm running my code in a container on the same machine in somebody else's container is there an issue there because we don't like this situation when we're running our own container orchestration platforms now I actually think the problem isn't as bad as it's made out I think I do have issues as running untrusted code on containers I think we can talk about the public docker hub for a couple of hours if you want to but that's a different story because we have some issues here because that what we're really talking about here is where I'm is my function and I'm worried that some other malicious function on the same runtime it's going to break out of its sandbox and a get hold of my processing access my data now firstly it's very hard to see how this would be a targeted attack because how are they going to launch enough functions to guarantee that their function is on the same machine where my function is going to be executed that's an almost impossible problem to solve from the outside because again you are limited about how many of these function invocations you can make and of course if my function isn't running it's sort of not there ish and also we're running in a sandbox you tend to actually be running inside a runtime now I will say kinda if your function is not running it's not there we know that's not actually true so firstly when so when languages like when support for things like Java and c-sharp came out for functions we had this cold start problem you know you'd hit the first team function invocation it'd take a while for that function to spin up and that could add a delay and so people were doing all these techniques to keep those functions warmed up so to be able to serve their traffic quickly well it's now happening to the cloud providers have kept those functions running on your behalf certainly you know for lambda and on Azure you can assume that your function isn't really going to the runtime isn't going to stop running it's for about an hour now you can't share state between them but the underlying runtime is still there it's not quite the same thing this has also got a lot better on things like azure because as you you have the option of running actually completely isolated containers within so completely isolated kernels which sort of blurs the line between virtual machines and containerization so if you look at things like vSphere containers I want to say it's the way you get isolated container isolated kernels within a container or a traditional Linux kernel containers all share the same kernel which is the the problem or at least the challenge with improving that isolation one of the last sort of big issues that comes up around any of the service offerings is this question of lock-in and I've had a chat with a couple of clients recently who loved the idea of what things like back-end as a service or function as a service can provide but their concern is well we're going to be stuck and this is always the issue when you buy into an abstraction it's how portable is this abstraction and if you look at the service products all of the servers of products are nor the majority of them are tied to a specific vendor in the way that lower level abstractions aren't necessarily yeah although a function is sort of portable all of the elaborate sort of management around that function isn't portable and if you've ever tried using and setting up your own lambda function without the help of a framework you'll realize what a giant train wreck that is so trying to pour that over is quite a pain and so there is this idea of okay where I'm gonna be stuck I'm going to be locked in and really sir people want portability that was always the promise the promise was that the cloud was going to be some sort of utility I would just flick a switch and change a provider and it all magically would work and that was the lie we were told on the early days of the cloud like in the UK I can go and fill in a form and say well this is how much I pay for electricity now click a button or you could pay this much roulette Risty I click another button and I have a new provider it takes about three or four days in the UK to do that whole process right because electricity it's the same stuff that comes out of the wall I don't have to change the plugs in my house I don't have to worry the voltage changing is all completely transparent that is true utility most of the services we use on cloud providers are not uniform in that way there isn't really an open marketplace we don't have transparency of switching and I don't think we should want that either necessarily I think the problem here is that we always phrase this conversation in negative terms we talk about lock-in we talk about locking right we say you know we are stuck we cannot go anywhere well it's distill that a little bit really it's not about being stuck and I think this is the wrong way of thinking about it you need to invert your think this actually comes down to option theory in a way what people are worried about is it might be great now but what if it's not great in the future I'm gonna have to move aren't I oh I'm scared about moving that might be a lot of work in the future the flip side to that argument though is to say that we're not going to take advantage of this call product on the cloud that could give us benefit today because we might have to move later and so by not taking advantage of that technology today we are paying a cost today in the hope that that will avoid is paying a larger cost later the issue is none of you can tell the future what ok some of you might think you can tell the future but you can't and so that's always a risky proposition I also I think it's much more rational to think in terms of different products within that cloud vendor some products are easier to port than others if you think about the Coyotes like to thinking about the cost of migration you know you think about moving something like blob storage so you moving binary large object storage that's fairly portable the abstractions are fairly similar as is compute as is the ability to spin up they know machines once you've got the Machine the machines behave more or less the same you start adding low balances in ok they're pretty portable I can the abstractions are the same you start moving into the function platforms that's an issue you know if I've got up if I'm running PA a PHP function on Azure which I can do I can't run that on lambda without some kind of shim execution for the backend is a storage solution is even awful even more awful dynamo does not exist anywhere other than Amazon cosmos does not exist anywhere than on Azure spanner does not really exist anywhere than on Google if you build your solutions to use those products to move them to a new vendor you often have to change the application code itself to make that migration and so if you are worried about the potential migration cost don't see it all as one size fits so it will break it down what are the things that you would like to use what are the things that are going to be harder for you to move and what other things that going to be easier for you to move ultimately the power of servers comes from abstractions and buying into an abstraction if you choose not to buy into that person's abstraction you don't get the benefit of it I think the service offering you probably should actually be all-in on one platform because there's a whole lot of issue across cross cloud usage now I think for a larger organization it certainly makes sense to have a multi cloud strategy for a single solution you're building it's rarely a good idea to be multi cloud purely because of you know latency and the fundamental challenges of disputed systems and networks let alone the deep skill set you need to build up to be truly effective on more than one cloud vendor there is this services you pay now or pay later thing you just got to get your head around a little bit have the conversation but don't necessarily believe that you can tell the future but if you break it down into this discussion I think it goes a lot more easily there's one last thought about this this whole area is that people assume when you're locked in it's always a bad thing when you're stuck in a place it must always be a bad thing we talked about the walled garden effect oh no I'm stuck inside the Apple walled garden you know what the Apple walled garden is really nice it's like a lovely beautiful manicured garden as a babbling brook I sit in my reclining chair and and someone brings me martinis I sip them by with a babbling brook listening to the cricket and I mean it's a fantastic lovely place to be some walled gardens are nice places to be don't get me wrong some are horrible right in some of them it's nighttime the walls are covered in barred wire and there are monsters behind every single tree and you can't escape and you want your pet right I have been in developer ecosystems like that before they're not fun right but some of these cloud vendors you might well be you know in your parents locked in but they give you so many amazing capabilities that you might actually be better off for it for some of the service offerings out there you are starting to get you get some options about running now in other places with a more portable abstraction and you can look at things like open fasts which is a platform to give you function as a service type capabilities on top of kubernetes that that could be good thing because you get because there are multiple vendors that will run kubernetes on your behalf you could use open fat to give you portability across those things this actually isn't this is getting really a load of interest right now but it's far from the only one we have we have open fast we have fast Nettie's we have fission we have open risk we have open lambda and there are probably five more since the end of this talk of all functions those platforms running on kubernetes guess what by far the least interesting thing about service look at the other things the other service products that we've talked about already you know things like messaging there are the ability to have a portable serverless messaging solution that gives me the ability to send you know point a point on multicast bigger issue much bigger ticket item are those service databases which even the main vendors are still struggling to get right there's no portability there right this is a real if you look at sort of the the cloud native computing foundations projects they're incubating this is the big hole the storage options around that stuff and the pure data plane options around that stuff are completely lacking because guess what that's the really really hard problem and so for those of you who are looking towards maybe wanting more portability across platforms and things like that so maybe using something like open fast or one of the other 58 kubernetes functions service frameworks might be a good way to sort of balance I get the functions but I get the portability but you're also going to have to potentially opt out of a lot of other cool stuff if you really want portability across those platforms you're going to be having to run your own message brokers on kubernetes your own database on top of kuben and that's not necessarily an easy thing to do so coming out of this whole thing I may be a little bit less confused in a way I sort of I sort of I'm quite fortunate that I was you know around when agile was just becoming a thing and you know then you know out of that we had DevOps and then we had micro services and it's been this whole sort of slew of new buzzwords that we and I think it's sort of interesting you see his transistor density doubles right you know you've thought about clock speeds getting faster and then we we talked about there being more cores per CPU I think the way that transistor density doubling every 18 months is now impacted the marketing world is that these terms become devalued even faster than before I think it took a good five or six years for agile to become meaningless about three or four years for DevOps micro-services was meaningful for about six months I think service you know maybe what two to three months at most you know I want you to get away from this idea about you know I can't use it because it is service I can't need it because it's not service just think about these things as better abstractions than we've had before this in many ways is nothing new I think there's a lot of people running around saying oh it's amazing it's going to change the world it's the best thing we've ever seen it's just another iteration on our ongoing battle to create higher order abstractions to allow us to get more things done there are no future where fewer service there's no future with fewer with fewer programs with less software we're growing the need for software way faster than when growing the developers that can write that software and so creating higher order abstractions that allow us to get more things done is really quite important and some view serverless products like that maybe the difference is that they're baby those abstractions are more tied to specific vendors than they were in the past and that might well be cause for concern for some of you I will say though if you really want to get the most out of this stuff you probably do right now have to throw yourself all in on one of the vendors really adopt and get to know as you oh really well really get to know land really well really get to know Google really well if that's where you choose to spend your time and energy that's what you were to take the most benefit the most advantage of these sorts of things the benefits in terms of reducing that undifferentiated heavy lifting are quite large and certainly of all the new abstractions I've seen over the last 20 years in the industry you know this is a big step change this is the biggest step change potentially for application developers as it was for operations people when the public cloud became apparent I think that sort of the order of magnitude just that we now get to feel a bit more of it I think it's also just about experience we're getting smarter are understanding what we want you know a few words of warning though the ideas behind this stuff are conceptually quite simple I still think the execution is really very poor when you look at say how that the details of how you get lambda working it's still nowhere near as neat as the idea is and most of database options are also incredibly poor in terms of how easy and turnkey they are go and spend some time with the calculator that Microsoft provides for working out your allocation units your cosmos usage and then come back to me and tell me oh it's easy I don't have to manage around use it's just scales up and scales down magically so I do think we can and should do better but just remember subtractions all the way down and that's absolutely fine I have just done a whole video series on a service of how service and micro services are related you can go watch that on Safari which is a writer's platform you can watch that for free why your builds are on and I'll be sending out the slides for this talk and everything else on Twitter afterwards if you don't know more information you can find that over at my website but thank you very much for your time [Applause] the types of questions I have few questions on on a app please remember to rate the talk I don't know and you can ask questions yes what's wrong with paper so you can actually ask with the mic in the middle or send paper notes that were teeny if you want questions for the app penny will strip out all the questions that I want to answer yeah I'll the first one yes can you give some advice what strategies to use to emigrate from analytic systems to serverless so I'm a consultant so what answer do you think I'm gonna give it depends of course I'll charge you a lot of money to say that it all depends I mean first it does Bend so firstly are you on the cloud if you're on the cloud then great then I start looking and I actually what I start doing this is why I've driven it with a few teams is get all of your developers to list out all the things that they do that aren't about actually writing code and identify where they spend that time and that energy for some of the clients I've worked with some of the biggest wins have been just switching from using a traditional database over to using something like dynamite or cosmos for other clients it's actually being piecemeal building new functionality on cloud functions but I would drive this by looking for the win right so think about the work and the effort you do everyone's application is a bit different and so it's harder to say exactly you should definitely do this and then this it's fit for you it might be not having to run your own RabbitMQ cluster and using SNS on on Amazon for example but I would list out what are your developers pain points and then see what service functions on your features and your crowd could address that I hope that was good hey Sam thank you for for your presentation so while in micro services the most fundamental thing was actually sociological one so basically we have different teams what like distributed across the wall they are like working on different parts of the system so here you are saying about billing isn't just it easier to charge for the CPU and memory usage it's right I think for me the billing isn't necessarily new it's a continuation of those ideas that we have coming out anyway and I think being charged for what we use is the interesting part here and so I think this is the thing is easier and you had the right question when we first had the cloud and it was like oh we can spin machines up with the spin machines down we're going to be charged for what we use and then you know your tuition operations department would go or your typical accounting department ago that's too confusing we need to know a year in advance how much money we're going to spend because they were used to buying machines there's a capital expenditure thing and writing it down against the accounts and when you started moving to per month billing with variable being cycles that was really disruptive and I think we're seeing a continuation of that with service because you're right it's easier it's consistent and it took a while for us to realize yes it can be consistent it's also way higher and so if you get better at that and you start thinking about those per month billing cycles and then you know I think it's an iteration I think those those part it will take time for you to get to that next level it will happen but it's not going to be easy or as you go that makes sense yeah but you know even with containers you can just calculate the amount of the CPU usage if it's not being used and there is no usage so you're not charging for that well only if you turn it off and that's the thing you have to turn it off that's the data which is the missing part Simon Wardley says you know you know service is less it's less DevOps more FinTech arguing that there's going to be whole new tribes that will appear inside organizations whose job it is to handle cost accounting for the use of service cloud products this may well create a whole new disappear in terms of IT accounting I hope I don't have to give that talk in twelve months from now though we'll do one more here no more switch let's say I want to take advantage of several s benefits like fine grained scaling and up to to the workload how would you make it work for data intensive low latency apps is it possible and if not so low latency is interesting because some of the issue with any of the cloud any cloud services suffer from theoretically can suffer from variable performance and a lot of the time your variable performance is due to so what else is happening on the cloud there was some really great stuff done by yang I think is here somewhere the burning monk online looking at variable performance of service functions at I'm John Chaplin did some work looking at variable performance over time where you would see like functions would be behaving differently at different times of day and it wasn't it wasn't necessary think you could plan for and so you'd often have to overcook your functions more memory than require to get guaranteed throughput and so when you start getting into low latency what you're often talking about is consistent behavior consistent throughput and performance and so if you're concerned about that what I would say is you actually get your only thing you can do is actually build the solution or build a sketch of the solution and see it out see what variability you get and so for me it's less about the service products necessary it's more about the fact that you don't have that control and you might be a bit more vulnerable to sort of the cloud weather effects but certainly my video so as I go through stand but if you look for yeah stuff done so if we were search for the burning monk and function performance or the work done by John shap in over like Fibonacci calculations you know he found was did you run the Fibonacci calculator on lambda and he would see with a lower memory footprint sizes he would see incredibly variable performance of that over time if he whacked up the memory to like 1.5 gig or something for the process he would see completely consistent performance and so you guessed that under the hood when you're giving these functions smaller memory what actually happening is they're using birth Commission's are underutilized and therefore going faster than you'd expect the Google site reliability engineers sort of have a mantra which is it's better to be consistently slow than in consistently fast because you can plan with things that the instance is low so that'd be my the data intensive stuff does not worry me in your question is the latency issue the last question yeah one more question so you know the server loss functions we already seen it somewhere so basically we have to request response context which is called PHP for example and it's you know they it goes not very well when you in the particular context do you see that there are gonna be some issues with that as well there were lots of others of challenges setting em function invocation so some of the other issues I think are legitimate are the developer workflow how do ID bug these functions this story from lambda for example is quite poor as you is head and shoulders above in terms of giving you the ability to debug locally and debug on the cloud whereas lambda has more of a just a local emulation model which is not the same thing I think that will get better I hope it will get better but at the moment I why part the reason why I think Microsoft are ahead on the cloud functions game one of the other issues is monitoring and again Yan's been doing some great work looking at how you actually understand and troubleshoot things that happen as service system you should also look at the work being done by charity majors looking at how you get sampling and tracing out of those systems things like app dynamics and Azure can help and cloud trail but they're not the whole thing so those are the kind of those are the two big ticket items that I see is being an issue that people are hitting and I think over time as we go from companies that have been having these solutions for like there are zero to one year problems and then there are one to five year problems I think we're starting to see the one to five year problems coming up now and we're seeing solutions coming out to solve them but those are the big two troubleshooting and the other one I said whatever I said troubleshooting in something else anyway alright thank you so much your time the slides will be out on Twitter so thank you very much for your attention and Cuenca [Applause]

Channel: GOTO Conferences

Views: 35,232

Keywords: GOTO, GOTOcon, GOTO Conference, GOTO (Software Conference), Videos for Developers, Computer Science, GOTOams, GOTO Amsterdam, Sam Newman, Serverless, O'Reilly, Security, Cloud

Id: Y6B3Eqlj9Fw

Length: 54min 36sec (3276 seconds)

Published: Tue Aug 07 2018