Dos and Don'ts for Serverless and Azure Functions - Jeff Hollan

okay everyone we will go ahead and get started I'm very excited to be here this is the last session of the last day of NDC Oslo so everybody here watching you are the real heroes of the conference this will be a good session though this should be a pretty fun one so I'm Jeff Holland I'm program manager working inside of Asher back in Seattle Washington and I'm on the Azure functions team so I'm program manager for Azure functions which is one of the core pieces of service for Microsoft Azure that said a lot of what I'm going to be talking today are going to be some of the concepts to know about for as your functions but they are not necessarily unique to Azure at all a lot of these practices are good for any even distributed system all around but especially around server lists so whether you do or don't use Azure this should still be useful for you but but do use a sure you won't be sorry about it so we're gonna go into a little bit of an intro of functions and server lists but we're gonna spend a lot of our times on best practices architecture and some examples so with that said in case you aren't already familiar what are as your functions in the very basic definition of this is as your functions much like other service providers allow you to have your code executed in response to some event so in this case I've got a really simple code sample this is in c-sharp and all this is saying is I'm gonna get an HTTP trigger request to some endpoint and whenever that request comes in I want you to run this little bit of code here now the code you might want to run might be you know scheduled based hey I want you to run this sequel query every day at 1 o'clock p.m. it might be message base like every time a message shows up on some queue or stream base like stream this data and constantly process it as it comes in but it gives you as a developer the flexibility to just write your code and not have to worry about the scale the hosting the resiliency of the service that's all managed for you out of the box from as your fun excuse me as your functions imma grab a drink real quick so with that said a term that you have probably heard a lot of this conference been some great sessions already very brief definition I'm gonna jump right through this which is what is serverless as your functions is a service technology and service is about an abstraction of servers first and foremost like when I publish that code that I just showed you I'm not worried about how to host maintain patch and secure the servers behind the scenes I just focus on my code it is event-driven the resources to run your code are provisioned automatically for you on-demand at whatever scale you might be requesting so you attach it to some type of event I've already given some examples of those already and you're only billed for when your code is actually running so if I did have a serverless API and nobody called it during the entire month I wouldn't be charged a dime and then if the next day all the sudden I got you know thousands of requests I'd only be charged for those thousands of requests now as your the service platform looks a little bit different than you might see in other cloud providers there's actually three core components we're not going to spend a lot of time in these right two but I think it's important to know as you consider what a service architecture might be formed of so the first one you have functions this is you know the code that you write and publish code that you want to run off of some event and Azure there's also something called as your logic apps I'll show you just briefly what one of these looks like in a second but these are workflows this enables you to orchestrate code or data in a service fashion and there's also a service called as your event grid which allows you to route your events to different pieces of your architecture so you might say hey I want to be notified whenever something happens on my virtual machine or on my storage account or I want to be notified whenever an employee is hired by my company so I can go do some something an event grid is the platform piece that lets you wrap that information now on top of it it's very likely that a full application isn't just going to be service code service workflow done events you're gonna need data you're gonna need security potentially for like user logins potentially some intelligence on top of it as well so you get rich integration with a bunch of other services inside of the Azure cloud to do things like store your data in a nose database or run machine learning to give like order recommendations like maybe you have an ordering API and you want to send them an email and say hey you might also be interested in these items weaving those types of intelligent services is very easy as well and then something we're very proud of an a sure that Microsoft's obviously always taken very seriously is the development tooling so I'll go through a bunch of the this today whether it's monitoring developing testing or even continuous integration and continuous deployment okay so that's kind of the overview of service many of you might have already known much of this let's talk about some examples and one of the reason that I personally really love service is just because it's changed how I am able to write applications so the first cloud app I ever wrote was actually an app for the Microsoft band if you were familiar with that that short-lived product from Microsoft this is a picture of the app here what it was was a simple app so that if you were a fan of sports you could open up the app and you could say hey I want to know about you know NBA scores or football scores or whatever scores it might be and your band would then talk to the cloud I would get the sport score of whatever game was happening and send it to your wrist for a point in time reference I built this in 2015 right around the same time that the Apple watch was released this was the first application I'd ever written that was callable through the cloud it wasn't just like some app that ran on a box it took me about a month to build I spent a lot of time building this it was a good learning exercise and I was spending $80 a month for the server behind the scenes that was powering this application when I had one user I was paying $80 a month and even you know this is actually a picture from a news article that picked up this app when Microsoft band was just taking off and when this news article hit I got a thousand downloads like in one day and it kept growing I was still paying $80 a month but I also struggled to kind of manage the spike in usage that I was seeing the other challenge was because of this was kind of built as just my VM with a bunch of code that I'd Jerry raked together one of the biggest pieces of feedback I got when I first shipped this was hey we want to be able to follow Premier League or world or whatever adults it might be and so to add additional features was taking me a really long time and was hard to update because I just again had this virtual machine so cool app fun experience but took me a long time to build and was painful to manage now if I look at this year some of the different apps that I built this is just one example of one of the apps that I built this is actually a picture of my doorbell at my house I have an IOT doorbell that has a camera on it and it will be able to detect motion or sound and I want it to be able to use that feed from my doorbell to do some intelligent stuff so what actually happens is when somebody comes to my door you can see an image of the end result I'll actually run this video feed through some intelligence services if you've given me access to know your identity then I can actually do some facial recognition I can detect the text if any text is being spoken and all this is just running monitoring my doorbell 24/7 this only took me two evenings to build I actually wrote a few blogs about how it was put together if you're interested and just like I mentioned with serverless this is only ever running when somebody actually comes to my door I don't have a lot of visitors I'm not super popular so this means I'm only spending about four dollars a month right now to constantly be monitoring my doorbell and a lot of that cost is actually from the no sequel database the functions themselves are actually zero dollars they're free because I'm not calling them enough this will automatically scale to any traffic and it's extensible and reusable so to give you a sense for what kind of a service app can look like in Azure I do want a piece apart this doorbell one ok so just an example this is Visual Studio code and for this app my door Bo one it's actually written in JavaScript so I just have here a very simple Java function which says hey on a timer I every 15 seconds go and check to see if anybody is ringing my doorbell if somebody is ringing my doorbell I want to send an event out to let me know that someone rang my doorbell and then it keeps running and keeps checking my doorbell so not a bunch of complex code I have to write just check to see if somebody's at my door if somebody's have my door notify my application some of the nice things to call out I talked about dev tooling I've got this extension directly inside a Visual Studio code for Azure functions or I can create updates and publish my functions all from within this experience I can even do things like set a breakpoint and f5 run and this will actually spin up the Azure functions runtime inside a Visual Studio code so that I could debug this live and hit a breakpoint so you'll see here with all this moving text this is just spun up it's now checking locally on my machine here I think I've set this one to see every 15 seconds so within about 10 seconds or so you'll actually see this breakpoint would get hit so some really nice tooling in here as well to help me build these in this case a JavaScript function now what happens when that event gets sent because this functions not doing a whole lot right that's not powering all that cool machine learning stuff I showed you a second ago and what's powering all of that this is the only part you're going to be able to see out your logic apps today so don't blink this is actually the workflow that gets kicked off from that event that's running in my subscription okay so you can see here's some of the different runs that have happened I can click into them and see the status if I actually click Edit this is what an Asscher logic app's looks like this is a very useful tool for functions and for service to orchestrate events so here I'm listening for that doorbell event so I'm saying hey wait to see if my function sees that someone's at my door bell and now I'm gonna execute this process which can be pretty long running the first thing is I'm gonna wait until the recording is finished this could take a few minutes until somebody finally leaves my doorstep once I know the recording is finished I'm actually going to upload that recording to an out-of-the-box connector here inside of azure so this is a connector for one of the cognitive services or artificial intelligence services in Azure they can actually take a video feed and automatically process emotions faces text all that other good stuff so we'll go ahead and push that up here I'll wait for that video to be processed and then I can store the data here one other thing I'll show you in case you haven't seen launch apps before if I wanted to continue to add some more stuff when someone rings my doorbell I've got all of these not only as your functions that I can call I could also call you know contain running an Asscher container instances I could call other connectors like dynamics github whatever service I might be using this is probably code I don't have to write so this is great it's awesome and now you know within two evenings I have this code that's constantly running and listening for when someone's at my door Bell so just an example of what a service app can look like okay now we're gonna spend the whole rest of the time talking about functions so that's that's the service architecture piece let's go deeper into functions now a few key concepts to note when you're writing as your functions the core component that you're dealing with is a function app and an app is your unit of deployment it's your unit of development it's also the unit of scale so a single app will be scaled as a unit you can have multiple functions in an app but your app is your primary unit of development deployment and scale and it can have many functions inside of it so again just to kind of tied the knots together if you come back here to my doorbell project this is my app here this is my function app but actually I have a few different functions I have checked the rings I have get the current recording I have store some data and Cosmos DB these are all different functions but they all live in the same app so I can develop them and deploy them together all of your functions will have a trigger this is the event source that it's listening for and the data that will be passed in to let your function know to execute every function can only have one trigger and the service itself will automatically be listening to that source so you're not you're not charged if it's listening to something like a cue waiting for a message to drop you're only charged once that trigger fires and your code actually runs the other piece which is unique to as your functions is this concept called bindings these can be super useful for use you're developing your functions bindings are data that you can pull in or push out of your function without having to integrate with that service directly inside of code so I have here a table these are like our top whatever seven or eight triggers and bindings that we have an example and I'll walk through this in just a second I might want to build a service API with has your function so HTTP would be the trigger I'm going to be listening for an HTTP Qwest but maybe I want to store some data from that request in a database in a no sequel database like cosmos DB now I don't necessarily want to have to write and manage my connection to cosmos DB so I can use a binding to automatically push data to cosmos DB as part of my execution and I can describe that integration through configuration I'll show that to you in a second alternatively maybe my endpoint I'm getting in a query parameter for like an ID or a path parameter for an ID and I know that that ID maps to some data in my storage account or like table storage maybe I actually want to pull in data from storage at the time of execution I can use an input binding to tell my function hey you're going to be triggered from HTTP but also pull in data from cosmos or storage or wherever else at the time of execution if that doesn't make sense right now that's okay I'm going to show it to you in just a second but just remember we have triggers that start your event and bindings which can allow you to flow data in and out independent of the trigger now one of the nice things about Azure functions is that you do have a number of different choices on where and how you want to develop I actually have some recommendations here though so you can write and manage hauser functions entirely in the azure portal there's a little code editor you can run and test I wouldn't recommend it for anything in production though it's great for trying things out if you have a really simple script and you just want to play around with out your functions or it's not a very important script it's fine but anything you're doing that will have the words production associated with it you're gonna want to use one of these other editors because then you can do things like source control and continuous integration and automated testing and all of the best practices that you've been hearing about so visual studio this is perfect for C sharp F sharp anything dotnet related there's a really nice visual studio integrated tooling that gives you debugging publishing integration with has your functions it's perfect for that you can use Visual Studio code I showed you I'm writing I'm using my Mac right now so I have Visual Studio code Visual Studio code does support c-sharp but if you have the option I do think visual studio is a more natural place to do that for now at least for me personally but if you're writing JavaScript Java or Python functions Visual Studio code is a great fit again it has that out-of-the-box tenth out-of-the-box extension we also have maven tooling for Java functions so you could write your functions in IntelliJ or eclipse if there's a different Java editor you care about and really the tools can be run anywhere so if there's an editor here that you like that I haven't listed it will still work you just need to make sure you hook it up to our CLI tools so that it runs for you ok so I want to show you what this looks like now I'm gonna do a really quick walk through a building function inside a visual studio this time from scratch ok and the one I want to build is actually what I described a little bit in the example so I want to build an API and I have here a cosmos DB no sequel database inside of my Azure account and I want to make an API so I can get a request have that function spin up on demand and then push that data directly into cosmos DB so let's go ahead and do that I'm gonna go ahead here and say file new project and create a brand new as your function project I can choose whether I want to do this in our version one I'll zoom in here I know this is really small in our version one runtime which is currently generally available which is written and runs dotnet framework functions or our version two one which is in preview and will be in preview for a few more weeks which is dotnet standard it actually runs dot in that core 2.1 which gives it some nice benefits i'm actually gonna go ahead and do this in the version to.net core one and let's go ahead and say we want it I'm just gonna do an empty project for now ok so here's an empty project inside a visual studio I don't have any functions here yet so let's go ahead and add one into this app remember this is my app and an app can have many functions and this is my unit of development deployment and scale so let's add a function into this project and there's a bunch of different triggers that I could choose from I already mentioned I want this to be an API so let's do HTTP triggered and this will start for us right here okay let me make sure our code is big enough so that you can see a little bit of what's going on 200% yes please I guess it doesn't like to zoom hopefully this won't crash if so that's fine we'll have to move on from there this is what I get for running inside of a virtual machine on my Mac on battery power it looks like it didn't like it that's okay don't zoom in your code live but that code right there was just an HTTP function I won't show you for the sake of waiting for this to reboot and making sure we have time to go into some other topics as we move forward but I could do the same thing I did in Visual Studio code set a breakpoint click the Run button and everything would would run for me and I'll do that in a second okay now this is great but I need to integrate now with with cosmos DB and I'm gonna use that concept called bindings Kay so I don't want to necessarily have to write the integration code with cosmos DB directly I'd rather leverage these bindings so one important thing to note is that functions you can pull in different extensions okay so we have an extension for cosmos DB we have an extension for service bus queues we have an extension for as your event grid these are all published through nougat so in this case the first thing I'm going to do is make sure I pull in the extension for cosmos DB and actually have a nice keyboard shortcut which is going to give me all the code here so I'm gonna say hey in my function app I want to install the extension for cosmos DB the reason that we make you install these extensions is because we want the base image to be as the lightweight as possible so you get really good performance so if you want to bring in any integrations with other services we have you install them in through nougat if you're using JavaScript or Java those two links have you know different ways to do those packages but I won't go into those now all right so now this is where I want to define how that integration works in my function this is where I'm actually gonna write my binding I'm gonna paste this in right now and then I'll walk through it okay so what I've done here is I've written an attribute on this function I've said hey I want you to integrate with cosmos DB I have a database called function I have a collection called function and I have a connection string which is going to be in an environment variable called cosmos DB connection okay I don't want to paste my actual connection string here because you don't want to put secrets inside of your code so I'm actually just referencing an environment variable which I'll set in just a second now what it's telling this to do is I'm gonna have this variable called Docs which is an out variable and my parameter whatever I set to Doc's in this case it's the string I could make it a document I could change the type depending on what type of type I want to use but let's just leave it as a string for now whatever I set as the value of docs that's what it's going to push into Cosmos DB for me okay so I'm going to set this value for docs and push that to this Cosmos DB database so the only code I have to write is hey let's set that Doc's variable to the request body that I get from the HTTP request okay so with that one line of code and this attribute up here now whatever is in my request body is going to create a record at my cosmos DB database obviously I couldn't manipulate and do whatever I want to do with it now the last part I need to do here before this will actually work is I need to make sure that I'm setting this environment variable okay so locally I have this handy file which you probably can't read very well called local dot Settings JSON and that will let me set local environment variables easier in the cloud they're called application settings but let's go ahead here and we'll say hey I have this environment variable and I want it to connect to this oops this really long connection string no one used this Cosmos DB database no one look at the video recording okay so that's it so that's my function that I wrote quickly here I use bindings to make my life a little bit easier and now let's go ahead and run this and test it out okay so I'm gonna go ahead and click the Run button I don't know where my solution Explorer went things kind of wigged out on me a second ago but it doesn't really matter okay so I'm now running this code I have my as your functions runtime which is spun up inside a visual studio this is the exact same runtime that we'll be running when I publish to the cloud so I know that if it works on my local machine it will work when I publish it because it's the same runtime that's running here it's given me this nice end point that I can test locally so here's my add your function now that I can test on my machine so let's go ahead and copy this endpoint I'm just gonna send a request using postman this Windows machine is really struggling but that's okay all right so I'm gonna send a request to that endpoint and I'm gonna pass in this request body the shift keys like locked right now but I'm not too worried about it I don't think so let's just keep going all right so let's go ahead and click send it's gone ahead now and fired off that function in fact if I come back over here to the logs it return back hello Jeff I could change this to whatever I want it to as well like hello NDC and go ahead and send this again and now it says hello NDC and I can actually see my logs that are been generated here just to prove that this worked let's come look at this cosmos DB database now in the cloud and if I click refresh we should now see two new documents that got created for me one with the request body of name is NDC and this is just the metadata that cosmos DB generates and the other one which is named Jeff so now I have this function that I could write check in the source control and then deploy once I'm ready to go which is now integrating with cosmos DB as an azure function okay so hopefully if you hadn't used functions before that's a bit of a walk through of how some of those concepts weave into what a function looks like okay so now that you've seen functions it's useful to know some of the common architectures where I've seen these fit in very well with some of our customers so here's the top five scenarios that I've seen the first one is any time you have an event stream that you want to be doing some processing on so these events might be IOT events coming from IOT devices or Machinery anything that's like in an event hub or kafka something that's pushing a ton of data diagnostic logs whatever it is because the nice part about functions is that I can just write the code I want to execute for a single event and click publish and now that piece of code will run for every single event that's happening so it's kind of very easy to just attach this to the stream and have it scale automatically to do some processing on top of that stream web api is a mobile backends is another very popular one one of our larger customers has a mobile app for a retail store and how it works is if you want to send somebody like a gift card you open up the mobile app and you're like hey I want to send Jeff $20 and you select my account and you click send when you click that send button and it's actually kicking off some as your functions behind the scenes which go and charge their accounts and the email do all those other great things and it's a great fit forum because they don't have to worry about managing this API and scaling it they just publish this as a function and it works you'll hear serverless often used and talked about as glue between systems you might have a bunch of data that's in some file system and you want that data in a sequel database or in a no sequel database being able to just really simply write a function and say hey whenever new data gets added or every hour go pull the data and push it over to this system is a really nice fit to just kind of glue things together if you're using pubsubhubbub ESB type patterns and then for just general automation or scripting if there's ever a task that you're doing that you want to automate something very simple like recently I was just on a call with a customer who uses functions to help automate their pull request process so they have it so that whenever anybody opens up a pull request it fires off a function that does some code checks very quickly for them it's not they don't have pull requests happening 24/7 but whenever they do now this code can run one of the customers I'll just share one use case Carl Zeiss who does very high-end electronics and imaging devices they've been using Azure functions there's a use case they published two strimer stop Microsoft comm that monitors and scales to field devices that are connecting to back-end REST API s and that's all powered by functions and just to kind of bring the point home and so I'm not the only one saying it the thing that they call out is that the scalability of Azure functions is amazing they can send millions of records in the platform adjusts automatically to the load so very low operations and very high scale okay now let's talk about some more advanced tips and best practices these are some of the common hiccups that I see people do when there's a support ticket that comes in or somebody says hey Jeff I tried to use functions and things aren't working as I expected these are some of the top things that I've seen and how to get around them the first is another concept to understand which is how instances and resources are managed actually I do want to show you this because it's a little bit easier to show so let's come back here to this let me open up my solution Explorer wherever that went what do I do view here view solution Explorer so this has your function okay very simple REST API this is in my app let's say I go ahead now and publish this to the cloud okay I have one just like this it's already published I'm not going to go through this now and what I do want to show though is I have here a live stream connected to application insight so one of the tips is hook your Azure functions into a monitoring solution like application insights which we integrate with out of the box because it will give you insights and observability into how your function is performing now what I'm going to do here is that I actually have a script ready to run which uses a load testing tool called artillery and I want to simulate 1,000 requests happening every single second at one of those function api's like I just wrote okay now nothing has been called yet like this thing is totally empty in fact just to show this I'll call it one time really quickly let's call this API and you'll see what happens here in our live monitoring so I can actually see that an instance has now spun up to execute that request right and it's already done you see I had one one execution that happened one lonely execution now I'm gonna run this load test I'm going push a thousand requests a second at this function and I want you to pay attention to the instances down here so there's a thousand requests coming in and you'll see how quickly and how many servers are going to be scaled up behind the scenes to process this load so suspense now is high let's go ahead and run this test let me wait for it to start up and initialize I think it should be going so we'll come back here yep it started up ICOC already before I finished my first sentence we're at four servers that have spun up online and are helping process the requests in fact it's seeing that the load isn't stopping so now it's at five servers six servers very quickly this has been able to scale up as the requests per second continues to increase seven servers you can see counter up here eight servers this is nice if any of you have had to deploy a server before it usually is not as quick as this sentence that I'm completing right now but already that function that we wrote a second ago is now processing across nine different servers and it looks like it's hitting about 500 requests a second right now the requester ations still really low so I don't know if my my computer is bogged down by me having too many things open and it's not sending more than 500 a second but that's okay regardless you can see I'm processing 500 requests a second across nine fully powered as your servers running right now it looks like this is very happy at 500 requests a second so I'm just gonna stop this right now to save my computer from blowing up okay so we'll go ahead and stop that okay so a few things to note is we talked about what just happened so as your functions will automatically add instances or servers as needed in this case for this load test it realized it needs nine fully powered servers running in the cloud to process that load each of those instances will have my app running on top of it okay so that same runtime you saw me running in visual studio those are running on all of those instances now every single one of those instances may be executing multiple things in parallel right these are powerful servers I don't need to just do one request per server I could be processing multiple multiple requests in parallel per instance so there is some parallel I'm here happening even on top of an instance for a single instance now the reason this is important and one of the best practices to know is that you can take advantage of the fact that yes there are servers powering server lists and you can reuse certain resources across executions so for example a very common example of this is a sequel client in your code maybe your code needs to talk to a sequel database in you're using sequel client a very common mistake that people make is that in their code they will do something like var sequel clients equals new sequel client and then my connection string now the problem with this is that every single time my function executes it's going to instantiate a brand new sequel client and have to establish a brand new connection to that sequel server but we're not throwing away your servers after every single execution in fact you'll notice if I even come back here to the monitoring solution we have I still have nine servers online all of the code that I instantiated all of the public or private static variables are still set they still preserve their values so if I would have done something like private static sequel client that sequel client would have only had to establish one connection and that connection could still be alive and I could reuse it for subsequent executions okay so again here's the anti pattern that I see a lot of people do HTTP client is the other one this comes up a lot this is an anti-pattern for a number of reasons if you're interested there's a link down here ok so here's my function code does it matter the language if I do something like using var client equals new HTTP client that means for every one of those thousand requests that are coming in I'm having to instantiate a whole new HTTP client my instance is going to be really inefficient but I'm also going to exhaust the available sockets pretty quickly because there's only a limited amount of sockets I have on one of these VMs and if I'm pushing high load I'm gonna exhaust them just because I'm not managing things very well ok so again there's a link here if you're interested just remember these instances are established but you can actually reuse resources across the you can almost have kind of simple caches as well so the better way to do this I'm gonna create an HTTP client as a singleton in this case a private static HTTP client this is thread safe and what this will enable me to do is now as multiple requests are coming in I can reuse the same client and make multiple requests so my codes gonna be a lot more performant it's gonna scale a lot nicer and I'm not gonna run out of a sockets okay so this could be sequel client this could be event hook client this could even be some type of an in-memory dictionary or cache and that's gonna hang around you can keep reusing that as you want so very very important tip that's gonna make a big difference and if you don't do it things might start working quirky for you okay moving right along because I want to give plenty of time for durable the last tip I'm gonna give in regards to scale I'm gonna paint a little story for you okay so if you think about this example we just did with the HTTP load test where unfortunately I wasn't able to push a big enough load that's okay imagine that we have two cities ok city a and city B and everybody on city a wants to get over to city B and let's say there's that 1,000 people who want to get across and I have one vote now the question that I have to answer is how many people can I fit on one boat for one trip to city B now if these are small children right maybe they're like 5 years old maybe I could fit like 30 people on one of those boat trips and that's cool because it means I have to take less trips back and forth but if they're bigger people maybe it's like a thousand people who are my size maybe I can only fit like two or three people on one of these boats before I get worried about it ok don't worry I'm gonna tie this back into functions I promise now even if I have multiple boats like maybe I have 10 boats or 20 boats there's still the question I have to answer which is how many people can fit on a single boat at one time maybe I have a thousand boats and that's ok but I want to make sure that I don't load too much onto a single boat by itself ok so just remember that things can scale out across multiple boats but I need to make sure I'm aware of how many things can fit on a boat at one time now what does this have to do with how's your functions so you just softer that HTTP example that I scaled out to a bunch of different instances and the service itself does as good of a job as it can to allocate and distribute your your load so that each of these instances isn't processing too much we do our best effort but sometimes our best effort isn't perfect okay and we understand that we're not perfect at this and so we may do something like push too many executions onto one instance at one time we would overfill your boat okay so you want to make sure that your function isn't having to execute too much on one of these single instances now when as your functions just so you know a single instance is one and a half gigs of memory and one core of CPU so you can think in terms of how much can I process in a gig and a half of memory in one core CPU if for some reason we've gotten the math wrong you're using some of the monitoring like I've just shown you and it looks like we're overloading an instance like it's memory utilization is bad or it's starting to timeout or get overwhelmed we do give you this configuration file which is called the host dot JSON file where you can actually tell us how much we can send and how much data we can push to a single instance and that will allow our scaling to compensate for it and scale out appropriately okay so in this case for like a very simple if I'm just doing simple addition like hey take value 1 and add it to value 2 I could probably be doing a hundred of those in parallel at a time it's not doing that much work but maybe I'm doing like Bitcoin mining a cryptocurrency it's not profitable don't do it on functions but if I was that's probably more memory or CPU intensive I might want to tell as your functions hey don't send too much on a single instance okay there's a lot more information here in terms of scalability so that you can control how functions scale to preserve things like downstream systems I'm not going to go into that right now but just note that there's a lot of config here that you should be aware of and this again applies to just distributed systems in parallel processing in general ok the other thing I'll let you know is that without your functions you do have a lot of fun flexibility and that you can run your function in your ideal environment so up into this point I pretty much talked exclusively about consumption functions these are functions that run in the service plan you only pay for what you use we only allocate resources for you on-demand but that's only one of the many different flavors that your functions can run in you can run your Asha functions in what's called an app service plan this is like the website plan inside of Azure and this plan allows you to reserve and have reserve capacity for your functions so instead of saying hey spin these up on the man you could say hey I want twenty instances that are my instances that I can manage how I want choose different hardware configurations maybe even have websites or other processes running on top of it you can still deploy your functions to run there you can run in your own dedicated isolated environments connected to things like V nets you can run on premises with Azure stack there's also an on-premises which is called as your functions runtime which is similar to the runtime I just had in Visual Studio which doesn't require the full Azure stack and finally you can run your Azure functions on IOT devices so that same function that I just wrote at the beginning if I wanted to when I've done demos like this before if I had a Raspberry Pi sitting next to me like a really simple IOT device I could deploy and have that function run on that IOT device as well connected or disconnected so you can have kind of take advantage of this programming model but then host it in the environment that best fits you ok so we've got about 20 minutes left we'll probably maybe even finished a little bit early in case there's questions and this is the last session so the last thing I want to talk about is how you can orchestrate processes with functions you saw at the beginning during that doorbell example how I used as your logic apps to kind of create that process to process the video but there's actually another way that you can process your your functions and orchestrate data and the other way that I want to spend the rest of the time talking about is a newer piece of Azure functions which is again very unique to Azure inside of service is called Azure durable functions I have screenshots of durable functions up here on the right or your left I'm sorry on the left as your directions are hard durable functions you'll see here I have one example here in JavaScript and one example here in c-sharp and what I've actually done here is I'm able to describe orchestrating and calling different functions and activities but I'm describing that all through code so to kind of help illustrate why this might be needed for one if I'm running in a microservices environment which serverless is a natural fit and some of these like single function single piece of functionality type mentality that microservices bring it's very often that to do something I'm gonna need to call multiple of these things so durable functions in logic apps are a great way to call multiple but the other thing to be aware of is for server lists and the server list plan a single function can only run for a maximum of 10 minutes which means if you have a piece of data or some processing that needs to occur that may be running much longer than 10 minutes that becomes very difficult to manage and you have to end up splitting up the task and without something like durable functions or logic gaps you end up having to create queues and do everything yourself here with durable functions and with logic apps you can actually run things that last for a very long time and we will manage the orchestration and state for you so that these can run indefinitely so let's talk about as your durable functions it's an extension that you can pull into your app there's a bunch of documentation on this I'd encourage you to look at look at that documentation as well because I'm not gonna be able to do it justice here in the last 15 minutes it enables you to run these long-running and stateful serverless orchestrations you can write these in c-sharp or in JavaScript right now this slide isn't fully up to date I guess JavaScript just got announced last month this will manage the state the checkpoints the queuing all for you it's built on top of some open source technology called the durable task framework which is just an open source framework to enable you to durably execute things so it's very nice to be able to manage things so a few patterns that it solves for just to give you another flavor it's kind of like function chaining I have function one I want to call function to them function three function 4 without durable I would end up having to do something like this where I would drop a message into a queue this one would have to pick it up from a queue and so forth that gets really challenging to manage and to monitor and to version with durable functions I can just write c-sharp code to say call function 1 then function to them function 3 another popular one this is what I'm going to show a demo in just a second of is doing some like distributed fan-out fanon or MapReduce type of patterns where maybe you get a bunch of data and you want to process it across a number of parallel executions and you can describe this all again in code a synchronous response is where you want to return back a status endpoint and be able to a pole until the long-running work is complete maybe you have some work that might take 5 minutes to execute and you don't want to have to keep an HTTP request open for five minutes durable will provide out-of-the-box patterns for you to say hey return back an immediate response and then do the work in the background and let them check to find out when the work is done super super useful pattern monitoring saying things like hey go watch this value for the next hour or the next day and tell me if something changes human interaction one of the cool demos and samples we have with durable functions is doing like a text validation pattern where you send somebody a code and you wait for them to put that code into like a website or to text that code to a number it might take them three days before they finally read that text message and send back the text but you could wait that long for that interaction and tie all that together obviously if it's like a authentication thing you don't want to wait that long but I'm just saying they can run for a very long time and again this is exactly something dropped that scared me this is an example of that first pattern function chaining but in c-sharp I just say hey call function 1 then call function to call function 3 I'm even passing in inputs so I can pull data and push data between and then all of the rest gets managed for me let's show it an example really quick before I do this piece I'll do this piece a little bit out of order so I want to show you this in action so I have an example very quickly we I have a website let me show you the website first this is a website that's powered by service technology it's beautiful it was written by one of the azure cloud advocates it's a simple store I could come in here and add some items go ahead and check out but the nice thing about this function is that whenever somebody Plato this is an email not not my credit card when any whenever anybody comes in here and places in order let me make sure I put in the right test credit card number it's actually right here when I push pay it's calling an azure function behind the scenes and processing this all server lessly so that the order process piece doesn't actually take any compute out of my website now this is using cosmos DB behind the scenes to store all of these orders and what I want to be able to do is generate a report so I want to go and say hey look at my database and give me the last seven days of orders and for each of those orders there could have been 1,000 orders or 10,000 orders I want you to go and get the current status of that order like hey is the payment been processed has it been shipped has it been delivered what's the status of that order and then returned to me a report now if I was trying to do this without durable this would be really difficult cuz this middle step here could take a really long time if I had a thousand orders that had been placed and I have to do a lookup of a database for each of those thousand orders that could take an hour I don't know it depends on how slow my databases there's no way I could execute that all in a single function but with durable functions I'm able to chain this together in code I'll show you briefly what this looks like and then show you what it looks like actually I'll show you the demo first let me open it up right here and while I wait for this to open let's go ahead and call that report so I have here a URL not this URL you can see a test of this I'm gonna go ahead and kick off this process to generate a report so I just put in the URL to that report up here and what it's immediately returned to me back is some data it said hey I've gone ahead and kicked off this process this is the URL you can follow to get the status if you want to add some data you can post some data here or if you want to terminate this I can terminate here let's just go ahead and click the status endpoint and hopefully I didn't talk too long and this will still be running it already completed so let's do it again it ran too fast let's run another one and I'm going to click that status endpoint cuz I want to show you it does so you can see right here it's still running right it's still right now it's looking at cosmos it's spun up a bunch of parallel functions to go look up the status of the database and it's telling me that it's running right now and if I go ahead and refresh this depending on how much data I had it could have still said running but in this case it's completed announcer deported data which said hey you have four that are preparing for shipment one that shipped four that are delivered and two that are being processed so I was able to build this pattern but have it all running as functions and the code here just to give you another sense of what a durable function looks like this is the orchestration all in c-sharp so the first thing that I do is I say hey go get all of the transactions from cosmos DB so that's going to call a function that gets all my transactions from cosmos I'm gonna do a for each loop to create a bunch of parallel tasks so for every single transaction I've had I'm gonna go and call the git process function and here I can just say wait for all of these to be done and again this when all could take an hour it could take two hours what's really cool here with durable function that's hard to wrap your mind around is that this function isn't actually going to be running the entire time whenever there's this awake keyword it's actually going to go to sleep and wait for them to all complete and that will trigger it to wake up and continue and then it will return all the details so a lot of tech here it's pretty advanced in a little bit complex but it's worth knowing it maybe to help visualize that the last thing I'll show you is is how this durable function is running behind the scenes what's actually happening here how are we able to run these long-running processes still have it be service not having to pay a bunch of money so here I have a very simple orchestration this is in c-sharp all I'm saying is create a new list call this function called say hello it says Amsterdam I should have updated my slides I'm sorry I've been doing too much traveling this week and then add that to a list okay but hello anyone who's watching this stream from Amsterdam once it's published and that's it so what actually happens is my orchestration is gonna start it's gonna run this first line of code and so it creates a new list and it's going to see this await word and when it sees this await word there's this thing called the execution history and the first thing it's going to do is it's going to look at the execution history and it's going to say hey have already done this action have already had this function have it say hello to Amsterdam and in this case I haven't so it's gonna say nope you haven't done this before the orchestrator is like okay cool I'm gonna do this then so it's gonna call that function it's gonna say hey function you have some work to do that activity functions and then sorry it's gonna call that function and then you're gonna notice that Orchestrator is gonna go to sleep it's gonna scale the zero it's gonna disappear it's done for now the activity function gets triggered it's been told that has some work to do it executes the work and in this case the response is hello Amsterdam should say hello Norway I'm sorry that functions done that function goes to sleep and you'll notice all of this has been stored in the execution history now once that activity function is done it wakes up the orchestrator so now my Orchestrator wakes back up but here's the part that's important to understand when that Orchestrator wakes back up it doesn't start from this line right here it starts from the very beginning again because there's no way to just like resume a program from the very middle after you've scaled to zero so the first thing it's gonna do is it's going to create a new list now it's going to see this awake keyword and it's going to look at the execution history it's gonna say hey have we already called Amsterdam or say hello Amsterdam in this case it has like it's already done that action so my history is like yep you've already done this thing Marcus Tudor says great I'm gonna move on and do my next step and then it's going to continue to do its next step and then return back the responses so every time one of these functions is completed my orchestration is going to replay now that's extremely important to know because this impacts what your Orchestrator can do it is very important to know that your Orchestrator code must be deterministic you cannot have values in your orchestration that will be different every time it runs so for example maybe I have an if condition I don't want to say if date/time dot now equal equals something right because every time that code gets replayed the date time is going to be different or if I generate a grid that grid is going to be generated different on every single replay and it can corrupt my state so never write logic that depends on random numbers date time or grids okay don't do i/o inside of an Orchestrator function do your i/o inside of your activities don't write infinite loops that's just a good rule in general I figured we should throw it in every tech don't write infinite loops make sure that these things terminate there are patterns though for all of these so if you have something that is running infinitely there is a safe way to do this endurable that doesn't result in you having infinite replays there are ways to do i/o inside of your activities and there are ways to do like get a current date time or generate a grid or generate a random number but you need to make sure you're leveraging the built-in pieces of functionality we have in the orchestrator that enable us to do this in a deterministic way okay so again a lot of kind of advanced concepts here with durable functions but they're extremely powerful for you the last thing I'll finish with is we've shown two ways to orchestrate data now we've shown just recently durable functions which they should do it in code at the beginning we showed you logic apps which lets you do it in that kind of visual designer that I showed you so which one should you use durable functions is a stateful process Orchestrator not necessarily an integration service so logic apps has a bunch of out-of-the-box connectors it has things like Edda fact and x12 and the types of things you would expect from servus almost like BizTalk server durable is not that rich in those types of features so if you're looking at something that's doing integration at that level logic apps may be the better fit however there is a lot of healthy overlap between the two so a lot of it comes down to personal preference in fact a lot of times I'll talk with developers and they're like you know what I really like how fast I can build things and chain them together and visualize them inside of logic apps so I'm gonna use this declarative workflow in other people who are like hey I'm writing code all day I think in code I'd rather be writing this inside of code in kind of an imperative fashion that's totally cool that's fine I do whatever one makes you happy understanding some of the differences in how they execute for example logic apps doesn't do that replay between every execution it will just do action than action than action than action durable has that replayability these to mix and match together very well and if you're interested there's even another duck there but just use whatever makes you happy that's just a general good rule for all technology okay so that's it for the content I wanted to share with you again thank you so much for coming I'll hang around here for a few minutes afterwards and answer any questions if you're interested in learning more getting started I'd encourage you to look at our documentation especially durable functions there's like ten pieces of Docs and a bunch of samples to get you started if you're interested in trying it out it's extremely powerful because you can get these long-running processes with the benefits of server lists we also have a number of QuickStart so if you haven't ever built a function before we have one where you can do it all within the portal we have some for visual studio we have some for visual studio code whatever editor makes you happy even some that just use the command line we've got a bunch of samples and then some of the advance patterns I've talked about like instance management concurrency control in order messaging error handling I've written a few of these in my own personal blog if you're interested which is at medium.com slash at jeff hall and it's also my twitter handle i'm more than happy to answer any questions you may have there so with that that's it so thank you all very much for coming I'll go ahead and answer any questions there's not a bunch of people in the room so if you have any questions go head and then please just as you leave you get the little feedback slips this is my first time coming to NDC I'm learning exactly how best to do this they stuck me at the very end of the last day because I was a late entry but even just those feedback slips will help me know how to improve and hopefully get me invited back as well so thank you all very much any questions I can answer while we're here are we all good oh yes one right here good question so the question for those who didn't hear is do the new dotnet core questions get around assembly redirect stuff so this is a good questions a very common one the the challenge that we had with the version one of Asher functions was that for dotnet function specifically the desire was to let you have access to very rich data types in fact if you look at this example that I had at the beginning where I was pulling in data from cosmos DB I could as I mentioned like I could say hey I want to push a string to cosmos DB but I could also say I want to pass in a cosmos DB document like this is a very rich data type that comes directly from the cosmos DB SDK and the way that this is possible is we wanted you to be able to pass rich data types between your function and the host process and there's a really good idea at the time and that's how it was built but the challenge with that is that it means that now your function has some dependencies on the assemblies that the host is using that host process so if you want to do something like use a newer version of cosmos DB then what the host is using if you try to do that you're going to get Assembly resolution errors when you try to run it especially when you try to pass data in between because it's like hey this is document version 2 and I use document version 1 so there's a lot of consternation there so one of the big things we've been doing with version 2 is helping improve that the short answer is just now the release that just completed improved a bunch of scenarios in fact 95% of the scenarios we were tracking are now much more possible today but we're not done yet the next piece we want to do which should be available in preview by the end of the year would allow you to run your c-sharp functions completely in a separate process doing whatever you want on those it does make it so that things like rich data type transfer would work differently because we're gonna have to like serialize and pass the data but the long story or I guess the short story is give it a shot in the B to one there's like you know a bunch of things that weren't possible that now we've we've worked very closely with the dotnet core team to have specific assembly load context for your function so that you can pull in whatever extensions you want and run them independently but we're gonna continue to work on this through the end a year so that we never have anybody who SSS is that so good question though definitely very high priority any other questions yep yeah good question so the question was again for those who didn't hear Python support so we actually have Python running right now in private preview which is funny to say I and we can give invitations to really anybody who's interested just just sent me a note on Twitter so yes it is coming it's it's available right now we have Python workers I built a few of these myself it's not yet in public preview yet it should be in the coming weeks but if you're interested in giving it a shot I'd encourage you to get onboard the private preview now and then keep an eye out in the next few weeks and hopefully you'll see an announcement where you can use it publicly and then as the version 2 becomes generally available it would come available after that as well ok any other ones yep go ahead functions with exceptions you said what happens to it yep so when your function has an exception is that like if you threw an exception in here oh and no reference in sight yeah so just like any except like if I wrote code here and I tried to like Poland request body and request body was no like what would happen yeah so Jen when exceptions get thrown your code just finishes and could go away and that's another really important thing to note is the trigger that you're using so for an HTTP trigger for example if I have an exception that I don't catch and handle in my code it's very possible that that request just vanished into in air and I would have lost that data okay so one thing is that you can like catch exceptions in your code directly some of our triggers though do have automatic retries so if this was coming from a cue instead of a HTTP trigger if you had something like a no ref exception we would actually automatically go and retry that a number of times and if all of them result in exceptions and we retry five times we'll actually go ahead and stick that in a poison cue or a dead letter Q so that you can see hey you know this message had an error and and it ended up being a poison Q so monitoring helps with that the trigger type is the biggest one on this one now if you have data that you want to make sure you don't lose make sure you use a trigger that can keep some state around like a Q does that help was that the question okay cool all right we'll go with one more go ahead mmm great question so I mentioned how these connection strings are environment variables in the cloud you would put this type of a connection string in an app setting which is encrypted but a lot of people want to keep these in key vault so by the end of this summer we plan to have a feature where instead of me having to put my connection string directly in an app setting I would actually be able to do something like key vault something something something and have an address to where that secret lives inside of my Azure key vault and we would fetch it out of key vault for you using the managed service identity so it's being worked on right now targeted to around the end of the summer where you could reference secrets coming from key vault instead of having to put them directly in your environment variables okay all right I think that's everyone then thanks again so much enjoy your weekend five twenty on the dot appreciate you all staying honestly I can't remember the last time I went to the last session of a conference so I appreciate you all being here if I had swag I would give it to you so thanks again so much

Channel: NDC Conferences

Views: 49,262

Rating: 4.9212122 out of 5

Keywords: NDC, NDC Oslo, 2018, Jeff Hollan, Serverless, Architecture, Cloud, Azure, Azure Serverless, Serverless App

Id: kvTostlJp7M

Length: 61min 22sec (3682 seconds)

Published: Wed Aug 29 2018