python vs dotnetcore - which one is faster?

which one is faster Python or dotnet core let's find out hi I'm Raphael and this is called a cave in this battle of the AWS lambda performance war I will pit Python three point eight against dotnet core 3.1 I have some code that creates a simple endpoint that receives get and post requests and I will bash this endpoint with a Dream Eater test to see which language fairs best in terms of cold stars and normal runs let's dive into the code this is the Python implementation of the lambda we have the main entry point which is discriminating by the HTTP method that is used in case of a post we want to extract the document that is part of the body of the request hash it and once we hash it we want to insert the whole object into DynamoDB so the whole object will have an ID a document and the hash and I want to insert doing a conditional put and I want to have it conditional by ID so in case there is an the same ID is already in the DynamoDB table the operation will fail and if it fails I will return in HTTP 409 which is a conflict otherwise I return a 201 which is a create and I return the ID which is set by the client and the hash of the document in case of a get request I get the ID as a Kersting parameter issue a gap on the dynamodb table by ID of course and then return the item to the client the.net implementation is very similar of course this is sharp so it's a lot more verbose than python the code is just going to do the same put and get from dynamodb now it's important to point out how the references to the DynamoDB client and the DynamoDB table are created in this code I'm using a static constructor so that I'm sure that this operation is done only the first time any of those properties are requested properties or variables static variables are requested so that I'm initializing the client and I'm referencing the table only once and once those references are created they are kept in memory so that subsequent executions of the lambda keep referencing values that are already existing which is similar to what happens in Python when we create global variables in in the lambda now let's have a look at the template for for this lambda the templates for my turn and dotnet core are very similar I'm setting a timeout to 10 seconds that's not too important for the function the.net implementation runs on a dozen dotnet core 3.1 this ensures that the lambda runs on a Amazon Linux two virtual machine I'm allocating a 256 megabytes of memory and reserving 10 concurrent executions this is going to be important for the test to keep things consistent for the rest just giving permission to put and read of course query into the dynamo DB table and through API gateway I'm defining the post and Gatton points on the hash end point and over here I'm declaring the dynamo DB table it's very simple it has ID and hash as keys I'm not really using hash but I'm keeping it there the Python template is basically the same thing the table is a copy/paste everything else is basically a copy/paste except of course the runtime that's a Python 3/8 this also runs on Amazon Linux 2 machine same memories same amount of reserved concurrency executions and with this done we can start deploying you can deploy both dotnet and Python alright with the deployment successful we can go back into AWS console and find the URI of our api's so I will start with dotnet I will find the URL in staging or use the production as usual and I will quickly show you how the API is supposed to work using postman so I have prepared request here this is the payload is an ID and document for the ID I am going to use a online tool to generate uu IDs going back to postman and would send this and when I'm posting I'm receiving back HTTP to 201 the ID and the hash of the document and if I'm using gate and I'm passing the ID as a query string parameter I will receive the document back so what we want to do now is to simulate this thing with jmeter so that it's happening a lot of times so doing a real load test alright so here we are in Dream Eater then Dream Eater I'm going to create a thread group a thread group with tiny concurrent users a ramp-up time of zero and this is going to be dotnet core and in this thread group I will add two samplers that are HTTP requests sampler is a component that lets me model a request so this is going to be a dotnet or post it's an HTTP endpoint and an hour endpoint is this training found in AWS console the method is post and path is hash the body is going to be similar but not exactly the same to what I used in postman so we can we can grab request I can grab this transfer it over to Dream Eater but now there is a problem I don't want to have a hard-coded UUID because otherwise I will always have only one document saved maybe I will have one successful create operation and all the subsequent one will be a 409 so what I'm going to do I'm going to create as a preprocessor we create a user parameter user parameter with a variable the variable name will be identifier and the identifier will be generated using a function of jmeter called the UUID because it's a function I open and close and then close the curly braces now this is going to create a variable that I can use in my test and I want to update it once per iteration now I want to reuse this in both the dotnet and the Python tests there won't be conflict because those are two separate stacks which means they have their own tables so the conditional insertion is not going to affect the the fact that there are documents with the same ID across stacks all right so now that I have my identifier I need to substitute that in my just body so I'm doing that by referencing the identifier to reference the variable I will use a dollar sign curly braces and the name of the variable and yeah this should be good enough now the second request is going to be the get request so again I will add an HTTP request sampler I will rename it as dotnet core gut uses an HTTP protocol here I could have used generic parameters for the requests but I'm not going to do that now okay so here I need to add a parameter the parameter is this is the query string so it's going to be ID and the value is going to be again our identifier so for those tests one request is putting a document and the next request request gets it all right I think there is nothing else needed here we need to do basically the same thing for Python I don't want to do it all over again manually so I will just copy/paste the dotnet core group I will rename things here and there [Music] now to see the results I will add a listener to the test plan a listener called summary report now the summary report is going to show me in a tabular form the result of my test all right and with that should be good to test let's cross our fingers alright so this first run with ten concurrent users one round shows that clearly there are cold starts and the dotnet has a far worse cold start than writin we are at very very long cold starts and this reflects really badly on the average execution time here we can see clearly Python destroys dotnet when when it's time to measure cold start so now let's change our test plan so instead of looping one time which means not looping let's loop okay maybe not 2000 time let's loop 500 times each thread group so the incidence of cold starts is going to be very small so let's erase the summary and run the whole test all over again [Music] all right so for the first run off of tests we see that the dotnet suffered a few cold start and yet the average execution is basically the same as with Python only the post endpoint is 1 millisecond slower let's run this whole test all over again now I'm pretty sure there won't be any cold starts in in this run [Music] all right the test is completed and we can see that python is considerably slower even though dotnet seemed to have some cold starts the post endpoint performs roughly fifty ten percent better then it's a Python counterpart and the same thing applies to the get end point so it looks like when the dotnet executions are warm they perform a good ten percent better than in Python the results are in Python is much faster at cold starts but dotnet core seems to be 10% faster consistently under normal conditions and keep in mind that you can overcome the cold starts issue by just using provision concurrency check my video about that you can find all the source code for this video at github.com slash coder cave check the links in the description now subscribe to this channel let me know in the comments down below what's your language of choice when you called AWS lambdas and follow me on twitter at color cave [Music] you

Channel: Coder Cave

Views: 4,411

Rating: 4.7647057 out of 5

Keywords: codercave, serverless, aws lambda, dotnetcore, python

Id: A7hiOUQhV9A

Length: 13min 53sec (833 seconds)

Published: Wed Apr 22 2020