AWS S3 File Upload + Lambda Trigger (Tutorial In Python) | Step by Step Guide

hey guys this is AWS simplified and today I'm gonna show you how to connect a lambda function to an s3 event specifically an object creation event so firstly let's look at how this thing is going to work so the first step is to create a lambda function I'm going to be calling this lambda function transaction processor I'm going to code up a lambda function such that it can process a file that looks like this after that I'm going to create an s3 bucket and we're going to call that s3 bucket the transaction store and the next step is to hook up the trigger so every single time someone uploads a file to this s3 bucket it's going to a synchronously invoke this lambda function and this lambda function is just going to do some light processing of the files content and the final step is upload the file to the s3 bucket and make sure everything is wired up and working correctly so that's how this is going to work let's go to the console and get started so here we are in the console page so I'm going to go over to find services here and type in lambda I click on that and we're gonna come over here and click on create a function and from here we're going to call my function named transaction processor and for the run time we're gonna pick Python 3.6 and permissions so what we need to do here is create a role that has s3 get object permissions so luckily one of the policy templates here actually has that for you if you click on policy templates down here and you type in s3 and you see this option your Amazon s3 object read-only permissions so that's going to give us the permissions that we need and I am going to name this s3 access role so this is going to associate this role with my lambda function so everything looks good here I'm going to go ahead and click create function and that may take a moment or so for the function to be created alrighty and here we are with the transaction process or lambda function so the first thing I'm going to do is put in our handler code so if I go over here I have some pre-written code that I wanted to insert directly into here so let's take a quick look at what this lambda handler is doing so first we're gonna grab the bucket ID and we're going to parse the events to grab the bucket name using this dictionary lookup notation and then we're gonna do the same thing for the key and again we're going to be looking up the key in the object and then from there we're gonna use the bucket and key combination to fetch the object from s3 and the fourth step is to deserialize the files contents and put it into in local object that I'm naming data in this case we're gonna print data so we can see what it looks like and finally the final step is to iterate over the contents of the JSON object and print a transaction type so that's exactly what we have here so I'm going to take that come over here and paste that in and I'm going to click Save and that is pretty much everything for the lambda function I can leave all these settings down here empty don't really need to worry about it so from there I'm gonna go to the s3 section of the console type in an s3 so I'm gonna click on create a bucket here and I'm going to name this bucket transaction store the WS simplified I'm gonna pretty much leave all the settings here default click Next click Next click Next create the bucket alright awesome so buckets created I'm going to click into the bucket and we're going to go to properties and from here we're going to the events section that's going to let us hook up rs3 put to our Latin function I'm gonna click on that and here is the dialog box so what we need to do here is click on add a notification and we get to pick which type of notification we want to trigger on but for this exercise we want to pick the put event so I'm going to click the put checkbox and I'm also going to name this event s3 object put so I can keep track of it later they also have some additional options here so say for instance you only wanted to apply this s3 trigger to certain sub-directories of your bucket you could specify the subdirectories here and similarly if you only want to apply this to certain types of files that are being updated and put into this bucket then you can do so by changing the file suffix here and this is actually the one that's important so we're we're going to send it to so we're going to be sending it to our lambda function and this should pre populate with the results yep so there's our transaction processor and I selected that's going to click Save and so it should show up as a active event here let me just refresh the page sometimes there's a bit of a delay yeah so there we go so we have one active notification that's now set up if I click on this guy you can see exactly what I just created so the s3 object put the put event type and the destination lambda so before I actually upload a bit I want to go back to lambda just to confirm that the s3 trigger is present so going back inside that's and there we go so now we have the s3 trigger that's present so when I click on this guy that we can see that we have the bucket name here social interaction story WS simplified we the event type and we have the notification name so now we're gonna go ahead and upload a file to that s3 bucket so I'm gonna go back - they click on the bucket in question click on upload add files and here's my transaction JSON file and here's the size gonna click next to upload it just giving us some warnings that's fine we can leave everything default standard tear looks good and upload and there we go our file is there so that should have triggered an s3 put and should have triggered our lambda function so let's go back to the lambda function monitoring section to see if there's actually an instance of the invocation going back here transaction processor going to the monitoring section and there we are so we have one data point here signaling that there was one invocation very quick duration did not error up so that's perfect so let's go to cloud watch to see the logs that were emitted and verify that we were able to successfully handle the message and long the contents so I'm going to the log section click on the log group and click on the log stream and here we go here's our logs so here's the first line where I was printing the data object so we're just printing the contents of the JSON file here and you can see here I'm iterating over the JSON content and extracting out the transaction type so the purchase and the refund and that's being extracted out here thanks so much for watching guys and if you have any topics you'd like me to cover drop me a message below in the comment section and please don't forget to Like and subscribe thanks so much again and I'll see you next time

Channel: Be A Better Dev

Views: 72,246

Rating: 4.954329 out of 5

Keywords: AWS S3 trigger, aws, s3 trigger, s3 trigger lambda, lambda s3 trigger, s3, amazon web services, aws s3 lambda, aws s3, aws s3 lambda trigger, aws lambda s3, aws lambda python, aws lambda s3 python, aws s3 put, aws s3 put trigger, aws s3 lambda tutorial, aws lambda s3 example, s3 put event, s3 put event lambda, aws s3 put event lambda, aws s3 trigger python, aws s3 lambda trigger python, awssimplified, aws simplified, serverless, serverless nodejs, aws developer, s3 amazon

Id: H_rRlnSw_5s

Length: 8min 12sec (492 seconds)

Published: Mon Aug 12 2019