How to set up SFTP access for S3 using AWS Transfer for SFTP

hi and welcome to another majestic dot cloud video today I'm going to show you how to set up SFTP access for an s3 bucket we will do this with AWS transfer for SFTP so what is a WS Transfer it's a fully managed service offered by AWS that allows us to transfer files in and out of an s3 bucket as if it was a regular SFTP server because it integrates with route 53 DNS we will even set up a custom domain name for this SFTP server and use that one instead of the default hostname provided another cool feature is that we can integrate custom identity providers and this allows you to set up a different authentication mechanism and use that instead of the default one so what are the use cases for such a service use cases revolve around existing legacy applications that need access to an SFTP server for reading data or outputting data to an SFTP server so if we have such a legacy system we can set up this SFTP endpoint on our s3 bucket and provide this bucket as a data store for that legacy application so what about the pricing the pricing for AWS transfer for SFTP is per hour you pay 30 cents per hour for every hour that your endpoint is provisioned and you also pay 4 cents for every gigabyte of data transferred either uploaded or downloaded it doesn't matter you pay for cents and that's it you also pay for your s3 for your regular s3 bucket you pay the storage space and you also pay for operations boots cats and stuff like that so let's not waste any more time and let me show you how you can set up AWS transfer for SFTP on your s3 bucket first of all we need to create the s3 bucket so let's go to a stream and create a bucket I will name this bucket majestic SFTP ok let's create it and that's it for all let's go to I am where we will create an I am role for the SFTP endpoint but before we creating the role we need to create a policy and in this case let's create a policy for s3 for the bucket we just created we need to provide list bucket access and this bucket get bucket location we will provide our bucket name majestic SFTP add ok so this is these are the back bucket permissions and we need to add additional permissions for the files so we choose s3 here again and provide read access so the get object get object permission and then we go to write permissions and we will provide delete bucket or delete objects or in its object and delete objects version and also put object okay and here we also need on the reads get object version okay so these are the five permissions we need for s3 files and here we need to specify the bucket name again adjusting SFTP and for object name we will choose any so but at this so this is our policy and we will review it and save it as myself DP I just think this will be the policy name let's create the policy and then we need to create the role so let's create a role here we need to choose the first option AWS service and here the service that will use this role will being transfer so we need to choose transfer here and then go to the permissions and let's search for the policy we just created previously and we will add this policy next click on tax we don't add any tax review and SFTP we will give the name for this role SFTP access for majestic a string bucket okay let's create a role okay so we created it please sit down whenever we go to this role and check here in the trust relationships we need to have this trust relationship with transfer Amazon AWS dot-com so let's update trust policy and this is it we have our role setup so now let's go to SFTP AWS transfer for SFTP we will open this in a new tab and for some reason it doesn't want to lose this okay now it works let's click on create server this will be our new SFTP server then point type will be public we will use a custom domain name in this case for service will it will be route 53 DNS all alias let's give our SFTP server the name SFTP playground majestic dot cloud okay AWS transfer will create the DNS records for us automatically the identity provider will leave the default one for this example and we click on create server ok while our server is creating we will go and setup our user but first let's check if the DNS record is properly created so if we click on this one we can see that AWS already created us the cname record in the DNS console and it should be ok let's go back and proceed with creating our user click on server ID and then here we have the option to add the user so let's add the user now the username will be majestic here we need to choose the I am role that we created previously so this one SFTP access for a majestic s3 bucket and here it at the home directory we need to choose the s3 bucket we have created for this purpose and for which we created the policy the access policy now here we need to specify a public key for our user because our user will be using private key to connect to the SFTP server so if you don't know how to create the key you can click on the info link here and here you have also a link which you can open and then here in generating SSH SSH keys you have the command that you will need to run in your terminal so let's do that let's create a transfer key I've changed the name so it will be SSH cajon and the name will be transferred so it did create the keys if I do a list we will see here the transfer key and the transfer key pub let's list the transfer key Pub public key this one we need to copy it until here and let's paste it in here so this is the public key ok let's add our user our user is now added and we need to configure our SFTP client locally so the witnesses SFTP client you connect to the SFTP server in my case this will be winscp this is an FTP client I use often and it has support for SFTP CP FTP and so on and you can download it for free and it's a very good little utility let's configure our SFTP server here it will be a soft TP playground majestic dot tau this is the hostname for the username I've put in majestic and we will not give a password but instead we will go to advanced and here at the authentication tab we will choose our private key file it can't find any private key because it is looking for the PPK extension I don't have the file in that format I will click on the transfer key file and it will automatically offer to convert it to the OpenSSH private key to party format so you just click on OK click here on save and it's automatically converted into a apk format mat and then you click on OK and let's save this connection so save so I have it for for future actions and this should be it let's try to login so it wants me about potential security breach because I've changed the hostname okay so it connected to the majestic SFTP bucket so let's try and copy some stuff over ferry head first we have this taxable CSV file let's copy up this to the last three bucket upload it in other words so it says the upload of a file was successful but error occurred while setting the permission and/or timestamp so our file was copied but the utility tool could not set the timestamp because we need to modify a setting here so we will keep on skip the file is copied and if I go to a string you see I mean the safety majestic SFTP bucket if I do a refresh you see I have the file here but in order to not to get those annoying alerts when we upload files let's go and change the options we go to preferences you know in winscp and here in the transfer settings if we go to the default and edit here we uncheck the preserved timestamp option and click on ok and if we transfer any more files then like this one for example we will not get alerts so let's copy more files I will copy the transfer key PPK so I have the shortcuts I'm using shortcuts here with F the the f5 key on your keyboard you can copy a file up so this is it we go back to the s3 bucket and we click on it and I mean click on the refresh button and you can see that I have all the files that I've uploaded with the SCP tool now let's go back and delete all these files I've selected them all and I can click on the Delete icon here or just push f8 and let's delete them delete them all for this is deleted from the s3 bucket itself if I go back to s3 and click on the refresh the bucket is empty so this is how it works now we will also stop our server because if we leave it running then we will incur 30 cents per hour so let's delete the server and delete you need to confirm it okay so now it's deleted so this is how you can set up an SFTP server on top of your s3 bucket and this is the hub you can use it easily so I hope you liked this video if you did click on the like button and also if you want to get updates when I release new videos on topics like cloud computing or AWS click on the subscribe button to stay updated thank you for your attention and have a great day

Channel: Majestic.cloud

Views: 23,178

Keywords: s3, sftp, aws transfer, aws, iam roles, custom domain name, s3 bucket access, s3 bucket

Id: P9CvdX8aMUY

Length: 16min 50sec (1010 seconds)

Published: Mon Sep 09 2019