Powershell part60 (How to copy files from Windows to Linux or Unix and visaversa)

hi there did you ever find yourself stuck on the windows machine trying to copy files from a unix or linux server and you did not find a way to optimize this uh be sure to hang on i will teach you how this can be done with powershell of course this is mr automation um go to presentation let's start from the top so how to copy files between any operating system from windows by any operating system i mean uh linux and unix those are the most common ones i think but that will also work on network equipment by the way next slide so ssh from windows so we will do that with the powershell module called posh ssh show you in a minute how that works how you installed it don't copy files from my windows machine team or linux box perhaps i have a crappy drawing here i can share that this one yes so imagine you're sitting here on your uh admin laptop right or on the laptop doesn't matter if you are the administrator you need to manage a bunch of servers here let's imagine there are five thousand servers here right your imagination you have like i don't know 2000 linux 1000 unix and 2000 windows machines right so imagine you're having a windows system sitting here and that needs files from this from a linux server for instance right on a daily base i imagine you need to copy a folder structure every day from a linux server to windows server and then do some analyzes and then perhaps pass that even through to unix box i applied it to there or the other way around doesn't matter but uh i'm working in large organizations for a long time and i've been doing this also for a long time and sometimes windows to unix or linux can be a little bit challenging you know getting files there or get or or putting files there or uploading or downloading basically and you have some graphical user interface tooling for that win scp for instance and i think there's a couple of others but if you want to automate it then yeah you need to have a script or some sort in place and yeah i created this video to show you how that is done so let's continue a little bit on this we're going to copy files from windows to linux and to unix by the way it can be automated as scheduled very important no need for win scp already told that it's a fine tool by the way it's nothing wrong with it but you just can't do automated tasks with it so in that department it's it's it's a little bit cumbersome right so you have a get sap item command that's inside of this powershell module and there's a set scp item also that's for the upload and the get is for the download and there is also an ssa agent on windows available these days but i did not use it for this demo so because this will also work on a 22l server for instance or even a 2008 r2 server for that case so this is a little bit more flexible this approach so let's go to the demo part you know it's boring to see you hear me talking and see me talking as well by the way uh so let's deploy three virtual machines we need a unix a linux box and a windows machine right for our test so i was good for that let's run that and run it a couple of times today already let's start with the linux box and we call it linux thousand doesn't matter are you centos by the way and i will pause this as you can see my cluster manager is now a linux uh i think i have zoom it's yeah i have that alt and then see that linux yeah now running and specifically it's here it's a virtual machine so that's being booted currently let's wait for that and that the scripts finalizes then we have a linux box to play with okay there we have our freshly deployed linux thousand so let's see if i can ping that guy linux and i can ping it so let's set up an ssa session as well do it so we have that available uh later in the demo i did not provide the correct password for my key okay ssh mark at linux yes where my house name right so i'm there and i can even elevate myself to root if needed so that's one let's clear the screen here and create a unix box as well unix then we use not centos but we use bsd so it will be a freebsd machine and uh unix thousand yes let's create that system and i'll show you again and i'm not cheating i don't have any unix you see that no unix systems here yet you see nothing here yet that's being created now on the back here with this script it's already setting some cpus there so let's wait a little bit actually being added to the cluster now and here here's the freebsd now booting and then the background the script is configuring the system and so we need to wait a couple of minutes and then we can use the unix system so we wait for that and actually deploying these unix and linux systems i'm also using the ssh module inside of powershell there we have it the unix thousand let's see if we can ping the guy ping unix thousands yes let's set up an ssa session to that as well so if that also available a little bit later uh ssh i don't think i have my normal account there so i need to do it with the roots for now uh yes yes write a password and i'm not sure i think it's this one where am i hostname run the unix server and uh if you don't believe see it's a freebsd server so we have that one in place now as well let's do the same for windows right we need the windows box as well so let's create the windows server and we do we call it windows thousands windows 1000 and this time we not installing but we install in 2019 i windows server 2019 and let's give that some time to install and this is the windows server that's popping up that's installing and in the background you see the script still running installing configuring the system and set so we can use it afterwards and there we have our windows 1000 also available so let's login to that one as well windows 1000 this time let's see and here we have a windows server so we deployed three virtual machines and now we're going to install the powershell module on the windows machine yes what you do is you launch powershell elevated so you need to run it as an administrator to install the module you can also download the module but it's beside the scope of this demo i guess so install module push ssh wait a little bit you get a prompt you can press yesterday you can also install module and then minus confirm false of course and then you suppress these kind of outputs messages but for now it's fine now we get another prompt to install the nuget provider so i'll do just an all there and now actually the module is being installed it is installed what i do now i launched in the iec script editor and perhaps for the sake of visibility i forgot that let me size up the font here quite large one so this is what i did install module ssh then i answered some questions and now we have that module available basically when we write some script now we can use that module okay so yeah let's create a small script to download some files that's the reason why i open the script editor of course and let me copy and paste something over here because it's quite boring to see me typing all that stuff so let's fix this first i get the proper size you should be able to read that i can even make it a little bit bigger with the nasty backtick here then put that on a new line yes perfect let's save it okay what we do get scp item that comes from that module so if you do a get cmd get command get sap and you will see that the source is that module that we just imported it is get sap item already told you that's for downloading files on this linux box we are going to the temp directory with some credentials which we don't have yet by the way so let's construct that and uh let's copy that over as well from here i want to type it all over again basically i can provide some credentials here and take some root credentials because those are available on both unix and windows on my on my end and what i can do again is for instance go to this linux box and go to the temp directory there right checking the temp directory we can look inside of that there's not much but we can we can for instance create a file mark.txt for instance we can open that file right nano mark txt and we run some rubbers in there and we save it mark.txt right so we have some content there as well so we created the file there and let's go a little bit back to this crappy drawing and we're sitting here on this admin laptop right i'm connected now to that windows server via idp right you saw me make a connection there rdp right i'm sitting now on one of those windows boxes and from that windows box we're now trying to let's take a different color for that we reaching out to this linux box to the temp directory specifically and we try to get everything back from the temp directory to this windows box to the c drive right in the temp folder we're going to store it there so that's that's what we're doing now let's go back to this guy so the first line credentials already showed you we have some credentials now i can comment this out it's not needed anymore uh if you want to automate it of course you cannot prompt for credentials each time so i created the video that can be found there on how to securely uh encrypt your input for scripts credentials passwords and such and so you can use them in a secure manner right encryption with certificates and on this line line 304 we're actually reaching out to that linux system here with that get scp item command we're using some credentials root credentials we just store that we go to the temp directory what we do then is destination so that's on the windows box and environment temp i will show you if we go to that folder here explorer environment temp you will see that currently there's no marg linux folder right there's nothing there pod type equals directory pod type directory it's a directory we want to retrieve i want to have the whole temp directory in all sort directories there and i already showed you what's inside of it right here that's the temp directory [Music] enough talking let's run this command and see if we get an error or not a destination oh i made a mistake here you know uh that's the nasty thing about backticks right i have a hidden space there and then your command breaks doesn't matter now it works if i terminate this one first try it again f8 now i reached out to that linux box here got some files from temp and we got it back on the windows box let's see if we actually have something there on this windows box environment temp see that we now have mark linux sorry i cannot size this up properly so you have to do perhaps i'm able to do it but i think you can read the jesus here mark linux that's not there so if i go inside of this folder there's temp right the tmp folder we copied over and inside of that there was some hidden hidden stuff you didn't see that on there on the terminal but there's also the mark right so this is our file let's go back to this let's edit this file once more we add another another line on the from the command line red doesn't matter save that go back to this linux to this windows server going to rerun this command and we are going to check this folder again this and this mark file specifically you will see now that this line also has been added okay so that part works so what we did now let's go back to the demo created the small script to download files but we only did linux right now we're going to reach out to unix and do the same so go back to the script editor we need to modify the command a little bit but i already did that in advance it's quite lame seeing me typing all these kind of syntaxes i guess we have better things to do so let's do the same here so what we do now is we go to the unix box we do the same we also go to the temp directory and let's go to the unit box first and that open here right this unix box and let's go to the temp directory there we do a listing there it's not much the same files actually so let's also create a file here mark.txt on the unix and we go to something inside of the file and we do that to mark txt okay now we have something inside of that file okay so we have something on that unix box now as well what we're simulating now take a black pen for now we're still sitting here on this windows box now we're reaching out to this unix server get the content of the temp directory there and copy that back to the windows server in the temp directory there right let's do that i'm using the same credentials here for unix and linux i'm using the same root password not best practice of course but it's beside of this side of the scope of this demo of course so let's run it and let's go to the temp directory again and now we go to the tempo we also have a mark unix right you see that mark unix and there you have a temp and then also we have some files and also one of them is mark something inside of the file okay that's quite cool let's go to the demo so we created some small script to download files okay fair enough now create a small script to upload files so now we're going to reverse the process so what we do now we pick some files up here from this windows box and we ship it over to the linux box right and we copy it over there we do the same then for the unix as well right we pick it up from the windows box and then transfer it to the linux into the unix server okay let's go back to the demo here i also prepared that a little bit so let me see this part so let's do it in two because prep loading needs a little bit more attention so to speak i'll show you why that is in a second so let's take the linux server first and this one as well and what we do first we set up an ssa session to the linux box with the same credentials that ssa session is from new ssa session commandlet it's also part of the module we earlier imported right the pos ssh module right and in that you can also set up an sshs and so we do that now so we reach out to the linux server and now if you look inside of ssa session you'll see that we have a connected true to the linux thousand what i can do then is actually create a directory because we're going to copy this stamp directory from my windows box over to the linux server but actually to a directory that doesn't exist yet and then uh it doesn't work because it cannot create the output structure there so what we do we first create that folder remotely on that linux server and i'll show you that it doesn't exist now so it's also sort of a bonus that we do some ssh as well so i do some listing here and there you see no mark linux folder available when i run this command and that's just a make there right this this command is just run remotely on that linux box and if we now go back and we do a directory listing here you'll see that we have a folder mark linux which is currently empty by the way as you can see total zero so that's currently empty but we're going to fix that if i now go back and for instance we go to that server and i upload everything from mark linux sorry so that the stuff i have now here on my windows box and let's create a copy of this one for instance we have now two files and we call it mark two right mark two for now okay and let's now uh set so now we're going to upload it back to the linux box can we do that now let's check the linux box now you see that so now we have temp here as well and inside of temp you have mark and mark 2 but that part works let's do the same for unix i prepared those commands as well works exactly the same of course i'll show that again first we set up an ssa session to that server do that now then on that remote server hang on let me fix that on that remote server here on line 15 we created unix directory mark unix inside of temp and i'll show you that's currently not there you see that this is the temp directory and what we do now is uh create that folder that we now look inside of here you will see a mark unix folder okay and if you then run the following command we're actually uploading everything that's in mark unix here locally on my windows box and let's create a mark iii here right doesn't matter much of course mark three okay and this upload everything now to the unix box from a windows machine so let's do some listing now using our mark unix and then we have temp folder there right inside of a 10 vf mark a mark 3 dot txt um yeah use your imagination of course i can now modify some files here of course and if we then uh upload it back let me check on the unix server here the content of mark 3 for instance hang on ah that doesn't cooperate at the moment yeah i was a little bit fighting with the terminal and the keystrokes on linux that looks a little bit strange and here you see something inside of the file with all these lines we just created here on this windows box right all this rubbish we just put inside of this file once more i add some more rubbish i save it we upload it once more then we check this file once more and then you see you see there's a lot of other rubbish lines now inside of this text file so yeah um let's see so we create a small screen to download to upload we can create perhaps create one script based on input arguments let's see if we have time for that we have a couple of minutes left so perhaps we change the script a little bit so we add some parameters down now specifically this one yes let's make it a little bit simple uh not too fancy uh let's call it this will be the server name okay some input for this uh script and let's give it a mandatory assignment so we call it mandatory okay and let's take another one i need to close this one and i also need to give it a type by the way i did not give it the type so let's make it a type string and this one is of a type of validate set i think validate set set i think you can then provide value one value 2 for instance let's see if this works also typing it on the fly and it will be the let's call it uh [Music] item for now it doesn't make sense yet but it will in a second because for instance we call this item upload and we've got this item download okay let's also provide a mandatory flag for the local path correct local part and one for the remote part remote and lock call must be local of course you need to close that one we need to close that one and this is not the validate set anymore that is a string and this is not the validate set anymore this is also a string now so let's all show so now we're enhancing the script a little bit this is not supported so now we have input server name for the script perhaps i size it up a little bit will take some time to see me type these kind of codes but it's perhaps quite interesting as well to see how i built these kinds of scripts so basically we have a server name now so we're reaching out to a server to get download or upload some information no one we specified it by item so the item will be upload or download so perhaps action is a little bit better than item right item is a little bit like action so the action will be upload or download then the local part you need to specify that you want to have the local part uploaded or do you want the remote part to be downloaded to the local parts here it's quite complicated but i hope that you get what i mean and what we can then do for instance uh you can use an if statements here right so we can do an if statement action action equals let's say a plot we do some action we just step out the code a little bit so we do some exits there and else that means there's an uh upload special a download specified you do something else right that's that's it that's it for now what what do we do if we do an upload we must do all this stuff right we all already found it out so let's cut that here and put it inside of the plot action for now okay just leave it there for now else it will be get action so then we get something right it will be download only removed that from there and we stick that there and basically this is all not needed for now so let's comment it out for now and put it out of the visibility so it doesn't bother us this credential needs to go a little bit down it makes no sense to have it above the parameters okay so actually what we're doing now this server name that's now variable you can use that here right so we can use the server name variable variable the credential still stays the same that's right we still need to have that you can also provide it as an input here of course at some point in time but for now that's not out of the i already told you here's a link on how to secure credentials i would suggest add it to the script you know if you have some interest in this call um so let's see what we do then is uh we make a dear still but we need to do that on the remote part right we need to create the directory so the remote part we created directory so we do that create a directory and then here we set the sap item again we can use the hard coded server name we can remove that i mean but use it to remove it credential status is saying this can also stays the same but actually that becomes local but if you don't want any hard coded values there it becomes a local path and the destination there becomes the remote path there that's it and basically we can do the same here and then a get item will be that server name replace that that will be the remote part then the local part in this variable we use that then here you can remove all this stuff there and i think this way we now created a quite powerful script that accepts some input when we run it then let's show how it works right let's see if it works at all okay let's try um it's called demo 60 minus server name right we have that on top there so let's take the unix thousands for now uh i can do full qualifier it's not really needed but i do it for now minus action it will be if i do a control space now you will see that only uploader download is allowed so i'll do in the cloud here all right download i meant download and then minus local pod what's the local bot and in my temp here i have a marg folder i use that now as a local part so we provide that as a value there then we also have a remote part and that's again our uh tmp directory remotely on that uh unix box let's run this and now it says user and which user it's line 10 here the get credential right that's prompting us now if you don't want the prompts i will tell it again i have a video here and how to secure those credentials so you can automate all these steps and now we did not got any error if you now go to this mark folder you will see temp then we have our wonderful stuff there so let's see if the upload works as well perhaps we brought the script right yeah so let's try to upload something now so we run the same script action is now upload we upload from the temp directory to the remote temp let's see what we get if we run this uh oh i have the debugger enabled doesn't matter much i guess um okay script completed let's see there there we have it mark3.txt i'm not looking in the correct wall hang on i am looking in the correct folder i was uploading that downloading so let's change a little bit of information in here let's see if we can upload that back without any debugging see all right still debugging i don't understand why but never mind there we have it i think i was looking in the wrong folder by the way so it helps indeed so let's remove everything inside of there let's run our script again okay now we did not have any debugging so that's good and let's see if this temp file you see that was just updated that mark three you see something inside of the file so that was the upload part so now uh yeah there we have it simple script with some input upload download local path remote but server name and based on that we download or upload information and if you automate this part the credential part with the video i created here then you can automate all these steps and this is very simple script see that 21 lines so i bet you can advance it a lot at least this is how it's done so how you can communicate with unix and linux from windows let's go back to the crappy drawing so yeah we did everything uh that was mentioned there we installed we created a small script we even created one script based on input arguments yes exactly so that makes me come to an end and thank you very much for watching and practice you again bye

Channel: IT Generalist aka Mr Automation

Views: 92

Rating: 5 out of 5

Keywords: automation, powershell, learn automation, how to automate, windows automation, linux automation, unix automation, powershell automation, Mr Automation, IT Generalist, configuration as code, infrastucture as code, orchestration, how to copy files from windows to linux, how to copy files from windows to unix, how to copy files without using winscp, PowerShell intro, powershell tutorials, powershell scripting, powershell intro, powershell tutorial, learn powershell

Id: qpN_PMAKClI

Length: 33min 17sec (1997 seconds)

Published: Sun Sep 12 2021