Setup and Review of SyncThing, The Open Source File synchronization tool

okay we're going to take a look today at sink thing same thing is a decentralized peer-to-peer file synchronization program that can sync files between devices on local networks or over the Internet it's completely free open source very active Eila actively developed and very secure while it does synchronize files similar to own cottage Dropbox it does so without the need for a server it is more or less an open-source version of the BT sync tool but has a lot more features these devices you share the folder will keep the same copy of the files as you synced now syncing supports Linux 32-bit 64-bit ARM arm 64 Solaris Windows Mac and Android as of September 2016 we're at 0.1 4.6 version secure private open development a lot of great reasons to use this now it's built on a framework referred to as the block exchange protocol and this is really really well written this is not just well written is well documented so this entire block exchange protocol is the base that all of this runs on top of now this whole project I can't believe the level of documentation it has for it I'm still going to do a video on it even though it's not hard to follow documentation to get things set up but I just kind of want to show it in action now so I got my demo machines fired up here and I already have it done here downloaded on a Linux box when I quit Firefox and just show you how you open it up so once you download sync think you just have to run it there's no config file you really have to do there's plenty of options you can do but we're just going to start with the basics of just kicking it off and run so you download run no wizard sync thing just starts it's not running in the background the first thing it does is launch a window directing your browser to HTTP essentially localhost or 127.0.0.1 authenticated username if we want authenticated password use HTTPS and you can pick your IP address to bind to so we can be externally accessible now if you wanted to do this on a headless server that does not have a web browser you can you launch sinking on a Linux box or a Windows box with a command-line option telling it which IP to bind to and then go in and change all these settings to the web interface and customize it or you can edit the config dot XML file directly which is in Linux located under dot config slash sinking config dot XML and on Windows it's your username local app data sync thing and it's actually the same config data XML file now you can also set your rate limits your outgoing limits internal limits so if you wanted to not have a set rate your bandwidth you can do that you can give your device a specific name it's just going to pull the name of the computer this is a VirtualBox installation called Thomas VirtualBox you can change the default listen address now this is some neat options here so natura versatile global discovery local discover and relaying what these things do is if you don't have a way to map your firewall and you wanted to connect devices and you just want it to be discoverable you can use and this is the default their own discovery servers now your data does not pass through these discovery servers per se the discovery servers just allow sync things the instances to sync with each other by registering with a global discovery server sharing out your IP address to the global discovery server and then allowing other people to connect to you now they can't just connect they do need to use a key so it's not like it's an unencrypted connection and I do not need your username and password I just need your unique key and then even once they have the key I'm going to show you how you accept the share so we have seen thing running right here and here is the default share folder which is a unique folder ID that's randomly generated on here now whenever you're doing a share you can call the folder label whatever you want default folder whatever but you have to leave the folder ID unique upon creation it can't be changed so once these are set they're set and we're going to save and we can call this this one's now referred to as default folder whatever other options on these folders you have a lot of different how often you want to scan it you can take it whether that this is a normal or master folder a normal folder means whatever's in this folder is synced with all the other devices and I say all it's it's not just one or two devices you can sync with it's many many some of these people have a thousand devices that they're syncing with but other option is master and if you set up as a master anything you put in here just be its propagated to all the clients that connect to it and changes they make don't go upstream back to this once we're leave it here normal file pull order you can say pull the newest oldest random or alphabetic list so that's how if you have a lot of them that are sync which order do you want them to sync in the file versioning is pretty cool now trashcan versioning is pretty simple which means when a file is deleted throw it in the trash can now this is very important if you're sharing this with someone else and they delete a file and they're completely synced with your folder it deletes it on your computer too but then you have an option to say throw that into the trashcan now the downside of trashcan versioning is you can set the number of days that it goes in there but let's say we have a file called time text and we delete it then we create another file called time duck text and we delete it it goes and over writes it that's where staggered file versioning there's an option where it also synchronizes every incremental change and we'll show how that works so we're going to set this one to staggered file versioning we're going to go ahead and leave the days at 365 leave the version path at default but this is a neat option you can take and have the staggered file versioning sent somewhere else now there's also simple file versioning which just keeps five versions of the files so every five edits of a file it deletes the sixth edit and full external versioning is you can set up a series of commands to execute every time a file is moved or deleted and then that just gives you another level of customizing we're going to use staggered for our demonstration here so save all right now we're going to fire up the same thing on our Windows machine so here's Windows already have sync thing downloaded here here is the location user's username app data local sync thing there's the config.xml file just like we have on the Linux machine under config so much the same there and it runs and launches the web browser actually to make this easier I'm going to go ahead and allow access to the other machine so we do that by doing this what's the IP address of this one nine two one six eight dot 103 so we go into the settings and change this to 192.168.1 oh three save now it's going to warn me of the danger because there's no password on us and it says people can easily access hackers pretty bla bla bla I don't care all right so now we have it over on this device here so this is our Windows machine and this is our Linux machine we're going to show the device ID now these devices are absolutely you should keep them protected that's one of the reasons I'm doing it in a virtual machine because knowing my device ID would be the step in towards accessing my existing machine so we're going to copy this and we're going to add a device and what's the device ID we're going to give a bit of that and it's the windows 7 v box now if I knew the address which we know it's TCP colon IP colon port now that's how you connect directly we're going to go ahead and let it figure out what to do we're also going to go ahead and tell it to share this right away so we're going to hit save here and this will allow this device which is the Linux box to find and discover we're going to go ahead and let it use the discovery servers and local discovery servers can hit save now it's looking for it now this does not happen instantaneously it's going to take a second and on here we're going to wait we're going to see a prompt come up that the devices want to talk right now they're going through in a block exchange protocol is slowly pinging out and finding the other devices and syncing them up and it's verifying with TLS security the keys that were generated and a way of verifies that we actually are trying to access this is because of that long ID when we into actions and show ID and of course if you're doing this with a phone that's why it's got a QR code as well so now the device is sound a new device this virtual box with this ID from here wants to connect would you like to add this device I can say ignore it or add it or deal with it later so it wants to know okay it's dynamic address this is fine do I want to share the folder I'm not going to share my folder but they're going to share with them so we're going to hit save and we're going to call this the Linux box so you know that's the Linux one because that we can add whatever device name we want so save now it's going to take a second because now the devices are connected with each other and we're going to see this show up in turn here this is on our Linux box and this is our Windows 7 box this does not happen absolutely instantly so I'm going to go ahead and show you the delay it does take up sometimes 60 seconds before the two devices really start talking to each other that is normal now it's found that says hey looks they want to share this folder so we're going to go ahead and add the folder now the folder path is pretty cool I like the way this works so you can say share it on C what do we want to say or what I'll type users it autocompletes for you it is case sensitive whoops users umm Tom's and we're going to call it the Linux share and if it does not have the folder it creates the folder on their default folder whatever that was what we called it on the Linux machine default folder whatever from a linux box then we're going to go to the Advanced Settings now the trash can set up is per device so I set up a trashcan for that share on the Linux machine but we didn't set up yet on here and so it's per device per share per folder share so we're going to go ahead and say we're going to just do something simple here a trashcan file versioning keep 11 days worth and hit save alright now we have a share and it says it's up to date so this one's still unshared and we're just going to for simplicity we're going to delete because you don't need to have the default share you don't have any share you can just share with this we're going to have here's from the Linux box it's under the Linux share and here's my VirtualBox here's my folder for it now let's actually look what's in the folder let's put something in a folder so we got desktop sync thing home sink right now it's empty so we'll do this it's going to rescan and find some files I can force it to it now by default it's set to every seconds 37 files and in a few seconds when this free scan so I'm going to go ahead and force it to speed things up it now has them indexed it's going to take a second and goes hey we are not up to date and have this happens a little bit slow when you first connect them once they've been connected for a while this happens instantly it also happens faster if you specify the TCP ports we've now transferred over to this one thirty seven items eleven point eight Meg's the protocol itself is using Delta syncing within the block exchange protocol and what that means is it syncs on the Delta so if you had a one gigabyte file and you change only a few bits into one gigabyte file it can sync in 128 kilobits the Delta changes of that file so the protocol is actually really fast so now that these two machines are completely talking each other and fast let's go in and make some changes on the Windows machine we drag a bag over here here's the Linux share let's go ahead and just delete something like this will delete it out of here let me go back to the Linux machine and now it's deleted out of here so it's synced up and it's deleted and now this like I said I've turned on so we can see the hidden here's that st versions folder and there's that folder we deleted so this is how it controls the versioning is for every share there's an ST versions folder and depending on the type of versioning you have set up that's where the files go so let's go ahead and create another file so we can have the readme text make it out a copy of it and rename it just testing now let's make some edits on this some edit from the Linux box save the file go ahead and force a rescan now by default it's set to 60 seconds I'm not going to set it to one second because it makes it kind of file intensive at that point and we can last file receive updated testing dot text that means on our windows box which we'll bring back over here there's our testing dot txt file some edit from the Linux box some edit from the windows box Save forced to rescan real quick rescan files make sure it's up to date yep this one updated it some edit from the windows box so now you go back and forth and as you save these now if we create a file conflict by doing that it will automatically create a conflict so we're going to say start conflicting from Linux say up don't want to save it so here's an edit ready to be saved and we're going to simultaneously edit and save over here before they can do anything so just make some edit here and we're going to jump real quick and hit save save then we're going to force them both to sync and it's going to give me a list of failed items because I tried to edit the same file at the same time this one synced first this one's saying out of sync and trying to figure it out all right now for the version conflicts I still save them so fast that it did create it and here it is we have a conflict 1 so it tells you about the conflict of this it doesn't do anything to the file itself it just creates another version so we can look here and see we have testing sync conflict and it tells you the time that the conflict occurred so we still have our original file and now we have one called conflict now also what was going on behind the scenes is in the version folder we have a another version from we saved over the same file so it tells you the date and timestamp of this version of that file so if we edit this file again so we're going to go over here to the Windows machine again and we're going to open up that testing dot txt file and let's just clean it up so let's get rid of all the crap inside of it delete it now it is mostly blank file exit save and we'll go ahead and move this out of the way and we're going to force a resync on these rescan priests can go into our sink folder look for this testing duck text file now it is mostly blank go back to the actual Rula folder look at this hidden st versions folder and here's the different variations so if i want to go back and look at what it was here or what i was here it now kept all the version histories of them so i synched it from I edited persistently on the Windows machine and it creates all of these hidden ones this is the staggered versioning definitely a great way to do it that way everything happens to your files or something gets edited on one of the other computers you need to go back they're all nice and date stamped so that's kind of the basics of how the syncing works you just share that on there this concludes same thing for kind of a quick overview of how to get started and how to get it working now I'm going to do another video this can be a follow up to this taking a deeper dive in it but this is just kind of an overview to get you started on this sink thing it's a great tool I really like it it's definitely been a solid performer for me having all the files that I have in there thank you for watching this review of sink thing if you like the content here like and subscribe and I love to hear feedback of how you're using sink thing or any other tools that I talked about thank you

Channel: Lawrence Systems

Views: 68,633

Keywords: linux, software (industry), sync, ubuntu, ubuntu (operating system), tutorial, bittorrent sync, syncthing, file sync, btsync, linux mint (operating system), syncing, peer-to-peer, file sharing, debian

Id: iFh27uPsYRw

Length: 18min 49sec (1129 seconds)

Published: Sun Sep 18 2016