Syncthing Made EASY

today's video we're going to look at one of my favorite pieces of software sync thing syncthing is a free open source file syncing solution for mac windows linux bsd solaris android but sadly not for ios and i will talk a little bit about workarounds for ios later in the video so the purpose of today's video is to show you how to get up and running this thing thing how to install it how to pair your machines together how to share files and so on and so forth but before we dive into the installation steps i just want to talk about two characteristics of think think that i think make it a really interesting solution for syncing and is the reason why i use it over something like dropbox or icloud so the first characteristic is that sync thing uses peer-to-peer syncing what this means is that your machines talk to each other directly and they share files with each other directly there's no centralized server which is what you have with like icloud or dropbox where each machine sends the central server and then the central server sends on to the rest of the machines in your little file sharing network the benefit of peer-to-peer is that you're not beholden on a central service or a central server or even a central service provider so if dropbox decides to shut down their service one day and use dropbox for syncing you are in a mess also if dropbox just has an outage you can't sync your files you don't get that with peer-to-peer syncing even if one machine is not working the rest of the machines can still talk and obviously because sync thing is free and open you are in complete control over your sync network so in certain cases two pairs in your network can't talk to each other so they transmit data via a relay and this brings me on to the second characteristic of sync thing that i really like which is that the data is end-to-end encrypted it gets encrypted on the client that transmits the data and then decrypted on the client that receives it there's no bit in between where the data is decrypted contrast this with solutions like icloud or dropbox or google drive where data is encrypted in transit so between your machine and the central server and it's encrypted at rest or when it's been stored on the central server but there's the small window when the data arrives from transit and before it's written to storage where it's actually decrypted and your mileage may vary on how much of an issue that is and certainly end-to-end encryption is something to strive for but it's not it's not absolutely necessary i feel much more comfortable with my data being a network that i completely control touching only my machines and always flowing end-to-end encrypted between those machines one of the very nice things about sync thing though is it's not an all-or-nothing solution you can still use it alongside your other file syncing solutions and we'll see it later on when we look at how to make sync thing work alongside ios by plugging it into icloud for now let's dive straight into installation so to install a mac you've got two main options you can install using homebrew which if you are using that that's a great way to install it and i've got a video on that up above if you're not using home brew and you just want to dive straight into sync thing then come to the sync thing website come to the downloads page and what you want to do is download this sync thing mac os application bundle this is the easiest and nicest way to use sync thing on the mac click on that to be brought to the downloads page and then download this dmg image here your version may differ if you're watching this at some point in the future once that downloads come to your downloads section click on the dmg image to mount it and then drag the sync thing icon into applications once that's inside applications if you open up your applications folder find the sync thing icon double click on it choose open and you'll see that a little sync thing icon pops up in the top corner here and we'll revisit that shortly when we look at how the control panel works now let's see how to install on linux so on linux you're likely to find that sync thing is in your package manager so it's a really easy installation i'm here on ubuntu so i'm just going to do a studio app to install sync thing doesn't take very long to install once that's installed you now need to start sync thing and if you're using an operating system that has systemd then you can do sudo systemctl start sync thing and it failed and the reason for this is sync thing is not a service that you run just once on your machine you run it once per user so instead of running system ctl start sync thing you do start and then at the user in question i'm using parallels because i'm running this on my virtual machine so that's started that if you want to check what your username is you can just run who am i like that and it tells you in parallels and you may also want something to start when the machine starts so rather than just running start here we'll also run enable so this will now auto start syncing for my parallels user when the machine boots okay great let's turn our attention now to windows so on windows come to the sync thing downloads page again and rather than choosing sync thing mac os choose this sync tracer this is a really nice wrapper around the default sync thing binary that gives you kind of like a tray icon and a file system watcher and launcher and all that kind of stuff it makes sync thing feel much more like a windows app so click on that scroll down here to the setup xe for your platform so you'll want x86 or x64 because i'm using sync thing on arm i can't actually use this package i'm going to install a different package but download that and install it and then you'll get to basically the same situation that i'm in if you're on arm what you want to do instead is come down to the base sync thing here and choose one of the base windows packages i'm on arm 64. i'm going to download that i'm going to open file choose file explorer come in here run sync thing and it will say that this is an application that's in a compressed folder so you can just choose extract all extract them all and now run the sync thing again when you first try to run it windows defender will complain you need to tell it it's absolutely fine so the button here says don't run you actually have to click on more info and then you can choose run it anyway and then choose allow access and there we go we've got sync thing running now when you run this default windows package you are running just the syncing app if you reboot windows it doesn't restart i will link below some instructions on how you can mount this as a windows service so it will automatically start but if you can if you're on x86 or x8664 you should try to use sync tracer it's a much nicer experience so now we've got three machines running sync thing windows ubuntu and mac now it's time to configure each of those machines and actually start to pair them and the way this works inside of sync thing is you use the admin console and the admin console is accessible by the browser and it's on each machine it's available at localhost colom8384 the first time you run it will ask if you want to allow anonymous reporting i never do and it will also tell you that a username and password has not been set for gui authentication by default you can only access the control panel from the machine that sync thing is installed on so you don't necessarily need a username and password when you're in that setup if you choose to make the control panel available to other machines which you can do then you should definitely set a username and password however i think there's a nicer solution and we'll see that a little bit later so i'm going to click ok for now and let's make this a little bit bigger that's perfect so you can see there's kind of three areas here there's an area where folders are so the nice thing about sync thing is you can pick exactly which folders you want to share and exactly which machines it's not like icloud or dropbox where you have one big massive folder you have to share everything you can pick and choose where you want these folders to come from and where on the machine you want them to go and we'll see some examples shortly there's a bunch of information about the machine itself what the download and upload rate is how much is being shared there's this identification token this is really important this is how machines find each other how you share one machine with another we'll see some more examples of that soon and then when we have a remote device it will show up down here up in the top corner here where it says actions you can find settings you can restart so on and so forth and if you come into settings one of the things you can do in this gui section here is set a username and password if you do want to enable other machines to access this control panel you can take where it says one two seven zero zero one here and change it to zero zero zero like that but i recommend not doing that and we'll see an alternative very soon so i've got my machine here on windows and i want to pair it with say with my ubuntu machine so i'm going to choose add remote device and it's already found those two devices when you have sync thing running on multiple machines on the same network they will automatically see each other so this is a really nice way of pairing machines if you can do them all on the same network you kind of just get this helper when machines are not on the same network they don't automatically see each other and we'll see an example soon of how you can pair machines in that situation i don't know which of these is my ubuntu machine so i'm going to switch back into that and then i'll open the control panel here as well again localhost 8384 let's also make that a little bit nicer to read so you can see similar control panel just a different device name and a different identification and i can see that the identification for this is asyfzo so let's come back to windows and we're going to pair with that machine af asyfo and you can see the real id is much longer but when using on the local network they use the short ids i'm going to press on that i can give it a device name so i'll say that's my ubuntu i want to and press save so when you first create it it says disconnected unused and that's because you haven't yet paired the machines together you have to when pairing machines and also when sharing folders you have to offer it on one side and accept it on the other side so we come back to ubuntu now what we see is that this little pop-up here says oh this new device wants to share with you wants to connect with you so i'm going to choose add device i can give this a name i'm just going to call it windows press save so now it says you're connected to windows so now back in windows it says that my ubuntu is connected but unused and it's unused because we haven't yet shared a folder so let's now share some folders now i like to get rid of the default folder so just come to edit on that and press remove yes and instead i'll add a custom folder so pressing the add folder button you can give it a custom label but we'll change that in a second the most important thing for me is the folder path where's the data coming from i don't want this randomly generated one i want a folder for actual windows and i want to share my windows documents folder i'm just going to call it documents and there are a bunch of other settings that we'll revisit in a second the most important one for me is file versioning but we'll we'll see that soon but you do want to come to sharing and choose which device are you sharing this folder with so let's share it with ubuntu press save so that says documents is up to date here and now if i come back to my ubuntu machine i'm now prompted do you want to share the folder documents with the machine windows so click add and it brings up a similar dialogue now i get to choose the values of that documents folder on this machine and i could if i wanted to put it somewhere completely separate so maybe i want to put it under sync like this and have it separate from my documents folder but no i'm going to use the default documents folder for now press save and i will just quickly clean up the default folder here also so now we've got a documents folder here and there's nothing in it so let's put something in it say on the ubuntu side and we'll see it appear on the windows side so to do that i'm just going to bring up my terminal and i'm going to come into my documents folder let's just say echo hello into a test.txt file and i can just check that that is there on this machine yep great and if i come back to the control panel start to see some action now what you want to see here is in rescans it gives you the period how often is it going to do a full reason of the folder but you most want to see this this little eye icon this says that sync thing is watching for file changes using your operating system's watch support so it will detect changes far faster if you don't see that little eye icon with the enabled next to it you need to install some additional software and i will link below what that software is i find now that most machines have file watching enabled by default so you don't really have to do it very often and you can now see that the global state and the local state show that there's one file which is six bytes and it also shows that my windows device is up to date and it took maybe three or four seconds to find that i'm not quite sure it usually happens well within 10 seconds to sync the files together if we come back to windows we'll see similar data on this side so it's saying that i'm up to date the global state and the local state are the same and it also shows you like what was the last thing that happened so the latest change is that i updated text dot txt and just to show let's come to the documents folder we can see test txt here gonna open it with notepad and there it is hello so now we've got two machines talking to each other and they're sharing a folder let's add a third machine into the mix and i'm gonna add my personal mac into this little syncing network and if you look at the control panel from the mac you can see i've already got a bunch of folders that i've shared and i've only got one remote device you might wonder why i'll tell you a little bit about that in a second but let's add a second device in come to our remote device i'm going to pick the ubuntu machine on too and before i press save i'm going to come to sharing and i'm going to make this the introducer and i want to be introduced to the other devices that the ubuntu machine knows i also want to auto accept folders that that machine is exposing we'll still need to go through and click a bunch of buttons but we don't have to specifically pair up with every single machine in every single folder so let's come to save come to the ubuntu machine you still have to accept that pairing on this side so press add device just going to leave the name as it is so now we've accepted the pairing on the ubuntu machine and it says that the machine is unused but connected which is exactly right if we go back to the mac we get the same thing it's connected but unused and i want to share a folder the documents folder between these two machines and we should see that the mac links to the windows machine as well because we made ubuntu an introducer so coming back into the ubuntu machine come to the pairing with the mac machine choose edit come to sharing and then share documents now i wanted to show you one thing very interesting here the ubuntu machine is able to expose the documents folder to the mac because that's the folders it knows about if i come to the mac and do the same thing come into the ubuntu machine come to edit come to sharing you'll see it's able to expose its folders so it can get quite messy when you're trying to pair up all these folders especially if you've got some machines that only see some folders and some machines that say another but this is the power of sync thing like you get complete control over what files go where what folders go where and what devices appeared but just something to be aware of coming back to ubuntu let's allow the mac to see documents press save so that's now up to date and if i come over to the mac you'll see it's now got the documents folder because we auto accepted the documents folder so we got we didn't have to accept it and you'll see it's got windows down here now it's disconnected so we've been introduced to the windows machine but it's not an auto pairing we still have to accept the pair on the windows machine this is just a part of the nature of how the end-to-end encryption works you are required to validate on both ends of the tunnel so coming over to the windows machine you'll see it's popped up here new device do you want to allow so let's add device press save once we accept the pairing between the windows machine and the mac machine windows is asked do you want to share your documents folder with the mac as well and this might seem a little bit counter-intuitive at first because we've already shared documents with the mac but we've only shared it from the ubuntu machine because this is peer-to-peer not only do you have to accept each peering relationship at the machine level you also have to accept it at the directory at the folder level as well so let's press share so now i've got this folder called documents i come into edit you can see that it's shared from this windows machine with these two machines so that's basically everything you need to know about peering machines and sharing folders on a local network before we turn our attention to dealing with machines that are remote to us and also deal with what happens when you leave your house and you're not on the same network let's just talk about two interesting configuration options one is how to ignore certain files for sharing and there is how to version files so we come back to the control panel just say in windows here and come to documents and come into edit the first thing that i think is interesting is file versioning so by default there's no file versioning whatsoever which means if you overwrite a file then it's overwritten everywhere you can enable trash can file versioning which basically says keep file keep one copy the previous copy of a file in the trash can for as many days as you want and it's zero means forever that's the default by default all versions go into this st versions directory but you can change that with the version path if you want you can also choose simple file versioning which is kind of the same as trashcan versioning but you can have a fixed number of versions so i think the default is five and you can set as many as you want you can also choose staggered file versioning this is a little bit more complicated and you can see the text here that basically says for the first hour a version is kept every 30 seconds for the first day a version is kept every hour and for the first 30 days the version is kept every day i use this for most of my folders it gives you a good compromise between the accuracy of your versions but also not taking up to disk space if your data is not that important you don't have to version it at all but my data is important so i typically version everything the other option for versioning is external file versioning where you can basically build your own command for handling the versioning i've never used this i think staggered file versioning is fine but just know that you can enable this versioning the other option is to ignore certain files and this is really really handy because often you get a lot of junk files like temporary files in your folders and you don't want them to be synced around and you can do that here by creating these little patterns so one thing i like to do is um i often call my files like dot temp if i'm working on something temporarily so i'll add a star star that means any directory in the path any subdirectory at all then a slash and another star.temp so anywhere in this directory where you see a file with a dot temp extension just ignore it from syncing completely so one thing to bear in mind about both versioning and the ignore patterns is they are specific to the machine that you set them on so they're not like a global property so if you want to also ignore say here on the ubuntu the same thing i have to come in and do the same pattern dot tmp and now press save and now if i can test this from here so if i open up my terminal i'm in my documents folder already if we do echo hello to test.tmp and come over to the control panel and open up documents what you'll see is that there's still only one file and it says it's been reduced by the ignore patterns so this is a really nice way of making sure you're not just propagating too much data and also that you're not losing the data because you've able to version the files there are other advanced settings but these are the two most important ones that i find myself using i like i say do have versioning on pretty much every folder and i do have an extensive set of ignore patterns because you just get so much trash in your folders if you use them on a regular basis let's now look at how we deal with machines that are not on the local network how would i pair my sync thing with a machine that's in the cloud for example so i've already installed something on a fedora machine running in linux cloud and i now want to bring it into my network now of course we can't access the gui because the gui is not accessible from external machines i could if i wanted to change that but i don't want to what i want to do instead is make an ssh tunnel from my local machine to the machine in the cloud and that way i can access the admin console over a completely secure encrypted channel so to do that i'm going to come in i'm going to ssh into the machine this is the ip address of my linux machine and then i'm going to add a dash l i'm going to create a tunnel i want the port on the mac that i'm on the local machine to be one eight three eight four you can try using eight three eight four but this max already running sync thing so it would clash so one eight three eight four i want that to access localhost on the machine i'm accessing so here localhost doesn't mean the mac it means this machine the one in linode and i want it to go to 8384 on that machine i should make sure i get that actually right okay cool so now we can do that and i'm connected to the node if i come over to my browser this is my local setup so it's rob's macbook pro but i can also now access localhost 188384 to access the lynode setup now if i come to add remote device on here it's not going to find any devices because it's not on a network with any other sync thing machines i want to add one of my machines i'll just add my mac in this case and i need to get the full device id so let's come over to the mac let's click on identification i'm going to copy the full id now i'm not going to blank these ids out because they're not really sensitive because you need to obviously accept the the pair on both sides you can keep yours private if you want the same thing documentation is quite clear you don't need to worry about that too much so these will not be blanked out in the video coming over to the lynode setup i'm going to paste the device id in here we'll call that mvp vp press save and if i click on it it's going to say disconnected unused and if i come over to the mac close that down there we go it's popped up do you want to add the linux device press add now if i want to share one of these these directories with something in the cloud i might want to also encrypt it because that machine might be sensitive so you can add an extra layer of encryption here so let's say for example i went to share this documents folder i could choose a password here foo and that way when you go to that machine you have to use this password to decrypt it i'm not going to bother for now i'm just going to leave that unencrypted i don't really use that very often so but i will share that directory nonetheless so press save come over to the node do i want to share the documents folder yes i do press add press save and come in here and we can see it's up to date so it's only got one file and remember that my extra file over in the ubuntu side is being ignored so none of these machines are seeing that temporary file so that's the nice thing about the ignore patterns and i can just quickly come back into the shell here and see what folders i've got go into documents and then we see the test.txt file that we created before so at this point you could basically stop you've got a working sync thing installed installation you've got some folders shared you've got some machines paid you're able to access machines that are remote and if you take your laptop out of the house it's just acting like that machine that we've got in lynode it's able to connect back to your house and you might be wondering how is that working these machines appear to appear how is the machine in the cloud accessing the machine in the house and this is what i talked about earlier on right the start of the video this idea of a relay sync thing has a set of public relays and when two machines can't connect to each other they can use a relay as a as a middle middle point the data is still end to end encrypted but they both talk via the relay relays are very convenient because they just work out the box they are quite a bit slower and you have to decide how comfortable you are with the slight trade-off of people not seeing your data but they are seeing your metadata they're seeing your ip addresses and they're seeing the pairing of the the ip addresses to the ids and also which machines are talking to each other i don't use the relays instead i open up ports on my firewall to enable machines to talk to each other so let's see how to do that the first thing we need to do to make our mac and the remote machine talk without using a relay is change the listen addresses for our mac these are the addresses it will actually receive sync traffic on but these are also the addresses it publishes to the world and right now it's publishing it's going to use a relay so let's come into actions come to settings come to connections here and where it says default under sync protocol listen address changes to tcp colon slash colon 22001. now i've chosen this this is the port number 2201 i've chosen this for a very specific reason if you've got multiple machines in your network that you want external traffic to hit you have to give them all a different port number because that port number then has to be exposed on your router the default port is two two zero zero and if you only have one device that's living in your house that you want external devices to access you can leave it at the default port that's fine when you have multiple devices inside your house that you want external devices to access you need to give them all a unique port because that unique port has to be folded on your router so we're going to use two two zero zero one for this machine and for every other machine i do this for i would give it a different port number so that's absolutely fine press save while i'm in these settings i also like to disable relaying so i'm going to uncheck enable relay here and press save so if i come back over to linux now and look at the state of the macbook pro connection you can see it says disconnected if you expand it you'll see that all the addresses report no network activity and the reason for this is although we've changed the listener address on the macbook we haven't yet opened up a port on our router to let that traffic come through that's the final step you have to do to make this work how you do this is going to be very specific to your router model i will show you how i do it in my unifi setup so you get an idea of what has to be done but the actual ue will be different for you so i'm going to come into my unify i'm going to come to the add new port forwarding rule i'm just going to call this st mvp and i want to forward two two zero zero one which is the port that i specified before and the ip address i want to forward it to is one nine two one six eight one one two five that's the ip address of this macbook i'm gonna forward it again to two two zero zero one like that click apply changes and then very quickly you see that the lynnode is able to connect so it's now able to punch through the firewall through that that port forward and connect to my macbook but not using a relay so let's now just talk very briefly about topology the kind of the layout of the syncing network you can create with your machines using sync theme as you've seen every device has to be paired with every other device if you want them all to share pay to pay every folder has to be shared explicitly between every device in that network and that can get onerous quite quickly also if you want to expose every device in your house to the external world and you don't want to use a relay then every device has to be given a unique port that port has to be folded on your firewall that can be quite a lot of admin to do what i've opted to do instead is to designate one machine in this case my nas as my central hub i pair every machine with the nas but not with each other this gives me a much simpler setup it means that every machine only has to pair with the nas and only has to share folders with the nars but it also means that only the nas is exposed on the file i don't have to expose every single device and machine that i use so you do lose a little bit of reliability with this setup obviously if the nas goes down then the machines are not syncing with each other but you can just lazily add any peer relationships you want during that time of the outage on your nas this is a really nice thing about sync thing the topology is so fluid that you can have full hub and spoke full peer-to-peer a hybrid you can move between them as you need to you're never fixed in one shape and just to make things even easier most nas software does have a sync thing package so i'm currently using a synology nas there's a sync thing package for that i know that qnap have a sync thing package i'm moving and building a new nas on top of truenas you can run something on that so pretty much sync thing is everywhere but that brings me back to the important topic of what to do about ios and it's very frustrating that there is no sync thing on ios and if you look on the app store you will see there is a third-party app called mobius sync which i have and i've tried it out it does look like development on that has stopped so i don't recommend buying it there are thankfully two to me at least reasonable workarounds so i think the simplest way to get ios and the sync thing working together is to use icloud as kind of like a bridge what i mean by this is you set up sync thing to sync your icloud drive and let me show you how i've done that so in my folders here i've got this icloud and the only thing you need to know is the special path where the icloud drive kind of is shown in your file system so if i just open up this in edit you can see that it's in under my home directory it's in library mobile documents com apple cloud docs and you can just treat this like any other folder inside of sync thing i've tested this for many many weeks and it works just fine but you still have to run icloud and i really don't want to have to an icloud at all so the second solution but what i'm only just testing is whether or not next cloud can be used as a front end on top of the sync thing files what i mean by this is i've installed next cloud on my nas it's got access to all of the folders that are synced to the nas over sync thing and because the iphone and the ipad have a really nice next cloud client they too can access those files it's not a perfect solution and i still am ironing out some kinks but i do think it will be the best solution until hopefully sync thing is natively supported on ios so there we go that's everything that i think you need to know to get up and running with a proper sync thing installation how to pair machines how to share folders how to set up remote access how to pair with remote machines how to even interface with things like icloud this reflects my setup the setup that i've been using every day for the last i don't know well over a year and i've had no issues with it i do wish there was a smoother integration with ios and ipad but i can work around that with the icloud sync i can work around that with the next cloud sync and hopefully in the future we'll see native sync thing on ios i hope you found this video useful i hope you found it entertaining if so please hit like please hit subscribe and maybe hit the bell as well so you don't miss out on any future content thanks so much for watching and i'll see you in the next video

Channel: Tech Craft

Views: 117,611

Keywords: syncthing tutorial, syncthing setup, syncthing tutorial linux, syncthing ubuntu, syncthing tutorial windows, open source file sync, syncthing setup guide, syncthing android setup, syncthing setup windows, syncthing setup linux, truenas syncthing setup

Id: PSx-BkMOPF4

Length: 28min 30sec (1710 seconds)

Published: Sun Sep 18 2022