How Does ZeroTier Actually Work?

[Music] yeah yeah zero terror has been awesome it's been a lot of fun um and and like i mentioned at the top of the show i've been working on a a small little mini tutorial series uh which sort of brings together all the tips and tricks of zero tier and even g just to help everyone sort of um get the most out of their labbing especially when they lab using google cloud as their platform as their infrastructure as a service um so i i wanted to talk a little bit more about it because i think last time we just dived into a demo which was relatively quick and just kind of just showed how quickly you can stand up um a a vpn between two end points and and that was wildly successful and we did it in less than five minutes and that's why i love zero tier and we didn't really actually talk about sort of how that works so we wanted to kind of sort of step backwards and talk about how that works a little bit so um over here i want to say on zero tier's website which is just zerotier.com um they have a a pretty good knowledge base and a lot of times when i'm going for this information this is where i'll start they also have another link that i will share with you guys um uh which is uh which goes into pretty good depth on everything uh that they do um one of the things i like to always point out is that these are the root addresses so i i am going to try and sort of draw if i can on my screen um draw the hierarchy that zero tier sort of operates in and at the very top of that is going to be the root servers these are owned and managed by zero tier so unless you're running your own controller or if you're running your own node as a moon you are using one of these so when you connect a node to the zero tier network it is communicating back to all of these addresses and it's telling them all what your node is and of course it's picking up your public ip address from that communication so all of your nodes are doing that and therefore this knows about these these controllers here these root servers know about the public ip address the cryptographic node id of each endpoint and so when you send your ping or your traffic from from one node to another it's actually going to relay through a root server and that's one of the reasons why we only have to worry about our outbound udp traffic because it's going to be coming back in that same udp channel that we have already connected to that root server now that's why this is just sort of stupid simple and always works and why it makes it so easy to connect two nodes onto separate networks however there's a more efficient way for nodes to talk to one another right and that's to actually go from peer-to-peer right to skip the relay altogether um i don't actually need to send uh encapsulate packets in xero tier send them to a relay one of their root servers just to go to my google cloud instance i should be able to send that directly well how that happens is that each node actually talks to the root server and the root server tells each node the other's address and then they'll start sending udp packets directly to each other using something called udp hole punching this is really cool this isn't anything that's brand new this is a very old technique but it's specifically designed for this and it works great with udp because udp doesn't have a sequencing mechanism the way tcp does so tcp is always going to have a sequence in a pending order or in ascending order i should say udp doesn't have a concept of that so we are able to actually use the udp hole punching method send packets to each other and they'll actually traverse the firewall that's really cool um so if anyone wants to read up on that just udp hole punching if you ever try to think like if you have a firewall or a gateway on your at your home network how does the zero to your packets come into the network right if they're coming from another node remotely um if you guys are doing direct peer-to-peer uh without opening a firewall port that's the whole that's the whole point here is without opening that firewall part how do you do it and udp hole punching is is a pretty cool technique and this sort of breaks it down here in the flow uh so again i i did a lot of research on this to try and understand what was going on under the hood with zero tier and there's a number of tricks that they employ and um and it's pretty cool a lot of it is sort of coming from the perspective written from the perspective of a developer sadly um i'm not a developer and i don't know that that language very well um so i'm more focused on the network communication and actually what that looks like so um the other thing i wanted to show you on xero tiers website if you go to their website zerotier.com you go to support and manual this is actually sort of a a many many many pages document that sort of breaks down what the architecture of zero tier is what the flow looks like as it goes from node to root server to node and and sort of how it handles the the virtual interfaces on each operating system so i think that's super cool i can't get into too much of it but i really wanted to understand um how that network flow was working uh so it sort of operates on on two levels the vl1 and vl2 v01 is actually sort of the the transport the encrypted transport and vl2 is more of the the interface on the operating system uh that your that your operating system um pushes packets through so uh i think that's very cool uh the root servers uh zero tier sort of has a little bit of its own nomenclature um obviously root servers you know you can visualize what those are similar to dns root servers um but there's a concept of worlds uh there's a world definition and there are planets and there are moons and right now i have i am a node my workstation rather is a node and right now it's connected to one planet which is our planet so all of the the root servers are connected to our planet which is earth it's we're not going to have another planet um but it it it affords us the ability to scale out um if we needed to start using another planet uh here on earth we could do that um but right now everything um while it's uh relatively small still operates on the um uh earth planet and um moons are just like the root servers but they are root servers that you would run locally that you would run within your data center within your enterprise and one of the examples it gives is maybe you don't want to be able to maybe you don't want to register to the root servers you don't want your nodes to register the root servers you can configure them to register to the moons to the controller that you operate um moreover if you lose internet connectivity if you lose connectivity between your node and the root server you can't register and therefore you can't talk node to node you can't send traffic but what if you still have reachability within your enterprise between two nodes that would typically be using zero tier if you stand up a controller within your environment that's called a moon those devices are able to talk to the moon and use that as their relay and use that as their their source of information to let each other know uh what their addresses are and how to reach each other so i think that's super cool i had to dive into it a little bit to figure out what was going on so one of the other things i did was i started uh capturing some packets that's what i like to do i like to study the traffic and understand how things work and this is this is how i do it so um i have a vps a linux vps um on a cloud provider and i joined it to zero tier and i have my local uh nick here in my home um from my dhcp server and i connect i distilled down i use this filter sorry i used a filter to just drill down to our two hosts and everything is just udp and in fact wireshark doesn't classify this traffic as anything maybe they don't have a dissector for um for zero tier and so i started looking at these payloads and as i go down the line here and start looking at these payloads as i go down packet by packet there's nothing clear text there's nothing i can look at there's no identifiers that i can see that is going to tell me what's going to what um so i use the strategy that i do when i'm trying to figure out sort of protocol fuzzing and figure out what's going on i used a t-shark to actually extract only these payloads and then i can stack them and compare them visually and see sort of what lines up what fields line up but as long as you have not variable length fields but but fixed length fields they'll always be in the same place and what was very interesting was i found some of those and i found what they were so here is a notepad document this is the hex dump of the payload this is with all of the layer four header removed uh layer four and up or i guess layer four and down i should say so this is actually starting the payload and these are packets of varying length but you'll actually notice that the this group right here one two three four five six seven eight packets are all equal length look they all finish the payloads all finish at exactly the same mark that's very interesting these two packets finish at exactly the same mark so these are some some interesting components to this protocol and i can't actually find any information on on how this is broken down on zero tier's website what i was able to find is there's a hex 88 right in the middle well it's not quite right in the middle but um about 20 bytes maybe 25 bytes in to each payload so what i did is i started separating bytes out from there right so now i have one fixed spot in the payloads uh and what that looks like when i start breaking them out is i started noticing this i started separating them and i could see that some were the source and some were the destination so in one notepad are my uh or my sources uh this is unidirectional traffic going from the source to the destination and this is the return traffic coming from the destination back to the source and what i found was this 88 identifier this hex 88 sort of delineates some fixed field in the packets what i found ahead of that was this is the 10 byte node id it's a 40 bit value of the source this is the 10 byte 40 bit id of the destination and so when i stack these you can see they flip flop every time i click on one it goes away let's do it like this pardon me should have had a little bit a little better window management okay uh so what i was showing was is that these flip-flop so what was the source becomes the destination and uh what was the the destination sort of becomes the source and while this isn't significant this is like the first step in being able to identify sort of what's going on here in the payload now the zero tier documentation says this is um fully encrypted um it's they're using pki for node ids and uh a bunch of crypto crypto something uh i i can't write code so i don't know what the hell they're doing um but there's a bunch of cryptology stuff uh so this is entirely encrypted but i again one of the interesting things is is that you know they're all fixed lengths they're not variable length and typically when you're encrypting things you know even if i encrypt the same thing the same message rather i will get different lengths out of the end of it unless you're using sort of i guess like a fixed hash or something a fixed hashing method um someone can drop me a message in the chat someone can shoot me a message on twitter and kind of break it down for me but i thought this was really cool because what this means is is no matter where you are if you're able to pull a pcap uh you're able to quickly identify whether or not the udp traffic that you're seeing is part of zero to your communication and then you can start to take a look at wait i have i can see these nodes so right now i would be able to build something in python that'd be able to strip all of the strip all of the zero tier node ids out of a pcap and be able to have that as some sort of evidence so i don't know what the what the leading values are yet i'm still going to dig into that and see what i can find but i thought that was very very interesting very very cool one of the other things i wanted to show you as i've been digging around in a lot of places that i shouldn't are in the uh uh windows driver let's say zoom in zoom in zoom in the menu uh this is the windows driver inf file that zero tier installs and they gave us a little hidden message here in the strings which is zero tier networks llc we're zero tier inc now but kernel mode certs are 300 plus so that and i thought that was just uh a little bit of programmer humor tucked into the uh the thing that's classified right there there's one here there's one other thing in here which is it says ndis5 yes ndis 5 is correct yep windows so so two little messages in there um i like to find that stuff out and there's another thing i want to show you guys which is i think this driver is signed with a certificate and maybe zero tier can correct me on this but here's the certificate that's in the same directory as the driver named the same as the driver and it expired in 2016. what's up with that zero tier see right here it's got the little red circle so maybe because someone can help me out with that um anyhow so uh so yeah so i just wanted to sort of break that down a little bit i wanted to talk through it a little bit more um i think there's a there's a a lot of very cool things we can do with this sort of architecture um this allows people excuse me nodes to communicate with each other via peer-to-peer because they get the information from a root server which tells the other end what their ip address is and and what port they're coming from so i think that is super super cool and a very clever way to get through to that um obviously you can open up the ports on your firewall that's a great way to do it too but i would say that would probably be only inside of a perimeter boundary probably not the outside of your perimeter boundary pass solutions has over two decades of experience building network monitoring systems designed to give you total network visibility total views automation means it can be fully deployed and configured in minutes monitoring every device interface and server in your entire infrastructure and it goes deep collecting performance configuration 19 different error counters poe and qos statistics on every single interface now all of this information is automatically analyzed to produce plain english answers problems get solved faster because more information and intelligence is brought to bear this is what total network visibility is all about knowing more about your network than with any other solution total view is easy to acquire as everything is included in the core offering netflow diagramming path mapping server monitoring network automation ipam cloud service monitoring and more putting your trust in a monitoring system can require a leap of faith from inception totalview has a more secure architecture and build process that helps to protect their customers contact pass solutions today where you can learn more about their competitive upgrade program total network visibility rapidly deployed find out more at www.pathsolutions.com [Music] you

Channel: Network Collective

Views: 4,709

Keywords:

Id: Lao9T_RQTak

Length: 18min 53sec (1133 seconds)

Published: Mon Mar 15 2021