DNS: The Protocols, The Myths, The Legends

all right so before I dive into the slides a minute got a couple of questions just to sort of gauge where we are as well how many of you folks are currently doing dual-stack DNS with before an v6 on your servers ok little lighter than we'd like to see and how many folks here are doing DNS SEC either validation we're signing ok and is anybody actually doing anything in production yet with doe or not ok excellent so yeah our next talk Casey will be talking about some interesting stuff with rollouts so I'm looking forward to that as well so the other part of this is I'm going to go through this relatively quickly because I'd like to have a little more time at the end for a question answer because the thrust of this really is there's been a lot of misinformation for the entire life of DNS of of what it does and does not do and what isn't isn't hard and that is mildly interesting but more interesting when it gets to the what can you do about it or what does that mean for you when you're actually rolling it out so I'd really like to get you guys feeding me back some of you know questions experiences whatever so I have been doing DNS in large scale since about 1990 I did start reading the RFC s and looking at the stuff back in 85 or 86 and at the time I was sort of amazed I'm now even more amazed that it's survived through this so let's start sort of with the how did things start way back in the dark ages and in those days obviously everything was bare metal and it was in some cases mini and mainframes 56k links were pretty fat pipes everything in hardware was expensive so preserving RAM preserving disk being very conservative what you sent over the wire were all things that were far more important than speedy performance it was also in the days where we knew each other if something broke we usually actually had a phone number where we could finger the host and get their office and call them back and say hey you you you did something bad here can you fix that for me in those days also you could actually read the DNS RF season weekend it was kind of a dull read but you could actually get through all of them Berek Hubert's dns camel the last numbers i saw from that we are now at 185 rfcs that are still current and 2,800 pages of specification if you in theory want to absorb it all and make sure that your implementation is RFC compliant woohoo everything was unicast anycast was not even a bad trick in those days and it was pretty much all UDP we knew that TCP might be useful at some point in time but again it was considered slower and more expensive of resources than any of us wanted to cope with and in those days if you had a security checklist they probably weren't even thinking about tcp/ip much less DNS and they certainly didn't actually talk to anybody he was doing DNS so in the list of things that have been wrong or never true one of the ones that started way back at the very beginning and we have still not squished is TCP everyone in the world of course knows that TCP is only for zone transfers and is never used for anything else whatsoever and DNS is all UDP and there are best practice jet lists floating out there that will tell you this to this day and I've even hit stuff relating to this with PCI compliance and a akka lists and of course the earth is flat and yes if you are unfortunately enough to encounter one of these people that at some point managed to get a decoder ring and a security audit checklist out of their box of cereal some morning and excited to declare themselves as a security expert yeah we can try and see if we can hopefully correct that obviously it with the large packets that you're starting to see these days Edie ns0 is helping and we have extended the life and utility of UDP but there are still times where setting the truncate bit and and sending over TCP is the correct answer and so there are a lot of reasons why and we'll talk a little bit more also about some of the DNS over HTTP and DNS over TLS where those of us who have mostly tuned kernels for UDP for many many many years are going to be getting to talk to our web folks and getting recommendations on how they got their servers to survive the HTTPS Everywhere challenge and so there are good reasons for doing this other than just a ik so far my XO far we have overloaded everything in text records for years I have also seen people who do really cheap load balancing by hoping to do round-robin and putting 20a records out for a particular label i actually i've seen all sorts of spectacular things in my experience in dns where it's awfully easy to get over 1280 or 1500 byte packet and unfortunately with ipv6 in general fragmentation has been a problem and while i'm not a fan of the we should drop every fragment because it's too easy to spoof and poison caches and dns and fragmentation should not mix in reality we need to be able to deal with it if we're going to ever get rid of before sometime in the next 40 or 50 years and there are some things that we're starting to do with TCP that are actually of use pipelining and and being able to eat the overhead of doing the TCP once but getting multiple DNS answers in the same packet or nailing up connections between particularly busy iBall recursive servers and very popular domains the Facebook's and Twitter's and various other things of the world that love having 200 dns lookups for every webpage there's a lot of advantages to that and also in today's world the whole oh well you can't do that with DNS and you can't load balance because TCP will break if you're not getting to the same host every time and the reality is is that load balancers also have gotten modern hardware modern software there are interesting techniques with hashing and sharding that allow you to much more consistently hit the same servers there's also a very good RFC on stateful DNS and why having state isn't necessarily broken for DNS ipv6 which is one of the other things that I have spent a lot of time working on has unfortunately done themselves some disservice --is I love the fact that intermediate boxes are not making decisions about dropping packets based on size unfortunately also in the cereal box level quality best practices be whole ICMP is evil and should be blocked is a problem when the six is signaling stuff like I dropped your packet because it's too big and you should resend it in a smaller size because only the sender can actually reef Ragman at a smaller size the folks at ap Nick Geoff Huston and George Michelson and Jo Thomas have been doing some very depressing research in what happens with fragments and how robust that is and I've seen numbers in terms of drops or failure to get the actual payload in double digits you know 15 20 percent with a lot of common stacks it's getting better as more people are upgrading and v6 is more mature but the reality is is that fragmentation is dangerous and not reliable one of the other things that they also did some research is what happens how many clients tax with stub resolver Zoar with recursive resolvers in cheap CPU routers will actually retry with TCP if you truncate and set the TC bid and that's also a depressingly high number where I think somewhere in the 80 ish percent success rate which these are okay but we kind of like to see a few more A's in the class so Deana sec just a couple things about DNS SEC first before we dive into what is and is not valid useful and true DNS SEC is basically using public key or asymmetric encryption um which means that there's a a public key which is published in the DNS and a private key that is held closely by whatever is doing signing public key encryption can do things to things you can use it to encrypt so that only people who have the other half of your pair can decrypt a file and read it but the other thing you can do is create a divisional signature where by using those keys you can validate that someone has not changed whatever it is that that that signature is for and DNS SEC signs are our sets the the resource record sets in DNS and supplies the keys so that you can walk the chain of validation and you can verify that what was put into the zone file is exactly what you're getting from your recursive resolver or your stub resolver once it's validated it is not magic if I put something stupid or wrong into my zone file and sign it you'll be able to validate that you got exactly what mistake I put in but at least you know that no one in the middle of all of that has monkey with the results and basically unfortunately because we wanted to be backwardly compatible all failures are our serf fails so the some went other than perfectly correct is what sort of fail means and there's no idea of of what else might happen there is a draft for extended error codes in the DNS op working group that is dangerously close to getting through and that's where we have attempted to actually give something more meaningful then it all didn't go perfectly and we'll be able to actually see that it was a network or a DNS SEC validation or some other failure and be able to hopefully do something more useful in our applications so DNS SEC the biggest problem is that when it rolled out way back when the folks that wrote the software were all protocol wizards and hardcore DNS people that felt that anything more user-friendly than VI or Emacs was a waste of their time and that that was their idea of the epitome of aux so signing and keys and key rollovers and key management we're all spectacularly hard about 10 years ago when I was working at ISC and doing training and DNS SEC we spent an entire eight-hour day walking them through the process of how to manually create keys sign the zone and get the sign zone to be served so shockingly enough DNS got this reputation of being really hard to do and because if you did anything at all wrong DNS just broke but if you didn't do DNS SEC it worked it was decided that boy that's fragile and complicated it's gonna drive up our support costs we're gonna get calls our competitors who don't do DNS X signing will not get those same support calls and what it fixed didn't seem to be sufficiently valuable to risk that kind of chaos and support burden ten years ago that was a fair assessment but we use other stuff that is crusty and nasty and fragile I love doing BGP I love keeping my web server up and running and functional under load that's all trivial stuff right just kind of work out of the box yeah its server software has gotten much better the open-source folks have all come up with tools and signing stuff that is much more turnkey these days you can add a zone in NSD or in bind or and not have four or five lines of config file reload the zone and other than the fun of getting your DS records up into the parent if it's not done by the same folks that are running your zone it just works and it continues to work there's also the fact that at this point a lot more large-scale deployment of both resolvers and validation and signing so it's not just a bunch of IETF wants you know sitting in whatever room they're in doing it we actually have very large companies with hundreds and thousands of employees who are all now familiar I can actually put out that I want DNS SEC background in a job hunt and I'll get candidates who all say oh yeah and who have done real work and on the public side you know the quad X servers quite a quad 9.1 people like Comcast are all doing validation and have for years now and I just confirm the latest numbers I had worked at Comcast redoing their recursive infrastructure they do six hundred billion queries a day and they are now about half the size of Google's traffic and just those two alone are well over a trillion and the DNS SEC validation failures that actually get up to a DNS engineer and require something to be done are in terms of single digit dozens per month and it's the I wasn't really good in science but I know with in scientific notation version we're like negative 12 negative 15 some something like that in terms of error rates I can also tell you that Google and Comcast both have engineering managers who if there were support calls that were driving up their support costs dramatically would be revisiting this so these days you could pretty much say that it is really robust the software is ready you can get real stuff done with it you can find expertise so the it's too hard argument has mostly gone away at least as much as any other protocol we use you can say the same thing and there are benefits cache poisoning is still relatively trivially easy to do host Kaminski in 2008 we did discover some things with randomness and entropy and various other things and at least we're not at the birthday attack we're at five to six hundred packets you have a really good chance of colliding and polluting the cache but eh there aren't still enough people that run crusty old stuff and B we slowed it down and made it harder so that it's in terms of hours not minutes but if you're determined and you've got the right route toolkits and stuff you can still dump stuff into cache and DNS hijacking which is very prevalent and very rewarding and there are several state actors who are actively doing this now this is one more thing that makes it easier for them to go through when the DNS peonage stuff was happening and PCH got cracked they got part of why they did was their mail server was exposed but the MX and a were in DNS SEC however they had two employees with Apple phones who were behind Hotel firewalls that would not allow them to use the DNS servers and didnõt DNS SEC validate and so they were able to spoof in lie and those folks didn't catch it and the phone's connected before they could come up with any VPN or do anything else secure which meant that they leaked their email credentials and once they had their email credentials daisy-chain hopped through get some emails read some stuff get into IMAP find a few things and see whether the passwords worth elsewhere and there they were so DNS SEC will slow that down Dane is becoming not as prevalent as we'd like there are a few places like Germany that are now making it mandatory but it is not as uncommon as it once was we're doing other certificates it's better protection for CAS folks like Lex let's and Crips are actually checking whether your zone is DNS X sign and you you will have better protection and cleaner automation if you're actually doing DNS SEC and last and sadly not least like I said I've been doing this stuff since 84 we have had two successful wide scale deployments of PK eyes and existence one of them was the Kerberos that wound up hiding under the hood for Active Directory that Microsoft has been shoveling for years and the other is DNS SEC we have not you know correct me if I'm wrong and I'd love to talk to you after my chat but there have certainly been other attempts but nobody else has actually gotten to an internet wide deployment of a useful and secure PKI so let's use it so okay so it's useful but what does it actually solve basic security this is sort of the triangle confidentiality nobody who's not supposed to see it should be able to see it in transit or either end integrity did what I receive match what was original he sent and availability when I need that information can I get it so DNS AK really mostly just solves one problem integrity what I put into his own file and sign anybody else who validates will be able to say yes this is what Paul stuck in his own file this must be what he meant for me to have it means that people can't get into my registry account change my name servers and set up false name servers unless they remembered to delete the DES record in my registry account which fortunately the bad guys don't always remember to do and with 24 48 hour cash there's still a window in which I can notice that my name servers have changed so it protects me from false name servers it protects me from cache poisoning and again those are not complete solutions but these days it is all about defense and layers you know anything you can lock down and make more secure is worth doing if it is not ridiculously onerous to do and the other side okay fair enough you know we're talking about myths and legends what doesn't it do well it doesn't do confidentiality it still goes in the clear it just goes in the clear with signatures for validating I can still overwhelm your authoritative server or a transit point and and make you unable to be able to get an answer from my authoritative servers it does not protect me from putting stupid stuff in and making mistakes or having script failures and generating my zone file sadly and it doesn't help with the parent your parent still also has to be secure and of course for DNS I they also have to themselves be sign and verifiable or there's a not a clean chain so if this doesn't work for confidentiality we're now also in the post Snowden era there are state actors who are interested and there are oppressive regimes that are interested in spying on us there are allegedly democratic regimes that are interested in spying on us there are commercial entities that want to spy on us and give us targeted ads sell our data correlate us identify us track us and anything that we can do to at least slow down the leakage of who we are what we're doing a more interested in the ITF several years ago said that's important and that's worthwhile work so there's an RFC that basically says in the face of pervasive monitoring we should encrypt anything that we possibly can and these days most of the pieces of the internet from the Wi-Fi that someone can war drive and camp on to all the way through our providers the websites we see are employers and all the rest they are watching we already know that so there were two efforts the first attempt with DNS said let's at least encrypt from the stub resolver to the recursive resolver because that's where the most dangerous leakage is because that if I can get to the stub resolver I can get to the home IP address I can now tie it to at least a home and possibly even to a device and that's very definitely PII Comcast knowing that six million homes are interested in resolving dub dub dub facebook.com is not PII and that's much less of a disclosure so there is software that was out there for years DNS curve done by DJ B that tried to solve that problem of the how do I get from my my Stuber solver on my operating system to my recursive resolver and encrypt it that really never got adopted widely though the algorithm we now use ECDSA is also elliptic curve and so gjb at least gets the credit for it you know getting our attention that elliptic curve was pretty cool algorithm it's you know it's relatively cheap computationally and it generates much smaller signatures than RSA and DSA so that was a good thing so we're already using TLS and if we go with the everyone should support TCP as a DNS server because it's in the RFC it always has been then TLS is just a TLS handshake and the and as a matter of fact if I'm recalling correctly unbound initial patch that someone submitted for making do t work on unbound was on order of 12 lines of C code something like that because most of the logic was already in there so that was a fairly straightforward one the downside was that it comes to a well-known port and so if you decide you don't want that you simply block that port and do tea breaks and while you can of course run it over a different port or you can even run UDP 53 on a different port I have certainly run a name server on port 443 before that had nothing to do with HTTPS because it works great at hotels it at least was a part of the problem and it was cheap and it was fast what then happened though was Mozilla got into the game and said well we have this piece of software that kind of knows what to do with HTTPS and we kind of like it and for us there's really not much more overhead than T less to do that and there are a couple of other cute tricks we might be able to do and they came to the IETF and worked with some other folks and within about a year we had DNS over HTTPS for better or worse it does solve confidentiality for a point-to-point link obviously again it doesn't deal with the recursive to authoritative part of the path and there is work in the IETF on what it's been called a dot which is which is authoritative ie D from the recursive server to the authoritative server and there are some trade-offs there as well but at least the PII leakage of my house or my laptop or whatever and what I'm asking my recursive resolver for now can't be sniffed however it does not solve the end-to-end data integrity problem it is a point-to-point confidentiality and so another in the myths that several people have gone through is oh we finally we don't need DNS seconding workers we have Doh and dot and no they are a different problem and we need both and we should use both it shouldn't be a choice of one or the other they both have advantages and usability and then obviously availability it's the same it's you know DDoS we'll take out name servers and it doesn't matter whether it's encrypted or not if you can't get a packet you will not get an answer so a little bit about what several folks are doing in major initiatives the one that really started the the the flurry and the gloom and doom of dough is evil and oh my god the world is ending was mozilla because they they did two things that upset people that were not necessary or part of the protocol they are an implementation choice and the two things they did one of them was it was opt-out knowing that 95% of all users will never touch anything that is not the default and so basically they guaranteed that everybody using Firefox with that release was going to be switching to doe and the second piece was they decided for us who that was going to be and they went with CloudFlare and of course Europe said you are now bypassing my ISP that in Europe we don't hate nearly as much as the u.s. hates their ISPs because a they've had a better relationship be they have gdpr for privacy and violations but now you're shipping it off to the US where a US warrant will expose my data without any GDP our protections whatsoever also scary is that it completely bypasses every thing in the OS so enterprises all said oh my god we we have group policies and ad and all this stuff to make sure that our users don't do anything bad with their browser and with their DNS and we have filtered DNS that works for us we like being able to block malware and botnet and command control and all the rest of it we feel that that's a good thing and so that that was the one that sort of got this to be so controversial they have somewhat slowed down one of them is they are now working feverishly to abide by group policy this does not help small and medium businesses where there is no IT staff and they have six people and they're all running Windows but they're running into them as unsecured with no ad unfortunately and for large ISPs and other folks they have what's being called a canary domain Firefox will fire up and it will check this domain and if it gets an answer we'll go oh you don't want us to use doe I will turn it back off we'll see if they say somewhat more soft in their stance the next two I think did a more nuanced an incremental upgrade and I think that that's probably a better path and hopefully Mozilla will start going through Google originally also was like over yeah we're gonna put in Chrome and we're gonna throw it out in release and then we're gonna turn it on you know I think it was October of last year and the world said Google's even more evil than Mozilla and CloudFlare this is horrible and then of course the Wall Street Journal started blaming them and they were somewhat surprised and they sort of went oh okay so the first thing I did was they said let's make it opt-in if you don't turn it on you don't get it and then the said also we don't mind dot you know we were already supporting dot in Android so we're just gonna do an auto discover ourselves there's no RFC for how to order to discover what protocol support there is with your recursive server what what encryption if any it will support but they say hey we'll just do it's been done you know DHCP and or Ras will tell us what our DNS servers are and we'll get IP addresses now we'll just try it and if doe works will switch to doing dough and if dot works we'll switch using dot and if not we'll do it in the clear and I think that that incrementally is a much more sane way of doing it and it's somewhat less of a surprise to everyone involved Microsoft also one of the other objections was I hate controlled the OS but this application isn't following the rules so Microsoft said well ok we also think encryption and privacy is a good thing and we think that having that option for our customers is something that they should be able to do but they should be able to do it for everything in one place with the dialog box that turns it on or off in the same section with the privacy settings that they already have for other things and their version of Chrome will just follow the rules so they are doing opportunistic though as well they will see if it's available and they will have some lists also short term of trusted providers and then the stub resolver in the OS will just use it so if you are in a company enterprise network ISP and you either don't want to lose access to the DNS data for debugging or or troubleshooting purposes or security purposes if you don't want to lose the ability to give customers filtered feed when they opt in parental controls malware blocking the rest of it or if you simply are at a place where you want to be able to catch the NSX filtration and tunneling over DNS the odds are what you should really be doing is is most returning Li obviously for Mozilla setup the canary domain in your DNS servers and for everything else simply enable dot Indo on the same ip addresses you're already handing out for traditional I would make the case that encrypted within the enterprise and visible only by IP who has access to the DNS server is a feature anyway I do SSH tunnels in my home and don't trust that my network is in any way safer clear and I I think there's an awful lot of security professionals would do the same all right so that's sort of the landscape what I'd like to do now is get feedback and questions from you guys yeah and so there are microphones there and they're that way they can get it into the recording there right behind you yeah and and the normal nanog state who you are David Belson with I sock for Chrome are sorry for Google is the dough enablement or the dough checking is that only an Android or is that also in Chrome for Mac OS and Windows currently my understanding is that is Android only okay then add the OSS right currently the browser is going to be doing just dope I Brian fields Nokia so my question about dough sorry that goes messing with my head ah my question about dough is doesn't that just move the potential for like my instead of my ISP being able to essentially spy on me doesn't it just move it to like CloudFlare or somebody so does that really solve the problem I mean I underst we trust CloudFlare maybe more than Comcast but you know there's a lot of companies that started out and said well we're not going to be evil and there are a little evil now yes indeed and and yes it's essentially who do you distrust will inform your choice because there's presumably someone you are more scared of having access to that data I think these days the reality is is even with the trusted resolver the TRR that mozilla is doing in the cloud floor has contractually agreed to of the we won't resell it we'll throw it out after a 24 hours all the rest of it but they still look at it for 24 hours and we are in in a state where Amazon probably knows more about me than any friend or relative does just due to my Amazon purchasing habits and browsing habits and so to some degree we have long since lost our ability to truly have privacy so what it really boils down to what is the most egregious thing to you so if government surveillance is it I think you're really screwed in the VPN is probably a much more usable solution than merely DNS anyway but if what you're more concerned about is just feeding the admah that we are creating and and B the ad capitalism that we have unfortunately allowed to flourish this at least slows it down and makes it more painful and more expensive so yeah much in the same way that we only get to vote for whoever managed to get through the primaries we don't really get to vote for who we actually want yeah I tale actually kind of surprised so ITF co-chair the DOE working group one of the things I'm surprised that Paul said the first I'm not surprised you said the first time I'm surprised he didn't just emphasize it again and his answer is there's nothing in the DOE specification that says you have to have centralized DNS correct this was a major issue with with the Firefox deployment because this was a design decision that they made to take this particular tactic and there in fact many other doe deployments including within major telecoms and ISPs to have a local tow resolver that is acceptably no different than using their current DNS resolver except now there's an encrypted channel within their network to that resolver yes thank you and that's that's why I repeated several times about why I thought that the Google and Microsoft approach which is you're not changing who you have the relationship with you're merely changing the transport is a much better incremental step and as a follow-on to that the ITF is currently in the process of trying to charter a new working group the adaptive DNS discovery working group where we will be co-chairing presumably once the iesg approves it where it is one of the the main work products that's intended from the working group is that we will have your a client resolver a client itself will be better able to understand this the resolver environment that it's in and make an informed choice about the kind of resolver it wants to talk to which would possibly be talking to a centralized tow server but could also then presumably be talking to local and cryptid servers for example yes and and in the slide deck in the further reading here are the current active working or mailing list groups dealing with all of this and we are very actively adding things as well a small correction to that however Eddy is not a ITF activity that is correct Eddy EDD is a separate independent activity in fact good straight Magnus what I came up here to say that there's a initiative for encrypted deployment initiative of how to get it right if you go to encrypted - and I know we hate dashes and names but yes go grab the hot link on EDI will take you to that website and allow you to do the mailing list and the rest right and so if you're just said I'm Glenn deed from Comcast this was initiative that we pulled together but it's made up of a lot of participants from across the industry the goal of Adi is to bring together a community of people who are interested in successfully deploying encrypted DNS whether it's doe or dot we don't play favorites but and do it successfully without breaking all the uses of DMS some which Paul talked about already and they go deep and we've been we've been clever for 30 years and using the DNS and the goal here now is to deploy encrypted DNS without breaking all that stuff yes so if you like to be part of that go to that website sign up it's free it's open its public we would very much welcome participation yes and some of the followings with that the odds of having DNS over quick and other transport are likely and are certainly going to as well be addressed in this particular initiative so I am a DNS user not a DNS administrator and you mentioned a couple of times that Android tends to do things differently I was wondering if you could expound upon that a little bit so your phone just like any other device uses both router advertisements and v6 and DHCP in possibly v6 and certainly in before to get its address its gateway and who its DNS servers are what Android does is once it gets that DNS server address it will then see whether that IP address works with dot and if it does it will use do T for all of its DNS queries if it does not it will fall back to a traditional unencrypted UDP and yeah so it's very interesting if you stand up a d-o-t server you will instantly know how many Android devices you have because that's suddenly how many new clients you have using that particular port other questions comments alright so yeah there are some further reading things yes III did ITF an email list I should have made it more obvious that they were separate things sorry Glen and but yes these are all the places where stuff is starting and as more things spin out if you're on those mailing lists you will have already heard about some of those there's another initiative that Steve Crocker is pushing DNS X provisioning which is the how are we going to potentially automate getting DSS into the parent in a more consistent fashion all of those are mentioned in these mailing lists and then you have some other rfcs and drafts that I think that if you're trying to figure out what the space is they're they're worth oh we need as well all right and if that's it then now let's see what Casey has to say and thank you very much

Channel: NANOG

Views: 903

Rating: 5 out of 5

Keywords:

Id: BqyYgqGdGAw

Length: 42min 53sec (2573 seconds)

Published: Fri Feb 21 2020