Why Does the Internet Need a Programmable Forwarding Plane with Nick McKeown

welcome to networking field day the presentation that you are about to watch from barefoot networks is being attended by a group of invited networking delegates who represent the community by asking questions offering opinions and discussing the technology that you are about to see you would like to see more information about this event please go to our website tekzilla.com and check out our youtube channel youtube.com slash tech field day I'm very happy to introduce Nick McEwen he's a professor at Stanford a co-founder here he spends a lot of time here I won't say how much because I'll get them in trouble and he'll he'll spend some time maybe giving an idea of why you should care thank you great excellent so I think I've met quite a few of you before last year I'm Nick McEwan and chief scientist and one of the co-founders here and as I think some of you may know I'm also a professor at Stanford in computer science and electrical engineering where I work with graduate students and run research programs in networking one of the research programs that we started a few years ago was called the clean slate program the clean slate program led to what is now called software-defined networking martine casado was one of my PhD students one of the better ones he's the clear or accomplished guy and after a few years we started the zero together you know I like to think of what we're doing here at barefoot as a logical continuation of what we started then so nasira Software Defined Networking brought to those who owned and operated network it handed them control over the control point what we're trying to do here is to hand those through own networks and those who build equipment for them the ability to define the forwarding plane there's really a continuation of what we saw before but I think to to really frame what we're trying to do here it's good start with a question and the question I want to ask you is it have you ever stopped to wonder why it is that it takes so long so many years to add new features and functions to networking equipment what is it that's going on that takes it so takes so long let's start with an example the excellent VX lamb was first defined in 2010 by Cisco and VMware so just look at the steps that needed to take place in order for that relatively simple function to be added to a switch or router so inside a switch or router here's a cartoon picture they all can they all have a switch operating system inside based on Linux or some variant of UNIX and then on top of that there will be user space processes that are implementing the control for those protocols OSPF BGP etc and then a driver that is communicating to add and delete entries into a fording chip so now imagine it's 2010 customers are clamoring for VX lamb and the switch one does need to figure out how you need to do in order to add this feature well this clearly they were to be able to add VX lab map as the means to talk to the outside world and to implement the control of this protocol they also need to change the driver in order to be able to add and delete entries into the VX lab and tables that presumably will be in the switch ASIC they also need to be able to update the ASIC this part up here few weeks work this down here for years it was four years until this feature became available in merchants switching silicon the ways it take so long never look at what's going on during that time so they've got to add that that that feature in this case it's not a niche feature the expand is not a niche feature that you just put on the back burner this is in that most highly invested in the most profitable the bleeding edge part of the networking industry so let's think about what happened the enterprise network folks go to their vendor and say I need this feature the equipment vendors all get together gather at the IETF as they do in order to try and figure out what is this feature that they're going to to implement they have to go and talk to their software teams and they'll say this is the feature we're thinking about and they'll come back and say yeah we can do that in a few weeks then they need to talk to the ASIC team or their third party switch vendor switch chip vendor and so they'll say these are the features we need they're pretty simple it's just a table just a new way of processing the packet and they'll say it's about four years or two to three years in order development added into a product and so it's four years before they hang up there really is kind of a a process that is at odds with the rest of our industry so it means your vendor can't just sell you a software upgrade that will be nice it takes years for them to develop that new feature and this is really a simple feature this is not like we're trying to add anything particularly magical or particularly complicated about its equivalent to an O reprogramming the hash interest for ecmp or adding a few extra counters or changing the way in which the resources these are all relatively simple it takes them so long but probably by then you've figured out a kludgy to get around it because you want to wait four years for this feature adding the code is never really a good idea so you made your never one's more complicated less reliable more brittle and eventually when they when that that upgrade is available it either is no longer solving your problem or you could say okay I'll go or go for it and then you want to replace all of your hardware which is expensive and all the other pains that come with making that change so that was true so when you look at it like this the whole process is kind of a little bit crazy and we've got used to it we kind of take it for granted now right but this is how our industry works because we're being fed a lion and we've been set aligned for a long time and it really is at odds with a lot of the areas of high tech let's look at what happened this look at what's going on with is that where this comes from it all stems from the fact that if you're trying to build a piece of networking equipment you start by looking at the various types of six function switches that are available they all come with a datasheet and it says this is how I process packets this is the features that have cells that I support and then you look and say well out of all the things that I would like to have which one is the closest to meets my needs it probably has a whole load of other stuff in there that you don't need but you look for the one that's closest and then from there bottoms up that will define the system that you can build so the consequence the entire products that you can buy build today unless you can sort out enough to build your own chips are defined by data sheets from third-party vendors so the features that are being defined in our systems are being figured out by chip designers it's not surprising the feature set that we end up with doesn't particularly fit with those who want to operate big networks because the features are being picked by chip designer those who are willing to put in all of that complexity into the chip so one way to think of what their foot is doing is trying to turn this model upside down in fact the whole p4 and programmable 14-play movement is trying to turn this website that's played down we want a programmable switch you'll seem a lot more of the pheno later and we'd like to define the behavior that we want by saying this is precisely how we want packets to be processed in a program this is a snippet of a p4 program and then we want to compile that and then drive it down in order to tell the chip that this is the behavior that we that we want seems actually kind of obvious and I'm belaboring the point here to make is to try and make it very clear but shouldn't it always be done this way in fact it begs the question why aren't all network systems built this way today as soon as you see it this way you you kind of can't unsee it because it makes you ask the question why doesn't everyone do this in this way and it took us a while to really convince ourselves and we would ask the question a bit timidly to start with then say we being silly but there's this conventional wisdom in networking that you hear time and time again and I've heard this one for 20-25 years programmable switches at 10 to 100 times slower than fix function switches they cost more and they consume more power we've all heard this well I can tell you that with going back and thinking about the basic operations that you need for networking instruction set the data models the way that ASIC technology is is is evolving we can say categorically this is not true and in fact we can say with confidence and we will show it to you with Tofino that you can build the chip which is as fast there's not cost anymore and consumes no more power and it's programmable by the users who build systems out of it the pheno operates at six point five terabytes per second and Totino is the fastest which chip in the world and it's programmable by a fuse putting context what six point five terabytes per second is just as I think it's kind of fun you were to watch the entire Netflix catalog from now it would take you four years to watch that catalog Patino will switch it in 20 seconds that entire catalog ok any 6.5 terabyte the second switch can do that the pheno will run a 4,000 line program on every packet it is a passes through it costs the same as the fixed-function chip it consumes the same power it's easy to program it's easy to program in the following sense we have had users who have never heard of p4 before they learn about it they learn to use it they write a program they contributed to the repository in less than 8 hours from beginning to end so it's not something which is for really deep specialists who know a lot about a specific chip the general way it reads like common sense where is that repository p4 org people will talk about a little bit later so why is this happening now part of it is this garage but there's another reason that it's happening is that networking is kind of catching up it's catching up with watch what has happened in other areas of IT for a while we're all very familiar with this model we write high-level programs for our computer we've all done this Java Python C whatever it is that we prefer to use we run it through a compiler it runs on to a chip below important note that that program up there written in whatever language you prefer is compiled down to the target without you having any knowledge as to how that compilation takes place down here could be x86 it could be an ARM processor it could be AMD you don't care so you're writing in a target is target independent portable fashion what some other domains have done is develop their own domain specific processes with domain-specific languages that are specific to that domain OpenCL for graphics for example compile down to a GPU I don't any of you are old enough to remember when graphics cards were these big fixed function cards from Sun and HP and Dec at that time there was a raging debate as to whether anybody could ever build something that was programmable that would be fast enough low enough cost and low enough power Along Came the GPU game over you'd never go back what they did was figure out a basic instruction set and a data model that they could build that domain-specific processor for graphics and so if you want to define a renderer this doesn't know what rendering is the rendering is to sign up here same thing happened in signal processing it used to be that in order build a radar or a base station or something like this it all be fixed function Hardware you wouldn't dream of doing it that way today the DSP from from Ti of from others running at multiple gigahertz was a domain-specific processor for signal processing doesn't know what an FFT is you tell it what an FFT is and you tell it in a language high-level language like C or MATLAB it's compiled down the main specific language domains to this specific processor you get the pattern same thing happening in machine learning the main specific language the main specific processor that's all happen in the last year what about networking who skip networking right so what happened so we don't have a common language until now and we certainly don't have the domain-specific processor which is a target for such a language and that's what we've been building the language is p4 it's not necessarily the only only language it's the one that is is gaining traction right now it's the main specific language for describing how packets should be processed in a network there is a compiler that will compile it down to run on a particular class of machine and the machine that we're going to be describing today is the architecture we call Pisa P is a for protocol independence which architecture the switch itself does not know what a protocol is you think of switch with no other protocol is but the key here is just like a graphics device doesn't know what rendering is this doesn't know what a protocol is the protocol is defined here if you need to add a new one you change the program you want to throw one out you change the program and then you compile down to run on here the first instance of a Pisa chip is Dafina so it's an instance of a Pisa architecture device to understand yes that implies that sort of employ a couple of different things one is you might expect other languages from p4 to emerge and you also might expect other silicon below-the-line to emerge as well it's like a fair assumption we're absolutely proven out after all and a follow-on question if I may we have GPUs we have DSPs but the market size for those is enormous right there you know measured in you know hundreds of thousands of units of shipments per month etcetera networking is a relatively small market like in terms of units shipped number of switches that we actually use and if if the market is not big enough the mechanics the market mechanics may not make it viable is that a concern or do you believe the market is viable we're very comfortable with the business opportunity for barefoot in this space in general one way to think about that the mark is just to think about what goes on in a datacenter well one of these devices may be comparable to the bisection bandwidth of the public internet inside a data center with say four hundred thousand servers there will be twenty thousand switches inside that right and there are a lot of data centers being built so the market size here is is easily enough to to support several players but we're not worried about them but we do expect that there will be over time that could well be other languages there will be other devices really this model levels the playing field so instead of hiding behind closed locking strategies no one can get access to our API for us it will open and and so that we can bring on this programmable model so in order to understand what this what sort of how this this works it's good to go back and look at how the fixed function switch chips work this is an example of a of a fixed function pipeline it will be pretty common pretty commonly used today arriving packets they arrive to a fixed parser that fix puzzle is being told at the time the chip was designed what set of packets that it could recognize once is recognize the middle pass those headers along to each individual stage here we see Ethernet max MPLS ipv4 ACLs whatever happens to be designed in at the time that the chip was first built the first stage will process this it will pass it along to get processed here down the pipeline as it goes if you want to add a new feature like my V excellent example you need to insert it in here somewhere but there's nowhere to put it because it's all fixed at design time so there's no way that you can do that even if you're not using MPLS as most people don't you can't say oh I'll use that resource that you can't move the resources around or even you say hey they put too many prefix prefixes in there I want to take some of that space and use that for vxi you can't do that you can't even get it to recognize the VX LAN header because it wasn't told about that at design time so features and table sizes are baked in at design time and essentially this is the way that it's been done for 20 years this is the way it's been done for such a long time you have to ask the question surely one area of tech cannot stay unchanged for 20 years so I have a quick question about that so wait what is your table size and you may be covering this later but that leads me you know being able to allocate it wherever you want what could the table size be of atrophy no switch yes so the table size overall is really determined by the physical dimensions of the chip at all these ships have the same physical dimensions so we will end up with Tofino as being able to 1 to 1.2 million prefixes okay if you were to only use it for that purpose right so then you can then choose how you use that result okay if you wanted to cross on back for IB v6 or for something else you can do those the Duggan example allocation for you you just make that choice that's right exactly yeah and if you want to change it while it's in the field you can do that okay so an example of that in the demo later well it allows us to have one hardware platform that can use multiple names exactly exactly so you can have it that you can change it you can upgrade it in a sense what's happening is the Vino is bringing you peace of mind you know we're a peace of mind company right you can change that later in the field in fact you can do things like I want to put a layer for low balance right to my top over X which just download any people program add it into that and upgrade that at that feature I want to step back for a second to M the fixed process sort of fixed architecture versus a programmable architecture it was common knowledge right that that fixed process would be faster by orders of magnitude yes and you're saying that that's different but why is it because the conception was all or is it because technology has changed as allowing us now to have a programmable architecture that's better I would say that as an industry we were lazy okay if you think about what happened with the GPU the GPU came along because a few folks step back and said I'm not going to try and modify a CPU I'm going to say what is the instructions that I need what is the parallelism model that I need internally and they knew enough about graphics to be able to figure out how to do that in the last ten years partly because of at the end if aggregation etc we've got much clearer idea of what is it that the forwarding plane does in a switch and this match action paradigm match on headers followed by an action is really arisen as a common abstraction for the way in which packets of process so now you need an instruction set that will allow you to declare what am I going to match upon and then the actions that I will perform and if you can get the right instruction set and Tofino has rough on the order of one to two dozen instructions internally that the compiler will use in order to process the the packets it's really spotting what they are figuring out the right model of parallelism to bake into the chip and that was the thing that happened haven't happened before people tended to say I'll build a programmable device by throwing lots of CPUs down under a piece of silicon that's not architecting for the common fur they feathers of the case for the specific domain of networking so using we just become more intelligent about the way we've narrow down the instructions that have network projects that has to do that's right okay that's right and really following the lead of the GPUs and the DSP because they're going to do the same thing so this is see the sort of the simplified figure of the PISA pipeline the thing to notice here is that each stage is identical I show four here just because that's what would fit on the PowerPoint in practice it could be four it could be 15 it could be 20 so the you should think of in terms of roughly a dozen stages each of these is identical both in terms of what it does and its dimensions each one is implementing a match plus action there are multiple match memories here so we can do multiple matches on different headers at the same time and then once it's matched upon them it can perform actions from that instruction set that I described earlier so that's done by these al use that are that are here so multiple matches multiple actions and then move on down the pipeline initially it doesn't know what a single protocol is we have to tell it by it by the way in which we program first the programmer has to declare which headers are recognized that's described described in the program and so it will download in here and say these are the headers if I want the X land in there as well that's what I would place that's where I place it next the programmer declares what tables are needed and how the packets that are be processed so all the stages here are identical and we have been adamant about keeping them identical simple uniform because this allows the compiler to have a very simple view of the underlying compiler target so we use a term here at barefoot we say we're building a compiler target and so long as we can keep in our minds to building a compiler target and we can make sure the compiler can do a good job let me just elaborate a little bit more on what's going on here imagine there's a packet coming from the outside the little colored squares at the beginning are just different protocol headers what the Pazza is responsible for is ripping that apart based on the description we've given as the headers that it should recognize identifies the headers and then it's going to pass them down this pipeline and the way to think of it is that each stage each stage of this pipeline is going to perform a a transformation of this set of headers so for example the first stage will do match plus action and we end up with a different set of header fields it could add some like pushing an MPLS tag it could modify them like decrementing a TTL and an ipv4 header so that's the transformation that happened here and then it's going to go down multiple transformations as it goes down matching action matching action and until it gets to the end and we end up with some modified set of headers as a consequence for the program that we've been told to run at the end these get put back into the header queued until its turn to leave and the normal packet scheduling algorithms and then it depart from the use block very great sorry I'll be very quick the header is what's being manipulated here not the payload not the not the data just the header action correct you should think of the sort of the p 0 and P 4 approaches is about the headers now it all makes a question of how much header can you look into and that's a that's an implementation specific choice but it's thousands of bits of the order of 4,000 bits of for headers I'm seeing in terms of an ipv6 header which can get quite large yeah and so this is size to make sure that for any anything that you could have ever imagined as a header yep so if you imagine it was 4 K bits why for example that's an awful lot of header that you can get yeah what yes it is but the point is it's programmable it's staged but we're manipulating the header we're not doing layer 7 that's right activities here we're doing l3 l4 correct or encryption for that matter which would inquire and require looking at the entire payload yeah that's right so that's the limit we're not the limit but that's what this is intended because you're and I guess you have to be I just like to clarify this because you're forwarding at 6.5 terabits per second you can't afford to be doing l7 you've got to pass this through it's really just a trade-off of bits per second versus packets second there's a certain certain part of the processing which is packaged a second all switch chips have to make this trade-off yet trying to figure out how they're going to find the sweet spot of both and so in order to do that without consuming too much power most go for processing the headers but not the payload is that a evolutionary path in the decades ahead I would fully expect that there will be switched chips that will be p4 programmed that will do payload processing but whether we see them at the very high end remains to be seen yeah so but it's an evolutionary path that might weave line evitable might reasonably expect to see you know line rate forwarding at very low latency doing l7 with a programmable architecture I would expect so you might not see it yet at the very top speeds yes and it will probably always lag a little bit behind because it consumes more power and that'll process and I think it and it would obviously be fair to say that years away that's not this year sort of thing yeah I would expect in yes there was another question area before the animation your slide said the programmer decides what tables are needed yes does that mean the programmer is choosing from some available most of tables or he's defining the table so and even the names and what they mean so it might say hey I want i pv for I want I want a table that's a V excellent avi when it's right I may say that I actually might design my own if you know doing some crazy experiments IPV Nick and I actually want to describe my own type of table maybe it has 33 bit headers instead of 32 the tables are from the programmers and whether they're from the program from the Hatter that's right thank however for the purposes of getting people going we have built and then actually open source a lot of libraries for things like ipv4 ipv6 all the common types of protocol just to get people going so that they can get a starting point so people wouldn't remove them or edit them and change them if they want these targets and transformation points is that like a hardware limitation how many using you can go through or how many stages yeah I'm a number of stages in a machine ok I will talk more about the insides of Tofino a little bit later that would be a implementation choice for a particular piece of chip the only reason for for here is a power point limitation right but I figured there had to be some yeah there will be a physical limitation in the machine yeah so from the users perspective I write a program there's my snippet from earlier it's got a type of table that I described here and then a type of processing I'm not going to go through the details of that and that program is then compiled down and loaded onto the chip right so well that that magic is happening underneath the user should be as about as familiar with what goes on in here as you are about the insides of an x86 when you're writing a Java program the idea is to shield as much as possible however make it available to them if they want to see exactly what's going on so in terms of the way in which they get mapped one mapping might be I put my Ethernet MAC address table here my MPLS I put my ipv4 over here and I've noticed that I don't need to do the ACL at the end of the pipeline I don't mind the order in which I put them so I put them here that would be one example of a mapping the compiler did that for me based on information that I that I gave it if later I want to be able to make room for ipv6 I can just shift up and insert ipv6 in there I may in fact go back and say I want my VX land feature so I just use that on an unused match action capability right there ok so given your view of this program ability you know roughly at a high level how it works in a few minutes stan is going to go into more detail of what out if you know actually does this more more specifically all right you know you might be sitting there thinking ok now I kind of believe that you could do all of this but remind me why would you do this why would you make it programmable what are people actually going to do I think you'll get an idea during the afternoon of some of the applications and we've gone through and talked to many many people about the applications that they're interested in but there is one I'm going to call it the killer app the one that we just keep hearing again and again and again it'll come as no surprise to you that it's about visibility it's about telemetry it's about knowing what's going on in my network before I tell you about it were telling a little story I see that tell the story sometimes it's a story we were visiting somebody on Wall Street and the meeting was delayed and they told us the meeting was delayed because they don't outage in one of their data centers and what happened was they were losing millions of dollars a minute or an hour I don't know how many much they were losing they were clearly panicked they put to compute people in a room to networking and two storage people and they said okay you need to figure it out within 30 minutes the compute folks they were trying to isolate the problem they had measurement going on they were trying to repeat the problem they were doing workarounds they were programming they were running in the real system the storage people almost the same not quite as flexible and the networking people were sitting looking very bashful because they had ping traitor out an SNMP it's the same measurement features that they had 20 years ago nothing had changed I mean pretty shocking and so I like to say the philosophy of making networks today was brought to us by yo-yo ma because you're on your own mate basically what we're doing I mean I think we should be kind of embarrassed for the state of measurement in networking systems there's when it comes to debugging a network what is it that you really need you don't want sending traceroute packets that are different from your data packets that take different paths within experience different queues you don't want things that are ready tell you anything at all and you don't SNMP that's averaging over long time periods you never really know whether it's doing it right what I like to be able to do is very simple I want to go into the network I want to pick out a packet and I want to look at that packet and say why are you here how did you get here what delay did it take did you encounter when you were getting here and if you got here slowly who got in your way there is no network in the world that can come even close to telling you that today so let me turn this into a small set of questions that we would like to be able to answer so imagine I'm in my network there's two two racks of servers that are communicating together and I'm going to go in and I'm going to go to one of these packets and I'm going to say which path did my packet take if I can get this ground truth this is certainly more than I can get from any system today I'd like the packet to say I visited switch one at this time switch two at this time switch 12 at this time that's what I'd like it to tell me I'd also like to tell to answer this question which rules did my packet follow as it went through all of these switches there was a set of tables like prefix table so here I matched on rule 75 for this particular prefix I'd like it to be able to tell me from that packet I followed these particular rules so I know the past it took the time that it got to each switch and the rules that it followed it's not obvious my CPU will tell me that about my programs I got my packets tell me that event of a network I'd like to go further I'd like to say how long did my packet queue at each of the switches that it visited I'd like it to say I would delay this much of the first one this much of the second one unroll this much at the third one there's a problem okay I see the problem now I want to try and diagnose the problem so I go to the switch I want to be able to do this and say give me the time series the entire time series of the occupancy of your queue right down to the packet right down to the nanosecond and tell me what happened here when the delays suddenly went up there was congestion was it my fault or was it somebody else's fault I'd like it to be able to tell me who did my packets share the key words in this particular case it turns out I was the green one I was perfectly well behaved the orange one was the problem I then VI liked to go and say there was an aggressive flow over here that was sitting there and then generating a whole load of packets these four set of questions if I knew them will give me ground truth about the behavior of my network which packet did my packets and passed in my packet take which rules did it follow how long did it cure at each switch who did it share the keys with so there is no network in the world that will tell you this today but I can tell you this in fact we're going to show you this the see now mp4 can answer all four questions for the first time at four line line rate without generating a single extra Paquette so once you've got that in you can go and find out everything about the behavior of your network so look for that a little bit later this afternoon so to come back to why is it that someone would do this you're going to hear us talk about this because we ask ourselves this a lot we want to always remind ourselves of why is it that someone programmability they want to add new features if they're in a data center and they've got some new clever idea or if they're a large enterprise and they've got some ways to do the net bill then that work better they might want to add stuff they might want to subtract stuff they want to remove things that they're not using to make them at work more reliable or to free up resources for some other purpose and the last thing is that they want greater visibility as I just described into the way in which their network is behaving any last questions before I wrap up one thing yet I wanted to ask earlier was as you push new faith features down to the chip how does that affect the forwarding of packets does everything stop the everything that's not in the tables I guess I how does that what's the effect sure so there's a there's a few different answers to that so that question in principle with this model with this piece of model and programmability through p4 in principle nothing needs to nothing needs to stop you never need to drop a packet there's nothing in the approach that would do that it all comes down to the particular implementation of the chip and the particular implementation of the compiler in the worst case for to see now if you wanted to change the entire program reboot and come up it'll take you less than 50 milliseconds that will be in the worst case in some cases you can actually change it on the fly without actually bringing it down it'll actually depend on what you're changing imagine you've got a long pipeline and you switch the order around completely that's too much so you would it would take you 50 millisecond hit but there's nothing in the approach that would mean that you couldn't do that it's just a choice that you make at the design of a particular chip or a particular complaint you're adding your vehicle an example yeah that probably would not be in a row would interrupt service would it depends whether you would left room for expansion at the time that you did the design in the inefficient way so it's not about it correctly the beginning that's right yeah so this is the right place for this question but by opening up the the chip for programmability you're also potentially adding the need for some kind of configuration management change management infrastructure is that thing you guys are thinking about so it depends on your equity right that you obviously need to think very carefully before changing your network for the majority of you we would expect this to be done by the equipment vendor right so you know a large fraction of our customers our equipment vendors who then handle that with their customers they make it may be done through you know quarterly revisions and updates it may be done through different profiles for different customers that's up to them to figure out how to do that and in a way it's very much in keeping with the way that somewhere and software upgrades will be done today does that be one approach so the MS VCS or of course our other customers right now they have their own ways of doing this for firmware and software would expect it to fit in with that so as they're doing the upgrades we're a little ways away from somebody who is who is doing this in a relatively small environment for themselves there is no reason in principle we fully expect that to sort of develop and come out over the next couple of years the more and more people will be doing this for themselves with the fabrication and white boxes but that's sort of beginning to play out now in the early days you would expect this to be largely done by the equipment vendors

Channel: Tech Field Day

Views: 11,255

Rating: 4.915493 out of 5

Keywords: Tech Field Day, TFD, Networking Field Day, NFD, Networking Field Day 14, NFD14, Barefoot Networks, data-plane, PISA, Domain Specific Processor, ASIC, Tofino, P4

Id: zR88Nlg3n3g

Length: 36min 38sec (2198 seconds)

Published: Thu Jan 19 2017