Juniper Mist Marvis Evolution - The Journey to an AI-Driven Enterprise​

all right uh good morning everyone and thank you for joining us uh in uh on mobility field day five we are super excited because i think uh today once again we're going to move the bar raise the bar on the industry and and for us that all starts with marvis marvis our ai engine is going to get a massive facelift today at mfd5 we're super excited for having all of you uh to join and listen in on this so uh once again my name is sudhir mata i'm the vp of products here at miss juniper uh i came from the uh the the miss team and now run the entire enterprise uh product management for the ex switches the uh uh the srx van portfolio and of course uh the missed wi-fi portfolio so uh i have two other elite presenters here in the initial section of course uh bob friday our cto co-founder uh the visionary elite in the industry uh bob's gonna start off with uh sort of you know you will actually find this slide very very familiar you know it's one thing to have a vision it's one thing to actually have a bold vision in 2016 and be executing year after year if you go back to mfd2 our slide on our vision is almost exactly the same thing we've we've now got fancier graphic designers outside of that the vision was exactly the same so bob now is actually leading uh the enterprise cto uh position for all of juniper's enterprise portfolio we're super excited about that and another very very very good presenter uh is our header data science uh dr ji xing vang uh ji xing leads our data science team a stellar cast of characters and and we're gonna go deep and we're gonna go wide with ai and ml and the transformation we're bringing for the wired wireless van network so without further ado uh bob i'm going to turn it over to you to speak to our vision here a little bit thank you severe you know what it is great to be back mobility field day five here you know as seder mentioned this is the premier event where we like to announce the latest and greatest and seeing all the old faces here is great and for those new to the miss journey you know one of the original inspirations for miss was really watson playing jeopardy and that's when suji and i really realized that this ai stuff was more than just marketing hype right when we saw that they could actually build a solution that could play jeopardy at championship levels on par with champions like kim jennings that's when we realized that we could build something that really could answer questions on par with networking domain experts and really the first step on that journey was really to making sure we had the data we needed to answer questions such as why is steve having a problem with his internet connectivity why are the associates in the store in phoenix complaining about their wireless experience you know why are the robots in the distribution center not connecting to the network correctly and really that is why we built an access point that could really send us back the synchronous and asynchronous user state data we need to really answer those questions so when you look at the beginnings of this journey it's really about getting the data you need to answer the questions that user i.t people need to get answered and that's really the change in the paradigm we've seen over the last 20 years right 15 years ago at airspace it was all about trying and controllers and managing network elements devices we're now moving into a paradigm where it's really about helping enterprises manage the end-to-end user experience of what's connected to their network and when you look at some of our competitors they talk about global data and the quantity and i will tell you after six years of working on marvis is global data is valuable but it's really about the quality of data no two sites are different if you train an ml model with global data that ml model is not going to work a particular site so it's really about getting the data you need for the particular network you're working on each network is unique so when you look at what's going on after you have the data the next critical part of this journey is really around putting that data into a framework that really marvis can use to answer questions and what you saw in the first version of marvis was we built an sle framework and that framework was really about allowing us to use things like mutual information that really allows us to figure out when a user is having a bad experience what network feature was causing that bad experience you know and that's what allowed us in the distribution centers to figure out that that reader was actually real the problem related that reader was down to a particular os right you know where you want to be able to look across 50 network features and figure out which feature is causing a problem and what you're going to see in today's session is really the introduction of a new framework graph theory graph database and really this framework is going to allow us basically to take the topology of a network and basically start being able to correlate user experience problems with elements in your network problems and so this is what's going to allow us to start answering even more questions or find root causes the problems like when that person changes the configuration the router on the other side of the network and it causes a problem on the user experience how would you ever figure that out someone in two buildings away screwed something up this is where marvis can now basically do the correlation and basically figure out that that router configuration it's what's stopping your user from connecting the network and finally the next piece we're announcing today is really around this conversational interface and this is really a paradigm shift of doing away with dashboards you know we saw the industry move from cli to dashboards dashboards are becoming way too numerous right now you know they are not effective in actually solving network problems because their networks are too complicated and the data is too much these conversational interfaces or is what gonna allow marvis to really become an integral part of the it team it's what's gonna allow marvis to really gain the trust of the i.t department and become a virtual member of that team and then finally you're gonna see on the self-driving action is really putting how customer support is being turned upside down you know we are basically taking the customer support model the days of you trying to argue with your vendor on whether or not it's a hardware software problem those days are over you know now it's going to be more about marvin sending you an email and letting you know that you have an r hardware problem and that you should already make that piece of equipment back you know so i'm happy to be here today a lot of exciting things coming your way uh with that back to you senior no thank you bob uh we're super excited so i'm gonna what i'm gonna do is sort of um uh level set baseline on sort of where marvis is today i'm gonna literally take one minute to explain marvis and then we're going to throw the gauntlet down it is time dashboards are for the the last decade the next decade of networking is going to be through an ai powered conversational assistant and here we go our competitors are going to launch this a couple of years from now so take notice so uh step one let's start with marvis as it stands today right marvis as it stands today is basically uh you you ask sort of a a natural language question to marvis and say hey what's going on you know what's happening with this particular site you know what's happening with a certain user and marvis does the leg work for you right marvis is actually you know putting the picture together was it dhcp was it radius was it dns was it the network was it the van right marvis was playing that network assistant for you right and you can ask you know who's having a bad wi-fi day we can answer without people opening support tickets we can answer you know who's having a bad wi-fi day and if so why are they having a bad wi-fi day one thing you will notice the truism in every single missed network that is out there and our customers hopefully you're on twitter uh can vouch for it every single place missed goes in we cut down their ticket volume dramatically i will give you two public examples servicenow there's a video on this there was a 97 reduction in the number of user open support tickets coming into the it help desk from the day we displaced their cisco network and went to a global pervasive missed wi-fi network that is the power of ai at work for you there are many other uh customers the in the retail space gap uh they had a significant more than 90 percent uh reduction in the number of point-of-sale errors that their of sale systems were complaining about that the network was not available um we were able to reduce that right so questions here consider your question if i could yeah so you know you're mentioning cisco and replacing them and problems go away and kind of attributing ai to being the solution i just have to ask are you seeing bad cisco deployments and you you know miss just got deployed better are you seeing networks that went in incorrectly or are you saying that it's purely technology under the hood of mess that made all of these problems go away versus the technology under the hood of cisco uh lee outstanding question thank you for interrupting uh i i'm i'm hoping the format tom is that we we do get interrupted so i appreciate it uh lee for starting that off um so so number one is of course there are a lot of bad designs out there right and and you know it uh you know we all know it in the industry but that's really not the point right so step one and we're going to talk about this lee more specifically so there is this notion that we're going to address on something we call self-driving network right uh it's it's the the epitome of marketing hyperbole terms that are out there right here's what we do that actually contributes to the lower tickets right so if you look at a cisco network aruba network any of the legacy networks there are problems that are happening that are intrinsic to the network as an example you know beacons get dropped multicast is dropped ethernet gets stuck radios get stuck right all of these types of things today you have to manually detect them remediate them you know i know of a customer a very large one that sort of every three days they reboot all their cisco aps just because right because it sort of clears out a bunch of this stuff what we are doing is we are actually baselining the experience of every single access point you're going to see in in specific detail that literally we are going to put out on notice to all our competitors what do we self-drive when we say we self-drive and so i would attribute the number one reason why the mis networks have fewer support tickets is we actually remediate problems in line as we are learning right the difference between why couldn't a controller do that and why can a cloud do that is the controllers it's groundhog day every day they just do not have the memory to store what happened you know two days ago five days ago and be able to you know look at that event holistically over a week or a month time the cloud can do that right controllers forget and that's where we are able to baseline and and come back and remediate i would say if there is one thing that attributes to our networks working better that's self-driving you know i may add lee you know cisco and aruba are tough competitors but what we're finding if you talk to miss customers what you'll hear is that they have more visibility than they've ever had before and that's really because of ai and marvis it's not directly because of ai it's because to answer those questions we had to get all the data back into the cloud and for the first time now the same data that marvis needs to answer those questions is the same data an it guide means right and so that's one thing you'll find when talking to miss customers they really appreciate having the data they need to answer questions even if marvis is not answering them yet they appreciate the fact that marvis is answering but they first of all they appreciate the fact that the data is in the cloud now for them to look at studio i've got a question since um since you've mentioned them a handful of times now something i brought up last year at mobility field day was there is still no documentation about how a wi-fi professional is supposed to design a missed network we were committed back then that would show up shortly to the best of my knowledge there is zero public-facing information about what we as wi-fi professionals need to do in order to make sure that when these networks go in they're fully supported by the missed support team what's your take on when you're going to enable the people who have to actually go out and build these networks and getting them documentation they need in order to do the designs that they need to do if i compare again you've brought up system cisco and aruba several times i can go to either one of those folks and say give me a validated design guide that's 500 pages long i can get a bible for how to design these networks i've got nothing i missed sam i think that's that's a fair point um um i think first you know the fact that it requires 500 pages is uh speaking to something but uh that notwithstanding uh your point is valid here's what we are doing right most of our our customers follow uh standard sort of you know best practice wi-fi design you we use ekaha extensively in almost all of our deployments we take acaha designs you know ryan last time spoke about how we're integra we've how we have integrated ecohow designs to be able to seamlessly be uh imported into the miss dashboard however it's fair i think it needs to be out there and and we need to make a statement about it so so we actually that's that's good one sam i think that's on us and we will do that um for the most part sam um you know our uh our partners and customers uh they follow standard design best practices and and you know been delivering good networks but uh having a document out there is completely fair ask thank you and we should do that uh so how about this um in the next month we don't do it uh um you know we have a problem so uh and we gotta deliver it uh on a platter uh to uh to all the mfd delegates one month hey this is wes uh we actually do have uh some best practices out there that we make recommendations uh sam you may have seen it um it's from our it actually was drills originally from next works uh in in november but it's uh we do have a best practices uh guide out there with our recommendations so i did check the missed documentation page there is some stuff on the on ble and beacons and locationing stuff but there's not a lick of stuff on mist.com that shows me where to place aps or how to place ap or design cell edges or overlap or any of that stuff sure sure i think that's it's a fair i mean the point does i don't care if it exists if people can't find it it doesn't exist right so so i i think uh it's a completely fair ask sam and it's on us so thank you um was there another question if not i'll keep going okay so so so so this is sort of where marvis is today right this is something i think a lot of you have seen and and a lot of you have experienced and and really we uh we appreciate um a lot of the uh the feedback along the process as well now um where we believe where the next generation of interactions are going to be is in sort of a conversational interface so without further ado let's actually talk about what does that mean right so this this is um uh um sort of the the new marvis if i may so marvis is someone and something now is joining your team marvis is a new member on your team that you can actually work with right so so i'm gonna i'm gonna walk slowly on on all the things that are happening here right uh so the idea is let's say when i first log in i i see at a global level you know we have something we call marvelous actions the marvis actions come together and and we say hey jishing's uh welcome to welcome to marvis you have 13 things that that we want you to act on and and you know uh you know do you want to take care of one of those things or are you coming in to have a conversation about um uh something else right and so so then you know g shin can say hey you know what you know what's happening with this unhealthy ap as an example um then essentially you know we're we're saying hey we've used sort of um uh you know log mining and we're actually going to show you a specific example of what we're talking about here you know our largest customer resort is about a hundred thousand aps at the scale of a hundred thousand access points every single access point every single crash file every single log message is going to the cloud we put that through um through a log mining engine which we're going to talk about in a minute and and it comes back and says hey i looked at this one ap and this one ap by the way for the 100 000 ap customer today if if you log in you will see that they have less than 10 issues truly at that scale right and so now you can say ah okay this ap is unhealthy we've determined based on everything happening locally on that ap we believe there's actually an updated and up uh an updated image that can make this ap healthier based on what what all is going on with the ap and and the things that we're seeing so now you could say hey marvis you know uh marvis is going to say do you want me to upgrade and and you know the user has a choice you could you could say yes upgrade or or no don't and right here essentially this is now instead of you hunting and packing and finding where where you go find the ap and upgrade the ap and whatever you could say yeah get that ap upgraded uh life's good and and and let me know if uh once the upgrade is done right simple stuff marvis is now like a an assistant to each member of your team and and you're working with um uh marvis a simple difference uh you know just for uh just for example you know there are a lot of chat bots that are out there and ultimately that's technology talk to a human that may help you a little bit here and there or not this is you that for the first time in the networking industry you're interacting with the ai assistant live and and and you're getting data from that you can ask say hey you know i'm having some issues i don't know which user it is we'll show you which top which what are the top users having issues and we could say hey let's go look at this one particular user and he's having some dhcp issues you know do you want to see when it started yeah i got it you know it started uh you know wednesday on july 22nd and you know and maybe you know you want to see more data we will take you contextually to exactly that user and and see every single failure that has happened ultimately if you take away one thing what is so fundamentally foundationally different about what mist has done architecturally what bob and team have done we've pivoted the entire company from day one on the user experience everybody says that everybody says we're about user experience nobody is at the granularity we have everything one more question on uh on all the you know all the uh goodness that is marvis and with the juniper acquisition how deep into the wired network is marvis going these days is that something you're going to cover and is it you know how tightly integrated is excuse me the um you know it feels like a wireless problem but it's really not and you know how granular into switch configs and you know router acls and other points of potential problem does does marvis go yeah so so we're gonna we're gonna address that in in in just two slides uh lee uh and the the short answer is today marvis is covering um wi-fi uh switching and the new thing that we're announcing at mfb is uh is van routing from the juniper sr axis and the juniper e access it does not speak to you know a cisco uh uh switch or a third-party router but if it is a juniper stack we're getting data directly from the e-access and the srx's that picture is complete and that's what we're launching today in the i may you know what i would add is the graph database that james thing is going to be talking about here really that is really part of integrating both the switch and the router into marvis now is for the first time now marvis will be able to take events from each network element in that topology so like if that switch has a bad cable that event will now be correlated to a user experience problem you know so that's the beginnings of bringing the switch in the router into the marvis framework and that graph database is a key part of being able to get the topology of the network and the elements in that network into a way marvelous can actually analyze them yeah so that's good i i if i add another question i apologize so we brought up nlp at the last mobility field day and then it was far from uh you know the n nlp was far from natural uh can you pull up um the marvis dashboard and say can you show me where lteu is impacting my network now yeah um uh we have that exact demo coming up sam uh fantastic thank you yeah yeah absolutely absolutely so uh so so um the other piece that just flew by here is if you say hey i'm i'm not satisfied with what you're telling me would you want me to open a support ticket and if you said yes i want to open a support ticket marcus will just open the support ticket for you you don't have to go hunt hunt and pack you know where the support interaction is and and all that good stuff that's built natively right so this is and you're going to see more demos around this on the conversation interface as we progress throughout the day so so specifically actually sam thank you the a great point on the n in nlp is how natural is that and and and where is that gun where are we going there the next two slides are exactly that uh g shang i'm going to ask you uh to speak to this slide yeah yes so and you can see this is a high level architecture of marvel's conversational interface there are three things i want to quickly highlight here you know from the left to right first is this unified interface you can see from the demos here just to show about all different use cases that this interface can handle you know for the day-to-day network of operational work and this is going to be the simple interface that we are just like acting like a team member you know for all of your day-to-day network operation and if marv is failed to answer for failure requirement it will open a support ticket to you with all of the contacts so just think of the google you know there are maybe one million things google can do but at the end google just give you a small black box to deal with no one needs to read a 500 page how to use google right this is exactly the user experience we want to bring into the networking world the second is another engine that's a very good question this is as you may know one big challenge of developing chartboards there are many of the bots in the world the market one common challenge is how to collect the feedback because the ai needs to improve feedback is critical so you know most of the user if you just ask them to give a feedback you know five star of this most of you that can not give the feedback or only give the buys the feedback when they are dissatisfied right so currently with this new unreal engine in the middle we actually build again a ml based segment tagging engine that can automatically tag or monitor each term of each conversation so in this way markets can really continue learning and improving based on your user interaction but not relying on your manual feedback you know last but not least this marvelous interface is different than any of the general purpose interface in the market that just answer your question about the general information no this is fully powered by the marvelous intelligence we have built for each customer over years for example where baseline if your ap htr uh each of the you know ap doing the anomaly detection you know this like all mentioned this temporal graph all of these are used in the background to empower this marriage into this awesome so so actually uh um uh to to sort of summarize uh what ji shing was saying uh but before i do that maybe nick did you have a quick question sorry we uh we didn't pause for you please oh no worries i was just trying to figure out the uh when you were setting up that support ticket there i saw zendesk and all that pop up is that the customer support desk or is that with missed support or what are the options there that we're doing it it won't say zendesk going forward that's uh that's just internal it's basically you're opening a ticket with the miss support team um and so uh just like you opened today a ticket with uh with miss support team if you are going down a rabbit hole and and you're not satisfied with the marvis answer or you know you're not getting anywhere and the issue is becoming hot you could just say hey i want to open a support ticket what marvis will do is it will package your entire conversation that was just the history of the conversation you had and send that to the support agent so you nobody has to ask you what user again and what are we talking about and and what data i mean we don't do that today we actually obviously have a lot more intent but even even the the history of the conversational interfaith the the conversation you've just had will be packaged part of that right i've got a question can you can you speak to um how you're not trying to take away wireless engineers jobs with your marvis right now and um and just um with that said um is it do do you see customers that just blindly and i know this is this isn't just a single vendor issue but just blindly accept the recommendations that are given to them um you know it's just and then do you see that people are still required to have rf knowledge obviously that's kind of a loaded question but because you should have rf knowledge and stuff like that before deploying anything like this but are you seeing where people still are like blindly taking recommendations and then causing problems because they don't necessarily understand what marvis is offering to them um are you seeing any issues like that yeah so so um uh uh uh thank you uh mitch uh so so basically um uh first um there are far fewer i.t engineers than problems out there right you all will admit that um you know there's there's you know you take a higher ed you take an enterprise there's more users more devices more applications every day and it's not growing at that same scale right so marvis is truly there to augment um on that team be a part of that team uh number two marvis is never wrong and i want to be i actually i want to make that statement marvis is never wrong why because it's it's data it's actually understanding data and interpreting data it is not making up a lot of the data marvis isn't always right meaning we may not get everything right so there is definitely white space and bobble's going to talk about our efficacy uh you know again this is how ibm watson was built and it is about measuring the efficacy and holding yourself accountable and being public about it and we've been public we've presented you know at last mfb in the previous mfps exactly what the efficacy is today our efficacy is at 65 um you know basically 65 percent of the support tickets that come in uh you know something north of 60 percent uh we actually are able to marvel able to get that answer the remaining 40 we don't need marvis has the data for it in some of the cases but so so we're growing up but what it says it says based on actual factual data i will never say hey there was a dhcp pool exhaustion when there isn't one right where uh you know you have a bad password when when there isn't one right so so that's that's where i feel it very strongly that you know people are not chasing you know uh um rabbit holes uh based on marvis leaving them the wrong path right doesn't happen right perfect thank you sadir that was that was a great answer to a poorly worded question no no no very good i do have one more that goes with what mitch had and i'll try to be quick because i know that your time is um short but so a lot of times you know out in the dashboard world we're told about all the problems we must have you gotta have problems we got a dashboard damn it by our dashboard you have to have problems and a lot of times the dashboards are just hyper aggressive and calling everything a problem when in fact they're not reproducible they're explainable um they become the problem what is the story with marvis quest for uh very good networks versus perfect how razor edge is marvis trying to be in calling out you know what are the tolerances if you will is marvis trying to deliver us good reliable networks or cutting edge perfect networks that if you're not cutting edge you get a lot of false alarms hopefully i'm being eloquent enough where you can follow what i'm asking i got it i got it uh so actually uh lee um so so most of our large networks including you know juniper's own network what marvis action uh you know puts out you'll see is is a very small number intentionally and uh does does that mean we're shooting for perfect and life is so perfect on a on a campus network or uh you know the 100 000 ap network absolutely not intentionally when we are in doubt we actually lean on not uh you know uh calling foul and we actually lean on being conservative in marvelous actions right so so consequently when marvis action says something is wrong take it to the bank or you know we see near 99 accuracy around um the the ability to uh to get that right so truly marvis actions i i ask our customers to chime in i ask our partners to chime in we take it to the bank when marvis says something so that consequently we will miss some we're not shooting for perfect because that will raise a lot of you know you know the the long tail of problems for you that you may not may or may not want to address right so we're being conservative so dear i have a question about some of the marvelous stuff here too yeah um so obviously marvis has come a long way and using it in production environments now you know we're seeing that so while i appreciate all of this road map and what's coming that's nice there are some things currently that i'm interested in knowing are you guys going to give us a little bit more granularity and depth for example you know if there's an authentication failure with 1x i don't want to have to dig through a packet capture or dig back through a radius server to understand is there an eep type mismatch was it a bad password is there a certificate that's invalid or missing you know something along those lines are are you guys going to be kind of bubbling that up for us to make it a little more easy um that is the goal uh uh jennifer uh i think um so we can obviously continue to be better at that uh today we just say hey dave 1x failure and blah blah you know we get to a certain level what this uh this uh the new sort of engine gives us an opportunity to do is actually have a translator uh if i may on hey if this is error code five on on this e transaction what does that mean if it is you know failure code 17 what does that actually mean for a help desk user so that's something i think uh yeah so that is the that is part of this uh new step uh uh jennifer yeah so uh 30 seconds we are 30 minutes late already so uh so go very quick uh because this addresses one of the questions sam talked about is actually getting the nlu the natural piece of nlp right so good yeah yeah next i will quickly talk about this because that's a very good question as you may know this analogy problem has been in the broader industry not networking for even longer than the networking problem we have you know for decades of the time so the technology we here use is called transport learning you know and this really helped us you probably saw from the demo all of these typos and it's really like a true conversation experience it's not q a you ask a question give you answer no you are talking with a true intelligent about so um the principle of the transfer learning can be easily explained by using a simple example of teaching kids you know if you have kids and you may still remember the tough time of teaching them some basics of like math like y one y equals two you know i do i still experience it every day now because i have a little one in the kindergarten so but what if if there's some technology which you can transfer really all of your learning math knowledge including calculus you know uh algebra to your kids in one night so your kids can just learn on top of that you know is that cool this is exactly what the transfer learning is doing in the ml and the unlp world you know and uh this uh this is also the reason why transforming is called this standing on the shoulder of the giants come back to our marvel interface we actually leveraging the state of art neutral language understanding framework which is the same framework used to empower google home and amazon alexa you know just with this framework you can see we are really truly delivering the best experience a true ai solution is supposed to do so and this has reached pretty good accuracy when doing a long time even with much of the really this interactive training yet awesome um so bob um i want us to go quick so so i'm going to have you have you speak to the question lee asked how much of this is end to end yeah so so lee sit here so what we're doing here and it's my analogy here is really around watson jeopardy self-driving cars if you look at any of these ai solutions it's really multiple algorithms to get to championship level and so what we're seeing here is our graph database framework being introduced and this is really the framework that's going to allow us to basically bring other network elements into marvis and the topology so that graph framework is really capturing the topology of the network from the user all the way to the internet to the internet and what that allows us to do now is whenever there's an event in that switch or that router we cannot correlate that with temporal correlation back to some user experience so ginseng has basically been working on this for the last year and so this is the latest framework into marvin's and this is what's going to take us another step closer to getting to that 90 championship level we're trying to get to this will allow us to basically find the root cause to all those misconfigured devices in the network that sam screwed up someone's user experience yeah so so uh in the spirit of uh time i'm actually going to show you what what does that mean right so so the idea here is is you can ask hey abi is having a teams issue right uh avi is having a bad teams call we will come back and say these are the six uh five teams calls or zoom calls or skype calls whatever it is that you're doing and essentially put the picture together for that specific call across the wi-fi switching and the and the routing environment right so so essentially you could say hey you know there's some wi-fi interference during that exact time that happened in the in that ap could that have interfered with that call and again you know and the srx during that exact time for this flow for this user you know we feel uh there may have been impact right so so this is fusing the the graph database is nothing but you know there's so many uh edges to this graph here on you know is is it is it the the window is it the device is it the ap is it the switch is it um uh the uh the srx we're putting all of that together this is coming and it is end to end uh as part of this experience now let me ask you for the shortest answer and again trying to respect your time um i understand you know missed his juniper juniper has messed is there a goal and you know i'm guessing road map may be off limits but is there a goal to be compatible with other vendors is that something that you can answer so lee today uh the the on the switching side uh we actually have um uh some compatibility uh with cisco switching we actually uh show some metrics not at this kind of level we don't stream data from cisco switches into the cloud but we are able to show you know our uh you know a lot of times we go into an environment where the missed ap's are connected to a cisco switching environment a meraki or aruba switching environment we from a switching perspective we bring in so five vectors that work for any switch vector number one are my wireless vlans on that switch or not we deduce that on any switching environment are all these switches running the same version just simple stuff you could do this through solarwinds or scripting but we can learn that as well using lldp and we do that you know yeah so the answer is yes lee the answer is yes we want to be able to answer any question whether it's juniper boxes or not right the goal is basically to make sure marvis can find the root cause whether it's the juniper box hp or cisco box and so like right now i'm working in the ognog in the industry right now is basically how do we solve the problem you know if i have a different sd-wan router in the network how do we basically start sharing data and so right now we're working on how do we virtualize data across different vendors right how does marvis get access to data from zoom how does marvis gets access data from velocloud so basically we are basically going to try to virtualize the data across all these vendors you know and use the data in place but the simple answer is yes marvis has to be able to work across all network environments it's not you know we don't expect that everyone to have juniper boxes everywhere thank you yeah thank you bob uh so so um jishing um we are almost actually at time so so but this was an important issue this is an important conversation on what it is that we do address that contributes to fewer support tickets so if you can cover this in uh in in a minute or so right yeah i will quickly you know cover this and also come back to uh just try to summarize the graphs uh saying like as soon as this is showing that what we call the aios demo right so if you just think um six actually give us the flow data now but flow data only has a source id how do we know rb had that uh teams call we correlated actually the flow data with our dhcp data which is from the ap so we know who owns that ip at that time and also we also called it with the ap data you know with the switch data so this is really the graph database where the distributed graph data will be installed the nodes to really track the end-to-end network topology for each client and each flow session so on top of that this is also this event action framework like bob said the goal at the end is to bring all different type of the events from this end-to-end network all together why just because each of these could be the root cause of the user experience problem and also in addition we can market can also take for the high confident events mavericks can take a proactive actions to remediate the network problem you might even before it is impacting users so yeah sorry great finish finish finish with that yeah i think it's uh you can finish then i will summarize later yeah so so i think uh uh net net let me summarize um you know um one of the things uh that we have intentionally not done uh because i think uh a lot of people would be interested but uh we've intentionally not done is self-driving actions you don't find them in our dashboard um uh just like you know when you're driving in a self-driving car uh you know it's not telling you all the nuances it's doing all the camera angles it's finding to make a certain decision uh we want that to be as simple as possible uh for network engineers so the self-driving actions we're gonna keep talking about are you know is healthy radio's healthy pre-connection issues rf issues config issues a lot of these are self-driving issues so if we can't take a self-driving action is when we bubble up to marvel's right that's the distinction between our networks and almost every other vendors network that are out there now um i will move you to the next slide here right i'm sorry yeah probably this is the answer to some of the questions i just heard of you know to summarize the ultimate goal of aiops is to really simplify the network operational work for the network admin you know you don't have to um so some of the vendors build the ai ups by just stacking up different different gears but the problem they have is like they keep them in this this period of silos and it will really let the user to kind of connect the dots you know marvels actually give you this single pane of the glass view of all these end-to-end network problems so through and what we build all this common framework a temporal uh entity graph event action framework and finally this new marvelous interface you know you don't have to worry about adding van like even as you ask the how about the certified vendor to us is the same as not as you give data this framework is neutral to any of the vendor you know all of this together on top of this cloud native infrastructure can really guarantee our marriage to provide a seamless user experience across this entrance network at the end so to summarize in one sentence the goal of the ai is supposed to do all of the complex work and deliver the simplicity for the end user awesome thank you when is it coming uh thanks to uh michael who actually asked my question to marvis show me where lteu is impacting my network and marvin said i'm sorry i don't know how to answer that question yeah so um uh sorry the is the question when is the conversation interface coming well either either when is it coming or when can my question be answered got it so so the marvelous conversation interface um that we're launching today uh is going to be you know launched to our customers as beta in in the next few weeks uh it's it's uh uh almost there and then within q3 is is basically publicly available within this quarter and and i'm going to get to some uh your question on cellular i i it's it's very specific answer there with a demo so um on the self-driving piece uh we we've talked a lot about this so i'm going to go quickly on this uh bob maybe uh you know a couple 30 seconds on rrm i i think the point here is you know back in my airspace days back then we were doing radio resource management it was really about trying to optimize the power between aps and clients really the big change in rm right now with ml is we're able to use these reinforcement algorithms now to really optimize the user experience so back when we were doing airspace 15 years ago we didn't really have the user state data now we actually have the user experience data in their state and we're actually optimizing the symmetry of the link between the ap and that client you know so that's the big addition into the rrm space right now where ai and ml is really making a difference in our resource management algorithms and then the next one here is i'm going to walk you through a specific example of uh and and you know any wi-fi vendor that tells you they don't have a software issues or hardware issues are flat out lying and we we've always been very forthcoming about it and and so so let's take the the top column the top row here right so this is an access point that's been running happy and healthy and suddenly starting to see a few ethernet pocket drops uh you know could be a a chip issue a driver issue a firmware issue whatever it is right so we quickly uh detect that and remediate it and if you follow the top row that problem went away and life is good this is a problem that happens in wireless networks i can tell you for sure because i used to be an access point engineer uh and wrote of the entire access point stack for one of the enterprise vendors and this happens in every single ap because uh that's how life works right now let's say the problem didn't disappear let's go to the middle column here middle row here and the problem was okay the ap is good ap is healthy now and it came right back out right oh maybe that is you know maybe this is a you know there's there's uh you know a leak happening in memory or something else happening in the software this is where we apply sort of log mining and and uh basically we're collecting all these logs from the aps we're now correlating with hey these errors are happening you know is it possible that um you know these these uh you know crashes coming from this ap or logs coming from this ap map to a certain bug um that we already know and and then we say ah we actually you know each of these dots is actually a software or a heart or a firmware issue it's like oh it maps to that particular bug we go to our jira version and then we come back and say you know what we've determined we can actually uh remediate this problem based on we've already found this bug elsewhere and we can fix it for you using an image upgrade right and so here's another example of the same graph let's say you know there's a certain number of aps where we've auto remediated life is good a certain number of ap's this is by the way a real factual graph uh we just redacted the uh the the y column here but um and the where the firmware upgrade fixed it the last smile here is maybe it is not firmware maybe it is not software think of your experience today it is your job to prove to the vendor that you have a dead ap on your hand that you need to rma we believe that should be the network of the last decade the network of the next decade truly should show you hey these three aps are failing we believe it's a it's a hardware issue and here you go we're shipping aps tell us which site you wanted to go to and um and and and by the way and along the way obviously a lot of our customers are on twitter i can tell you confidently and factually our hardware failure rate is far below the industry average but one ap that is rma is one too many and and you shouldn't i.t shouldn't have to prove to the vendors you know level one tax level two tech and level three tax saying i have a hardware issue can you send me an ap none of that right the ai should assist you with that right so so um again uh basically all of this sums up to you know when we cannot address it in uh sort of a self-driving action uh basically that's when we bubble up into marvellous actions right so marvis actions itself continues to evolve we're adding more and more use cases every day we've actually added some of the anomaly stuff here you could say hey i'm uh you know i'm resolving this thank you very much uh you know we got a radius issue going we fixed the radius server life is good the problem happened you know at such and such a time that's okay what we do is we say okay that's good you said you fixed the issue not good enough we will validate that you added the missing vlan that you in fact did change out the bad cable you in fact did you know you know change scope reboot dhcp whatever you did on the dhcp server the ai engine will come back and say now your ticket is closed right this is the driver assist part of the self-driving piece right so you said yeah you can go sorry go ahead have a question from ravel and rauel asked you if you can look sfp problems or these stats or only the ethernet part of the ep abi and or wes uh i want one of you guys to answer that question if you are uh if you can abby yep uh the answer to the second part of the question yes do we do detect ethernet errors today and the sfp problems are coming soon yeah so the sfp is definitely part of the roadmap that we're working on thank you sorry jennifer i was going to ask you the one thing i've noticed is that the um a lot of the marvelous actions are triggered by you know certain volumes of data and it might not trigger with smaller volumes of data um so i'm just curious if you guys especially knowing that you already have deployments that you know obviously some of them are you know small remote sites um at several sites and then large campuses will you guys have some either type of baselining or setting um to kind of figure out the size of a site and tune that a little bit it truly uh we can show you marvis actions with a one ap site uh we do baseline and and then that becomes the baseline for the site and we learn and adapt um so um so basically uh it doesn't matter what is the number of aps at a given site our efficacy is just as good uh and we do have a lot of retail locations that have maybe two or three aps per site uh even 180 percent yeah awesome all right uh this is uh to the questions sam uh um one of the questions you were asking is um is is marvis sdk this is a brand new thing we're launching today we're super excited uh cinolini i'm gonna i'm gonna ask you to take this maybe actually bob uh why don't you start with just a little bit of high level vision on marvel's sdk and then suddenly i'll ask you to take this yeah so samuel i think if you look what we're doing right now right we're extending marvis across the juniper portfolio and beyond beyond juniper the other thing we're doing right now is bringing the client's view into the network and what we do by bringing that client to what that solves a particular problem when you mention lte is hey when someone goes into a retail store and they call up the it administrator and says hey all the users are complaining about their mobile app the first question you have to figure out is whether or not is it a cellular or a wi-fi problem you know so what by bringing that client's view into marvis that is now allowing marvis to basically get more granular on the root cause and what it's really doing is saving trip travel time we actually had a customer who had to fly all the way to california to figure out it wasn't a wi-fi problem it was really a cellular problem so the marvelous sdk is what's allowing us to start to answer those cellular questions you know am i having a cellular am i having a wi-fi problem in the store uh and certainly you want to give them some a demo here to actually show you what's in action now absolutely thank you bob thank you sudhir so again mfd is always about the first things that we launch and it's always about industry first especially for missed marvelous sdk again is another industry first where we're taking the client experience to the next level so sam to your point you know how do we know if the link is bad or what's impacting the client experience what we've done with marvis sdk is we have this sdk that sits on on a device and android devices specifically currently where we can get data directly from the device itself and feed that into our our marvelous ai engine feed that into our sles so what you're seeing in the demonstration here is the marvis sdk running on an android moto g device we're able to tell you exactly the experience of the client as it sees the network now we know other vendors have created or purchased other companies where they go and deploy overlay sensors to try and get that view of what is the view from the ground or from the client perspective what we've done is go straight to the client we don't need to deploy an older network the clients themselves feed this data into our ai engine into our sles what what you're able to get with this essentially is the view of apart from the network seeing the clients and what the perceived rssi is we can also now tell you what is the view from the client perspective and you can actually see where the data aligns and what it doesn't align what we also provide in addition is and it going exactly to your question of lte you we can show exactly when the client was on the wi-fi network and when the client is actually not on the wi-fi network but on the seller network and give you a signal strength again from a client perspective when it's on the seller so when you're you know your remote branch office calls in or the remote store calls in and says hey my app's not working the wi-fi sucks and we all know wi-fi is always the first finger to blame we can actually now go into uh marvis and say hey marvis what's happening with this client and the source of data for marvis to answer that question is this marvelous sdk where marvis now has the power to say oh this client experience was bad because it was a not on the wi-fi network b it was on the cellular network and c while it was on the cellular network the experience was pretty bad or if it was on the wi-fi network even though the ap are saying life is good you know they're transmitting what they're said to what is the client perceiving of that same signal strength so now you have for the first time in the industry the clients themselves talking to the network providing that full feedback loop of what is the network thinking the client see versus what the clients are seeing in addition to when the client is on wi-fi when the client is on cellular so the story bob mentioned it's a true story techs have gone on site to respond to this problem of hey why is the app bad it's an app server issue is it a seller issue is it a wi-fi issue is it really a wi-fi roaming problem turns out it's not and again as we go into our sle framework right we are able to tell you exactly what the issue is now based on the data coming in from the client directly but even go into details of is it as we call it device generic or device specific so with the miris sdk we're able to pull out not only i mean the device type device os everybody gets but on android as you know even getting to the detail of the device os version or the radio characteristics is now possible when we do this on windows on mac addresses we'll be able to get to driver versions so when the typical question comes up to say hey go update your driver we know exactly which devices are having issues and why that is is it a driver specific problem last but not least the the nirvana of it all why do clients make the roaming decisions they make right network vendors do a lot of uh mechanization in fact even standards allow for hey how do we help clients make the right roaming decision but you never know what the client is seeing why did the client go to the ap it went to and now with this marvelous sdk we're feeding into our ai engine the ability to see okay we know that these aps were the strongest for this uh client but how are the clients those ap so we can actually do a mapping of all the aps that the client is scanning for and what is the client perception of the strongest ap it is hearing and why it made a roaming decision so when you have roaming issues it's not just from a network perspective even from a client perspective now we can say the client made a roaming issue or a roaming decision to go to this particular ap because of all the ap's it hurt this was the best it could get to so that's the part of the client sdk it's it's going to help us take mars to the next level by having this full feedback from client to cloud for wi-fi to seller for you know device specific issues driver related issues as well as roaming decision clarity no now sam let me let me just on lte you is your question whether or not we can detect ltu in the unlicensed span or are you talking about yeah yeah yeah this this was completely not what i was asking i was saying show me where microwave ovens are hurting my users or show me where i'm experiencing interference by lteu or show me where i've got a narrow band jammer going off in the wi-fi network or something something that that i as a wi-fi professional or even a reasonably competent help desk person would want to know not not where is my client failing on the lte network right yeah so that that will be in the rrm piece of the puzzle right so i mean rrm is basically going to be breaking down all the interference into connected wi-fi unconnected wi-fi unidentified wi-fi lteu i suspect we will be able to identify it when actually shows up in the band you know this is really focused on the user experience right most of our problems right now are end in user experience on the cellular side yeah that piece what you're talking about sam we did that four years ago uh honestly speaking that one interference stuff and avoiding interference today uh we're going into some of the most mission critical networks and and i think as bob said they are our mps and that that adaptation around some of this stuff because we have long-term listen and long-term memory we're able to adapt that i really don't believe again it's cute for i.t engineers to be able to continue to see that but what do we do about it ultimately is how do we avoid it right and so that's what we've been focused on and i would tell you sam i think we are probably the only one we have one person dedicated to rrm we've been involving rm since day one since we started the company rm is still being involved right now and that's where you start with reinforcement learning and yes the interference detection stuff will be added to it all you know we'll continue to evolve the interference detection piece of it well and so studio you said you've done that for four years but i just got done asking marvis will show me lteu on the map and it's not there yeah so so so uh the point is is we are detecting and avoiding interference already right you know why should people have to manually figure out where that is and what are we going to do about it right and that's where i feel uh trying to over complicate user experience with all the data that we see is is probably the wrong way of going about it right so um that's my mute i i would say sam that is a fair question and marvis should be able to answer that question and so we will add it to the list and marvin should be able to answer the question please show me what interference sources are in the network right now if i can jump in there real quick and uh so you were kind of pointing down a road that concerns me just a little bit um i whether there needs to be an expert mode that we can enable and disable um but but that all of that data needs to be available for those people who who are the experts and will go looking for it um i i absolutely i i appreciate what you guys are doing with marvis and i think it's it's cutting edge there's no question about that in my personal mind however um the experts still need to be able to be the experts and and make the changes and and troubleshoot as experts without that uh that middle system getting in the way between them and the data sometimes jonathan uh fair enough uh we will we will specifically answer that question uh christmas comes every wednesday at mist uh that is a whole that's the whole process of how we've been able to be agile so uh so maybe in the next wednesday release uh you'll see where is my ltu interfering uh we i believe we actually already have uh certain marvelous commands that will show interference where it's happening but we will we'll take that in advisement right so uh so make sense

Channel: Tech Field Day

Views: 2,227

Rating: 4.7894735 out of 5

Keywords:

Id: 8AKTca76CB4

Length: 63min 56sec (3836 seconds)

Published: Sat Aug 01 2020