Bob Friday, Chief AI Officer, and Sudheer Matta, GVP Product Management

Juniper Networks Marvis Update - Mini's Digital Experience Twins

Summits AI & ML
Bob Friday Headshot

Juniper Networks Marvis Update - Mini's Digital Experience Twins

In this presentation by Bob Friday, Chief AI Officer at Juniper, discusses the paradigm shift from client to cloud AIOps and highlights the latest Marvis announcements.

Show more

You’ll learn

  • How Marvis is being expanded to include extensions for WAN routers, the data center, and NAC

  • How Juniper has extended Marvis Minis AI-Native Digital Twin capabilities to the wired domain

  • The next innovations on the horizon for Marvis

Who is this for?

Network Professionals


Bob Friday Headshot
Bob Friday
Chief AI Officer, and Sudheer Matta, GVP Product Management
Sudheer Matta Headshot
Sudheer Matta
GVP Product Management


0:09 uh very first right up front for us is is truly Marvis our Crown Jewel uh you

0:14 know this is uh I'm going to introduce Bob Friday Bob's the chief AI officer for uh Juniper Networks uh was obviously

0:21 the co-founder for M uh where stoked will start with Marvis and we'll be uh we'll be hidden off so Bob come on up

0:28 thank you Sadir you know as and Su say this is my favorite event of the year because this is where we talk about the

0:33 latest and greatest Innovations around Cloud aops networking now when Su and I

0:40 started this adventure you know it was really with the observation that there was a fundamental Paradigm Shift we still have

0:47 to keep all these boxes greens AP switches and routers got to keep all that stuff running but what we learned

0:53 from the customers was Wi-Fi was going from a nice tab to a must business critical and they wanted to make sure

1:00 whatever critical app they put on network whether it was going to be a consumer app or robot that that was going to have a great experience and

1:06 that was the cloud AI Ops Paradigm Shift yes we keep all these boxes green but we

1:11 got to really start focusing on why that user is having a bad experience regardless of it's one of our

1:17 boxes now someone talked about the AP the reason that we started the company with the access point and we built that

1:24 access point because we wanted to make sure that we're going to get the data we needed to answer the question of why

1:29 you're having a bad mobile experience bad Wi-Fi experience additionally we

1:35 remember Miss we also believe that indoor location and connectivity were converging and so we put a very special

1:41 B antenna to make sure that we had location and connected conversion since we started at Juniper you've seen us

1:48 basically extend Marvis across the access point the switch and the SD Wan

1:54 router and as we ingest more data this lets us get to more granular root causes

2:00 right that access point has a lot of information about the user state that sdw rer has a lot of information about

2:07 layer three and the application's running on it and last year what we brought to the party was Zoom this is

2:14 bringing the end in Loop to the end of the party right now we have complete visibility on what that actual end user

2:21 experience is going on for our zoom and we're going to be now talking about the innovations that we've done around our

2:28 continuous learning aspect what we're announcing new this year is starting to extend Marvis across these

2:34 Wan routers which is going to bring bring more Wan features in Marvis you know when you look at what's going on in

2:40 your network you know it's not usually the wireless problem the other big a here is the WAN connectivity what is

2:46 that mobile operator doing when something goes wrong half the time is somewhere in that Wan router in the

2:52 mobile operator Network that's connecting you to the internet the other big announcement that

2:57 we're bringing to the party this year is extending Marvin over the data center this is really bringing branch and data

3:04 center Better Together story these are for business critical apps like point of sale right you have a business critical

3:11 app running in your branch it goes down you need to immediately identify what layer of the network is causing a

3:18 problem is it in my Branch campus or is in my data center so this is the next critical Innovation that you'll see

3:24 coming down this year

3:30 and finally we're bringing Knack you know I think a lot of you have already seen that this was announced last year

3:36 this is our new Cloud Knack radius authentication Service and this is

3:41 really going to bring into Marvis more visibility on that preconnect you know when you use connecting when you have

3:48 the Miss Knack we're going to get better visibility on how that user experience is going so the take away from the site

3:54 is Paradigm Shift client to Cloud this is where the networking I think everyone has now realized it 10 years ago wasn't

4:00 quite so obvious now Clyde aops is the Talk of the

4:07 Town now I think heard the journey you know you've seen this slide I think I've

4:13 shown the slide for six seven years ever since we've been doing MFD and the reason this slide has not changed is

4:20 because the foundational pillars of cloud iops has not changed right and the

4:26 first element of this perod we talked about is data right and the reason we built that access point you know I think

4:32 the SS are everyone knows me I make a barrel of wine great wine starts with great grapes great AI starts with great

4:39 data and this year what we're bringing to the party is Marvis minis this is

4:45 probably the most exciting announcement of the year what this is bringing is what do you do when there is no users on

4:52 your network Marvis Min is basically the digital twin for your preconnect and

4:58 postcon user experience and we'll be talking more about this but this is what allows Marvis now to make sure that your

5:04 network is ready for when the business opens at 8:00 in the morning so this is

5:09 probably one of the critical announcements AI starts a data Marvis minis is bringing a whole new set of data when we do not have users on the

5:16 network to provide that data next big thing is really around this continuous learning experiment

5:23 right this is basically that s SLE service level experience which was

5:29 another fundamental shift when you look at the SLE inside of Marvis they're based on the user

5:35 minute you know 20 years ago when I did airspace it was all about keeping track of the controllers and APS and the

5:41 network elements we've moved that Paradigm now the fundamental unit is user minute you're keeping track of that

5:48 user State at all times every minute is that user having a good or bad experience on the network continuous

5:54 learning is starting to bring that experience to the to our party now and

6:00 we'll be talking more about the innovation in that area you know data science you know

6:06 we're all using the same data science algorithms in the industry we all have access to the latest and greatest data

6:12 science algorithms the big change here is really around bringing those out to

6:17 the SLE what we're doing with our customer support and our data science team right this is where Miss has really

6:24 differentiated itself organizationally bringing these two teams together and you'll see it in what we call our new

6:30 application SLE so this year you'll see a new application SLE page that brings continuous learning zoom and teams up to

6:38 the Forefront uh for our it users and then finally conversational

6:44 interface you know since we started this we've had conversational interfaces since 2018 you know we fundamentally

6:52 believe that natural Lang is the next Generation user interface in the industry in networking and other

6:58 Industries we'll be talking about the three critical innovations that we're bringing into Marvis right now to make

7:05 Marvis more humanlike you know how do we make it the interface of choice for interacting with your network and

7:11 getting information from your network and finally what you'll hear from West

7:16 and sun is what are the latest Marvis actions this is the self-driving component of Marvis this is where we're

7:23 starting to see customers trust Marvis to actually start issuing trouble tickets and taking action on their

7:28 behalf when they see when Marvis sees problems inside the

7:34 network now when you look at Marvis Marvis has three critical customer

7:39 facing components and we've talked about them we have the conversational interface this will become the new

7:46 interface of choice for most networking teams going forward you know you look at the industry we all grew up with CIS

7:53 2030 years ago we moved to dashboards and I know it's hard to say eventually

7:58 you have to give up on that CLI I know it's like prying you know something from your dead hands but you know we have to

8:04 move on to better ways of interacting because these networks are getting more complicated continuous learning you'll

8:10 see like three new Innovations in this area you know you will talking about adding teams into the mix and we'll be

8:17 talking about bringing a general model to to the party and a new application lle dashboard here and then the action

8:24 framework we'll hear from weson weson soon more about what new actions Marvis is bringing to

8:30 table when you look under the hood um SAR others looking he said Bob no one's

8:35 cares about your little data science models I finally give agree most people do not care about these data science

8:41 models but what you will realize when you look under the hood machine learning is going on under there the thing that

8:48 is differentiating is going to be the Deep learning we saw it with chat GPT

8:53 you will see in networking that these deep learning models are going to start to differentiate what's happening inside of

9:00 networking and when you look at AI what I tell people this is really

9:06 the next step in the evolution of automation right the only thing with this step is we're starting to build

9:14 solutions that can actually do things on par with humans these Solutions are starting to appear to have cognitive

9:20 reasoning you know you see in your chat GPT when you're working it's like how did it do that that's what these deep

9:25 learning these big language models are bringing to the party they start to let you automate in new and different ways

9:31 this is why you may not need to understand the math under it but you should understand what they're capable

9:37 of doing to confirm this includes Rag and Vector database capabilities yeah if

9:43 you look under the hood of we'll go into C I am you'll see a whole Rag and there there's multiple use cases that whole

9:49 large language model stuff is bringing a whole set of new use cases to the party we'll talk about here in a bit very good

9:59 the other thing I always remind people is you know one of the reasons that suj and I left Cisco to start Miss um we

10:07 knew that we wanted to build have a blank sheet of paper We Built This Cloud AI architecture but the other fun reason

10:15 organizationally if you want to solve this Cloud AI Ops problems you have to make sure you get your data science team

10:21 next to your support team your support team is the domain experts they know

10:26 what's going on in these networks they see it every day your data science experts are the ones who understand the

10:32 math and how to apply math to the problem you know and you look and fundamentally when we move into a cloud

10:39 architecture you find out that your support team is really a proxy for your customers because once we have the data

10:46 in the cloud you know everyone has access to the same data the customer the

10:52 support team anyone troubleshooting that is the power of moving to the cloud and you look what happens this is

10:59 my favorite chart you know this is you know stopping when you start this journey and my favorite comment is if a

11:08 vendor is not using its own clay iops and support they have not started the journey to Cloud aops this is the first

11:15 step in the journey we started the journey in 2018 that orange bar shows

11:20 you how long it took me to figure out why people RIS sshing into APS and switches you got to stop it you got to

11:27 get that data back to the cloud you know and that's a cultural change for those who are in the networking business taking away SSH Keys is this

11:35 hard to do for your support team the blue red shows you it's an endless battle right up and down this is a

11:42 week-by-week battle on making sure that the data science and support team are reviewing the latest greatest problems inside of the

11:48 network the a lot of people ask us why does the blue which is the good here that Marvis was able to answer the

11:55 question why does the blue go down and what happens is is as we solve problems

12:01 and Marvis is able to answer questions users are not going to open support tickets with that same problem so

12:07 essentially the way you interpret this is every single week we are we are

12:13 solving newer and newer problem sets and and consequently yeah the the the it

12:19 gets harder and harder and harder because the easier problems are solved ahead in time so Marvis is able to

12:24 answer questions so every single support ticket at Juniper missed 100% of them they first ask the question

12:32 of Marvis and then say hey did Marvis have an answer did it directionally tell me the answer you know did I is the

12:38 answer somewhere in the McLoud is it in the back end of the cloud or did I actually have to get to the AP and you

12:43 could see we still haven't conquered you know any of those vectors in the sense but continually making Improvement this

12:50 is a scorecard for how AI is doing for us this has been an honest to goodness

12:55 the absolute best thing in terms of measurements we've done is me ourselves okay that does not count against my

13:02 minutes yeah how do you judge the that the questions are getting more difficult

13:09 yeah is there some Metric that lets you know yes it's kind of flatlined on the

13:14 blue yeah but things are getting more complex is there some Metric to also know we're not answering easy questions

13:21 anymore so I review so I I we review this with the data science

13:26 and support team once a quarter uh we go go through a list of all the questions we can't answer and then we make pie

13:33 charts of why we can't answer that and what you quickly find out is we're getting to the bottom of the barrel you

13:39 know the the easy hanging fruit those are getting farther we're getting to a tail of hard

13:44 questions so so it it we have categorization for every single ticket there's categorizations of of a a broad

13:51 category and and a microscopic category and then each week we're trying to knock off some of this right the the power of

13:58 the Mis Cloud so we did 15 production pushes major ones last year and that's

14:04 15 opportunities for us to fix and move forward that's that's where a controller based platform or a cloud platform that

14:12 you have to take offline every every quarter to update you know a legacy microservice architecture does not get

14:17 it done right so for us the agility of the cloud actually helps us move forward it's all in the categorization and I

14:23 would say back to the comment about Hardware you know by having our own access points switch and rout there's a

14:29 whole device health element to this where Marvis actually keeps track of the health that's you know a chunk of those

14:36 support tickets are divide into the hardware there's a hardware component that we fixed what so it's uh it's Rocky

14:43 on the Stream So what what are those hard questions what are the things that

14:50 that Marvis can't answer today yeah so so Rocky great question uh so examples

14:56 are where there's a lot of dependency on the customer side you know uh adjacent

15:02 subsystems right so if let's say you know their uh core router is dropping

15:08 packets on a certain VLAN when we switch to the backup data center this happened at one of the largest universities we

15:15 run in the country uh uh from a you know from a complete wireless network

15:20 perspective you know and and that was so far out of the realm of what we touch

15:26 and see and and so that's a question they'll say hey yeah that no Marvis didn't get the answer so it's it's as

15:32 the problem space expands beyond our Subspace that's where it becomes hard yeah uh so when you talk about an answer

15:40 somebody uh is having a problem do they get an answer or is an action also taken by Marv on the infrastructure great

15:46 question so so so if you look at uh to Drew's question how does you know you

15:52 know in in in in the first customer's case we replaced that was Gap 24,000 APS

15:58 from a different V without adding a single extra access point or adding a new cable we've cut

16:04 their you know site visits by 85% service now we cut their you know support incoming support tickets from

16:10 users by 90% how do we do this is it just all marvy's telling you hey you got to take this action that action no we

16:17 are the first and honest goodness true self-driving Network in the industry

16:22 what do that mean what that means is when we see a problem that could be a a

16:28 problem problem that and these are intrinsic to our system that means a software on an AP a hardware on an AP a

16:34 switch or a router where let's say you take a software process if the software process is crashing as Bob said we watch

16:41 the health of every single you know access point every single software process on the access point and if we

16:47 could reset it without waking up it we do I'll give you an example if the ethernet chip on on an AP suddenly was

16:54 doing it was happy and healthy and suddenly see CRC error suddenly see you know spikes of stuff right and then we

17:01 go we say hey I could reset that ethernet chip without waking up the IT team we do if the problem space comes

17:08 back after several tries and it's like hey this thing is coming back we issue a proactive aoma to our customer and say

17:14 hey we have an AP in your ceiling that we won't like back and here's a brand new AP to you proactive R but absolutely

17:20 100% we take actions uh you know remediating software and hardware issues

17:25 intrinsic to our system that accounts for lot of fewer support tickets yeah so what happened in

17:31 the middle you have you you know you see all this you see these you see how you guys have evolved over and over time and

17:37 in the middle you see that oh you started not be able to answer the questions and then this Spike like what

17:43 was going on there every single week is a brand new week it literally could go down next week right it's because the problem space is different that's so so

17:51 we and that's why this is Bob's way of holding ourselves accountable saying don't fool yourselves yeah you have you

17:58 know that middle was 50% yeah it's a harder problem space that we hit yeah you know my theory was the customer

18:04 suppor team is a proxy for our customers if we cannot keep them happy there is no way we're going to keep our customers

18:10 happy have a question yes sir you said that based on you know um

18:17 monitoring what's happening with the access point in terms of resources or any bug is right so who takes the action

18:26 of doing that like is it the team assigned to your account or who does

18:31 that action no no no yeah so uh the the the automated actions I were talking about that's the that's a data science

18:37 model that's running in the cloud and it's in see we can't really nearly keep going rebooting resetting process we got

18:44 it and so one of the things the the boldest statement I I ever make and people will call on it and

18:50 that's okay we want them to be a customer and challenge us Marvis is never wrong why is why how do we make

18:57 that statement because Drive efficacy it takes six weeks to write Corde maybe six weeks to test code we take N9 months to

19:04 drive efficacy into every single Marvis action and so anything that is self-driving it's near 100% efficacy we

19:12 have determined that with extreme high confidence if we take that action we're going to only do good right and so

19:19 that's it's the efficacy that we've built into each of these Marvis actions at the largest our largest customer

19:25 today has 300,000 app access points deployed in one production Cloud you

19:31 know for their Global Network no other cloud in the industry can touch you know even you know 100,000 APS in one

19:37 instance right that customer can tell you Marvis is never wrong right and that's the efficacy we've driven Bob I

19:43 know we they're going to kick us out of keep going let me make up some time here so this is what I call the collaboration

19:51 Canary in your network coal mine you know when your network starts to have

19:56 throughput latency Jitter problems it is going to be your video collaboration users that first experienced that

20:04 problem right and last year what we announced is basically continuous learning where we're taking Zoom data

20:12 right and join it with network feature data from your client from your wireless

20:17 network from your switch Land network and from your sdwan routing Network taking those Network features and

20:24 building a model that can now accurately predict video plus or minus 20

20:31 milliseconds at 90% accuracy now once you have a model that can actually do

20:36 that actually predict something accurately then you can use what we call shapley explainability algorithms to now

20:44 basically give you a ranking feature of what network feature how each Network feature is contributing to that

20:51 prediction right now what we're bringing to the party this year and

20:56 announcing is teens so last year we basically heard everyone say hey that's great but how about teams

21:03 so now we're introducing similarly a model that takes your teams session

21:09 information combines it with your network feature data and predicts your

21:14 Microsoft teams performance level now this will basically allow us to bring the C collaboration Canary Coal Mine

21:22 feature to 90% of our Fortune 500 customers almost 90% of all our customers either have zoom or Microsoft

21:28 teams now as their video collaboration tool the other thing we're bringing to the party is an application SLE page

21:36 right this will now give you direct visibility of how many bad Zoom teams minutes are running on your network and

21:45 it'll quickly start to be at a site level it will start to let you quickly figure out what layer of your network is

21:52 causing post connection Zoom users the most pain is it your client is it your

21:57 wirus link is is it your land link is it your Wan connection this is the problem

22:03 with most of these features is it's not one thing or other it's a combination of things that's causing pain in your

22:08 network there are occasions we'll talk about on the client side where there may be something like a VPN this vpm Ser we

22:15 found that will cause a lot of users pain that this will quickly find the needle and the haystack problem probably

22:21 the most exciting thing that we're bringing to the party this year that that that that bringing that that to

22:27 party um is body call General model what you see here there is no Zoom there is no teams

22:34 data right this is similar to the large language model this is a model that can

22:41 basically predict what the performance of your zoom and teams users will be on

22:47 your network now we actually had a very large retail customer was having a problem we applied this to that Network

22:54 to see what was going on basically informed us that we need to basically take the channel bandwidth from 40 MHz

23:00 down to 20 MHz right because we're seeing a lot of interference That's The Power of a general model that can look

23:06 at a network and start to predict what is a zoom team's experience going to be like on this network this question are

23:12 you are you planning on heading this direction or the reverse you did teams

23:18 and um Zoom were you were you looking at going and hitting all the other SAS providers individually or is the goal to

23:26 go and and go more General role rather than individual hitting each one by itself I'll let Sun talk to the road map

23:34 I would say there are Visions right now where to take this next right you know zoom in teams are probably the big

23:40 applications uh the other thought is to work with app monitoring companies start bringing in a more General look across

23:46 all the apps but but the short answer Keith is uh the most stringent apps on

23:52 our Network or the collaboration apps like voice video apps in fact the problems faced with voice and video apps

23:58 right now is is you know if your CEO had a teams issue everybody's going to jump up and down and go and fix it but if the

24:03 average user in the Enterprise is having a teams issue or Zoom issue we never get to the root cause of it it just passes

24:09 by because these things are transient right so uh we took the hardest one so we can get this data and build truly a

24:17 model that right now we're consuming no joke millions of user minutes on a every

24:24 single day perspective and so we're learning with millions of users minutes the analogy there is large language

24:30 models learned on billions of you know trillions of data points right and so we're learning this and if we could

24:37 crack this it's game over then I don't need to connect to Salesforce and and a SAS applicational service now something

24:44 else we're that's the goal this is building a general model that can truly

24:49 become the game changer how much of this data that you're collecting is from the

24:55 actual application itself so you got Zoom you got teams are you are you

25:01 taking that from the client that's on the device or are you doing some sort of

25:06 API tie in to these applications these are Enterprise customers uh taking it

25:12 directly from the zoom teams API so zooming teams both have apis that keep data per minute or per session on all

25:20 the sessions on the network I think this redate what uh Sadir said you can visualize that if you look at chat GPT

25:27 they built a a very large language model trained on trillions of words of data to predict the next word what we're doing

25:34 here is taking billions video collaboration data points from Zoom team users and building a model that is

25:41 predicting your Zoom team's experience so that is the power of deep learning models right this is what is going to

25:47 trans you know differentiate and transform what's happening in networking so so should it actually I mean do um U

25:54 should we expect to see this uh I mean I'd love to personally see it in Marv's Mini for example I mean I initially

25:59 showed application SES for example right so I know Marvis Min kind of show some

26:05 stuff in there but but I mean I'm I'm visioning and thinking like this showing

26:12 up in Marvis minis because Marvis you know you're you're using I mean we're using your network I mean U you I'm

26:17 going to get there in like Miss devices right so it knows what exactly we are uh uh you know accessing different

26:23 applications right so it can tell me which applications are having issues and problems basically we'll we'll speak to that okay yeah so so this is you know

26:30 roel to your point large language model that ingested words large experience

26:37 model that's ingesting The Experience minutes from these real-time applications first time in the industry

26:43 right so so there's real opportunity here so um anyway so uh um Ali let me

26:49 start with what does this do we are super excited because um you know uh for

26:54 the last seven years in the M dashboard you saw Wireless wired and Van as the three service

27:01 levels that you saw uh with insights and stuff today for the first time we're

27:06 revealing the an application SLE right and what this is doing is it's basically

27:13 you know collating all the user minutes we see at a given site uh or a given

27:18 access point you know at any uh uh level of uh granularity and say okay I had

27:23 4,700 total user minutes on on that site today from a you uh you know uh uh

27:28 collaboration perspective 233 minutes were bad and then you say okay uh you

27:34 know let me go dive into exactly why so this is basically saying let's dissect

27:41 that 233 minutes it says van was a a huge contributor for that and then you go in and say okay now show me more is

27:48 this uh is this a van by the way in this particular example only the Wi-Fi was

27:53 Juniper misk right the wir network the van Network this is the power of the shapley model we can look at so many

28:02 parameters that we don't have to actually directly get this is again if

28:07 you do some research on this Baba and and nav or data scientist introduced us to this thing and it's it's gamechanging

28:13 in that I could say I could say hey this cohort of users are having an issue

28:18 what's commonality for them ah they're all sharing the you know the same vling or they're sharing the same APS in an

28:25 IDF they're sharing the same SS ID same whe and you know maybe they're all on VPN they so it's it's developing cohorts

28:33 and making meaning of those cohorts that that's where this is this is headed and so we're launching this with zoom and

28:40 teams this should be available for you know missed customers as part of their Marv subscription in the next uh you

28:45 know 90 120 days or so um but but brand new SLE will will basically get you down

28:52 to Aha you know uh Jane in finance had an issue with themes no one's going to

28:57 jump up and down and saying ah let me go get down to that particular team's call no I can take you down to you know

29:04 Jane's laptop and say hey what was the Lost latency in Jitter I have an enumeration of every single team call uh

29:12 uh that that uh she's gone through pick an individual team call that she may have complained about and say aha during

29:19 that specific time there was a van issue or maybe client side right so there's there's incredible value in going from

29:26 at a site level at an org level saying what is what's all the that is happening and then taking it down to an individual

29:32 call first time in the industry we're taking all the way down to an individual call and the experience right so now

29:39 it's um it's Rocky again yeah go excuse me um I see clients listed and all of us

29:47 in wifi know that a big percentage of the issue is clients so how deep and how

29:53 much data are you gathering without a client agent in that hey it's the client

30:00 super good question um so on the client side actually what we're doing is we're

30:05 not installing any new agents we're actually leveraging the agent that is the teams agent and the zoom agent and

30:12 we're basically getting from uh zoom and teams Cloud to Cloud so we don't have to

30:17 ask for a new agent to be installed Cloud to cloud and say uh on the client side we get things like what's the CPU

30:25 during the exact team call that was happening so at one of our customers the CPU on certain clients was actually

30:31 contributing to it uh and and how does that correlate for example if I'm downloading a big Microsoft update or a

30:37 Mac OS update during the time I was having this teams call a normal user would not correlate those two things as

30:43 having impact but it does have impact right another thing we get from client Rocky is is is VPN as an example was the

30:51 client using VPN where are they ingressing egressing that kind of stuff sorry go ahead so let's say it's not

30:58 teams and zoom yeah that are the issue how much data are you able to gather and

31:05 see about a client so let's say that it's the you know senior director VP in

31:11 the corner office and their Spotify is flipping right and we get the call how

31:17 much are we able to see from the network side about the client and how much data are you pulling in to see that uh again

31:26 we have a different agent for if they are not teams in Zoom we use our Marvis

31:31 client agent if some if people are are willing to install that we do have that agent that gives us RSSI driver level

31:39 detail driver type detail all of that kind of stuff you know one of our customers has about 40,000 APS they've

31:45 deployed this on 200,000 handhelds our Marvis client and so they they you know

31:51 they are able we're able to get that kind of data but not application data Marvis clients today focus on Layer Two

31:57 what about the so there is a Marvis client you just aren't as much utilizing

32:02 it for teams and zoom or you're enriching we're not installing another agent but for uh if if the option exists

32:10 for and Slava is going to speak to hey um you know uh some of the onboarding stuff we're doing there is an aspiration

32:17 for a grander client that Slava is going to uh you know speak to I won't steal his Thunder but that will also have

32:23 Marvis I'll I interrupt you sorry oh no it's fine I was just so Marvis agent right I know for handheld it is

32:28 available um I know I mean then there was like a Windows agent that correct what's the status and development of the

32:34 windows agent and Mac possibly I think there was an an announcement and and with with iOS 17 there's possibilities

32:41 on Mac uh sorry on iPhone right so Flav will address all of those in the timing of that yeah all

32:46 righty this is Kevin Fran from the BR um you mentioned these are Enterprise

32:52 customers is there any interaction needed to enable this or this just works

32:58 question so Kevin U so they do connect their teams or Zoom they get an we we

33:03 get an API token and and um it's a base subscription on teams and zoom zoom had

33:09 has additional options they don't have to choose those we're connecting with their base Zoom subscription using an

33:15 API token that they would share uh they would input into the M dashboard uh another question um you

33:22 mentioned that some of this visibility is is done done with just the aps if this was a full junip site do you have

33:28 more visibility so so I think the question Kevin uh uh was you know does it need to

33:34 be a full Juniper site the answer is no uh if as long as just Wi-Fi was Juniper missed uh everything you saw uh is is

33:42 still possible no so the question is if it is a full Juniper site are you able

33:48 to uh see more into the network and have more visibility yes yes absolutely absolutely yeah so if there is sdban

33:55 from us uh we get application uh input into this Chaple model if there's switching from us uh um we we we take

34:01 additional input into the model yeah awesome a couple of quick customer stories uh I think we're severely late

34:08 at this point but uh uh but basically uh service now this was a great example this is there uh where the Hyderabad uh

34:16 you know call center essentially uh you know people were some people were complaining specific cohort of people

34:22 were complaining and the shle model said hey these guys their VPN is egressing out of uh Sydney while the users were

34:29 sitting in Hyderabad you know true story and it was it was interesting that nobody actually car to look in that kind

34:34 of format uh Dartmouth College this is a great example uh Brian Ward a lot of you

34:39 guys know uh uh basically this was a corner office the executive has uh you know uh his own uh dedicated AP and so

34:48 you know why would there be a capacity issue life was good and and but the this model was saying hey there's a capacity

34:55 issue and Brian said no I don't think so let's let's duke it out and turns out uh

35:00 what what ultimately was happening was uh you know dmouth runs the largest hotspot uh uh in in higher ed verle they

35:08 have a wide open network and so passer by passing that building constantly connecting through the AP uh from this

35:15 uh executive and that was actually uh challenging so I'll skip some of the example Bob come on up we got to catch

35:21 up on time Marvis minis okay so as a data scientist Marvis minis is probably

35:27 the most exciting announcement of the year because it's bringing data to the party that we did not have before the

35:33 other reason why this is exciting is because similar to the general large experience model you know we are now

35:40 able to predict a user's experience before they actually get on the network the other big thing for me personally

35:46 was making sure this was part of our Cloud architecture and what does that mean that means speed of innovation that

35:53 means we can innovate Marvis minis during our weekly production pushes right this is not part of my firmware

35:59 build this is part of my cloud architecture now secondly we're bringing it to the switch so that also brings

36:05 more data from our switch wir Network into Marvis now of what's going on inside that that Land network and

36:12 thirdly it looks at both the preconnect and postc connect experiences preconnect

36:18 is focused on making sure that authentication dhp DNS service is up and running and going we're also making sure

36:25 critical applications up and running before the door opens and we're bringing a speed test to marvus minis this starts

36:31 to giv us visibility into that Wan connection and throughput connection before the doors open up in the morning

36:38 with that I'm going to turn it back over to Sadir to catch us up let's go did my part quick question on that will we be

36:43 able to customize the speed test possibly I probably want to use a I don't know like a server in my data center or something like that like iper

36:49 or use any not right now not right now it's uh yeah so so for those that may

36:54 not have uh seen or known what Marvis min is let me actually just give you a very quick uh you know view of what this

37:01 is what today Marvis actually does an extremely good job of understanding users experience and and and pinning

37:08 down on what was happening with an individual user but a lot of times I mean if we as the more and more cios we

37:15 speak to their number one uh you know concern and issue is I want to know

37:20 before my users have failure I want to know before anybody experiences failure

37:26 and that's where we launched the the first sort of digital twin uh um uh in in the industry uh from a user

37:32 experience perspective the industry generally when they speak of digital twins they speak of network uh you know

37:38 emulation this is we we've always been focused on the user and this is about

37:44 understanding the users's experience and so being able to actually assert problems before users come on the

37:49 network or experience failure and so what what we are super excited to launch

37:54 today is and and again I think a lot of the Marvis minis piece is um you know

38:00 classic Mist style we want this to be super easy to deploy meaning there is no

38:06 configuration you you deploy missed access points and and you know if you have the Marvy subscription minis are

38:12 just automatically uh you know running what does that mean we learn your ssids we learn your vlans we learn your DHCP

38:19 your DNS your radius this is not you know our grandfather's sensor Network where you have to set up everything or

38:25 if something else changes on the network I to go tweak that this is a modern Cloud native digital twin experience and

38:33 so if a failure happens I actually have a dynamic packet capture waiting for me in the cloud right so so really cool

38:40 stuff well that's great that's what we said you know earlier in January what are we launching today what we're

38:46 launching today is Marvis minis you know is now incorporating you know the

38:53 switching authentication from the switching side so this is step one and I'll get to Oliver you're talking about

38:59 the experience bits so today Marvis minis is basically saying okay I can

39:04 actually a certain you know DHCP DNS radius some level of applications um

39:10 where you know we we learn the the major SAS applications and we are validating this is an example where DCP was failing

39:16 I have a pcap waiting for you now let's take this to um the van speed test today

39:23 we and this is the early days the first version we're launching a basically access points running you know speed

39:31 tests once a once a day just to establish what is the the the maximum bandwidth at this site you know uh and

39:38 you know obviously you can customize this thing and then switches starting to do radius validation every hour to say

39:45 I'm able to reach radius and able to authenticate with radius so again a little bit more data to Bobs point the

39:52 more data the better Marvis is now let's talk about uh where where uh you know

39:57 this could get even more interesting this is brand new that we're launching today where we are now basically saying

40:04 how do we get more visibility into application performance right and so uh

40:10 the idea is is essentially leverage your infrastructure use this digital twinning

40:16 experience to say Okay I want to define or I want to learn you know some of the

40:21 these are my some of the top applications at this site let's say as an example I picked teams as an example

40:26 and say okay I want to know the performance of teams across all my sites

40:31 across L across you know loss latency Jitter so these are all my sites I wanted to know and bucke tize my sites

40:39 for me on what's my average latency across my entire organization and so you

40:45 can immediately get to the outliers on saying which sites are actually having

40:51 persistently you know low lat bad latency or larger latency than others

40:56 right so first gives you an organizational view of site bucketization of application performance

41:03 then you go in and say okay I want to now pick you know a particular site across all my applications I want to

41:09 know in that particular site what is the uh um loss latency Jitter across all

41:15 these applications so now um you know again this is um this is another

41:20 application this is the bucketization of the latency for those applications again no sensors no additional Hardware

41:27 nothing to set up constantly always running and and having the data at your fingertips and so this is getting you

41:35 proactive if something changes at a site where it jumps from you know its normal

41:40 latency to something uh something new we catch it and now I zoom into an individual site here right this is this

41:47 is the same site all five applications that I'm showing they're continuous you know uh experience so constantly we're

41:54 validating application performance uh and this is where I'll leave we are getting closer to you know not requiring

42:00 additional servers and sensors just using the network to validate app performance yeah I like the idea of

42:06 testing the application I know Marvis minis um are initiated from a

42:12 neighboring AP to another one but will you incorporate data because I know you're showing average latency and all

42:19 that will you be able to show us to like the AP it's connecting to and how many clients were connected to that AP cuz I

42:25 I would like to know if that ly is driven by whatever else is going on with the clients Associated good yeah great

42:33 question this is Phase One Step one right lots of possibilities uh you know

42:38 we you know we when we launched Marvis minis in January I you know I genuinely

42:44 feel we have a three-year road map of things that we could do that could truly be gamechanging right so this is step

42:51 one we are laying down the foundations and infrastructure the one difference between January and and and May 24 is

42:58 we've made Marvis minis completely sort of a you know the very first time we've asked our customers to upgrade to a

43:05 certain firmware version from now on Marvis minis is an extension of the cloud new functionality don't have to

43:10 touch firmware it's just a you know a new push from the the cloud type of stuff so we're we're laying down the

43:16 foundation this is the very first phase of it lots of possibilities Yeah you mentioned something about is when you

43:22 mentioned about radius right so is that from the switch but is it also testing like from the access point it does

43:27 access point we already have cuz you know you run into sometimes a scenario for example where um somebody wants to

43:33 add access point IPS in the radius or right like individually or something like that or somehow it got missed or

43:39 something like that it will test that as well access point so so the way Marvis minis so so obviously it's testing all

43:45 the time uh this application performance as an example we're doing this every 5 Seconds we're constantly validating

43:51 everything from a uh um a radius test perspective essentially what we do is

43:57 you know we test and if everything is okay you know you know no harm no foul but if we find one AP failed the radius

44:05 test immediately we start a blast radius determination oh is that one AP is it

44:10 just these APS on the switch is it the APS in this closet is it this floor is it this building that site so we have

44:17 Marvis minis is designed to slowly expand scope to tell you hey all of the

44:22 Eastern Corridor is actually having a radius issue because the clear pass for that thing is down right something like

44:28 that so we we it has the blast radius determination buil and the last question is like will the user will ever be an

44:35 availability where I can maybe possibly add a user and some credentials in there to test against not yet but uh we'll

44:41 we'll get there but today you can instigate a a a test now the test now button actually can go do all of the

44:48 tests uh right now okay yeah awesome all right uh I'll I'll skip some of the uh the the stories on Marv's minis in the

44:54 interested time sorry sorry um I was just going to ask like how is this data like distributed out to uh become

45:01 actionable like like if you know that it's a radius problem can you just set up Marvis or the dashboard to package up

45:07 some screen grabs or reports or data and like send it over to the radius team so they can fix their stuff or the teams

45:13 people so that they can know or the W people so that you're not just digging down into the weeds to like look at

45:18 charts and feeds super yeah but like do something about it correct great question so the way we've actually uh um

45:25 uh done this is all of the Marvis minis um from the rest of the system

45:30 perspective looks like an actual user and so if we determine that hey radius is actually unavailable for this segment

45:37 of the network um that today turns into a Marvis action which which throws a web

45:43 hooks into uh a service now or their automation tool and that that's how it becomes actionable and then we take all

45:49 this data and attach to that right and so yeah so that's how so the consumption not bound for from our system is always

45:56 web hooks through Marv action awesome uh I think again you get

46:01 the stories in the interested time we have one more section uh Bob so so uh you know I'll let you uh speak to

46:09 that so we're going to do a little bit more on conversational interface I think someone brought it up earlier you know

46:15 so this journey as I said started back in 2018 we started this journey with

46:20 natural natural Lang natural language understanding technology back in 2018 we

46:26 started started with realtime troubleshooting questions so we got very good at understanding what people were asking us and being able to respond to

46:33 these realtime try what we announced last year was really around knowledge based with open Ai and chat GPT what

46:40 they brought to the party was natural language generation they gave Marvis a

46:45 voice we're very good at understanding the question now we're getting very good at answering the question like a real

46:52 human and where that's leading to is a much more natural dialogue inter face with Marvis in the future going forward

46:59 when you look under the hood of Marvis what you see is we basically have built

47:05 a platform an architecture that has multiple llm pipelines for multiple use

47:12 cases so depending on what use case you're solving you'll find different open source large language models under

47:18 the hood of Marva C the first use case that we mentioned was real-time troubleshooting right this is where we

47:25 have a query classifi ation algorithm that classify what type of query question is the customer asking us

47:32 realtime trouble shooting goes down One path what we talked about last year was knowledge base right that is where we

47:39 basically started getting much better we've always been able to retrieve documents for user questions we're

47:44 starting to summarize those documents right and that's what chat jpt brought to the party was solving that doc search

47:51 summarization use case and that's what everyone in the industry is working for working on and I would say if there's

47:57 anyone working on this we're right at the Leading Edge please look me up I'm happy to compare notes I think we're all

48:02 in the same boat trying to figure out what's going on with do language models that are changing weekly the other big

48:08 thing we're announcing this year is customer support similar to that real-time efficacy chart you will see an

48:15 efficacy chart for support questions now for knowledge how well are we answering them back to that same theory if

48:22 customer support's not happy customers are not going to be happy and finally what we're looking forward to in the future is starting to put large language

48:29 models on top of SQL this is basically letting our customers start to explore

48:35 their own network data think this is an alternative to bi large language model will become another way to know what's

48:42 going on your network on top of your data and I would say this is the other

48:47 new chart you'll be seeing basically for every knowledge-based question that comes into our support team about half

48:54 of it half the questions that come in realtime troubleshooting the other half are some sort of how do I configure my

49:01 4400 where do I find something these are the questions that llm is going to get

49:07 very good at answering the green and the yellow start to show you how well Marvis is doing at answering those tickets for

49:13 every support ticket that comes into our our Network right now you can see in the scheme of things same thing like other

49:20 one we want to get up around 80% we're probably the blue here represents hey this wrong

49:26 good news is hey it's only 23% so we're off to a good start we want to get that down to 80% going forward hey Bob who is

49:33 asking these questions are these trained it operators that know how to properly

49:39 form a question and know what they're looking for are these first level help desk when I go to talk to Enterprises they're like yeah no we don't we don't

49:45 let our help desk type questions into the into the chat bot in order to get data back they're like heck no we don't

49:50 want them to do any of that and so it it you know if the quality of question is coming from a trained source then that's

49:57 way different than Joe Blow going why is my Wi-Fi broken I would say what we found is a use case what we call bring

50:03 your own question like a uh an SE or a customer asking a general Network question those are relatively easy to

50:10 answer the questions we see coming into this customer support team are usually much harder much more specific you know

50:16 once we once someone issues a support ticket it's usually and it's a range of beginner to expert that means that they

50:23 were not able to find the answer inside the documentation usually to to your point Sam u a lot of them are they they

50:30 haven't spent the time they they're either new to us or new to the industry and they just want to get something done

50:36 fast and the fastest way instead of searching and analyzing is hey support can you tell me how I do this right and

50:43 they could search in our documentation do it themselves they open a ticket today and and and maybe Bob if you click

50:49 one more uh essentially what we're trying to do is uh is essentially get to

50:55 support ticket deflection uh um to to leverage um uh this piece for uh so

51:01 essentially today you know uh you could ask these questions within Marvis people

51:06 can selfs serve uh within the Marvis dashboard ask a certain question but if

51:11 let's say they didn't or they they they didn't know how to or they didn't have Marv subscription or they they didn't

51:18 want to do this you know how do I actually when when I open a support ticket and if I type in hey what the

51:24 hell do these three blinking LEDs mean this is is not a a you know Jan you know

51:30 Finance kind of question this is someone in it in helped ask you know saying hey I got to set this IP up or something and

51:36 and you know that that deflection right now we're at 11% so 11% off our

51:43 questions are getting deflected by we provided the right answer and they

51:48 didn't even open the ticket but I would tell you Sam for anyone who complains about

51:54 documentation this put a big bright light on it I bet it does I bet it does 11 11% is right now we have aspirations

52:01 to do a little bit more but uh is this like an internal CH

52:08 GPT um so we are exposing this to our customers through uh uh through uh through uh the Marvis conversation

52:15 interface but we're going to embed it into everywhere right so Juniper documentation will have an llm uh front

52:22 end to it right our support tickets will have an llm front end to it but we got to get the efficacy uh uh to the right

52:29 place where we feel comfortable to throw people over that because again Hallucination is a real thing with open a eye and chat GPT and and you don't

52:36 want to send people on Wild Goose chases right soini uh let's catch us

52:42 up yeah there you go so sudhir said he can speak fast I will speak even faster I have roughly 30 seconds to cover this

52:48 one minute um but the really cool thing when Bob started the session he talked about

52:55 more telemetry coming into Marvis to always answer the question of what is impacting the end user experience we

53:01 talked about how we are pulling zoom in teams labeled user session data what we are doing with this now again taking it

53:09 to the next level is pulling in Telemetry from the data center because often times you have customers who have

53:14 apps SAS applications as well as data center applications fore an end user

53:20 sitting in the branch and the campus when an issue happens how do you know which layer of the network is the

53:26 problem so with what we've done here what used to be Marvis applications now it says

53:31 Marvis data center applications and now I can actually go from my Marvis ofren

53:37 campus into Marv the Data Center and see exactly what the issues I have in my

53:43 Juniper abstract Control Data Center and pull that back into my branch and campus

53:48 solution so for your network operations team Sam you spoke about the level one

53:53 held desk they now have access to Marvis they can actually see when the ticket comes in oh Wireless is clean wir is

53:59 clean van is clean I have a data center problem let me escalate like Jen like you said escalate the data center team

54:04 to say hey the problem is right here it's impacting these five users in my branch do something go do something but are you're not pulling your data center

54:11 gear into the dashboard you're just asking abstra I'm pulling anomalies from abstra into Market but you're not using

54:17 it for like inventory control and you know configuration and all that stuff that stays in AB stays in Abra so from a

54:23 day two perspect perspective from an aops and to and a stance perspec Ive data center was our next sort of

54:28 building block we're pulling that in into the branch and campus solution so again all about the end user all about

54:34 the end user experience how do we give that seamless view to say where is the issue impacting these end users

Show more