Day 0: Designing your AI data center with Juniper Networks

Introduction

0:00 welcome everybody i'm Kyle Baxter i'm

0:02 going to be talking about how you design

0:07 your AI ML fabric with

0:11 Abstra and so many of you aware probably

0:15 challenges that that are out there um

0:18 there's the AI space is moving so fast

The challenge

0:22 that there's a demand to deploy it now

0:25 and the challenge is is it's the network

0:27 the the the industry is moving so fast

0:30 that there's not enough adequate train

0:32 resources out there when they are

0:34 there's hundreds and hundreds of pages

0:35 of this documentation how to configure

0:36 this or that or or this pro

0:39 protocol and there's no time to read up

0:43 on that become an expert and that's

0:45 where Aster can come play and be your

0:48 expert assistance to help you deploy um

expert assistance to help you deploy um

0:51 and so when we're talking about

0:53 deploying we're we're going to be

0:54 looking at rail optimized designs we've

0:58 covered this in some other sessions on

0:59 why rail optimized designs this came

1:01 from Nvidia designs to to make the most

1:04 out of the network to get the most most

1:06 throughput reduce latency reduce

1:09 congestion all that get all those

1:11 benefits

1:13 but when designing it you know how many

1:16 of you could design it and say you know

1:18 give me a give me a design and give me a

1:20 cable map right

1:21 now right but with Abstra I can show you

1:25 how to do it in five minutes or

1:27 less who wants that

Designing with Abstra

1:30 yes I do and so we start with a simple

1:34 simple template designer where with just

1:36 a few basic inputs that that imagine as

1:40 an architect you're looking at this

1:42 you're probably going to know all these

1:43 you're going to know how many servers I

1:45 want to use how many GPUs per server

1:47 it's going to be eight but maybe you

1:49 don't want to use all um but that is

1:51 configurable um how many servers per

1:53 stripe what is your GPU nick speed what

1:56 is your over subscription ratio you want

1:58 so pretty simple things that any

1:59 architect will know and They can put

2:02 this in the input and it will tell them

2:04 exactly what are your design options

2:07 based on those inputs and it could be

2:10 different things you can see there's

2:11 some that are using you know PTX 10K16s

2:14 as as leaves or do you want a QFX 5240

2:17 it gives you options that say depending

2:19 on what devices you want to use or how

2:21 many of them want you can see some of

2:22 them have can be different quantities

2:24 potentially and so you can pick the

2:26 devices that make the sense for you that

2:29 match your design

2:32 and you can preview that to see what is

2:34 that going to look like how is that all

2:36 going to be connected so they can say

2:38 Yep that makes sense that's the

2:40 template design I want to use. And then

2:43 once you save that design you can have a

2:45 list of saved designs you can see I have

2:46 different designs i have a,000 GPU a

2:49 4,000 GPU an 8,000 GPU some collapsed

2:52 fabric that are smaller like 128 GPUs i

2:54 have different designs that I want to

2:56 use that I can then take those in in

2:59 seconds i can deploy them and have them

3:02 ready to

3:04 go so it only takes me just a few

Building a cabling map

3:07 minutes to to design it and deploy it

3:10 but then what do you need to do next now

3:12 what like that's great you built me a

3:14 template um you know you picked the

3:16 devices I need that's great but what's

3:19 next well the next thing you're probably

3:22 gonna want to do is you're gonna want to

3:23 have a cabling map to know how to

3:24 connect all these things because there's

3:26 a lot of connections when you're talking

3:27 about you know like a 4,000 GPU cluster

3:30 there's a lot of connections and so what

3:33 we can do in Abstra is even before you

3:36 have hardware that shows up before

3:38 you've ordered anything you can you can

3:41 build that design in Abstra without the

3:43 hardware where you can have that cabling

3:45 map and you can see what it is you can

3:48 export it whether you want it in JSON or

3:50 CVS cs CSV not CVS CSV um depending on

3:54 how you want to use it and obviously

3:56 it's going to be a big table because

3:58 it's a lot of things but it allows you

4:00 to take that cabling map and then give

4:02 it to whoever is going to be plugging in

4:04 the hardware when it arrives so you have

4:06 that full design but even if somebody

4:10 you know goes and doesn't plug it in

4:12 exactly right even afterwards once it is

4:15 connected we can use LDP to see did it

4:17 actually get plugged in the way it was

4:19 intended right because we that's the

4:21 whole intentbased networking design we

4:22 have in Abstra is we validate and

4:24 continuously validate that the running

4:27 network is operating as intended

4:28 including cabling map including config

4:30 drift all those things we can validate

4:32 and continuously validate so we can make

4:35 sure that Kal map's right will you also

4:37 like put up a picture of the switch and

4:39 highlight the port and say this one's

4:41 wrong yes thank you yes we do that we we

4:45 have the the K map and we can show

4:46 exactly which ones are wrong um and show

4:48 the mapping of hey you know this spine

4:50 this leaf or this leaf to these GPU

4:52 servers we can map all that and show all

4:54 that absolutely excellent yeah great

4:57 question and so that cabling map is is

5:00 really nice as well but the other thing

5:01 we can do is we can look at the

5:04 configuration for those that still love

5:06 looking at CLI and don't trust that

5:08 automation is going to deploy what they

5:10 expect you can go look at the CLI and

5:12 you can go look at the config and see

5:14 everything that that is generated um and

5:16 then back to our multi- vendor point is

5:18 we can generate for multi vendors so not

5:20 only just Cisco but we can generate it

5:22 for other vendors as well including um

5:24 Cisco and Arista and Sonic based devices

5:27 we can cover those and so it allows you

5:30 to see the cabling map see the

5:32 configuration all before you've ever

5:33 ordered a single piece of

Demo

5:37 hardware so let's see a little demo of

5:40 this so I'm going to show you here in

5:42 just a couple minutes how we take an

5:44 input on how many GPUs we have build a

5:48 template deploy that template get the

5:50 cabling map and we're done i mean that's

5:54 amazing we can do that in five minutes

5:56 so let me go over here and I'm going to

6:01 play a little video so that way we can

6:04 um walk through it so I'm going to go to

6:06 Abstra i'm going to go to templates and

6:08 there's a create an AI cluster template

6:11 and so just like we talked about earlier

6:13 I can have inputs and automatically

6:16 updating on the top you saw the number

6:17 of GPUs automatically changed as I added

6:19 different settings and then I can see in

6:22 here the different combinations so again

6:25 you can see some different combinations

6:27 of if I want to use you know one of the

6:29 PTX chassis I can reduce the number of

6:31 devices but if I don't want to buy the

6:33 chassis I could buy 5240 from the QFX

6:36 line and have those question yes brian

6:39 Martin Signal 65 um you have servers per

6:42 stripe what's a stripe

6:46 so so simply a stripe is is a collection

6:50 of of rails so it's it's how many do you

6:54 want in that in that stripe to make it

6:56 up because that rail is is a single

6:58 combination of all the the GPUs

7:00 connected and then a stripe is is a

7:02 combination of the rails okay so that's

7:05 basically the number of ports that'll be

7:07 used for a given rail i would go with

7:10 the servers yes okay thank you yeah

7:12 excellent thank you

7:14 Brian so now that when I decide I want

7:18 hey I want to use all 5240s i don't want

7:21 to use a PTX for whatever reason i maybe

7:22 don't want to buy a chassis um I can

7:25 look and see here and I can hover over

7:27 and see how everything's all connected

7:29 um it's a lot of connections because

7:30 we're talking about a thousand GPUs so

7:32 there's a bunch of servers in there that

7:34 are connected but we can see how it's

7:35 all connected we can see then here the

7:38 number of spines the number of of rack

7:40 types we have we can see the devices

7:42 it's picked it's 5240s as we expected so

7:45 then I'll save it give it a name and

7:49 we'll save that template so that way we

7:52 can use this to rubber stamp and just

7:54 repeat the design over and over if I

7:56 wanted to um and so we had that design

7:59 saved so now I'm going to go to my

8:01 blueprints and in abstra terminology a

8:04 blueprint is your your your design of

8:08 what you want your data center to look

8:10 like so just like a blueprint of your

8:12 house is the model of your house looks

8:14 and gives you all the specs a blueprint

8:16 here in the data center is blueprint of

8:18 how you want your fabric to look and

8:21 operate so I'm going to give it a name

8:24 and I'll pick my template so like I said

8:27 in just a few clicks we can deploy that

8:29 template and look it looks exactly like

8:31 what we had in the template no surprise

8:34 um so it matches exactly what we built

8:36 in our template and we can now go

8:39 quickly in just a simple button create

8:42 and it's going to go scaffold that whole

8:44 design for us it's going to use some

8:47 default pools of like IP addresses and

8:49 ASN's you can configure that to however

8:51 you want um but it defaults to some

8:53 default pools so that way you have

8:54 something to start with and then we can

8:57 go right into our staged area because in

9:01 Astra terminology we have the concept of

9:03 stage and active as the names indicate

9:06 active is what you actually have

9:07 deployed it's what's active stage is

9:10 where you can play with the design see

9:12 what it's going to look like and this is

9:14 where we can look at things like the

9:15 cabling map and the configuration all

9:18 before we even ever have a single piece

9:20 of

9:21 hardware and so we'll then look at the

9:24 links we can see the the cabling map um

9:28 we can then export it like we already

9:30 saw and talked about into CVS or JSON

9:33 files we can see exactly what interfaces

9:35 everything is connected on what IP

9:37 addresses they'd be used so it gives

9:39 basically an exact blueprint here that

9:41 we built in just 5 minutes and say "Hey

9:43 I'm already ready for building you know

9:46 a thousand GPU server here's the cabling

9:48 map." When you're ready to order the the

9:50 hardware order it plug it in as I

9:52 specified and it's just going to work

9:56 so question on the back backing up a

9:59 second yeah please have I already made

10:01 some decisions on the racks and all of

10:04 the DJX systems that I need in this

10:06 particular fabric right because if this

10:07 is from the network operator perspective

10:10 how have I ingested what systems I'm

10:12 going to use in this fabric or are you

10:14 saying I build the fabric before I

10:15 decide what DGX systems I'm going to

10:17 plug into it where's where's the the

10:19 deciding factor of how I build the

10:21 fabric mhm mhm so so we're focused more

10:24 on the on the networking side obviously

10:27 or networking company but um but we do

10:29 model also and we do have agents that

10:32 live on the GPU servers and that's how

10:34 when we get to it we'll show some of the

10:36 the metrics we get on the the nick and

10:38 the GPU and and we can monitor that um

10:42 so we we model it as what we call like a

10:45 generic system so that way it's anything

10:47 that that can that can fit that kind of

10:50 idea which is needs to be you know some

10:52 kind of it's like all of them are like

10:54 Abuntu servers so the agent lives on

10:56 that Abuntu server and we can then

10:58 connect to it um so really what you're

11:00 looking for is what is the nick speed

11:03 that you need to get there um beyond

11:05 that whatever you know DGX you know

11:08 server you have or GPU server you have

11:10 is is transparent to that that process

11:13 so there's some sort of management plane

11:15 connectivity right to those systems that

11:17 I'm ingesting here so I've got it like a

11:19 data feed perspective so when you when

11:20 you actually get the hardware you need

11:22 to then assign it in in in abstra and

11:26 say "Hey this for like the the switches

11:28 this serial number matches this leaf

11:30 that I want to use corresponds to this

11:32 correspond." Yeah and then the same

11:33 thing with the GPU servers you

11:35 correspond hey this one you know goes to

11:37 this location in in that template design

11:40 so there is obviously some assignment

11:41 that you have got the GPU operator

11:43 design as well that would kind of mirror

11:45 perfect thank you yeah yeah good

11:47 question so while you're configuring are

11:49 you also then choosing which of those

11:52 many uh load balancing techniques we

11:54 just learned about or does it

11:55 automatically default to the the RDMA

11:58 aware one or Yes yes we do and we'll get

12:02 definitely more into that um but but yes

12:04 we do have load balancing configuration

12:06 um so this first section was really all

12:08 about how you get the initial design

12:11 built and then to a cabling map and be

12:14 able to see the config so that way

12:15 you're you're ready to order and start

12:17 you know plugging in we'll get right

12:18 into in the next session on on how we do

12:21 things like load balancing and how we do

12:23 assigning virtual networks all those

12:25 things to to actually get into some of

12:27 the what we call like the day one where

12:29 you're starting to um get into not just

12:31 the the design but into kind of in more

12:33 in the deployment phase you mentioned so

12:37 go ahead spritzky CDC um just curious

12:40 are we're here basically looking right

12:43 GPUs are the problem in this situation

12:46 right but are we also going to take a

12:48 look at well I've got this database

12:50 server I've got to plug in I've got

12:51 storage I've got other you know possibly

12:54 a public network that I need to plug in

12:58 uh does this does appra also handle say

13:01 security aspects you know and and uh

13:04 things like Okay yes so so so back to my

13:08 point earlier for no matter what kind of

13:09 network it is we can we can manage it

13:11 we're focused a lot today because it's

13:12 AI infra field day on how we would

13:15 deploy you know the backend training

13:18 that's what's new but but we've been

13:19 doing it for years of any other kind of

13:21 network like you said storage network or

13:24 even you know a front-end inference

13:25 network which follows a more traditional

13:27 design um that's not you know not always

13:29 the rail optimized you know kinds of

13:31 designs so absolutely we can do that and

13:34 you can use it in the same single abstra

13:36 instance um you can have multiple

13:38 blueprints for like your your backend

13:40 your front end your storage your and so

13:42 you have then you can move around just

13:44 in your single UI and be able to see

13:46 that and even do things like data center

13:48 interconnects when that's necessary that

13:50 was my question yeah and then on on your

13:52 point on security you know we have

13:53 security policies that can limit you

13:55 know what do you want to talk to what or

13:56 not talk to what and then we can even

13:58 look at flow data to see you know is

14:01 there potentially something that's

14:03 deviating or looks malicious inside the

14:05 network so we talked a lot in the past

14:07 session about you know how do we you

14:08 know prevent some of that stuff you know

14:10 going in and out of the network okay but

14:11 in the network we can look at east west

14:13 traffic and to see you know is there

14:15 potentially something malicious going on

14:18 to inspect that being by looking at some

14:20 of the the flow data like like SF flow

14:21 packets and things like that got it okay

14:24 does this um yeah take into account the

14:27 specific type of nicker you using or

14:29 what your interfaces are going to be or

14:31 is that just assumed that you know what

14:33 that is

14:35 it in it does take in account some we're

14:38 not doing nick configuration at the

14:40 moment um so we're more looking at

14:42 getting metrics and information from the

14:45 nick so we can look at things like

14:47 congestion and out of sequence and and

14:50 then also the health of the general

14:51 server um nick configuration is

14:54 something we're we're looking at in the

14:55 future okay that we can get to so so in

14:58 that when it gets to configuration yes

15:00 then that would obviously matter what

15:02 kind of nick it is um but um but when

15:04 we're looking at you know all the the

15:06 Nvidia GPUs which is the the big play in

15:09 the the space um we can get all the

15:12 common GPU metrics and nick metrics from

15:15 the the Nvidia servers okay and the the

15:18 second question along those lines is if

15:20 you're coming up with a a cabling

15:22 diagram um if you actually know the

15:26 layout of your data center can you also

15:27 come up with cable lengths out of the

15:29 configuration

15:31 we don't look at cable lengths um is if

15:35 you have a cabling diagram yes next the

15:37 next question you have is what are the

15:38 lengths of the cables yes yes that's a

15:41 very valid question um we we don't have

15:44 that yet in the product for like cable

15:45 lengths um but we have like where what

15:48 should plug into what you know interface

15:50 this on this on this interface on that

15:51 one um but that's that's a a great

15:54 question because obviously you're going

15:55 to need to look at and you know how far

15:57 cable links especially when you start

15:58 getting into some of these high-speed

15:59 ones that that cable lengths can make a

16:01 difference okay well I I figured it

16:03 would be like the next progression of of

16:05 what of the information you have would

16:07 be show me how the data center is laid

16:09 out mhm yeah yeah i mean we can show it

16:13 from like the physical connections of

16:15 what's connected like I said but not the

16:16 the cable links yet i think there was

16:19 another question or a couple over here

16:20 that

16:22 So quick question so you mentioned that

16:24 sits on top of iuntu or you can connect

16:26 to iuntu do you need to have versions

16:28 for Red Hat and for SUSA and for all

16:31 sorts of things our our agent is is

16:33 agnostic to the That's all I'm going to

16:35 cover you okay you just mentioned Auntu

16:37 servers because that's I mean Abuntu is

16:39 typically what a lot of the NVIDIA GPUs

16:42 or servers are running and they're

16:43 they're I think they're all on Abuntu at

16:45 the moment um but but our agent is

16:48 agnostic to to any Linux version it can

16:50 be deployed on other things mhm okay

16:52 maybe you get um something uh after you

16:56 stage you the design thereation process

16:59 of the what is connected at so probably

17:02 partially the answer it's okay so yeah

17:05 yeah if is there any validation process

17:08 or after stage yes yes so we do lots of

17:12 validation um I'm actually going to show

17:14 it in the the next one on how we catch

17:16 errors because the first one I'm going

17:18 to show is how we catch errors of things

17:20 where like virtual networks aren't

17:21 assigned okay Um and so we'll see some

17:23 of that and how we we have validation in

17:25 our stage before we push it to the

17:27 active to that production and we prevent

17:29 you from pushing bad bad code to your

17:33 production network oh yeah gotcha but

17:35 yes yes we do all that validation we can

17:38 also do um commit checks um so if you're

17:41 familiar with the Junos CLI you can do

17:42 commit checks on on Junos commands we

17:44 can help you run all that um on the the

17:46 staged area before you deploy it okay

17:49 okay so in addition to you know other

17:50 validations that we can do we can do

17:51 things like commit checks and and

17:53 validate you know syntax and all that

17:55 okay got it yeah you got another one

17:59 what's under the rack tab the racks tab

18:02 the Oh the racks tab is that Is that

18:05 where we figure out how many servers per

18:08 rack per rack yes good exactly we can

18:11 map it out physically yes yeah yeah yeah

18:14 yeah yeah yeah we can map it out

18:15 physically just not the cable links yet

18:17 yeah yet

18:21 okay all

18:23 right so um so a couple more things that

Flexibility

18:27 I wanted to point out that um we asked

18:30 what if I need some more flexibility um

18:33 a common one we get asked on is what if

18:36 my server counts low enough that um I

18:40 don't want to have um or I only don't

18:42 want to have multiple stripes i want to

18:44 do everything in one stripe so meaning

18:46 having leaves only and no spines um so

18:50 then everything connects and and runs

18:52 through those leaves and the NVL link on

18:55 the the Nvidia side and we can do that

18:58 as well um so we can build that um in

19:00 the template designer where you can have

19:03 um 128 GPU servers to get to a thousand

19:07 GPUs but only have eight leaves and then

19:10 you don't have to have the the spine

19:12 layer on top for additional connectivity

19:14 so it saves you some on hardware cost

19:15 optics cost all those things so if you

19:17 have a small deployment we can do things

19:19 like that to give you that kind of

19:20 flexibility kind of like a a collapsed

19:22 fabric in a way where we're collapsing

19:24 everything in and reducing from a from a

19:26 a three-stage to a you know a single

19:28 single layer would that be the collapsed

19:30 rail design click button there yes um so

19:33 you have L3 claus and collapsed rail no

19:37 straight rail or rail optimized or is

19:39 that what class is rail class is rail

19:42 okay yeah so no straight cloths yes in

19:45 the solution okay yeah yeah thanks

19:49 and then this gets back to the other

19:51 point we're talking about is what if you

19:52 know a lot of people ask what if I want

19:54 to do something else with my you know my

19:56 my my front end my storage network all

19:58 this so you know we can do your typical

20:01 designs that you know aren't the you

20:02 know the rail optimized that we're

20:04 talking a lot about today so we've had

20:06 that in the product for for years so I

20:07 just wanted to make sure we cover that

20:09 so any kind of network design you want

20:10 to do whether it's reoptimized or not we

20:13 can do that as

20:15 well so to summarize what what we've

20:20 what we've seen is Astra can be your

20:24 expert to help you deploy faster we saw

20:27 how we could deploy you know from from

20:30 specs to design to cabling map in a

20:33 matter of minutes and we were able to do

20:36 that all without having a single piece

20:38 of hardware where we could we could look

20:40 at how the cable map how the config is

20:41 all going to look we can validate all

20:43 that before the hardware arrives so we

20:46 know that when the hardware arrives

20:47 we're ready to go plug it in and let's

20:49 start using the equipment and start

20:51 building some some exciting models um so

20:55 any final questions on this section

20:59 i had one info advisor um so you

21:03 mentioned I picked up one you mentioned

21:04 you're doing all this in the contextual

21:06 graph database yes and is that something

21:08 you've developed yourselves or are you

21:10 leveraging uh another product or no it's

21:13 proprietary we built um within within

21:15 Juniper and perfect thank you and I

21:18 might have missed this but what type of

21:20 a uh APIs and SDKs are needed for this

21:22 and if so do you have so we have full

21:26 REST based APIs we also have Python SDKs

21:30 um so whatever you want to build we have

21:33 a lot of people that are doing

21:34 integrations um we also have a Terraform

21:37 provider that's becoming very popular in

21:40 kind of the networking world to to

21:42 simplify and make managing private

21:44 clouds very similar to how you would

21:46 manage public clouds um but then you

21:48 have the advantage of it's your private

21:50 cloud it's your infrastructure but you

21:52 have the same kind of operating

21:53 capabilities um we actually demoed that

21:55 at a a previous field day session um so

21:58 anybody that wants to go look at how

22:00 Terraform works we we've done that

22:02 before as well or you can just simply

22:04 you know search for Juniper apps for

22:06 Terraform and you'll see the the

22:08 Terraform module out there and all the

22:09 documentation out there with it and

22:11 Anible support is that coming yes oh yes

22:13 we do have Anible support as well all

22:15 right um so thank you for bringing that

22:17 up we do have answerable support as well

22:19 for those that uh still want to use that

22:26 one more

22:27 question um in the beginning you

22:30 mentioned that there are typically three

22:33 networks there is the storage network

22:35 the front end network and then the GPU

22:37 network yes

22:39 um when we see all this now uh in that

22:42 blueprint is everything connected to the

22:45 same switches or are these dedicated uh

22:48 separate uh networks so the GPU has

22:51 their own dedicated switches or are

22:54 those three networks all combined

22:56 converged on the same switches they're

22:58 they're typically running on different

23:00 switches different networks but they

23:02 they there is sometimes connections

23:04 between them um or they're sharing

23:06 potentially GPU resources um but they

23:08 are they're thought of and modeled with

23:10 an Abstra as separate network separate

23:12 switches okay mhm and so Abstra is also

23:16 aware of this of these three different

23:18 networks yes okay thanks