Using the Services SDK jbuf Library for Packet Processing

Because network packets vary widely in size, networking applications need a specific memory management mechanism that is different from those required by the rest of the operating system.

In FreeBSD, the mbuf facility provides this mechanism; the JUNOS software implementation of the same functionality is provided by the jbuf library.

A jbuf is a data structure that describes a block of data that can vary in size depending on its contents. jbufs are used for holding packets, in addition to other data relevant to network protocol handling.

A packet can be comprised of multiple jbufs chained in a singly-linked list, which lets applications add or trim network headers with minimal overhead. This structure lets you link data together that is not stored contiguously without copying the data from one buffer to another. The next node in the chain is specified by the jb_next pointer in the header.

When you are moving more data than will fit in a single packet, jbufs and jbuf chains can, in turn, be combined to form a larger singly-linked list that represents a stream of packets. You can use the jb_scratch field in the jbuf structure, described in the next section, to locate the next packet stream.

Structure of a jbuf

A jbuf consists of a variable-sized header and a data buffer. The header contains metadata about the jbuf. The data buffer's total size, JBUF_SCRATCH_SIZE, is a constant defined in the sandbox/src/junos/lib/libmp-sdk/h/jnx/jbuf.h header file.

The jbuf header has the following fields:

The following header fields are only valid in the first jbuf at the head of a jbuf chain:

Note:
Always use the jbuf accessor functions rather than accessing the fields of the jbuf structure directly.
The data buffer follows the header and is defined as:

    char  jb_scratch[JBUF_SCRATCH_SIZE];

Operations on jbufs

You can perform the following operations on jbufs:

Most commonly, your application will need to copy data into and out of a jbuf. Be careful not to cause the buffer to overflow.

You can also modify jbufs to encapsulate or decapsulate packets by adding or removing headers, or to retrieve data. If you are modifying an existing jbuf, you should also use the checksum functions. If your modifications involve changes to packet header information (for example: protocol field, address information), your application must swallow the packet entirely and reinject it.

Local jbufs are used for packets originated by data-handling applications (the data component). These applications use the jbuf for copying packets that need to be queued for later processing (for example, for IP reassembly).

System-owned jbufs are the same as local jbufs in all other respects.

Encapsulating and Decapsulating a jbuf

To add a header to a jbuf, you call the jbuf_prepend() function. The following code from the jnx-gateway sample application prepends an IP header to a packet. pkt_ctxt is declared to be type jnx_gw_pkt_proc_ctxt_t, the definition for which is shown first:

typedef struct jnx_gw_pkt_proc_ctxt_s{


    struct jnx_gw_data_cb_s*                app_cb;             /* Pointer to the Control Block */
    msp_data_handle_t                       dhandle;            /* Data thread handler */
    struct jbuf                            *pkt_buf;            /* Pointer to the packet received */
    u_int32_t                               ing_vrf;            /* Ingress VRF of the packet */
    u_int32_t                               eg_vrf;             /* Egress VRF of the packet */
    jnx_gw_data_gre_tunnel_t*               gre_tunnel;         /* Pointer to the Gre Tunnel */
    jnx_gw_data_ipip_sub_tunnel_t*          ipip_sub_tunnel;    /* Pointer to the IPIP sub tunnel */
    struct ip*                              ip_hdr;             /* Pointer to the IP Header in the Packet */
    jnx_gw_data_vrf_stat_t*                 ing_vrf_entry;      /* Pointer to the ingress VRF entry */
    jnx_gw_data_vrf_stat_t*                 eg_vrf_entry;       /* Pointer to the egress VRF entry */
    jnx_gw_gre_key_hash_t                   gre_key_info;       /* Gre Tunnel Key */
    jnx_gw_data_ipip_sub_tunnel_key_hash_t  ipip_sub_key_info;  /* IP-IP Sub Tunnel Key */
    jnx_gw_gre_encap_header_t*              ip_gre_hdr;         /* Pointer to the Outer IP&GRE Header */
    jnx_gw_ipip_encap_header_t*             ipip_hdr;           /* Pointer to the Outer IP-IP header */
    jnx_gw_stat_type_t                      stat_type;          /* Pointer to the stat type to be incremented */
}jnx_gw_pkt_proc_ctxt_t;



...


if (pkt_ctxt->gre_tunnel->tunnel_type == JNX_GW_TUNNEL_TYPE_IPIP) {
    
    /* We have a pre-computed IP header in the IP-IP Tunnel
    * to be appended to the packet
    */
    pkt_ctxt->pkt_buf = jbuf_prepend(pkt_ctxt->pkt_buf,
                                           sizeof(struct ip));

    /* Copy the outer IP header to the packet start */
    jbuf_copy_from_buf(pkt_ctxt->pkt_buf, 0,
       sizeof(struct ip), (char*)&pkt_ctxt->gre_tunnel->ip_hdr);

Removing a header or trailing encapsulation fields can be done using the jbuf_adj() function, which adjusts a jbuf's total length by the number of bytes you specify. The following code from the sample application removes the outer IP header and GRE header from the packet and then passes the packet to another function to process the IP layer.

decap_len in the following code is calculated as follows when the GRE packet is processed:

decap_len = (pkt_ctxt->ip_hdr->ip_hl << 2) +  GRE_FIELD_WIDTH;

    jbuf_adj(pkt_ctxt->pkt_buf, decap_len);
    
    /*
    * Get the information about the Egress VRF and how to encapsulate the rcvd
    * packet
    */
    pkt_ctxt->eg_vrf_entry = pkt_ctxt->gre_tunnel->eg_vrf_stat;
    
    /* Get the pointer to the inner IP Header */
    
    pkt_ctxt->ip_hdr =
    jbuf_to_d(pkt_ctxt->pkt_buf, typeof(pkt_ctxt->ip_hdr));
    
    /* Do the processing for IP Layer */
    if (jnx_gw_data_process_ip_packet(pkt_ctxt, TRUE) == JNX_GW_DATA_DROP_PKT)

Making jbuf Data Contiguous

After execution of jbuf_prepend() and jbuf_copy_from_buf(), the packet data could be placed in physically discontiguous memory. You can use jbuf_pullup() or jbuf_pulldown() to make sure that your memory references cover the entire length of the data to be read in the same jbuf. A sample function call is

pkt_ctxt->pkt_jbuf = jbuf_pullup (pkt_ctxt->pkt_jbuf, pkt_ctxt->pkt_jbuf->jb_total_len);

It is then safe to typecast the memory area of such a length by calling jbuf_to_d(), as the following sample illustrates.

jb = pkt_ctxt->pkt_jbuf;

if (jb->jb_len < sizeof(struct ip))
    {
        if ((jb = jbuf_pullup(jb, sizeof(struct ip)) == NULL)
        {
           syslog(LOG_EMERG, "jbuf_pullup returned NULL\n");
        }
    }

    pip = jbuf_to_d(jb, struct ip*);

    printf("SIP=0x%x -> DIP=0x%x\n", pip->ip_src, pip->ip_dst);
    printf("LEN=0x%x TOS=0x%x\n", pip->ip_len, pip->ip_tos);
    printf("ID=0x%x FRGOFF=0x%x\n", pip->ip_id, pip->ip_off);
    printf("TTL=0x%x \n", pip->ip_ttl);

Transmitting a jbuf

Before transmitting a jbuf, you need to set the jb_xmit_subunit field. The jb_xmit_subunit field is an overloaded field: applications can specify either the outgoing VRF or the subunit.

If you do not know the value to set, use the functions in libvrfutil (defined in sandbox/src/lib/libvrfutil/h/jnx/vrf_util_pub.h as follows:

  1. Call vrf_getindexbyvrfname() to return the VRF index according to the interface name or the VRF name

  2. Call jbuf_setvrf() to set the value that is returned. The kernel will use the first available subunit in the VRF for transmission.

If you know the subunit, use code like the following:

jb->jb_flags &= ~JBUF_FLAG_XMIT_VRF;
jb->jb_xmit_subunit = xmit_subunit;

The following examples show some configurations with and without known VRFs or subunits, and correlate them to your application code.

Configuration Using Next-Hop Route-Based Forwarding

The topology for this example is as follows:

                 [                        Routing Engine                    ]
    
                 [                 Control MS PIC at ms-5/1/0               ]

IPv4 client ---> [   I/O MS PIC          Data MS PIC         I/O MS PIC     ] ----> IPv4 server
                     at so-5/1/1         at ms-1/0/0         at so-5/1/2

The configuration is as follows:

 user@host# show system
  system {
      extensions {
          providers {
              jnx;
          }
      }
  }
  syslog
  {
    file {
        messages {
              any;
              debug;
          }
     }
  }

[edit]
user@host# show chassis
fpc 1 {
    pic 0 {
        adaptive-services {
            service-package {
                extension-provider {
                    control-cores 1;
                    data-cores 7;
                    package jnx-test-ctrl;
                    package jnx-test-data;
                }
            }
        }
    }
}

fpc 5 {
    pic 1 {
        adaptive-services {
            service-package {
                extension-provider {
                    control-cores 7;
                    data-cores 1;
                    package jnx-test-ctrl;
                    package jnx-test-data;
                }
            }
        }
    }
}



[edit]
user@host# show interfaces
 so-5/1/1 {
    unit 0 {
        family inet {
            address  10.10.10.10/24;
    
    sonet-options {
        fcs 32;
    }
}

so-5/1/2 {
    unit 0 {
        family inet {
            address  20.20.20.20/24;  
        }
    sonet-options {
        fcs 32;
    }
}


ms-1/0/0 {
    unit 0 {
        family inet;
    }
    unit 1 {
        family inet;
    }
}

ms-5/1/0 {
    unit 1 {
        family inet;
        {
            address  5.5.5.20/24;  
        }

}

[edit]
user@host#show policy-options
policy-statement dummy {
    then reject;
}

The following are optional settings for this configuration:

user@host#show routing-options

static route {
    30.31.32.33 {
        next-hop
           ms-1/0/0
        }
}

static route {
    100.100.100.100 {
        next-hop
           so-5/1/1
        }
}

jbuf Calls in Your Application

The following jbuf code appears in your application for this scenario.

This code forewards ingress packets in jbufs to a certain egress ms- interface subunit:

/* init xmit using current ms interface subunit number */

  static inline void jbuf_setvrf(struct jbuf *jb, if_subunit_t msif_subunit) {

  jb->jb_flags &= ~JBUF_FLAG_XMIT_VRF;
  jb->jb_xmit_subunit = msif_subunit;
 }

A sample function call is

jbuf_setvrf(jb, 0) /* remote client loopback */

Configuration Using Virtual Router Forwarding

The topology for this example is as follows:

                 [                        Routing Engine                    ]
    
                 [                 Control MS PIC at ms-5/1/0               ]

IPv4 client ---> [   I/O MS PIC          Data MS PIC         I/O MS PIC     ] ---> IPv4 server
                     at so-5/1/1         at ms-1/0/0         at so-5/1/2
                     VRF vrf-1                               VRF vrf-2

The configuration is

 user@host# show system
  system {
      extensions {
          providers {
              jnx;
          }
      }
  }
  syslog
  {
    file {
        messages {
              any;
              debug;
          }
     }
  }

[edit]
user@host# show chassis
fpc 1 {
    pic 0 {
        adaptive-services {
            service-package {
                extension-provider {
                  control-cores 1;
                  data-cores 7;
                  package jnx-test-ctrl;
                  package jnx-test-data;    
                }
            }
        }
    }
}

fpc 5 {
    pic 1 {
        adaptive-services {
            service-package {
                extension-provider {
                    control-cores 7;
                    package jnx-test-ctrl;
                  }
            }
        }
    }
}



[edit]
user@host# show interfaces
 so-5/1/1 {
    unit 0 {
        family inet {
            address  10.10.10.10/24;
    
    sonet-options {
         fcs 32;
    }
}

so-5/1/2 {
    unit 0 {
        family inet {
            address  20.20.20.20/24;  
        }
     sonet-options {
        fcs 32;
    }
}


ms-1/0/0 {
    unit 0 {
        family inet
    }
   unit 1 {
        family inet
    }
}

ms-5/1/0 {
    unit 1 {
        family inet
        {
            address  5.5.5.20/24;  
        }

}

[edit]
user@host#show policy-options
policy-statement dummy {
    then reject;
}

In the following settings, the ingress routing instance is vrf-1 and the egress routing instance is vrf-2. The final static route is optional.

user@host#show routing-instances

routing-instance {
    vrf-1 {
        instance-type vrf;
        interface ms-1/0/0.0;
        interface so-5/1/1.0;
        vrf-import dummy;
        vrf-export dummy;
        route-distinguisher 1:1;
        routing-options {
            static {
                route 5.5.5.5 next-hop ms-1/0/0.0;
                route 100.100.100.100 next-hop so-5/1/1;
        }
    }
}

routing-instance {
    vrf-2 {
        instance-type vrf;
        interface ms-1/0/0.1;
        interface so-5/1/0;
        vrf-export dummy;
        route-distinguisher 1:2;
    }
}

In this example, the traffic received on any I/O PIC interface included in vrf-1 (so-5/1/1) is routed by performing a route lookup in the vrf routing table. Traffic is forwarded to the data PIC using optional static routes. Any routes in the inet-0 table are ignored.

Traffic that is received on any other I/O PIC interface not included in vrf-1 (so-5/1/0) is routed using a route lookup in the inet-0 table.

Routes are added to the inet-0 table in either of two ways:

jbuf Calls in Your Application

Following are the options for using jbuf calls for this scenario.

/* init xmit using egress routing instance (vrf) with specific id */
 static inline void  jbuf_set_xmit_subunit (struct jbuf *jb, if_subunit_t vrf_id) {
	jb->jb_flags |= JBUF_FLAG_XMIT_VRF;
	jb->jb_xmit_subunit = vrfid;
}

A sample function call is as follows:

(void) jbuf_setvrf(jb, 0); /* for client remote loopback */

/* init xmit using egress routing instance (vrf) uniquely identified by specific vrf name and protocol family */ static inline void jbuf_set_xmit_vrf_name(struct jbuf *jb, const char *lrname, int af) {
    jb->jb_flags |= JBUF_FLAG_XMIT_VRF;
    jb->jb_xmit_subunit = vrf_getindexbyvrfname(const char *vrfname, const char *lrname, int af); }

Sample function calls:

(void)jbuf_set_xmit_vrf_name("vrf-1", NULL, AF_INET); /* for client remote loopback */
(void) jbuf_set_xmit_vrf_name("vrf-2", NULL, AF_INET); /* for forwarding to the server */

/* init xmit using egress routing instance (vrf) uniquely identified by specific ifname and protocol family */ 
static inline void jbuf_set_xmit_ifname(const char *ifname, int af) {
   jb->jb_flags |= JBUF_FLAG_XMIT_VRF;
   jb->jb_xmit_subunit = vrf_getindexbyifname(const char *ifname, int af); }

Sample function calls:

(void)jbuf_set_xmit_ifname("vrf-1", AF_INET); /* for client remote loopback */
(void)jbuf_set_xmit_ifname("vrf-2", AF_INET); /* for forwarding to the server */

Usage Considerations for jbufs

When using jbuf functions, note the following considerations.

Tracking jbuf CPU Usage

The system allocates jbufs per CPU pool. jbufs are allocated at kernel initialization, before FIFO registration. This affects all jbuf_get_chain() and jbuf_free() operations, as well as other jbuf calls that indirectly allocate jbufs.

The system could allocate from this local jbuf pool dynamically (no limits at this time) for IP fragmentation, IP defragmentation, or when using internal system plugins. Therefore, you should actively monitor usage of jbufs to ensure that enough jbufs are available for your monitoring or transit application.

Applications can track jbuf CPU usage according to the configuration (pools are shared equally among the pool of allocated data CPUs). jbufs are evenly distributed to data CPUs as follows:

	   Number of cores    Number of jbufs per data CPU
     ---------------    ----------------------------

	    1                  2500 (7500/ 3)
	    2	                 1250 (7500/ 6)
 	    3	                 834 (7506/ 9)
	    4	                 625 (7500/12)
	    5	                 500 (7500/15)
	    6	                 417 (7506/18)
	    7	                 358 (7518/21)

The debug CLI ( mspdbg-cli ) also indicates the total number of jbufs. For example:

	    MSP-DEBUG> show msp jbuf            
	    Jbuf ref count    : 0
	    Local jbuf count  : 7518   <<< total number of jbufs
	    Jbuf max ref count: 6500

Allocation per CPU pool enhances performance for the following operations, which are performed with no locking:

You can use the following code to retrieve the number of per-CPU jbufs allocated on each pinned thread:

int get_num_data_cpus ()
{
    int 			num_dcpus = 0;
    int       next_dcpu = MSP_NEXT_END;

    do {
        next_dcpu = msp_env_get_next_data_cpu(next_dcpu);
        printf("msp_env_get_next_data_cpu ret %d\n", next_dcpu);
		    
		    if (next_dcpu != MSP_NEXT_END) {
            num_dcpus++;
        }    
    } while ( next_dcpu != MSP_NEXT_END );
    
    return num_dcpus;
}

int get_max_local_jbufs()
{
    int             max = 0;
    int             num_dcpus = 0;

    num_dcpus = get_num_data_cpus(); <<< see previous function
    max = jbuf_get_max_jbufs()             / num_dcpus;
    return max;
}

2007-2009 Juniper Networks, Inc. All rights reserved. The information contained herein is confidential information of Juniper Networks, Inc., and may not be used, disclosed, distributed, modified, or copied without the prior written consent of Juniper Networks, Inc. in an express license. This information is subject to change by Juniper Networks, Inc. Juniper Networks, the Juniper Networks logo, and JUNOS are registered trademarks of Juniper Networks, Inc. in the United States and other countries. All other trademarks, service marks, registered trademarks, or registered service marks are the property of their respective owners.
Generated on Sun May 30 20:26:47 2010 for Juniper Networks Partner Solution Development Platform JUNOS SDK 10.2R1 by Doxygen 1.4.5