Using Flow Affinity and Thread Affinity

For an overview of flow, thread, and session affinities, see Flow, Thread, and Session Affinities.

For maximum efficiency, your code should create thread pools bound to the available CPUs. There can then be one or more "accepting threads," which perform the following actions:

  1. Call accept() to get the socket from the kernel.

  2. Use the flow affinity API to query which CPU the socket is bound to.

  3. Use the thread affinity API to transfer control to the appropriate pool of threads.

The following pseudocode illustrates how this works:

main()
{
int cpus, i, s, newconn, boundcpu;

struct thread_pool *thread_pool_array;

cpus = get_number_of_cpus_in_system();

thread_pool_array = malloc(cpus * sizeof(*thread_pool_array));

for (i = 0; i < cpus; i++) {
   thread_pool_array[i] = make_thread_pool();
   }

s = bind_listen_socket(); // make listen socket.

for ( ;; ) {
   newconn = accept(s); // get new connection
   boundcpu = msp_cpu_affinity(flow_selector);   // Call the flow affinity API to get the CPU to which 
                                                 // 'newconn' is bound
   thread_pool_queue_socket(newconn, boundcpu);  // This function calls pthread_attr_setcpuaffinity_np()
                                                 // to send this socket to thread pool 'boundcpu'
  }
}

The following code from the IP Snooper sample application binds a thread to a CPU:

 pthread_attr_t attr;

...

 /* Create session thread and bind it to control CPU. */
    pthread_attr_init(&attr);
    if (pthread_attr_setscope(&attr, PTHREAD_SCOPE_SYSTEM)) {
        logging(LOG_ERR,
                "%s: pthread_attr_setscope() on CPU %d ERROR!",
                __FUNCTION__, cpu);
        goto error;
    }
    if (pthread_attr_setcpuaffinity_np(&attr, cpu)) {
        logging(LOG_ERR,
                "%s: pthread_attr_setcpuaffinity_np() on CPU %d ERROR!",
                __FUNCTION__, cpu);
        goto error;
    }
    if (pthread_create(&thrd->ssn_thrd_tid, &attr,
            (void *)&session_thread_entry, thrd)) {
        logging(LOG_ERR,
                "%s: pthread_create() on CPU %d ERROR!",
                __FUNCTION__, cpu);
        goto error;
    }

This sample application is located at src/sbin/ipsnooper when you install the example in your development sandbox. This code is in the session_thread_create() function in the ipsnooper_server.c file.


2007-2009 Juniper Networks, Inc. All rights reserved. The information contained herein is confidential information of Juniper Networks, Inc., and may not be used, disclosed, distributed, modified, or copied without the prior written consent of Juniper Networks, Inc. in an express license. This information is subject to change by Juniper Networks, Inc. Juniper Networks, the Juniper Networks logo, and JUNOS are registered trademarks of Juniper Networks, Inc. in the United States and other countries. All other trademarks, service marks, registered trademarks, or registered service marks are the property of their respective owners.
Generated on Sun May 30 20:26:47 2010 for Juniper Networks Partner Solution Development Platform JUNOS SDK 10.2R1 by Doxygen 1.4.5