Synchronizing Runtime Data

Replication allows applications running on two different Routing Engines or Multiservices PICs to copy their runtime data from a master to a backup by placing the data in the libjunos-sync subsystem.

The data is always sent from the master to the backup.

The libjunos-sync library must be linked with your daemon and initialized before it can be used. Libjunos-sync is thread safe, so there could be multiple instances in one process.

The library creates a TCP connection between two peers. If there are multiple instances within one process, there will be multiple TCP connections between two PICs using different ports. Thus, each PIC can pass its own data without affecting others.

Libjunos-sync runs on the Juniper Private Routing Instance.

Using the Replication Library

When you register with libjunos-sync, your application becomes part of a subsystem in which any data queued is sent to from the master's synchronization subsystem to the backup's synchronization subsystem.

An application first calls junos_sync_init() to initialize the synchronization subsystem, passing a pointer to a set of callbacks defined as follows:

typedef struct junos_sync_callbacks_s {
    junos_sync_ack_notify_func_t
        sync_ack_notify_func;          /*  Ack notify routine */
    junos_sync_data_decode_func_t
        sync_data_decode_func;         /*  Data decode routine */
    junos_sync_data_getnext_func_t
        sync_data_get_next_func;       /*  Getnext routine to walk your
                                            replication database */
    junos_sync_conn_status_change_func_t
        sync_conn_status_change_func;  /*  Connection status change
                                           notification routine */
    junos_sync_init_sync_done_func_t
        sync_init_sync_done_func;      /*  Initial sync done notification routine */
    junos_sync_bind_done_func_t
        sync_bind_done_func;           /*  Bind done notification routine */
    junos_sync_log_msg_func_t
        sync_log_msg_func;             /*  Error log message routine */
    junos_sync_malloc_func_t
        sync_malloc_func;              /*  Application-specific malloc routine */
    junos_sync_free_func_t
        sync_free_func;                /*  Application-specific free routine */
    junos_sync_db_clear_func_t
        sync_db_clear_func;            /*  Routine to clear your replication database */ 
} junos_sync_callbacks_t;

The application also provides state as part of the initialization, by supplying the data shown next:

typedef struct junos_sync_state_s {
    bool is_master;             /*  Flag to indicate if it is a master or backup */
    bool no_resync;             /*  Set if application does not want automatic resync */
    uint32_t server_addr;       /*  IPv4 Server address in host byte order.
                                       Supply this value only when is_master is FALSE */                                     
    uint16_t  server_port;      /*  Server port to use; Supply this value only 
                                       when is_master is FALSE */
    uint32_t keepalive_time;    /*  Keepalive time interval in seconds; if there is no 
                                       data/keepalive message received during this period 
                                       of time,the library will declare the TCP
                                       connection as broken. */
} junos_sync_state_t;

Synchronization Workflow

The workflow on a master Multiservices PIC is:

1. Initialize the libjunos-sync subsystem:

junos_sync_callbacks_t sync_callbacks;

junos_sync_state_t sync_state = {
     true,  /* master*/
     false, /* Do automatic resync*/
     NULL,  /* Let system to find the server adress. */ 
     0,     /* find a free port */
     10     /* Keepalive interval - 10 seconds */
};

/* Initialize the libjunos-sync subsystem, remember its context */
junos_sync_context_t *sync_context = junos_sync_init(&sync_state,
                                         &sync_callbacks);
                                         
 

2. Call junos_sync_start() to initiate and listen on a TCP connection. (A connection must be established before the data is queued.)

If the application is declared as a master, the TCP server then binds with a free port in the system and indicates the server address and port to the application through the sync_bind_done_func callback. The system then issues a resync request to synchronize all the existing data in the application database:

junos_sync_start(sync_context);

3. Encode the data to be synchronized in a type-length-value (TLV) structure, junos_sync_tlv_t. That is, the data must be in a linear buffer that also contains other information such as type of the data and operation associated with this data (add/update/delete). (See Example later in this topic for a more complete code snippet.)

junos_sync_tlv_t * tlv_data;

tlv_data = calloc(1, sizeof(junos_sync_tlv_t) + sizeof(replication_data_t));

4. Queue the data by calling the junos_sync_queue() function. This sends a copy of the data to the libjunos-sync subsystem:

junos_sync_queue(sync_context, data, data_len, true);

The workflow on the backup Multiservices PIC is:

1. Get the server port from the name resolution system.

2. Initialize the libjunos-sync subsystem and remember its context:

junos_sync_callbacks_t sync_callbacks;

junos_sync_state_t sync_state = {
    false,        /* backup */
    false,        /* Do automatic resync */
    server_addr,  /* Use the address from service resolution system */ 
    server_port,  /* Use the port from service resolution system */
    10            /* Keepalive interval - 10 seconds */);
    }

junos_sync_context_t *sync_context = junos_sync_init(&sync_state,
                                         &sync_callbacks);

3. Start a connection with the server system:

junos_sync_start(sync_context);

4. The system calls the sync_data_decode_func callback when the data copy is received from the master. The application can then process the received data (normally, add, update, or delete the data from its database).

The syntax for this callback is:

sync_data_decode_func(data, op);

Example

The following code from the IPTV Monitor sample application, which is in your development sandbox at sandbox/src/sbin/monitube-data/monitube-data_ha.c, encodes and queues replication data:

void
     update_replication_entry(replication_data_t * data)
     {
         replication_data_t * current;
         junos_sync_tlv_t * tlv_data;
     
         if(rep_ctx == NULL)
             return;
     
         LOG(LOG_INFO, "%s", __func__);
     
         // create a new tlv with the trailing data
         tlv_data = calloc(1, sizeof(junos_sync_tlv_t) + sizeof(replication_data_t));
         INSIST_ERR(tlv_data != NULL);
     
         tlv_data->tlv_len = sizeof(replication_data_t);
         current = (replication_data_t *)&tlv_data->tlv_value;
     
         // populate data to enqueue
         memcpy(current, data, sizeof(replication_data_t));
     
         replication_data_hton(current);
     
         junos_sync_queue(rep_ctx, tlv_data, false);
     }

The following code from the same application decodes the data on the PIC:

static void
     decode(void * data, size_t len)
     {
         INSIST_ERR(!rep_state.is_master);
     
         if(len == sizeof(replication_data_t)) {
     
             LOG(LOG_INFO, "%s: update message of length %d", __func__, len);
     
             replication_data_ntoh((replication_data_t *)data);
             add_flow_state((replication_data_t *)data);
     
         } else if(len == sizeof(delete_replication_data_t)) {
     
             LOG(LOG_INFO, "%s: delete message of length %d", __func__, len);
     
             delete_replication_data_ntoh((delete_replication_data_t *)data);
             remove_flow_state((delete_replication_data_t *)data);
     
         } else {
     
             LOG(LOG_ERR, "%s: unknown message of length %d", __func__, len);
         }
     }

The data is owned and freed by the synchronization subsystem after it is sent and acknowledged by the backup Multiservices PIC. The synchronization subsytem only guarantees that the data will be sent to the backup; it is up to the application to parse the data and operate accordingly.

The functions that applications call are in your backing sandbox at sandbox/src/junos/lib/libjunos-sync/h/jnx/junos_sync.h. For additional information, see the SDK Library Reference for libjunos-sync.

For complete sample code that performs replication between Multiservices PICs, see the IPTV Monitor sample application in your development sandbox at: sandbox/src/sbin/monitube-data/monitube-data_ha.c.

For sample code that performs data replication between Routing Engines, see the HelloPICs sample application, in your development sandbox at sandbox/src/sbin/hellopics-data/hellopics-mgmt_ha.c.

Note:
If you are reading this as part of an overview of the high availability functionality, go here next: Reliable Configuration Download.

2007-2009 Juniper Networks, Inc. All rights reserved. The information contained herein is confidential information of Juniper Networks, Inc., and may not be used, disclosed, distributed, modified, or copied without the prior written consent of Juniper Networks, Inc. in an express license. This information is subject to change by Juniper Networks, Inc. Juniper Networks, the Juniper Networks logo, and JUNOS are registered trademarks of Juniper Networks, Inc. in the United States and other countries. All other trademarks, service marks, registered trademarks, or registered service marks are the property of their respective owners.
Generated on Sun May 30 20:26:47 2010 for Juniper Networks Partner Solution Development Platform JUNOS SDK 10.2R1 by Doxygen 1.4.5