Help us improve your experience.

Let us know what you think.

Do you have time for a two-minute survey?

 

Slave Collector Installation for Distributed Data Collection

 

When you install NorthStar Controller, a master collector is installed, for use by Netconf and SNMP collection. You can improve performance of the collection tasks by also installing slave collector workers to distribute the work. Each slave collector worker starts a number of worker processes which is equal to the number of cores in the CPU plus one. You can create as many slave collector servers as you wish to help with collection tasks. The master collector manages all of the workers automatically.

Slave collectors must be installed in a separate server from the NorthStar Controller. You cannot install slave collectors together with the NorthStar application in the same server.

To install slave collectors, follow this procedure:

  1. On the slave collector server, run the following:
    rpm -Uvh rpm-filename
  2. On the slave collector server, run the collector.sh script:
    [root@ns-slave-coll]# cd /opt/northstar/northstar_bundle_x.x.x/
    [root@ns-slave-coll northstar]# ./collector.sh install

    The script prompts you for the NorthStar application IP address, login, and password. If the NorthStar application is in HA mode, you need to provide the VIP address of the NorthStar application. The IP address is used by the slave collectors to communicate with the master collector:

  3. Run the following command to confirm the slave collector (worker) processes are running:
  4. Optionally, use the config_celery_workers.sh script to change the number of workers that are installed.

    The collector.sh script installs a default number of workers, depending on the number of CPU cores on the server. After the initial installation, you can change the number of workers installed using the config_celery_workers.sh script. Table 1 shows the default workers installed, the number of total celery processes started, and the amount of RAM required.

    Table 1: Default Worker Groups and Processes by Number of CPU Cores

    CPU Cores

    Worker Groups Installed

    Total Worker Processes

    Minimum RAM Required

    1-4

    4

    8-20

    (CPUs +1) x 4 = 20

    1 GB

    5-8

    2

    12-18

    (CPUs +1) x 2 = 18

    1 GB

    16

    1

    17

    (CPUs +1) x 1 = 17

    1 GB

    32

    1

    33

    (CPUs +1) x 1 = 33

    2 GB

    To change the number of workers, run the config_celery_workers.sh script:

    Use the -w worker-groups option to add a specified number of worker groups. Since this installation is on a server dedicated to providing distributed data collection, you can increase the number of workers installed up to the server storage capacity to improve performance. The following example starts six worker groups: