Help us improve your experience.

Let us know what you think.

Do you have time for a two-minute survey?

 
 

APM Installation

APM Installation Overview

Juniper Address Pool Manager (APM) is an automated, centralized, container-based cloud-native application that network operators and administrators use to manage IP address resources. APM works with managed broadband network gateways (BNGs) to monitor address pools on BNGs. When the number of free addresses drops below a set threshold, the BNG raises an alarm. The alarm triggers APM to allocate unused prefixes from its global list of prefixes and provision a subset of the prefixes to the BNG as new pools.

Note:

The term BNG in this document also applies to the BNG CUPS Controller.

You can deploy APM on any hardware that meets the requirements. The following sections describe:

  • APM installation requirements

  • How to install APM

  • How to adjust APM setup parameters

APM Installation Requirements

To install APM, you need the following hardware and software:

APM Requirements

APM installs on a Kubernetes cluster comprised of physical or virtual machines (VMs). For availability, you must have at least three nodes in the cluster. APM requires the following minimal resources from the Kubernetes cluster:

Table 1: Cluster Requirements
Category Details

Storage

Storage Class or PVs capable of backing 100 mebibytes (MiB) RWX PVC for configuration

Network load balancer addresses

One for APMi

Node port address

One for optional CLI/SSH access

Container/registry storage

Container/registry storage 2.5 gibibytes (GiB)

Worker node resource consumption (specification)

APM resource consumption on each Worker node:

  • CPU: 8 cores
  • Memory: 2 gibibytes (GiB)

  • Storage: 2.5 gibibytes (GiB)

Jump host

  • Ubuntu version 22.04 LTS or later

  • CPU: 1 core

  • Memory: 8 gibibytes (GiB)

  • Storage: 128 gibibytes (GiB)

  • Installed software

    • Python3-venv

    • The Helm utility

    • The Docker utility

    • OpenShift CLI (if you are using a Red Hat OpenShift Container Platform cluster)

Node specification (minimum of 3 nodes)
  • Operating System:

    • Ubuntu 22.04 LTS (for a BBE Cloudsetup cluster)

    • Red Hat Enterprise Linux CoreOS (RHCOS) 4.15 or later (for an OpenShift Container Platform cluster)

  • CPU: At least 8 cores

  • Memory: At least 16 GB

  • Storage: 512 GB storage partitioned as 128 GB root (/), 128 GB /var/lib/docker, and 256 GB /mnt/longhorn(application data

This specification establishes a cluster that can run APM as well as its companion applications such as BBE Event Collection and Visualization and BNG Controller simultaneously.

Note:

To create the cluster, you can use either of the following applications:

  • BBE Cloudsetup release 2.1.1 or later

  • Red Hat OpenShift Container Platform release 4.15 or later. An OpenShift Container Platform cluster also, requires the following:

    • A container registry

    • A network load balancer with at least one IP Address Pool

    • A storage class named jnpr-bbe-storage

Additional Requirements

The BNG is a Juniper Networks MX Series router, or a Juniper BNG CUPS Controller (BNG CUPS Controller). We recommend that the BNG is running Junos OS Release 24.2R2 or later.

For APM, confirm that you have a juniper.net user account with permissions to download the APM software package. Download and install the APM software from a machine that will not be part of the Kubernetes cluster.

Install APM

Use the procedures in this section to install APM for the first time.

Before you begin, confirm that you have met the requirements for the APM installation.

We recommend that you use a secure connection between APM and the BNG.

You have the following two options for installing APM:

Before starting the APM installation, make sure that you have the following information:

Required Information:

  • Container registry details:
    • If you are using a BBE Cloudsetup created cluster.

      • External registry address.

      • External registry port number (usually 5000).

    • If you are using a Red Hat OpenShift Container Platform cluster:

      • External registry (fully qualified domain name)

      • Internal (Docker) registry address

      • Internal (Docker) registry port number

Optional Information:

  • APM initial configuration file. If a configuration file is not supplied, a basic configuration file is automatically generated.
  • Storage class name for permanent virtual channel (PVC) creation (default is jnpr-bbe-storage).
  • PVC Size (defaults is 90 MiB).
  • Archival configuration details. This is required if you are planning to mirror a copy of the APM configuration to an external server.
    • Either the name of the SSH private key file or the name of the Kubernetes secret that is present in the jnpr-apm namespace containing the SSH private key.

    • The Secure Copy Protocol (SCP) URL of the server where the configuration file will be archived. An SCP URL takes the form of scp://user-login@server-fqdn:server-port/absolute-file-path (for example, scp://user@host1.mydomain.com:30443/home/user/configs/apm.

  • Syslog server details. This is required if you are planning to export APM logs to an external syslog collector.
    Note:

    If BBE Event Collection and Visualization is detected running on the target cluster, the address and port values of the ECAV deployment will be suggested as the default.

    • Syslog server address.

    • Sysylog server port number.

  • Network load balancer details. This is required if you are planning to use a specific network load balancer pool and address for APMi.
    • Network load balancer pool name.

    • Network load balancer pool address.

  • APMi Details:
    • Port (default is 20557)
    • TLS details. You will need one of the following:
      • None (insecure)

      • Either the key and certificate files or the name of the Kubernetes secret that is present in the jnpr-apm namespace that contains the key and certificate information.

  • Number of worker processes for the provman microservice (default is 3).

Install APM Using the APM installation Utility

You use the procedure in this section if you are installing APM on a cluster that was created by the BBE Cloudsetup utility or by Red Hat OpenShift Container Platform Console.

  1. Download the APM software package from the Juniper Networks software download page to the jump host.

    APM is available as a compressed TAR (.tgz) file. The filename includes the release number as part of the name. The release number has the format: <Major>.<Minor>.<Maintenance>

    • major is the main release number of the product.
    • minor is the minor release number of the product.
    • maintainance is the revision number.
  2. Unpack the APM TAR (.tgz) file on the jump host by entering:
  3. Run the loader script after you unpack the TAR file.
  4. Use the sudo -E apm link --context context-name --version apm-version command to link to the cluster. The link command associates the loaded APM software package to the cluster in preparation for the setup.
    • context-name is the Kubernetes context (cluster name).

    • apm-version is the software version.

  5. If you are installing APM on a Red Hat OpenShift Container Platform cluster, log in with the OpenShift CLI and then proceed to the next step.
    If you are installing APM on a BBE Cloudsetup created cluster, proceed to the next step.
  6. You must authenticate with the container registry in order to be able to push the APM container images. How you authenticate to the registry varies depending on if you are installing APM on a BBE Cloudsetup created cluster or on an Red Hat OpenShift Container Platform cluster (see the respective documentation for details).
  7. Run setup to configure your installation. The setup command does the following:
    • Collects information about the cluster environment such as; Container registry contact information, keys and certificates needed to secure external interfaces, persistent storage resources, and other information relevant to supporting APM features.

    • Initializes the APM configuration.

      If you did not use either the bbecloudsetup option or the template file-name option with the setup command, you need to complete these prompts during the setup:

      • If you are using BBE Cloudsetup to create your cluster.

        • External registry address.

        • External registry port number.

      • If you are using a Red Hat OpenShift Container Platform cluster:

        • External registry (fully qualified domain name)

        • Internal (Docker) registry address

        • Internal (Docker) registry port number

    Note:

    When running setup, you can interact with the setup process by entering ^d.

    If you want to change a value after entering it, enter ^d. After entering ^d, the value you previously entered is removed and the default value is automatically used for the question. You can use the ^d operation for any setup questions that are optional or for which a list of values can be provided.

    • context-name—The Kubernetes context name.

    • update—Prompts for only missing values (primarily used after loading a new software release).

    • bbecloudsetup—Uses the default values provided with the charts. so that you do not need to answer prompts during setup. The bbecloudsetup option is used when installing APM on a BBE Cloudsetup created cluster (see the BBE Cloudsetup Installation Guide for installation instructions.

    • template file-name—Uses the values provided in the YAML file for the Red Hat OpenShift Container Platform cluster, so that you do not need to answer prompts during setup. The template file-name option is used when installing APM on a Red Hat OpenShift Container Platform created cluster. Table 2 describes the information that you need to enter into the template configuration file.

    • ssh host:port—Hostname or IP address of the cluster (any of the cluster’s nodes) and open port for ssh access to the CLI. SSH access to the CLI is provided by the MGMT microservice.

      Note:

      Enabling SSH access requires the MGMT microservice to run in privileged mode.

    • config config-file-name—Name of the initial configuration file used for APM at startup.

    Note:

    You can use an initial configuration file to start and roll out APM. You use the configuration file through the –-config config-file-path switch on the utility script’s setup command.

    When APM is started or rolled out, the configuration file that you supply during setup is used to initialize APM. If you do not supply a configuration file, APM starts with the factory defaults. The factory defaults include the bbe-ecav syslog server configuration, if the BBE Event Collection and Visualization application is detected running on the cluster.

    The supplied configuration file is stored on the jumphost’s context repository. This allows the configuration to be preserved across APM start and stop events. Commits to the initial configuration are not automatically saved to the persistent location on the jumphost. To update the configuration at the persistent location, use the utility script’s save-config command.

    Using the save-config command ensures that the latest configuration is used the next time that APM is started and rolled out. In order to restore the initial configuration back to its factory default, enter setup interactively and enter ^d to the startup config ... question.

    The action in the parenthesis changes to remove. Press Enter to accept the removal of the deployed configuration. APM reverts back to the factory default configuration after a stop and then rollout command sequence.

    When you change the initial configuration file using the utility script’s setup command, you must perform a stop and then rollout command sequence for the change to take effect.

  8. Verify the APM installation apm version --context context-name [--detail] .
    • context-name is the Kubernetes context

    • detail detail adds information about available releases in the software repository.

    Table 2: Setup File Field Descriptions

    Field

    Description

    External registry address

    The external registry address is a fully qualified domain name (FQDN) that the container images are pushed to.

    Internal (Docker) registry transport address (fqdn:port)

    The internal registry transport address is the address from which the container images are pulled from during rollout. This address is typically different than the external registry address.

    (Optional) Initial APM configuration file

    The configuration file that is used at APM startup.

    (Optional) Cluster storage-class name

    The name of the Kubernetes storage class to use for creating Persistent Volume Claims (PVCs). The management microservice uses a PVC to record the configuration state.

    (Optional) Cluster storage size

    The PVC size in mebibytes (MiB).

    (Optional) Configuration archival server

    When you configure the Configuration archival server option, APM archives a copy of the updated configuration to an external server after each successful commit.

    To configure the server information where configuration file changes are archived, you must enter the following information:

    • ssh-key information. Provide information for one of the following:

      • The name of a Kubernetes Secret in the APM namespace that contains the SSH private key data.

      • The name of the SSH private-key file.

      Note:

      If a secret name is supplied, you will not be prompted for the SSH private-key file.

    • The Secure Copy Protocol (SCP) URL of the server where the configuration file will be archived.

      Note:

      The URL must use the following format: scp://user-login@server-fqdn:server-port/absolute-file-path (for example, scp://user@host1.mydomain.com:30443/home/user/configs/apm).

      Upon successful commit, an SCP transfer of the candidate configuration is transferred to the archival URL as a compressed file with the name:

      apm-identifier_YYYYMMDD_HHMMSS_juniper.conf.n.gz

      • apm-identifier is the external IP address of the APMi interface.

      • YYYYMMDD_HHMMSS is the time stamp in Coordinated Universal Time (UTC).

      • n is the number designation of the compressed configuration rollback file.

    (Optional) Syslog Details

    If you want to export APM log information to an external syslog collector,enter the following syslog server information:

    • IP address or fully qualified domain name

    • Port number

    Syslog information is included in the generated factory default configuration file. If you did not use the generated factory default configuration file, and used your own initial configuration file, you must include the system syslog host stanza containing the connection details for the syslog server.

    (Optional) Network Load Balancer Pool

    If you want the APMi external address to be allocated from a specific network load balancer address pool, enter the following network load balancer pool information:

    • Network load balancer address annotation

    • Network load balancer pool annotation

    (Optional) APMi port

    The APMi port number (default is 20557).

    (Optional) APMi secrets

    To secure the APMi (recommended),enter one of the following:

    • The name of a Kubernetes secret in the APM namespace that contains the TLS secret data (root Certificate Authority certificate, certificate, private-key)

    • Key files (root Certificate Authority certificate, certificate, and private key)
    Note:

    If a secret is provided, you will not be prompted for the Key files during installation.

    (Optional) Number of worker processes

    The number of provman worker processes determines how simultaneous processes provman deploys to handle the entity workload. We suggest that you plan for 20 entities per process. Each process can consume a CPU core on the node it is running on. Therefore, the nodes in the cluster must have sufficient CPU cores to support the number of provman processes (plus any other workloads that may be running on a node).

    You can configure 1 to 10 worker process (default is 3).

Start APM

Use this procedure to configure and to start APM.

  1. Enter rollout to start the APM installation. You need to use the rollout command with sudo/as root. The rollout command also validates that all the values needed for the new releases are present and loads the new release container images to the registry. Use sudo -E apm rollout --context context-name to start APM services. For example:
    • contex-name is the Kubernetes context (cluster name).

    Note:

    By default, APM starts with the values that you provided during setup. Unless the configuration was saved, the initial configuration is what was provided during setup. All other persistent states (logs, database keys, and so on) are cleared.

  2. Enter apm status --context context-name [-o|--output json] [--detail] to verify that the APM services are up and running. For example:
    Note:

    Collect the logs for a service and contact the Juniper Networks Technical Assistance Center (JTAC) when either of the following occurs:

    • The service is not running.

    • The service’s uptime compared with other services indicates that it has restarted.

Install APM Without Using the APM Utility

The instructions in this section describes the installation steps for installing APM on a preexisting Kubernetes cluster of your choice. This process is a manual process and does not use the APM utility that comes with the APM installation package.

  1. Download the APM software package from the Juniper Networks software download page to the jump host.

    APM is available as a compressed TAR (.tgz) file. The filename includes the release number as part of the name. The release number has the format: <Major>.<Minor>.<Maintenance>

    • major is the main release number of the product.
    • minor is the minor release number of the product.
    • maintainance is the revision number.
  2. Unpack the APM TAR (.tgz) file on the jump host by entering:
  3. The container images needed by APM are stores in the images subdirectory. You must push the images to the registry where the scheduled application images will be pulled from. Depending on the type of container registry being used the commands may be different. The following commands illustrate one method of pushing container images to the registry:
  4. To prepare APM for deployment, you must create a YAML configuration file for each microservice. Each microservice's configuration file contains the specific configuration settings for the microservice. The YAML configuration file is called values.yaml and the file is located under the charts subdirectory, with each microservice. You should create a separate values.yaml (for example, new-values.yaml) specific to your configuration for each microservice. Table 3 describes the fields in the microservice's configuration files (values.yaml).
    Note:

    If you do not want to create multiple values.yaml files, you can create a single values.yaml that contains information for all the microservices. The single values.yaml is located under the umbrella chart in the apm/apm/charts/address_pool_manager folder. The procedures in this section describe how to configure an individual YAML configuration file for each microservice.

    Create a new values.yaml file for each of the microservices, by making a copy of the file and then saving the new file. Update each file according to your Kubernetes cluster's information.
    Following are the microservices and their values.yaml file location:
    • redis microservice—Located at apm/apm/charts/redis

    • mgmt microservice—Located at apm/apm/charts/mgmt

    • addrman microservice—Located at apm/apm/charts/addrman

    • entman microservice—Located at apm/apm/charts/entman.

    • provman microservice—Located at apm/apm/charts/provman

    Table 3: Microservices Configuration File Field Descriptions

    Field

    Description

    Microservice

    APMi port

    The APMi exposed port number.

    provman

    APMi secrets

    • name—Name space secret to mount

    • certificate—Certificate file name

    • key—Private key file name

    • rootca—CA certificate file name

    provman

    apmInitVersion

    APM init software version.

    • mgmt

    • redis

    archivalUrl

    The Secure Channel Protocol (SCP) URL of the server where the configuration file is archived.

    mgmt

    db master updateStrategy

    Only RollingUpdate is supported.

    redis

    evictionToleration

    The node's unreachable tolerance (in seconds).

    • addrman

    • entman

    • mgmt

    • provman

    • redis

    init_wait_for_sync

    Indicates whether or not to wait for all entities to synchronize during startup.

    provman

    log_level

    The default logging level.

    • addrman

    • entman

    • mgmt

    • provman

    • redis

    nlbPoolAnnotation

    The network load balancer pool name.

    provman

    nlbPoolIpAnnotation

    The network load balancer IP address

    provman

    pvcs config

    • meta—Permanent virtual channel (PVC) for configuration file storage.

    • size—PVC size (MiB).

    mgmt

    registry

    Registry information:

    • host—The registry contact for the cluster pulls.

    • port—The registry port number for cluster pulls.

    • addrman

    • entman

    • mgmt

    • provman

    • redis

    resourceRequestsEnabled

    Whether or not to accept the resource request.

    • addrman

    • entman

    • mgmt

    • provman

    • redis

    resourceRanges

    Required resource ranges:

    • cpuRequest—The minimum millicores that are required to operate the system.

    • memRequest—The minimum mebibytes (MiB) that are required to operate the system.
    • addrman

    • entman

    • mgmt

    • provman

    • redis

    • redis

    sentinelCount

    The number of sentinels to start.

    redis

    startup config

    The configuration to use for system startup.

    mgmt

    storage_class

    Name of the storage class for PVC.

    mgmt

    tlsEnabled

    Indicates if TLS is enabled.

    provman

    workerProcs

    The number of worker processes that you want started.

    provman

  5. After you have made all the desired changes to your new values.yaml files for each microservice, the microservices must be deployed with the new values.yaml files.

    Run the following commands:

  6. To complete the configure of the provman microservice, you must run the helm upgrade command twice. The double configuration is required because the provman microservice uses the external IP address assigned to the APMi load balancer service in the protocol exchanges with its entities. The first helm upgrade command establishes the external IP address and the second helm upgrade command passes the external IP address to the provman microservice, which allows it to initialize.

    Run the following commands:

  7. Verify the APM installation by running the Kubernetes Command Line Tool command kubectl get pods and verify the APM pods are running.
  8. Verify that the services are present. Run the Kubernetes Command Line Tool command kubectl get services.