Help us improve your experience.

Let us know what you think.

Do you have time for a two-minute survey?

Navigation
Guide That Contains This Content
[+] Expand All
[-] Collapse All

    Troubleshooting: MACsec Using Preshared Key Hitless Rollover Keychain on MX-series Routers

    Participating Nodes Time not Synchronized

    Problem

    Description: If the time is not synchronized in the participating nodes, MKA session is not established and packet drops happen.

    Solution

    Use the following command to check whether MKA session is established:

    user@host> show security mka sessions

    If the session is not established, check whether the participating nodes are time synchronized:

    user@host> show ntp associations
    user@host> show ntp status

    If the servers are not time synchronized, execute the following command in the participating nodes.

    user@host> set date ntp ntp-server-name

    Sample Output

    user@host>show security mka sessions
      Interface name: et-1/1/0
          Member identifier: F1B80E06749B07B3A936AD94
          CAK name: 100000000010
          Transmit interval: 2000(ms)
          Outbound SCI: EC:13:DB:BB:5B:97/1
          Message number: 169        Key number: 0
          Key server: no             Key server priority: 16
          Latest SAK AN: 2           Latest SAK KI: 6FF71A62FDAEE9EAA1CCB6BE/1
          Previous SAK AN: 1         Previous SAK KI: C4E299F428AF74932CDDCB77/1
          Peer list
           1. Member identifier: 6FF71A62FDAEE9EAA1CCB6BE (live)
              Message number: 167 Hold time: 5000 (ms)
              SCI: 3C:61:04:09:17:18/1
              Lowest acceptable PN: 0
    
    ...
    
    user@host> show ntp associations
       remote         refid           st t when poll reach   delay   offset  jitter
    ===============================================================================
    *ntp2.juniper.net
                      .GPS.            1 -   14   64  377  225.786    0.167   6.603
    
    user@host> show ntp status
    status=0644 leap_none, sync_ntp, 4 events, event_peer/strat_chg,
    version="ntpd 4.2.0-a Wed Feb 14 17:40:53 PST 2018 (1)",
    processor="amd64", system="FreeBSDJNPR-11.0-20180127.fdc8dfc_buil",
    leap=00, stratum=2, precision=-23, rootdelay=225.786,
    rootdispersion=11.380, peer=26276, refid=66.129.255.62,
    reftime=de3a5ba9.bb50dfab  Fri, Feb 23 2018  1:21:45.731, poll=6,
    clock=de3a5c05.97bfead1  Fri, Feb 23 2018  1:23:17.592, state=4,
    offset=0.167, frequency=36.223, jitter=3.327, stability=0.003
    

    Start Time is not the Same in the Keys Used in the Participating Nodes

    Problem

    Description: If the start time is not same in the keys used in both the nodes, MKA session is not established.

    Solution

    To check whether the same keychain start time is specified in both the nodes, execute the following command in both the nodes and check the start time of the key:

    user@host# show security authentication-key-chains

    If the key start time is not same, change it accordingly following the procedure at Configuring MACsec Using PSK Hitless Rollover Keychain on MX2020 and MX2010 Routers (Recommended for Enabling MACsec on Router-to-Router Links):

    user@host# set security authentication-key-chains key-chain key_chain_name key key_name start-time start_time
    user@host# set security authentication-key-chains key-chain test_key_chain key 1 start-time 2018-01-28.07:20

    Sample Output

    user@host# show security authentication-key-chains
    key-chain pskchain1 {
        key 0 {
            secret "$9$RmycevM8Xx-VyrwY2gJZFn6/p0O1RESrCAevMWx7UjiHP5Tz3n9Aq.p0OBEh-Vbw4aJGDjk.Y2P5TQn6Srl"; ## SECRET-DATA
            key-name 100000000000;
            start-time "2018-2-23.01:06:35 -0800";
        }
        key 1 {
            secret "$9$VcsoJZUjiqm2gfTQz6/yleKLx7-VbYgMWoJZGiHCtpuIEhSrlvWOBLx7NbwqmPfFn69At0BTQIEhcle24o"; ## SECRET-DATA
            key-name 100000000001;
            start-time "2018-2-23.01:08:35 -0800";
        }
        key 2 {
            secret "$9$1WMEreKM8L7-cSVwsYoaQF3nApuO1IhS/9reKvLXZUDj.PfTzF69HkApu0IR7-dV24oJGUikws.Pf5F3Srl"; ## SECRET-DATA
            key-name 100000000002;
            start-time "2018-2-23.01:10:35 -0800";
        }
    }
    

    Secret Mismatch between the Keys Used in the Nodes

    Problem

    Description: The secret used in the keychain must be same in both the participating nodes. If it is not same, CAK mismatch entry is recorded in the config trace logs.

    Solution

    Check whether IFD is up and running in both the nodes using the following command:

    user@host> show interfaces xe-0/0/0 terse

    If the IFD is up, then check the current active key being used in both the nodes, using the following command:

    user@host> show security authentication-key-chains

    Verify the keys being used. If the keys are not same, check the config trace logs (set/security/ and set security/interfacename) and search for the CAK mismatch entry. The presence of the CAK mismatch entry indicates that secrets do not match in the keychains. Set the same secrets following the procedure at Configuring MACsec Using PSK Hitless Rollover Keychain on MX2020 and MX2010 Routers (Recommended for Enabling MACsec on Router-to-Router Links).

    Sample Output

    user@host> show interfaces xe-4/0/8:0 terse
    Interface               Admin Link Proto    Local                 Remote
    xe-4/0/8:0              up    up
    xe-4/0/8:0.100          up    up   bridge
    xe-4/0/8:0.32767        up    up   multiservice
    
    user@host# show security authentication-key-chains
    key-chain test_key_chain {
        key 1 {
            secret "$8$aes256-gcm$hmac-sha2-256$100$k9da4zO9sG0$tHJe4kl3RVygIaWHmoHohQ$gW7pzH69/BbwDznGJN9jtw$40an5KZce1U"; ## SECRET-DATA  
            key-name 2398137739;  
            start-time "2022-4-28.07:20:00 -0700";
        }
    }
    
    

    MACsec Session Drop During Key Rollover

    Problem

    Description: For every successful key rollover, a message similar to the following is logged :

    DOT1XD_MKA_SA_KEY_ROLLOVER: Macsec secure association key rolled over on interface ge-0/0/1

    For every unsuccessful key rollover and when old key is being used, a message similar to the following is logged :

    DOT1XD_MACSEC_SC_EXPIRED_KEY_IN_USE: ifd: ge-0/0/1 using expired cak:

    Solution

    If there is a keychain mismatch due to wrong time specified in the keys, network time synchronization issue or wrong keys, the keychain rollover does not happen and the old key is used to retain the MKA session. A log similar to the following is recorded to indicate that the old keychain is being used:

    DOT1XD_MACSEC_SC_EXPIRED_KEY_IN_USE: ifd: ge-0/0/1 using expired cak:

    The old key is used only until the time specified in the MKA session expire timer.

    Sample Output

    user@host> show log messages | match DOT1XD_MKA_SA_KEY_ROLLOVER
    Feb 23 14:26:01.990297 macsec_update_keychain() active key presents. Kick key change to 2000000000000000000000000000
    Feb 23 14:26:02.195293 DOT1XD_MKA_SECURE_CHANNEL_CREATED: Macsec receive secure channel created for 64:87:88:f6:b0:a4 on interface ge-0/2/0
    Feb 23 14:26:05.586742 DOT1XD_MKA_SECURE_ASSOCIATION_ESTABLISHED: Macsec secure association established with an:0 on interface ge-0/2/0
    Feb 23 14:28:02.004648 macsec_update_keychain() active key presents. Kick key change to 2000000000000000000000000001
    Feb 23 14:28:04.625660 DOT1XD_MKA_SA_KEY_ROLLOVER: Macsec secure association key rolled over on interface ge-0/2/0
    Feb 23 14:30:01.012548 macsec_update_keychain() active key presents. Kick key change to 2000000000000000000000000002
    Feb 23 14:31:11.012951 DOT1XD_MACSEC_SC_PRE_SHARED_KEY_NOT_ACTIVATED: ifd: ge-0/2/0 cak: 100000000002 not activated
    Feb 23 14:31:11.013155 macsec_update_keychain() active key presents. Kick key change to 2000000000000000000000000002
    

    MACsec Session Drop Due to Mismatch in Key Start Time

    Problem

    Description: If there is a mismatch in key start time, NTP synchronization issue or wrong secret, the old key is used to retain the MKA session.

    Solution

    This session would expire once the MKA session expire timer threshold is reached. A log similar to the following is recorded to indicate that the old key is being used:

    DOT1XD_MACSEC_SC_PRE_SHARED_KEY_NOT_ACTIVATED:

    The old key is used only until the time specified in the MKA session expire timer.

    Sample Output

    user@host> show log messages | match DOT1XD_MACSEC_SC_PRE_SHARED_KEY_NOT_ACTIVATED: 
    Feb 23 14:26:01.990297 macsec_update_keychain() active key presents. Kick key change to 2000000000000000000000000000
    Feb 23 14:26:02.195293 DOT1XD_MKA_SECURE_CHANNEL_CREATED: Macsec receive secure channel created for 64:87:88:f6:b0:a4 on interface ge-0/2/0
    Feb 23 14:26:05.586742 DOT1XD_MKA_SECURE_ASSOCIATION_ESTABLISHED: Macsec secure association established with an:0 on interface ge-0/2/0
    Feb 23 14:28:02.004648 macsec_update_keychain() active key presents. Kick key change to 2000000000000000000000000001
    Feb 23 14:28:04.625660 DOT1XD_MKA_SA_KEY_ROLLOVER: Macsec secure association key rolled over on interface ge-0/2/0
    Feb 23 14:30:01.012548 macsec_update_keychain() active key presents. Kick key change to 2000000000000000000000000002
    Feb 23 14:31:11.012951 DOT1XD_MACSEC_SC_PRE_SHARED_KEY_NOT_ACTIVATED: ifd: ge-0/2/0 cak: 100000000002 not activated
    Feb 23 14:31:11.013155 macsec_update_keychain() active key presents. Kick key change to 2000000000000000000000000002
    

    Modified: 2018-03-02