MC-LAG lab – advanced IRB functionality

MC-LAG lab – advanced IRB functionality

For simplified Layer 3 gateway services, where Layer 3 routing protocols are not run on the MC-LAG peers, you simply configure the same Layer 3 gateway IP address on both MC-LAG peers and enable IRB MAC address synchronization. This IP address is used as the default gateway for the MC-LAG servers or hosts.
For more advanced Layer 3 gateway services, where Layer 3 routing protocols and Layer 3 multicast operations are required on the MC-LAG peers, you configure unique IRB interface addresses on each MC-LAG peer and then configured the virtual router redundancy protocol (VRRP) between the peers in an active/standby role.
To help with some forwarding operations, the IRB MAC address of each peer is replicated on the other peer and is installed as a MAC address with the forwarding next hop of the ICL-PL. This is achieved by configuring a static ARP entry for the remote peer.

IRB interfaces configurations:

vQFX1:
irb {
    unit 100 {
        family inet {
            address 172.16.0.1/24 {
                arp 172.16.0.2 l2-interface xe-0/0/3.0 mac 02:05:86:71:35:00;  ##vQFX2 IRB MAC
                vrrp-group 1 {
                    virtual-address 172.16.0.254;
                    priority 200;
                    accept-data;
                }
            }
        }
        family inet6 {
            address 2001:dead:beef:100::1/64 {
                ndp 2001:dead:beef:100::2 l2-interface xe-0/0/3.0 mac 02:05:86:71:35:00;
                vrrp-inet6-group 1 {
                    virtual-inet6-address 2001:dead:beef:100::a;
                    priority 100;
                }                      
            }
            address fe80::1/64 {
                ndp fe80::2 l2-interface xe-0/0/3.0 mac 02:05:86:71:35:00;
            }
        }
    }
    unit 200 {
        family inet {
            address 172.16.1.1/24 {
                arp 172.16.1.2 l2-interface xe-0/0/3.0 mac 02:05:86:71:35:00;
                vrrp-group 1 {
                    virtual-address 172.16.1.254;
                    priority 100;
                    accept-data;
                }
            }
        }
        family inet6 {
            address 2001:dead:beef:200::1/64 {
                ndp 2001:dead:beef:200::2 l2-interface xe-0/0/3.0 mac 02:05:86:71:35:00;
                vrrp-inet6-group 1 {    
                    virtual-inet6-address 2001:dead:beef:200::a;
                    priority 200;
                }
            }
        }
    }
}


vQFX2:
irb {
    unit 100 {
        family inet {
            address 172.16.0.2/24 {
                arp 172.16.0.1 l2-interface xe-0/0/3.0 mac 02:05:86:71:00:00;  ##vQFX1 IRB MAC
                vrrp-group 1 {
                    virtual-address 172.16.0.254;
                    priority 100;
                    accept-data;
                }
            }
        }
        family inet6 {
            address 2001:dead:beef:100::2/64 {
                ndp 2001:dead:beef:100::1 l2-interface xe-0/0/3.0 mac 02:05:86:71:00:00;
                vrrp-inet6-group 1 {
                    virtual-inet6-address 2001:dead:beef:100::a;
                    priority 200;
                }                      
            }
            address fe80::2/64 {
                ndp fe80::1 l2-interface xe-0/0/3.0 mac 02:05:86:71:00:00;
            }
        }
    }
    unit 200 {
        family inet {
            address 172.16.1.2/24 {
                arp 172.16.1.1 l2-interface xe-0/0/3.0 mac 02:05:86:71:00:00;
                vrrp-group 1 {
                    virtual-address 172.16.1.254;
                    priority 200;
                    accept-data;
                }
            }
        }
        family inet6 {
            address 2001:dead:beef:200::2/64 {
                ndp 2001:dead:beef:200::1 l2-interface xe-0/0/3.0 mac 02:05:86:71:00:00;
                vrrp-inet6-group 1 {    
                    virtual-inet6-address 2001:dead:beef:200::a;
                    priority 100;
                }
            }
        }
    }
}

I configured VRRP for two IRB units for both IPv4 and IPv6 address families. Notice static ARP entries for IP/MAC addresses of other peer’s IRB pointing to the ICL-PL.

VRRP status:

alex@vQFX1> show vrrp          
Interface     State       Group   VR state VR Mode   Timer    Type   Address
irb.100       up              1   master   Active      A  0.728 lcl    172.16.0.1    
                                                                vip    172.16.0.254  
irb.100       up              1   backup   Active      D  3.130 lcl    2001:dead:beef:100::1
                                                                vip    fe80::200:5eff:fe00:201
                                                                vip    2001:dead:beef:100::a
                                                                mas    fe80::2        
irb.200       up              1   backup   Active      D  3.130 lcl    172.16.1.1    
                                                                vip    172.16.1.254  
                                                                mas    172.16.1.2    
irb.200       up              1   master   Active      A  0.890 lcl    2001:dead:beef:200::1
                                                                vip    fe80::200:5eff:fe00:201
                                                                vip    2001:dead:beef:200::a


alex@vQFX2> show vrrp
Interface     State       Group   VR state VR Mode   Timer    Type   Address
irb.100       up              1   backup   Active      D  3.386 lcl    172.16.0.2    
                                                                vip    172.16.0.254  
                                                                mas    172.16.0.1    
irb.100       up              1   master   Active      A  0.199 lcl    2001:dead:beef:100::2
                                                                vip    fe80::200:5eff:fe00:201
                                                                vip    2001:dead:beef:100::a
irb.200       up              1   master   Active      A  0.253 lcl    172.16.1.2    
                                                                vip    172.16.1.254  
irb.200       up              1   backup   Active      D  3.386 lcl    2001:dead:beef:200::2
                                                                vip    fe80::200:5eff:fe00:201
                                                                vip    2001:dead:beef:200::a
                                                                mas    fe80::205:8600:c871:0

Don’t forget to delete mcae-mac-synchronize command, if you configured it before. Vlans configuration:

alex@vQFX1> show configuration vlans
v100 {
    vlan-id 100;
    l3-interface irb.100;
}
v200 {
    vlan-id 200;
    l3-interface irb.200;
}

CE interface configuration:

alex@MX2# show interfaces
ge-0/0/0 {
    description vQFX1;
    gigether-options {
        802.3ad ae0;
    }
}
ge-0/0/1 {
    description vQFX2;
    gigether-options {
        802.3ad ae0;
    }
}
ae0 {
    description to_MC-LAG_vQFX;
    flexible-vlan-tagging;
    encapsulation flexible-ethernet-services;
    aggregated-ether-options {
        lacp {
            active;
            periodic slow;
        }
    }
    unit 100 {
        vlan-id 100;                    
        family inet {
            address 172.16.0.22/24;
        }
        family inet6 {
            address 2001:dead:beef:100::22/64;
            address fe80::33/64;
        }
    }
}
lo0 {
    unit 0 {
        family inet {
            address 222.222.222.222/32;
        }
        family inet6 {
            address 2002:222::222/128;
        }
    }
}

[edit]
alex@MX2# show logical-systems
second {
    interfaces {
        ae0 {
            unit 200 {
                vlan-id 200;
                family inet {
                    address 172.16.1.22/24;
                }
                family inet6 {
                    address 2001:dead:beef:200::22/64;
                    address fe80::3/64;
                }
            }
        }
        lo0 {
            unit 2 {
                family inet {
                    address 222.2.2.2/32;
                }
                family inet6 {
                    address 2002:2::2/128;
                }
            }
        }                              
    }
}


To check correct forwarding functionality, I configured routing protocols between both MC-LAG peers and CE device.

OSPFv2/v3 for Vlan 100:

alex@vQFX1# show protocols
ospf {
    export bgp;
    area 0.0.0.0 {
        interface lo0.0;
        interface irb.100;
    }
}
ospf3 {
    export bgp;                        
    area 0.0.0.0 {
        interface lo0.0;
        interface irb.100;
    }
}


alex@vQFX2# show protocols
ospf {
    export bgp;
    area 0.0.0.0 {
        interface lo0.0;
        interface irb.100;
    }
}
ospf3 {
    export bgp;                        
    area 0.0.0.0 {
        interface lo0.0;
        interface irb.100;
    }
}


alex@MX2# show protocols
ospf {
    area 0.0.0.0 {
        interface lo0.0;
        interface ae0.100;
    }
}
ospf3 {
    area 0.0.0.0 {
        interface ae0.100;
        interface lo0.0;
    }
}


BGP family inet/inet6 for Vlan 200:

alex@vQFX1# show protocols
bgp {
    group mclag {
        type external;
        local-address 172.16.1.1;
        family inet {
            unicast;
        }
        export ospf;
        local-as 100;
        neighbor 172.16.1.22 {
            peer-as 22;
        }                              
    }
    group mclag_v6 {
        type external;
        local-address 2001:dead:beef:200::1;
        family inet6 {
            unicast;
        }
        export ospf;
        local-as 100;
        neighbor 2001:dead:beef:200::22 {
            peer-as 22;
        }
    }
}


alex@vQFX2# show protocols
bgp {
    group underlay {
        type external;
        export direct;
        local-as 65022;
        multipath multiple-as;
        neighbor 192.168.0.2 {
            peer-as 65001;
        }
        neighbor 192.168.0.6 {
            peer-as 65002;
        }
    }
    group mclag {
        type external;
        local-address 172.16.1.2;
        family inet {
            unicast;
        }
        export ospf;
        local-as 100;
        neighbor 172.16.1.22 {
            peer-as 22;
        }                              
    }
    group mclag_v6 {
        type external;
        local-address 2001:dead:beef:200::2;
        family inet6 {
            unicast;
        }
        export ospf;
        local-as 100;
        neighbor 2001:dead:beef:200::22 {
            peer-as 22;
        }
    }
}


alex@MX2# show logical-systems
second {
    protocols {
        bgp {
            group inet {
                type external;
                local-address 172.16.1.22;
                family inet {
                    unicast;
                }
                export lo0;
                peer-as 100;
                neighbor 172.16.1.1;
                neighbor 172.16.1.2;
            }
            group inet6 {
                type external;
                local-address 2001:dead:beef:200::22;
                family inet6 {
                    unicast;
                }
                export lo0;
                peer-as 100;
                neighbor 2001:dead:beef:200::1;
                neighbor 2001:dead:beef:200::2;
            }
        }
    }


Now it’s time to check routing protocol adjacencies.
OSPFv2:

alex@vQFX1> show ospf neighbor
Address          Interface              State     ID               Pri  Dead
172.16.0.22      irb.100                Full      222.222.222.222  128    39
172.16.0.2       irb.100                Full      22.22.22.22      128    33


alex@vQFX2> show ospf neighbor
Address          Interface              State     ID               Pri  Dead
172.16.0.22      irb.100                Full      222.222.222.222  128    36
172.16.0.1       irb.100                Full      11.11.11.11      128    35


alex@MX2> show ospf neighbor
Address          Interface              State     ID               Pri  Dead
172.16.0.1       ae0.100                Full      11.11.11.11      128    31
172.16.0.2       ae0.100                Full      22.22.22.22      128    39

OSPFv3:

alex@vQFX1> show ospf3 neighbor
ID               Interface              State     Pri   Dead
222.222.222.222  irb.100                Full      128     35
  Neighbor-address fe80::33
22.22.22.22      irb.100                Full      128     35
  Neighbor-address fe80::2


alex@vQFX2> show ospf3 neighbor
ID               Interface              State     Pri   Dead
222.222.222.222  irb.100                Full      128     36
  Neighbor-address fe80::33
11.11.11.11      irb.100                Full      128     39
  Neighbor-address fe80::1


alex@MX2> show ospf3 neighbor
ID               Interface              State     Pri   Dead
22.22.22.22      ae0.100                Full      128     29
  Neighbor-address fe80::2
11.11.11.11      ae0.100                Full      128     38
  Neighbor-address fe80::1

BGP:

alex@vQFX1> show bgp summary
Groups: 3 Peers: 4 Down peers: 2
Table          Tot Paths  Act Paths Suppressed    History Damp State    Pending
inet.0              
                       1          0          0          0          0          0
inet6.0              
                       1          1          0          0          0          0
Peer                     AS      InPkt     OutPkt    OutQ   Flaps Last Up/Dwn State|#Active/Received/Accepted/Damped...
172.16.1.22              22         19         20       0       3        6:13 0/1/1/0              0/0/0/0
2001:dead:beef:200::22          22         18         19       0       3        6:06 Establ
  inet6.0: 1/1/1/0


alex@vQFX2> show bgp summary
Groups: 3 Peers: 4 Down peers: 2
Table          Tot Paths  Act Paths Suppressed    History Damp State    Pending
inet.0              
                       1          1          0          0          0          0
inet6.0              
                       1          0          0          0          0          0
Peer                     AS      InPkt     OutPkt    OutQ   Flaps Last Up/Dwn State|#Active/Received/Accepted/Damped...
172.16.1.22              22         19         14       0       0        4:13 1/1/1/0              0/0/0/0
2001:dead:beef:200::22          22         18         14       0       0        4:08 Establ
  inet6.0: 0/1/1/0


alex@MX2> show bgp summary logical-system second
Groups: 2 Peers: 4 Down peers: 0
Table          Tot Paths  Act Paths Suppressed    History Damp State    Pending
inet.0              
                       5          3          0          0          0          0
inet6.0              
                       3          1          0          0          0          0
Peer                     AS      InPkt     OutPkt    OutQ   Flaps Last Up/Dwn State|#Active/Received/Accepted/Damped...
172.16.1.1              100         30         35       0       3        7:46 2/3/3/0              0/0/0/0
172.16.1.2              100         13         19       0       0        7:45 1/2/2/0              0/0/0/0
2001:dead:beef:200::1         100         29         34       0       3        7:38 Establ
  inet6.0: 0/1/1/0
2001:dead:beef:200::2         100         13         19       0       0        7:36 Establ
  inet6.0: 1/2/2/0


And finally show routes on the CE:

alex@MX2> show route table inet.0

inet.0: 7 destinations, 7 routes (7 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both

11.11.11.11/32     *[OSPF/10] 00:07:52, metric 1
                    > to 172.16.0.1 via ae0.100
22.22.22.22/32     *[OSPF/10] 00:07:52, metric 1
                    > to 172.16.0.2 via ae0.100
172.16.0.0/24      *[Direct/0] 00:08:10
                    > via ae0.100
172.16.0.22/32     *[Local/0] 01:10:46
                      Local via ae0.100
222.2.2.2/32       *[OSPF/150] 00:07:52, metric 0, tag 0
                    > to 172.16.0.2 via ae0.100
222.222.222.222/32 *[Direct/0] 01:10:35
                    > via lo0.0
224.0.0.5/32       *[OSPF/10] 01:10:55, metric 1
                      MultiRecv


alex@MX2> show route table inet6.0  

inet6.0: 8 destinations, 8 routes (8 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both

2001:dead:beef:100::/64
                   *[Direct/0] 00:08:12
                    > via ae0.100
2001:dead:beef:100::22/128
                   *[Local/0] 01:10:59
                      Local via ae0.100
2002:2::2/128      *[OSPF3/150] 00:07:55, metric 0, tag 0
                    > to fe80::1 via ae0.100
2002:222::222/128  *[Direct/0] 00:17:37
                    > via lo0.0
fe80::/64          *[Direct/0] 00:08:12
                    > via ae0.100
fe80::33/128       *[Local/0] 01:10:59
                      Local via ae0.100
fe80::246:d30f:fc45:5300/128
                   *[Direct/0] 00:17:37
                    > via lo0.0
ff02::5/128        *[OSPF3/10] 01:11:07, metric 1
                      MultiRecv

We can se here addresses of CE’s logical system loopback interface, learned from OSPF (222.2.2.2/32 and 2002:2::2/128).

Everything looks fine, so now it’s time to ping between loopback addresses of CE:

alex@MX2> ping 222.2.2.2 source 222.222.222.222 count 3        
PING 222.2.2.2 (222.2.2.2): 56 data bytes
64 bytes from 222.2.2.2: icmp_seq=0 ttl=63 time=45.194 ms
64 bytes from 222.2.2.2: icmp_seq=1 ttl=63 time=28.353 ms
64 bytes from 222.2.2.2: icmp_seq=2 ttl=63 time=64.465 ms
--- 222.2.2.2 ping statistics ---
3 packets transmitted, 3 packets received, 0% packet loss
round-trip min/avg/max/stddev = 28.353/46.004/64.465/14.754 ms

alex@MX2> traceroute 222.2.2.2 source 222.222.222.222          
traceroute to 222.2.2.2 (222.2.2.2) from 222.222.222.222, 30 hops max, 40 byte packets
 1  172.16.0.2 (172.16.0.2)  15.044 ms  13.808 ms  14.638 ms
 2  222.2.2.2 (222.2.2.2)  1195.272 ms  897.564 ms  1854.099 ms


alex@MX2> ping 2002:2::2 source 2002:222::222 count 3    
PING6(56=40+8+8 bytes) 2002:222::222 --> 2002:2::2
16 bytes from 2002:2::2, icmp_seq=0 hlim=63 time=715.811 ms
16 bytes from 2002:2::2, icmp_seq=1 hlim=63 time=708.803 ms
16 bytes from 2002:2::2, icmp_seq=2 hlim=63 time=1297.396 ms

--- 2002:2::2 ping6 statistics ---
3 packets transmitted, 3 packets received, 0% packet loss
round-trip min/avg/max/std-dev = 708.803/907.337/1297.396/275.828 ms

alex@MX2> traceroute 2002:2::2 source 2002:222::222
traceroute6 to 2002:2::2 (2002:2::2) from 2002:222::222, 64 hops max, 12 byte packets
 1  2001:dead:beef:100::1 (2001:dead:beef:100::1)  125.623 ms  924.959 ms  1027.946 ms
 2  2002:2::2 (2002:2::2)  1276.239 ms  926.132 ms  2238.168 ms


Perfect! CE loopbacks have reachability to each other.


So, that’s it. MC-LAG configuration not so complex, you just need to remember all parts of it (ICL, ICCP, MC-LAG interface, Vlans, switch-options), and understand concepts for advanced IRB functionality (how and why to configure static ARP entries).

P.S. Have you noticed IPv6 link-level addresses configuration on irb.100? I think you should be able to understand it’s purpose. If not, try to lab this topic yourself.

3 thoughts on “MC-LAG lab – advanced IRB functionality

  1. have you encountered any issues i overlapping your VRRP virtual address with your interface addressing?

    mx1 – irb.100 – 10.0.0.1/24 vrrp virtual 10.0.0.1/24
    mx2 – irb.100 – 10.0.0.2/24 vrrp virtual 10.0.0.1/24

    1. Nevermind, you mentioned it in the previous article:

      “For more advanced Layer 3 gateway services you need to configure unique IP addresses on IRB interfaces on each MC-LAG peer.”

  2. I have one small doubt if we have N# of irb interfaces with vrrp configured in both MX1 & MX2, in this case, what is the static-mac entry required to use from both MX1 & MX2 side.

    As per my understanding, we need to configure irb mac address’s of MX’s routers on each other routers.

    Hope, below command output mac-address, is common for all irb interfaces static-mac entry.
    “show interface irb” or do we need to use identical mac address of each irb interfaces like “show interface irb.1”, “show interface irb.2”.

    Please clarify this one.

Leave a Reply

Your email address will not be published. Required fields are marked *