In another part of his never-ending EVPN/BGP saga Ivan Pepelnjak argued with Juniper fanboys once again about sanity of iBGP-over-eBGP and eBGP-over-eBGP designs and all that fun stuff. I’ve already written my opinion about that topic in my previous post and numerous comments to Ivan’s posts (TL;DR: iBGP-over-eBGP design has its advantages, just implement it wisely – don’t place RR on spine switches).

But there is one thing that worries me. In almost every one of his posts Ivan talks about some mythical Junos limitations that don’t allow Juniper to support eBGP only (over single session) design. So let’s find out what these limitaions are.

Juniper has freely available version of vQFX for Vagrant. There are a few lab topologies available on GitHub. I will be using full-2qfx-4srv-evpnvxlan topology in this post.

This topology comes with Ansible playbook that configures vQFX switches with iBGP-over-OSPF EVPN. Standard Juniper configuration, just for reference:

If we want to convert it to Ivan’s beloved iBGP-over-eBGP design – it’s pretty simple (you can quickly do the same – just use “load patch terminal” and don’t forget to change config for vQFX2):

Also pretty standard Juniper config described many times in a lot of places. If you didn’t read the whole Ivan’s saga, BEWARE: there are local-as knob in this config, which means COMPLEXITY. But let’s not go that road this time.

Instead, let’s implement another complicated design option – eBGP-over-eBGP (overlay eBGP EVPN session between loopbacks). Diff from previous config on vQFX1:

And the whole bgp config for vQFX1:

We use different AS per vQFX in this design (configured under routing-options) and multihop option for evpn_overlay bgp group. But unfortunately that’s not all. In eBGP overlay design EVPN auto-RT feature doesn’t work by default. There are several ways to make it work, but the most simple one is to disable auto-RT and manually configure single Route Target for all VNIs (don’t forget to apply similar changes to vQFX2):

After that change servers can ping each other and EVPN successfully distributes routes:

So, to recap, what we achieved so far: iBGP-over-OSPF works like a breeze without any nerd knobs; to implement iBGP-over-eBGP we should use local-as; to make work Cisco-style eBGP-over-eBGP design it’s necessary to use multihop eBGP session and find a suitable way to assign RT to routes.

Now it’s time for the main topic of this post – let’s try eBGP only design option. First let’s try simple configuration:

And at first sight it seems to be working! Kinda… At least routes are propagating. But hosts can’t ping each other. Let’s take a closer look at one of the routes from vQFX1:

This route is advertised with next-hop of interface between vQFX switches. Which make sence, because by default Junos always rewrite next-hop for routes advertised over eBGP session. But this is the problem because VTEP is using IP address of loopback interface. It’s not obvious how to change this behaviour of eBGP, but here is working configuration:

We need to configure and apply policy to explicitly change the next-hop for EVPN routes to address of loopback interface. But, confusingly, this policy doesn’t take effect until multihop no-nexthop-change option is applied (as far as I understand, without this knob Junos always change next-hop for advertised eBGP routes to ip address of source interface). And to recieve such routes (with changed nexthop) accept-remote-nexthop option is necessary.

After that we can see EVPN route with correct nexthop:

Just to recap, here is the full working configuration for eBGP only design:

So, I think the myth about some limitations of Junos to not being able to support eBGP-only option is finally busted. Of course the configuration is convoluted with a couple of really nerd knobs, but it works. If you going to tell me about other vendors that provide a sane default for such type of configuration, keep in mind couple of things. First of all, EVPN in Junos is not only about datacenter, EVPN-MPLS also plays a big role in this world. Defaults that work great for DC can be really unsuitable for Inter-AS sessions. And, generally speaking, the sanity of any solution and default values is highly debatable thing. But I really confident that best product is the one that supports all the options.

From all the above, I want to make only one conclusion. Please, don’t blame Junos developers for simply offering more options. There are a lot of other things they should be ashamed of, but not this one. As for the marketing and educational departments for their promoting of iBGP design with RR on spines – that’s another story…