Spine-Leaf (Clos) with Cumulus VX (vagrant up) – Part 2:underlay + routing on the host (RoH)

This blog post demonstrates how to quickly deploy a lab with routing on the host (RoH) connected to a two-tier L3 spine-leaf (Clos) using Cumulus VXVagrant and BGP (Quagga).  This post is primarily focused on deploying and configuring routing on the host leveraging Ubuntu 14.04 with a L3 spine-leaf (Clos), my previous post of building only a L3 spine-leaf (Clos) without RoH can be found here.


Software Used To Build This Out…

  • Vagrant (I used version 1.9.5)
  • VirtualBox (I used version 5.1.22)
  • *Cumulus VX image for Clos (I used version 3.3.0)
  • *Ubuntu Trusty host image for RoH (I used version 14.04)
  • **Cumulus Quagga package for RoH (I used roh-3.3.0-ubuntu1404.tar)

NOTE:  The images (*Vagrant boxes) will automatically be pulled down from Atlas by HashiCorp and the **Cumulus Quagga package for RoH will automatically be downloaded from Cumulus RoH download section based on the configuration within the Vagrantfile (shown below)

>>>I installed all of the above on a Windows 10 machine<<<


Building is quick & easy with Vagrantfile

To start this build we will need the following two files in same local directory:

  1. Vagrantfile – this file will do all the magic of deploying and configuring four Cumulus VX leaf switches, two Cumulus VX spine switches and two Ubuntu 14.04 hosts with RoH, all peering via eBGP.
  2. 99quagga_defaults.conf – this file is needed to setup routing-on-the-host (RoH) as described here in the Cumulus documentation Installing the Cumulus Quagga Package on a Host Server.

Both files can be copied/cloned from my GitHub account found here: https://github.com/michaelc0n/cumulusvx_bgp_roh_lab_2


Vagrant Up!

After both files (Vagrantfile and 99quagga_defaults.conf) have been copied to your local machine, navigate to the directory containing these files and issue a “vagrant up”.  After a about ten minutes you should have the topology shown below with SSH access into all switches and hosts…



  • Username/Password for SSH access: ‘vagrant
  • Once connected enter “sudo vtysh“, this will drop you into Quagga where you can issue BGP commands such as “show ip bgp summary“, “show ip route“, etc.
  • See topology diagram below for all IP address information

Sample output of “show ip bgp summary” from ubuntu1 :

ubuntu1# show ip bgp summary
BGP router identifier, local AS number 65501 vrf-id 0
BGP table version 3
RIB entries 2, using 272 bytes of memory
Peers 2, using 42 KiB of memory
Peer groups 1, using 72 bytes of memory

Neighbor V AS MsgRcvd MsgSent TblVer InQ OutQ Up/Down State/PfxRcd
leaf1(eth1) 4 65101 198 202 0 0 0 00:09:47 1
leaf2(eth2) 4 65102 175 180 0 0 0 00:08:36 1

Total number of neighbors 2
Sample output of “show ip route” from ubuntu1 :
(ubuntu1 RoH with ECMP learned default-route via both leaf switches)
ubuntu1# show ip route
Codes: K - kernel route, C - connected, S - static, R - RIP,
 O - OSPF, I - IS-IS, B - BGP, P - PIM, T - Table, v - VNC,
 V - VPN,
 > - selected route, * - FIB route

B [20/0] via, eth1, 00:13:35
                   via, eth2, 00:13:35
K>* via, eth0
C>* is directly connected, eth0
C>* is directly connected, eth1
C>* is directly connected, eth2
C>* is directly connected, eth3
Sample output of “ping” (ubuntu2) from ubuntu1 :
ubuntu1# ping
PING ( 56(84) bytes of data.
64 bytes from icmp_seq=1 ttl=127 time=0.630 ms
64 bytes from icmp_seq=2 ttl=127 time=1.00 ms
64 bytes from icmp_seq=3 ttl=127 time=1.12 ms
64 bytes from icmp_seq=4 ttl=127 time=1.02 ms
--- ping statistics ---
4 packets transmitted, 4 received, 0% packet loss, time 3006ms
rtt min/avg/max/mdev = 0.630/0.945/1.125/0.191 ms

 Happy Labbing!!!

If you want to dig into all the knobs of how the devices in this lab are actually configured I recommend reviewing the Vagrantfile, the Vagrantfile details all the commands necessary to stand this up.  Please let me know if you have any questions, comments or feedback.


Leave a Reply

Your email address will not be published. Required fields are marked *