BrassHornCommunications is 99% OpenBSD and in January Vultr announced support for OpenBSD virtual machines, not only that but they also support BGP peering from virtual machines!

After earning some credit for writing the OpenBSD BGP guide I set about automating the BGP configuration of new VMs to use my IPs.

Vultr allows IPv4 announcements as granular as a /32 but IPv6 is a minimum of /64, I allocated a /36 of IPv6 space and a /24 of IPv4 space and updated the RIPE route records for AS28715 to declare a ROUTE-SET specifically for Vultr;

remarks:        +-------------------------------------------------------------+
remarks:        VPS Infrastructure
remarks:        +-------------------------------------------------------------+
import:         from AS20473 accept ANY
export:         to AS20473 announce RS-ABLATIVEHOSTING

Configuration Templating

With that done it was time to take the minimal OpenBGPd config and turn it into an Ansible template.

bgp.conf.j2

AS 28715
router-id 185.104.123.{{ ipv4_last_octet }}

# holdtime 180
# holdtime min 3
listen on 127.0.0.1
listen on ::1
# fib-update no
# route-collector no
log updates
network 185.104.123.0/24
network 185.104.123.{{ ipv4_last_octet }}/32
network 2a06:3000:1000::/36
network 2a06:3000:1000:{{ ipv6_64_subnet }}::/64

# restricted socket for bgplg(8)
# socket "/var/www/run/bgpd.rsock" restricted

neighbor 169.254.169.254 {
        remote-as      64515
        descr           "Vultr IPv4"
        announce IPv4 unicast
        announce IPv6 none
        tcp md5sig password {{vultr_bgp_password}}
        multihop 2
        local-address {{ ipv4_source }}
}
neighbor 2001:19f0:ffff::1  {
        remote-as      64515
        descr           "Vultr IPv6"
        announce IPv4 none
        announce IPv6 unicast
        tcp md5sig password {{vultr_bgp_password}}
        multihop 2
        local-address {{ ipv6_source }}
}

You’ll notice that 185.104.123.0/24 and 2a06:3000:1000::/36 are statically announced with ipv4_last_octet and ipv6_64_subnet being allocated on a per-host basis. With an entire /64 announced from this one virtual machine we have 18,446,744,073,709,551,616 addresses to play with (VMs, virtual hosts, containers, whatever).

To prevent people hijacking your prefixes the BGP sessions are authenticated, this password is held in vultr_bgp_password which is, as one would expect, encrypted using Ansible Vault. The BGP sessions are initiated from ipv4_source and ipv6_source respectively.

Playbooks and Roles

The playbook for this is quite simple as OpenBSD already ships with OpenBGPd installed and because I am using supervisord instead of rc.d there is no need to override rc.conf.local or utilise rcctl.

roles/bgpd.yml

---
  - name: Copy across bgpd config
    template:
      src: templates/bgpd.conf.j2
      dest: /etc/bgpd.conf
      owner: root
      group: wheel
      mode: 0600
      validate: /usr/sbin/bgpd -n -f %s
      backup: yes
    notify:
    - restart bgpd

  - name: Configuring bgpd Supervisord
    template:
      src: templates/supervisor/bgpd.j2
      dest: /etc/supervisord.d/bgpd.ini
      owner: root
      group: wheel
      mode: 0644
    notify:
    - restart bgpd

  - supervisorctl:
      name: bgpd
      state: present

  - name: Create dummy interface
    template:
      src: templates/hostname.lo1.j2
      dest: /etc/hostname.lo1
    notify:
    - restart networking

At the end of the play we create a dummy interface, the template it is copying over is;

templates/hostname.lo1.j2

inet 185.104.123.{{ ipv4_last_octet }} 255.255.255.0 185.104.123.255 description "Brass Horn Communications"
inet6 alias 2a06:3000:1000:{{ ipv6_64_subnet }}::1 64

This create a loopback interface (all machines usually have loopback0 which is where ::1 and 127.0.0.1 bind) named lo1 (or loopback1) and binds the IPs we are announcing that are unique to this VM.

With that done you can bind httpd, nginx, relayed, OpenSMTPd etc etc to your IPs rather than relying on the IPs from the provider!

Influencing the Global BGP routing table with ~70 lines of Ansible!