{"id":138,"date":"2026-08-17T00:26:06","date_gmt":"2026-08-17T00:26:06","guid":{"rendered":"http:\/\/127.0.0.1\/en\/docs\/how-openstack-was-installed-kampos-network-bgp-osa-haproxy-k8s-cert-manager\/"},"modified":"2026-08-17T00:56:50","modified_gmt":"2026-08-17T00:56:50","slug":"openstack-lab-deployment","status":"publish","type":"post","link":"https:\/\/wp-nv.radut.info\/ro\/2026\/08\/17\/openstack-lab-deployment\/","title":{"rendered":"OpenStack Lab Deployment"},"content":{"rendered":"

Scope:<\/strong> this walks through how this lab’s OpenStack cloud was actually built, end to end \u2014 physical network \u2192 MikroTik BGP \u2192 OpenStack-Ansible install \u2192 internal HAProxy. Real values from this deployment (VLAN IDs, AS numbers, CIDRs, commands) are included throughout; credentials, private keys and tokens are intentionally omitted \u2014 see the referenced files directly in the repo for those, and rotate anything you find that’s ever touched git history.<\/p>\n

1. Architecture at a glance<\/h2>\n

One physical trunk carries every VLAN into a single MikroTik RB5009, which does two jobs: it’s the L3 gateway for each VLAN, and it’s a BGP router peering with every cluster’s CNI (Calico) so pod\/service CIDRs get real routes instead of NAT tricks. OpenStack itself is a 3-node OpenStack-Ansible (OSA) deployment where each of the 3 physical hosts plays every infrastructure role at once (no dedicated controller\/compute split).<\/p>\n

2. The physical network: one trunk, many VLANs<\/h2>\n

Every VLAN rides the same trunk interface into the router’s bridge. From the live RouterOS config:<\/p>\n

vlan-id  8   lab-int             (management uplink to the switch)\nvlan-id  9   bmc                 (server BMC\/IPMI network)\nvlan-id  11  mgm                 (OSA management network, br-mgmt)\nvlan-id  12  vxlan               (OSA tunnel network, VXLAN overlay)\nvlan-id  13  k8s-a               (k8s cluster A VLAN)\nvlan-id  14  project-a           (OpenStack tenant network A)\nvlan-id  15  project-b           (OpenStack tenant network B)\nvlan-id  16  k8s-b               (k8s cluster B VLAN)\nvlan-id  18  os-net-bgp          (OpenStack provider network used for BGP peering)\nvlan-id  19  lbaas               (reserved for Octavia LBaaS, not yet enabled)\n<\/code><\/pre>\n
On the OpenStack-Ansible side, the provider VLAN range for tenant networks is configured as a contiguous block rather than the whole 4094-VLAN space \u2014 deliberately narrow, matching only what’s actually wired to the switch:<\/p>\n
# openstack_user_config.yml\n- network:\n    container_bridge: \"br-vlan\"\n    type: \"vlan\"\n    range: \"14:18\"   # tenant VLANs; 19 is reserved for lbaas, 20 storage, 21 osd\n    net_name: \"provider\"<\/code><\/pre>\n
3. BGP: how the router talks to each cluster’s CNI<\/h2>\n
Every Kubernetes cluster and the OpenStack tenant network peers with the router over eBGP, not static routes. The AS numbering follows one rule consistently: router-side ASN = node-side ASN + 2<\/strong>:<\/p>\n\n\n\n\n\n
Peering<\/th>\nRouter AS<\/th>\nNode AS<\/th>\n<\/tr>\n
OpenStack tenant (compute hosts)<\/td>\n65014<\/td>\n65015<\/td>\n<\/tr>\n
k8s project a CNI<\/td>\n65023<\/td>\n65021<\/td>\n<\/tr>\n
k8s project b CNI<\/td>\n65027<\/td>\n65025<\/td>\n<\/tr>\n<\/table>\n
Each BGP instance gets its own router-id and its own accept-only-this-prefix filter chain \u2014 nothing is accepted by default, each peering can only advertise the one CIDR block it’s actually supposed to own:<\/p>\n
\/routing bgp instance add as=65023 name=\"k8s project a cni\" router-id=10.14.0.1 routing-table=main\n\/routing bgp instance add as=65014 name=\"OS tenant\"     router-id=10.12.0.1 routing-table=main\n\/routing bgp instance add as=65027 name=\"k8s project b cni\" router-id=10.15.0.1 routing-table=main\n\n# one connection per node \u2014 example, k8s-a masters\/nodes:\n\/routing bgp connection add as=65023 instance=\"k8s project a cni\" local.address=10.14.0.1 \\\n  remote.address=10.14.0.21 .as=65021 templates=k8s-a-cni-template name=k8s-a-master-1\n# ... repeated per node (10.14.0.21-23 masters, .31-.36 workers)\n\n# the filter is the real security boundary \u2014 only this cluster's own service\/pod CIDRs are accepted:\n\/routing filter rule add chain=k8s-a-in rule=\"if (dst in 10.13.0.0\/18 || dst in 10.13.64.0\/18 || dst in 10.13.128.0\/18) { accept }\"\n\/routing filter rule add chain=k8s-a-out rule=reject   # never advertise anything back<\/code><\/pre>\n
The OpenStack tenant peering additionally uses tcp-md5-key<\/code> on the BGP session (the compute hosts sit across VLANs from the router, multihop=yes<\/code>) \u2014 the k8s CNI peerings don’t need it since Calico’s own BGP password support via a Kubernetes Secret covers that layer instead.<\/p>\n
4. Installing OpenStack (OpenStack-Ansible)<\/h2>\n
Standard 3-node OSA, deployed from a bootstrap\/deploy host with SSH access to all 3 physical servers.<\/p>\n
4.1 Base OS prep<\/h3>\n
Before OSA touches anything, every target host needs to match OpenStack-Ansible’s own “prepare the target hosts” prerequisites: short (non-FQDN) hostnames \u2014 long names blow past LXC\/SSH naming limits \u2014 the en_US.UTF-8<\/code> locale, SSH key-based auth with passwordless sudo, and the base package set OSA’s docs call for (bridge-utils<\/code>, lvm2<\/code>, openssh-server<\/code>, tcpdump<\/code>, python3<\/code>). The docs also call for four network bridges: br-mgmt<\/code> on every host (OSA’s own management\/API traffic), br-vxlan<\/code> for the tunnel network, br-vlan<\/code> for VLAN\/flat provider networks, and br-storage<\/code> for Cinder\/Ceph traffic.<\/p>\n
On top of that baseline, this lab’s own host-prep adds: openvswitch-switch<\/code> (the bridges above are OVS bridges here, not Linux bridges \u2014 see the lxc_net_bridge_type<\/code> override in \u00a75), ceph-volume<\/code> (these hosts double as Ceph OSD nodes), KVM kernel modules auto-detected from CPU vendor, and persistent iptables rules.<\/p>\n
4.2 Bootstrap Ansible itself, then generate secrets<\/h3>\n
export LC_ALL=en_US.UTF-8\nexport LANG=en_US.UTF-8\nscripts\/bootstrap-ansible.sh\n\n# auto-generate every OpenStack service password\/token \u2014 never hand-write these\ncd \/opt\/openstack-ansible\/scripts\/\npython pw-token-gen.py --file \/etc\/openstack_deploy\/user_secrets.yml<\/code><\/pre>\n
4.3 The 3 real install playbooks<\/h3>\n
cd \/opt\/openstack-ansible\/playbooks\nopenstack-ansible setup-hosts.yml           # 1. prepare hosts + LXC containers\nopenstack-ansible setup-infrastructure.yml  # 2. galera, rabbitmq, memcached, internal haproxy\nopenstack-ansible setup-openstack.yml       # 3. keystone, nova, neutron, cinder, glance, horizon...<\/code><\/pre>\n
A real gotcha hit during this install:<\/strong> instances had no internet connectivity until this was set:<\/p>\n
sed -i 's|is_metal:.*|is_metal: false|g' \/opt\/openstack-ansible\/inventory\/env.d\/neutron.yml<\/code><\/pre>\n
5. Non-default configuration (openstack_user_config.yml \/ user_variables.yml)<\/h2>\n
What actually deviates from OSA’s defaults in this deployment:<\/p>\n