Projects · Mini Hostpapa

Volume 2 Lab Environment

Turn the Volume 1 network design into a running simulated datacenter on one laptop. OVS as the top of rack switch, nested KVM, NFS storage, and Apache CloudStack on top.

Updated Aug 4, 2026 · 11 min read

This plan is applied and ready to use

All of the planification in this document (and the rest of Volume 2) is implemented as runnable Ansible, Terraform, and CloudMonkey helpers in mini-HostPapa/virt-dc. Clone that repo, follow its README, and you get the fabric, the rack VMs, CloudStack, the zone path, and the first-instance gate without retyping these snippets by hand.

Volume 2 Lab Environment

Volume 1 ended with a complete network design. Traffic types, VLAN assignments, IP ranges, bridge names, port matrices, an IPv6 plan. Every number I needed was written down.

And not one packet had moved.

That is the honest situation at the end of Volume 1, and I want to name it clearly rather than glide past it. A design that has never been built is a hypothesis. It is a good hypothesis, researched against the official documentation and reasoned through carefully, but it has never once been contradicted by a machine. That is not the same thing as knowledge.

What this volume is for

Volume 2 exists to make the Volume 1 design wrong in specific places, so I can fix it before it matters.

The output is not a document. The output is a running environment: Apache CloudStack installed, a zone created, a hypervisor host registered, system VMs green, and a real instance booting inside it.

Why this is not a Linux fundamentals volume

The original plan for Volume 2 was Linux administration. Users, groups, SSH, systemd, filesystems. I scratched it, and I want to say that plainly, because "I skipped the fundamentals" sounds like a red flag until you know where those fundamentals actually live.

Here is the decision in one sentence: this knowledge base already has a home for Linux fundamentals, and duplicating that material inside the HostKid project would make both places worse. Enormous docs that repeat the same systemd explainer twice help nobody. Volume 2 exists to get straight to the point — building the lab, wiring the network, standing up CloudStack — not to re-teach Linux from zero inside a project track.

If you need the fundamentals first, that is not a gap in this volume. That is what the Linux foundations section is for: a progressive path through the operating system itself, separate from any one project. Read that when you want the mental model. Come back here when you want the rack on your laptop.

Linux still shows up here on purpose

This volume is not "Linux-free." By the end I will have configured netplan bridges and VLAN sub-interfaces, written systemd units, tuned kernel parameters through sysctl, managed NFS exports, debugged firewall forwarding, and read journald logs in anger.

The difference is scope. I assume you can open the foundations track when a concept needs depth. Here, every command serves the HostKid lab — nothing extra, nothing repeated from a textbook chapter you already have elsewhere.

The constraint, stated honestly

I am building a datacenter simulation on the laptop I also work on. Here is what I actually have, measured rather than assumed:

ResourceTotalReality check
CPU8 threads (Intel i7-1165G7, 4 physical cores)VT-x present, so nested virtualization is possible
RAM15 GiB usableAlready sitting at 13 GiB used with my normal workload running
Swap11 GiBAlready 8.9 GiB used
Disk64 GiB free on a root filesystem that is 86% fullMy self-imposed ceiling is 50 GiB, and I will land at 44 GiB
This is genuinely tight, and pretending otherwise would waste your evening

Look at that RAM row again. 13 of 15 GiB were already in use when I measured, with 1.5 GiB available. A CloudStack management server plus a nested KVM hypervisor needs roughly 11 GiB.

The arithmetic does not work while my IDE, browser, and containers are running. So the lab is not designed as something that hums along in the background. It is designed as a mode I deliberately switch into, with a documented shutdown of everything else first.

I would rather tell you that on page one than have you discover it when the OOM killer reaps MySQL at 2am.

The whole resource ledger, and the discipline for living inside it, is worked out in lab architecture and resource budget.

What gets simulated, and what gets faked

The interesting engineering question in a lab like this is not "how do I make it run". It is which parts must be real and which parts may be pretend. Get that wrong in either direction and the lab is worthless: too much pretending and it teaches nothing, too much realism and it never boots.

My rule was this: anything that CloudStack touches, or that I would have to configure on real hardware, must be genuinely real. Anything that only exists because production has redundancy or physical separation may be collapsed.

LayerLab treatmentWhy
802.1Q VLAN taggingReal. Open vSwitch does actual tagging on actual trunk portsThis is what CloudStack manipulates. Faking it teaches nothing
Bridges cloudbr0 and cloudbr1Real. Real Linux bridges, same names as productionCloudStack traffic labels must match Volume 1 exactly
Nested hypervisorReal. KVM inside KVM, /dev/kvm present in the guestThe agent, the system VMs, and live migration all depend on it
NFS primary and secondary storageReal. Real exports, real mounts, real MTU 9000 pathStorage mount failures are the most common CloudStack setup problem
Edge firewall and NATReal. nftables masquerade, real internet egressThe Secondary Storage VM must genuinely reach the internet
Top of rack switchCollapsed. One OVS bridge on the laptop plays the ToRA physical switch is not simulatable, but its behaviour is
Redundancy (bonds, A/B power, MLAG, dual firewall)Faked or skippedRedundancy protects against failures the lab cannot have
Physical separation of serversCollapsed. The laptop is also the storage arraySaves roughly 1.5 GiB of RAM and buys no learning

Every collapse above is recorded in a deviation register in the architecture document, with the production equivalent written next to it. That register is the thing that stops a lab from quietly becoming a lie you believe.

The three levels of nesting

This is the mental model that makes everything else in the volume readable. There are three floors, and confusing them is the single biggest source of "why is this not working".

Preparing diagram
Here is how I keep the floors straight

When something breaks, my first question is always which floor am I standing on.

A VLAN problem at L0 is an ovs-vsctl problem. The same symptom at L1 is a netplan or bridge problem. At L2 it is a CloudStack traffic label problem.

Three completely different toolboxes, one identical symptom of "the ping does not work". Naming the floor first has saved me more time than any single command in this volume.

The software I chose, and the one-line reason

The full reasoning, including what I rejected, is in the architecture document. The short version so you know what you are installing:

JobChoiceThe one-line reason
Hypervisor at L0KVM / QEMU / libvirtNative, free, and the only sane way to get nested virtualization on Linux
Network fabricOpen vSwitchIt performs genuine 802.1Q tagging and trunking, so the VLANs are not a simulation
Fabric definitionlibvirt network XML with OVS portgroupsDeclarative VLAN access and trunk ports in a file I can version control
VM lifecycleTerraform with dmacvicar/libvirt 0.9.8Declarative, and it warms up the Terraform muscle this project needs by Volume 7
Guest OS configcloud-initThe industry standard, and CloudStack templates use it too
Host and server configAnsibleIdempotent, and it is already the Volume 21 choice
Rack server OSUbuntu 24.04 LTSOn the CloudStack 4.22 support matrix, and cloud images exist
Tenant instance OSAlpine Linux 3.24A 190 MB image that boots happily in 192 MiB of RAM. This is what makes the budget work
The platformApache CloudStack 4.22The Volume 3 decision, deployed exactly as Volume 1 designed it
Why Terraform and Ansible instead of one tool

This split is deliberate and I use the same rule in production.

Terraform owns whether a thing exists. Virtual machines, disks, networks, cloud-init seeds. Things with a lifecycle, where "destroy and recreate" is a sensible operation.

Ansible owns the state inside a thing that already exists. Packages, config files, services, mount points.

Mixing them produces the classic mess where nobody can answer "what happens if I run this twice". Keeping the boundary clean means I can always rebuild the whole lab from zero with two commands.

How this volume is organised

Four documents, in build order. They are meant to be executed in sequence, not browsed.

#DocumentWhat you have when you finish it
1Lab architecture and resource budgetThe full design, the RAM and disk ledger that proves it fits, the deviation register, and the tool selection reasoning
2Host preparation and network fabricA laptop with nested virtualization on, an OVS ToR carrying real VLANs, working SVIs, NAT to the internet, and NFS exports
3Provisioning the rackTwo booted, addressed, SSH-reachable servers built from a Terraform plan you can destroy and rebuild
4Installing CloudStack and the first instanceThe socle: CloudStack running, zone created, host added, system VMs green, and an Alpine instance with an IP

The gate that ends this volume

I do not consider Volume 2 finished because the documents are written. I consider it finished when these are all true on my own machine:

  • terraform apply from an empty state produces two running, SSH-reachable servers with no manual steps
  • terraform destroy followed by terraform apply reproduces the environment, proving the lab is disposable rather than precious
  • The whole lab footprint stays under 50 GiB, verified with a single du -sh
  • The lab starts and stops with one command each, so switching into lab mode is not a ritual
That fourth tab is the one that matters

Three of those tabs are checklists. The last one is the actual point of the volume.

If I get to the end and nothing in the Volume 1 design turned out to be wrong, I should be suspicious rather than pleased. It would almost certainly mean I built the lab to match my document instead of building it honestly and letting it argue back.

Next

Start with lab architecture and resource budget. It is the only document in this volume with no commands in it, and it is the one that decides whether the other three work.

After this volume, Volume 3 revisits the platform decision with the benefit of having actually run the thing, and Volume 7 picks up the CloudStack API that this volume leaves authenticated and waiting.