Infrastructure

waffle.tech runs primarily on a Proxmox VM host, located in a datacenter in Las Vegas, Nevada.

We need some rules to keep things orderly. Here are some standard engineering practices for WT infrastructure.

• NAT: unfortunately, it is a reality of our modern world that most VMs need to be behind NAT. The vmbr1 bridge on the VM host is the private network, behind NAT. It is possible to forward ports from the VM host's main IP to VMs if required.
• IP address management: Proxmox assigns an ID number to each VM. The vmbr0 private network uses 10.50.0.0/16. As a convention, VMs should use a statically allocated IP that corresponds to their ID number in 10.50.1.0. For example, the VM with ID 106 should use the IP address 10.50.1.106. This makes it easy to figure out what IP a VM is on, and what IP a new VM should use.
• Each VM should have a record under infra.waffle.tech pointing to its internal IP, with a subdomain the same as the VM name. For example, the mail server is called 'mx' and 'mx.infra.waffle.tech' points to its internal IP.

Unfortunately external IPs are a scarce resource, but here's the set of them:

• 104.225.250.114: “primary” external IP, used to connect to the VM host itself and most services via port forwarding.
• 104.225.250.115: the login VM is directly on the internet with this IP. It is also used as the “outbound” IP for VPN users, and has the PTR (reverse DNS) name users.waffle.tech.
• 104.225.250.116: used by wikimap. It probably doesn't need to be this way but that's future work to change.
• 104.255.250.117: used by nein, the plan9 cpu+auth+file server on the network. Accounts on this system are managed separately and must be requested.

The main type of inbound connection is HTTP. For web access, we take advantage of nginx as a reverse proxy to “fan out” traffic to the correct VM. The proxy machine (proxy.infra.waffle.tech) receives http/https/http2/http3 connections to the primary external IP. It serves some websites (e.g. waffle.tech itself, user homepages) directly, by NFS mount from the login server for user homepages. Other webpages are reverse proxied to internal VMs as appropriate, e.g. idm.waffle.tech is reverse proxied to the IDM server idm.infra.waffle.tech.