Posted: 2026-07-28
I finished my fourth racking on the 24th. I added Odin and Braggi. I remembered to replace the CMOS batteries this time! I also found out that without a fresh-enough CMOS battery, these motherboards may not power on automatically after a power cut. Yikes! Both have up-to-date BIOS versions.
These are the first two hosts that have 64GB of memory and 2TB of storage. Previous hosts have all had 32GB and 1TB. They draw nominally more power. They have the same CPU as Loki.
Loki is fairly close to full. It's been performing quite well, but if I get another surge of sales I want to have a place to put new VPSs.
I've done some load testing under OpenBSD 7.9, filling up hosts. It seems like 7.9 schedules better than prior versions. This load testing was also done more realistically. The max I typically see on a host is one VPS using about 10% of one core. I found that 1MB/sec of gzipping /dev/urandom approximated that use. While there was no disk or network use in testing which makes it less realistic, I was able to get up to 49 of such servers going successfully.
50 or more started running out of TTYs, and I haven't looked into the numbering/lettering to see if more is possible.
At 49 things were stable, but load average was over 6. Though I seemed to see servers keeping up at the throttled (with pv) 1MB/sec of gzip.
At 40 VPSs, load average was always under 6, which I like considering the host has 6 cores. It was pretty performant.
This makes me think that I can probably exceed 4 VPSs per CPU core without much issue. I will try to pad them with larger servers to help minimize CPU spread, but at least with this last round of testing things were promising. Of course with disk and network at play, and some VPSs occasionally using a lot more CPU, it's a balance of how conservative to optimistic I should be to offer both a profitable (ha, eventually!) and performant product.
Switching to SSDs was a wise move. They're readily available used, draw less power, and they're particularly fast at "seeks" -- all nice in a VPS environment. While they shed data faster than mechanical disks when shutoff, that's generally an issue with this kind of usage. They do cost more, but I think it's worth it.
Moving forward, my plan is to (eventually) consolidate Prometheus and Ullr into Odin, and decommission them. They've been good, but I think the Opteron fleet is more trustworthy and power stable. I'll use their chassis, of course. Most of that will be migrating my own servers.
On a more ambitious topic, I am not thrilled with 1U chassis after doing some thermal testing (in a prior post.) If I could use 1.5U or 2U short depth chassis, they wouldn't need rails (I'm pushing the limits on this with my 1U chassis not having them) and they would need less power. Also, in the event that HVAC were to fail and temperatures were to climb, they would be stable for longer without throttling.
1.5U or 2U also solves my heatsink dilemmas nicely, as I know of used HP heatsinks that fit with modification. They make a lot of sense in 1.5U, but require excessive and questionable modification for 1U.
Now with Freya and Thor, they are truly short depth 1U servers and I have no sagging issues with them. But spare is held in with just screws and sags a fair bit. If all stacked, there's no issue, but being able to pull a bottom or middle host without a spacer seems wise. Although a spacer could be a fair solution (maybe a 2x4?) it's not my favorite. The wider screw spacing with 1.5U makes it a lot more stable. Also a shorter configuration to begin with, generally.
I'm also curious about switching to direct DC power to further improve efficiency. Both combined could be something! This is already pretty efficient, but I want to optimize things where I can.
Anyway, though, I don't think I can move to 1.5U or 2U (if I can even find the chassis to work with in the first place!) without a bigger cabinet. I modeled the 1/3 cabinet setup I'm in around 10 hosts, plus a spare. I can't fit that if the hosts are taller. But in a full cabinet, I could have a very reasonable power draw and a density that makes sense for Slow Servers.
Thanks for reading my rambling!