vps performance vibe check
#author_luna #sysadmin-notes #vps #living_article
as part of bsky dot, months ago I ran some experiments in regards to how much performance vps systems provide, because spinning up a gpu at the time wasn't something I wanted to do due to cost.
the benchmark is as follows:
- https://github.com/lun-4/bsky-dot
- its all a mess there, but
testrunner.pystarts up an api running thecardiffnlp/twitter-roberta-base-sentiment-latestmodel - setup nginx, authentication, etc
- then I run the main (go) webserver on my desktop against the server and only that server and see its sentiments per second rate.
IMPORTANT NOTES
- this benchmark is compute and memory(?) intensive, but not disk intensive, different methodologies apply for workloads that need the i/o
- the actual value of a VPS goes beyond its performance, and that MAY matter to you more than it does to this type of workload, including but not limited to:
- support by the compute provider
- Acceptable Use Policy
- a lot of providers will execute you for slamming the CPU all day!!!
- compute liquidity (since you do have fixed budgets in a data center)
- I'm running off upcloud's free trial of 7eur, other clouds were paid for with my (or others) money. somehow upcloud gave me 100eur, probably because of their fucked marketing strategy, but I'm normalizing by the same cost and making my conclusions with it in mind.
data #
| compute type | max rate (sentiments/second) | price per month | sentiments per percond per euro | cpu | ram |
|---|---|---|---|---|---|
| my desktop | 12.13 | 0 (1) | infinite | 12 | 32gb |
| nat's ovh server (2) | 11.71 | 29.69eur | 0.394 | 8 | 32gb |
| nat's hetzner server (3) | 12.32 | 44.05eur | 0.279 | 8 | 64gb |
| scaleway DEV1-S | 3.64 | 10eur | 0.364 | 2 | 2gb |
| hetzner CPX11 | 10.10 | 3.85eur | 2.62 | 2 | 2gb |
| hetzner CX22 | 5.43 | 3.29eur | 1.650 | 2 | 4gb |
| hetzner CX22 | 6.48 | 3.29eur | 1.96 | 2 | 4gb |
| hetzner CX22 | 6.68 | 3.29eur | 2.03 | 2 | 4gb |
| scaleway DEV1-S | 3.76 | 10eur | 0.376 | 2 | 2gb |
| scaleway DEV1-M | 6.17 | 19eur | 0.324 | 3 | 4gb |
| ovh VLE-4 | 20.27 | 11usd (9.45eur) | 2.14 | 4 | 4gb |
| kimsufi ks-a | 12.32 (4) | 5.60 | 2.2 | 8 | 32gb |
| upcloud DEV-1xCPU-2GB | 8.29 (5) | 8eur | 1.03 | 1 | 2gb |
| upcloud DEV-2xCPU-4GB | 11 | 18eur | 0.61 | 2 | 4gb |
| upcloud 1xCPU-1GB | 3 (5) | 10eur | 0.3 | 1 | 1gb |
| upcloud 2xCPU-2GB | 11 | 20eur | 0.55 | 2 | 2gb |
(1) power bill not included. it'd be difficult to calculate and I don't care
(2) OVH SYS-1-SAT-32, Xeon D-1520. was sharing with other applications but was mostly idle
(3) Hetzner server auction, Xeon E3-1275 v5. relatively busy with other apps
(4) I should rerun this one, but lets leave it at that for now. dont care that much
(5) both these results are both suspiciously good and suspiciously bad for the given price points. my current hypothesis is that DEV/non-DEV boxes may be scheduled to good/bad CPU hosts (AMD EPYC 9575F), impossible to know without knowing upcloud's scheduler. this article may be updated over time
conclusions #
- hetzner CPX11 remains the most cost-efficient VPS in this type of workload. it is what I've been defaulting to for the project (even though right now the project is stalled due to focus on other projects. see discover feed fever dream)
- if hetzner is not "liquid" enough in terms of compute, I can fallback in this order:
- OVH (either normal or Kimsufi/"OVH Starter")
- Upcloud
- Scaleway
- I can also fallback to my own desktop or other server HW as well (including GPUs if needed!), this workload is heavily focused on just getting as much compute as I can for a data processing job