Benchmark between Hetzner, DigitalOcean, Linode, Vultr, OVH and UpCloud (January 2023)

The different types of cloud providers

In our June 2022 article, we took a closer look at the three different types of cloud providers. By this we mean hyperscalers such as Amazon AWS, Google Cloud and Microsoft Azure. In addition, the mid-segment providers in which we find players such as DigitalOcean, Linode, Vultr, UpCloud and Hetzner. The last group of cloud providers are represented by local players such as Combell, Snel.com, TransIP and Versio.

Hyperscalers build and maintain their own data centers from which they offer their services. In their data centers they build their own networks and install their servers, which they often design themselves.

The mid-market players mostly use existing data centers (Equinix, Interxion, TelecityGroup, etc.). They rent space in these data centers (colocation) and build their own network there, with their own servers. Hetzner is the odd one out here, because they have 3 data centers (two in Germany and one in Finland) of their own. In addition, Hetzner also rents space at other locations in the US.

The local providers also use existing data centers, but often use smaller data centers that are close to their users. They also use their own servers and build out their own network, but on a smaller scale than the mid-market.

In this benchmark we compare a number of mid-market providers. This comparison was made in January 2023. Newcomer in this benchmark is UpCloud.

What we compare

In this blog post we make a purely technical comparison based on benchmarks. Is that a complete comparison? No, certainly not. There is much more to the choice of a cloud provider than just the technical performance of a virtual machine. A lot also depends on your needs: Do you only need virtual machines and block storage? Or do you also want to run Kubernetes nodes? Perhaps you also want managed databases based on MongoDB, MySQL or PostgreSQL? Some providers only offer a few products, others offer several products. Do you want to host everything in one location, or do you want to use data centers all over the world? And not unimportant to include: the monthly price.

In this comparison, we therefore only focus on benchmark results of a comparable virtual machine and its monthly price. We make no further comparison between other products they offer, the accessibility and quality of their service desk, the possibility of SLAs, their reliability, the geographical presence, and so on. We leave that to you.

We compare the providers below (with the package in brackets) and the country of the head office. All the virtual machines (VMs) have 2 vCPUs and 4 GB of ram. If we could make the choice, we chose an Intel CPU. In each case, we selected a VM with a 'shared CPU' and local storage space.

• Hetzner (CX21), Germany
• DigitalOcean (s-2vcpu-4gb), United States
• Linode (Linode 4 GB), United States
• Vultr (2 VCPU 4 GB), United States
• OVH (VPS Essential), France
• UpCloud (1 vCPU 4GB), Finland

In terms of cost, we pay to Hetzner, OVH and UpCloud in Euros and to DigitalOcean, Linode and Vultr in USD. In the table below we have converted USD's to Euro's. These prices are exclusive of VAT.

In the graph below, we compare the monthly cost per provider with our benchmark in June 2022. The small variations of Linode and Vultr may be related to the USD - EUR exchange rate.

The virtual machines had these specifications (so 2 vCPU and 4 GB ram each time):

How we compared the virtual machines

We ran the benchmarks on an Ubuntu 22.04 LTS installation. We used the Ubuntu installation as it was installed by the provider. We installed all updates, but made no further changes.

For the benchmarks we used Phoronix Test Suite v10.8.4. Phoronix performed each test three times. When a standard deviation (deviation) greater than 2.5% was observed between tests, an additional test was performed until the standard deviation fell below 2.5%, with a maximum of 40 tests. The average value was recorded as the result.

We performed the benchmarks below, with a focus on the system, a single and multi core CPU, the ram memory, the storage space and the network.

• pts/apache (focus on the systeem). This is a test of the Apache HTTPD web server. This Apache HTTPD web server benchmark test profile makes use of the Golang "Bombardier" program for facilitating the HTTP requests over a fixed period time with a configurable number of concurrent clients.
• pts/hint (focus on single core CPU). This test runs the U.S. Department of Energy's Ames Laboratory Hierarchical INTegration (HINT) benchmark.
• pts/compress-7zip (focues on multi core CPU). This is a test of 7-Zip compression/decompression with its integrated benchmark feature.
• pts/stream (focus on the memory). This is a benchmark of Stream, the popular system memory (RAM) benchmark.
• pts/postmark (focus on the storage). This is a test of NetApp's PostMark benchmark designed to simulate small-file testing similar to the tasks endured by web and mail servers. This test profile will set PostMark to perform 25,000 transactions with 500 files simultaneously with the file sizes ranging between 5 and 512 kilobytes.
• pts/speedtest-cli (focus on the network). This test profile uses the open-source speedtest-cli client to benchmark your Internet connection's upload/download performance and latency against the Speedtest.net servers. We did not manually select the Speedtest.net servers.

Results

Benchmark: pts/apache

Focus on the system. A higher score is better.

Benchmark: pts/hint

Focus on single core CPU. A higher score is better.

Benchmark: pts/compression-7zip

Focus on multi core CPU. A higher score is better.

Benchmark: pts/stream

Focus on the memory. A higher score is better.

Benchmark: pts/postmark

Focus on the storage. A higher score is better. Each VM used ext4 as its file system.

Benchmark: pts/speedtest

Focus on the network. Download & upload: a higher score is better. Latency: a lower score is better.

Conclusion

It is not our intention to designate "the best" and "the worst" provider based on these tests.

We made a percentage comparison of each test score. We made a total percentage distribution of all those scores in order to arrive at a final comparison. You can see this in the graph below.

We see that Linode, Vultr and UpCloud score above average and Hetzner and OVH fall below the average monthly cost.

We see that Hetzner, DigitalOcean and OVH have similar scores, but the prices are very different. Linode, Vultr and UpCloud are the strongest in this technical benchmark, of which UpCloud is the most expensive.

Does your workload mainly require CPU power? Then you will be satisfied with UpCloud. Linode shows its strong side when it comes to memory power and speed of its storage space. Vultr stands out for its good bandwidth. Hetzner, DigitalOcean and OVH are evenly matched in many areas, but OVH provides the least bandwidth and Hetzner gives a bang of 20 TB data traffic.

In terms of price-quality, Hetzner makes a very good move.

Try it yourself?

Want to try one of these cloud providers? With the (affiliate) links below you will already receive a starting budget.

• Hetzner - €20
• DigitalOcean - $200, to be used within 60 days
• Vultr - $100, to be used within 14 days
• UpCloud - €25