Report: Overall AWS, Azure, and GCP Performance Assessment – Which is Best?

An annual report prepared by Cockroach Labs finds a statistical dead heat among the big three cloud providers. The report evaluates the performance of AWS, Azure, and GCP for general online transaction processing (OLTP) workloads. Tests show that all three cloud providers offer price-competitive options as well as best-in-class performance for OLTP applications, indicating a shift in the market as the industry operationally matures and standardizes linear pricing models.

This year, the report evaluated 56 instance types and 107 discrete configurations, and ran more than 3,000 runs across the following broad axes, including OLTP performance, CPU performance, network performance, and storage I/O performance. In addition, this year’s report improved the test design configuration for all tests and added tests for different sized instance types; cross-region latency; memory testing with fsync; variable OLTP workload complexity; and 12 times longer throughput test duration, from 60 seconds per test previously to 12 minutes per test.

The report shows that AMD outperforms Intel in overall performance: machines with AMD Milan (EPYC Gen 3) processors took the top spots in both CPU benchmarking and OLTP tests for both large and small instance types. In recent years, Intel has been the leader in overall performance, while AMD has competed in price/performance. This year, AMD-based instances were both the overall performance leader and the price/performance leader.

Additionally, all three clouds have competitively priced offerings: in terms of price (and performance), all three clouds were in a statistical dead heat. Even slightly more expensive instance and storage combinations are potentially very competitive depending on the needs of a particular workload.

Finally, storage and transmission costs have an outsized impact on the total cost of ownership: “Hidden” costs for storage and data transmission can impact the total cost more than the price of the instances themselves, especially when it comes to building a highly resilient stateful application.

Read the full Cockroach Labs report.