MLPerf 4.1 Benchmarks: Google Trillium Provides 1.8x Performance-Per-Dollar to Enhance AI Training
The constantly evolving generative AI models are putting unprecedented strain on hardware accelerator speed and efficiency. Google unveiled Trillium, their sixth-generation Tensor Processing Unit (TPU), to satisfy the demands of next-generation models. In order to facilitate training on an extraordinarily large scale, Trillium is expressly built for performance at scale, from the chip to the system to its Google data center deployments.
Today, Google released the first MLPerf training benchmark results for Trillium. Google Trillium provides an impressive 99% scaling efficiency (throughput) and up to 1.8x higher performance-per-dollar than the previous generation Cloud TPU v5p, according to the MLPerf 4.1 training benchmarks.
This blog offers a brief performance analysis of Trillium, demonstrating why it is currently the most cost-effective and efficient TPU training option. The first system comparison parameter that Google briefly examines is traditional scaling efficiency. Google lists convergence scaling efficiency as a crucial factor to consider in addition to scaling efficiency. It assesses these two factors in addition to performance per dollar when contrasting Google Trillium with Cloud TPU v5p. It concludes with suggestions to help you make informed decisions about your cloud accelerators.
Conventional performance indicators
Throughput scaling efficiency, effective throughput, and peak throughput are some of the metrics that can be used to evaluate and compare accelerator systems. Despite being helpful indicators, none of these metrics take convergence time into consideration.Specifications of the hardware and peak performance
In the past, comparisons mostly focused on hardware attributes such as peak throughput, memory bandwidth, and network connectivity. Despite setting theoretical boundaries, these peak values are not very effective in predicting real-world performance, which is mostly influenced by architectural design and software implementation. Because modern machine learning workloads typically involve hundreds or thousands of accelerators, the most crucial metric is the effective throughput of a system that is the appropriate size for a given task.Utilization performance
System performance can be assessed using utilization metrics like memory bandwidth utilization (MBU) and effective model FLOPS usage (EMFU), which contrast achieved throughput with peak capacity. However, there is no clear correlation between these hardware efficiency indicators and business-value measures like model quality or training duration.Scaling efficiency and trade-offs
A system's scalability is evaluated using both weak scaling (efficiency when workload and system size increase correspondingly) and strong scaling (performance improvement with system size for fixed workloads). The ultimate goal is to generate high-quality models as quickly as feasible, which sometimes justifies foregoing scaling efficiency in favor of faster training periods or better model convergence, even though both metrics are helpful indicators.Efficiency in convergence scaling is required.
While hardware use and scaling indicators provide useful system insights, convergence scaling efficiency focuses on the main goal of training: successfully reaching model convergence. Convergence is the point at which the error rate stabilizes and the output of a model stops getting better. Convergence scaling efficiency measures how well more computing resources accelerate the training process to completion.We assess convergence scaling efficiency using two essential metrics: the scaled scenario, where N₁ accelerators require time T₁ to converge, and the base case, where a cluster of N₀ accelerators converges in time T₀. The ratio of the growth in cluster size to the acceleration of convergence time is as follows:
Efficiency of convergence scaling
The convergence scaling efficiency is 1 when the time-to-solution rises by the same proportion as the cluster size. Consequently, it is desirable to have a convergence scaling efficiency as close to 1 as is practical.In order to understand our ML Perf proposal for the Google Trillium and Cloud TPU v5p training challenge for GPT3-175b, let's use these concepts.
Trillium's performance on Google
Google provided the GPT3-175b training results for four different Google Trillium installations and three different Cloud TPU v5p settings. In the study that follows, the findings are grouped by cluster sizes with the same total peak flops for comparison. For example, the Cloud TPU v5p-4096 configuration is compared to 4xTrillium-256, the Cloud TPU v5p-8192 configuration to 8xTrillium-256, and so on.All of the results in this study are based on MaxText, Google's high-performance reference solution for Cloud TPUs and GPUs.
Low efficiency of scaling
For increasing cluster sizes with proportionately larger batch sizes, Trillium and TPU v5p both offer almost linear scaling efficiency:Comparison between Trillium and Cloud TPU v5p's weak scaling
Efficiency of convergence scaling
Convergence scaling efficiency, as discussed above, accounts for time-to-solution, while weak scaling is useful but not a sufficient value signal.Comparison between Cloud TPU v5p and Trillium convergence scaling.
For the largest cluster size, it was discovered that Cloud TPU v5p and Google Trillium had comparable convergent scaling efficiency. With a CSE of 0.8 in this instance, the rightmost configuration's cluster size was three times more than that of the base configuration, and it took 2.4 times less time to converge (2.4/3 = 0.8) than the base configuration.
The last criterion is that, while Google Trillium and TPU v5p have comparable convergence scaling efficiency, Trillium is superior at offering convergence at a lower cost.
The price of training
Even while weak scaling efficiency and convergence scaling efficiency demonstrate the scaling features of systems, Google has not yet looked at the most crucial factor, which is the cost of training.
Cost-to-train comparison between Cloud TPU v5p and Trillium based on wall-clock time and on-demand list pricing.
Cost-to-train comparison between Cloud TPU v5p and Trillium based on wall-clock time and on-demand list pricing.
Google Trillium reduces the training cost by up to 1.8x (45%) while achieving convergence to the same validation accuracy as TPU v5p.
Choosing a Cloud Accelerator with knowledge
This article explored the intricacy of comparing accelerator systems, emphasizing the importance of looking at more than simply measures to ascertain true performance and efficiency. It found that while peak performance metrics provide a starting point, they often fall short of accurately predicting usefulness in practice. Instead, metrics like Effective Model Flops Utilization (EMFU) and Memory Bandwidth Utilization (MBU) offer more detailed information about an accelerator's performance.It also underlined the critical importance of scaling features, including both strong and weak scaling, in evaluating how effectively systems perform as workloads and resources increase. However, because it ensures that it is comparing systems based on their ability to accomplish the same result rather than just their speed, convergence scaling efficiency is the most objective statistic it identified.
In their benchmark submission with GPT3-175b training, Google Cloud showed that Google Trillium lowers the cost-to-train while achieving similar convergence scaling efficiency to Cloud TPU v5p and higher employing these techniques, resulting in 1.8x better performance per dollar. These results highlight the importance of evaluating accelerator systems according to a range of efficiency and performance standards.
0 Comments