In a recent MLPerf GPT-J inference benchmark, Intel's Gaudi 2 achieved near-parity performance with Nvidia's H100, reinforcing its position as a formidable alternative.
The Intel® Gaudi® 2 accelerator is a heavyweight contender in the AI accelerator arena, designed specifically for deep learning training and inference. It delivers two times the performance of A100 on Computer Vision, NLP, and large scale models.
Gaudi 2 shrinks the process from 16nm to 7nm, increases the number of Al-customized Tensor Processor Cores from 8 to 24, adds FP8 support, and integrates a media compression engine.
Additionally, Intel remains the exclusive server CPU vendor to submit MLPerf results. The 5th Gen Intel Xeon processors have shown an impressive 1.42x improvement compared to their 4th Gen counterparts in MLPerf Inference v3.1.
 |
| Intel Gaudi 2 |
Developed by Habana Labs, now part of Intel, the Gaudi 2 is equipped with a whopping 96GB of HBM2E memory offering ample space to store and process massive datasets. Additionally, the high bandwidth of 2.45 TB/s ensures smooth data flow during training and inference processes, minimizing bottlenecks.
Habana Labs is considered the center of excellence for AI solutions at Intel, and Intel acquired the company in 2019 for approximately $2 billion.
It is said that Habana Labs is now working on Gaudi 3, which is expected to offer a significant performance boost.
Price-to-Performance: It provides strong performance-per-dollar.
MLPerf Results: Intel is the exclusive server CPU vendor to submit MLPerf results, showcasing Gaudi 2's capabilities.
Scalability: Gaudi 2 integrates 24 100-gigabit RDMA over Converged Ethernet (RoCE2) ports, making it cost-effective and easy to scale out training capacity.
Networking: These ports enable efficient communication within the server, enhancing throughput.
Performance: Gaudi 2 is about twice faster than the NVIDIA A100 80GB for both training and inference.
Habana Labs is considered the center of excellence for AI solutions at Intel, and Intel acquired the company in 2019 for approximately $2 billion.
It is said that Habana Labs is now working on Gaudi 3, which is expected to offer a significant performance boost.
Intel Gaudi 2 Vs NVIDIA H100 / A100
Gaudi 2 vs. NVIDIA H100
Performance: Gaudi 2 remains the only benchmarked alternative to Nvidia H100 for generative AI (GenAI) performance.Price-to-Performance: It provides strong performance-per-dollar.
MLPerf Results: Intel is the exclusive server CPU vendor to submit MLPerf results, showcasing Gaudi 2's capabilities.
Scalability: Gaudi 2 integrates 24 100-gigabit RDMA over Converged Ethernet (RoCE2) ports, making it cost-effective and easy to scale out training capacity.
Networking: These ports enable efficient communication within the server, enhancing throughput.
Intel Gaudi 2 vs. NVIDIA A100
Inference: Gaudi 2 matches the latency of Nvidia H100 systems on decoding and outperforms the Nvidia A100 in large language model (LLM) inference.Performance: Gaudi 2 is about twice faster than the NVIDIA A100 80GB for both training and inference.
Available on the Intel Developer Cloud for easy access.
Memory Bandwidth: Gaudi 2 achieves higher memory bandwidth utilization than H100 and A100.
In summary, Intel Gaudi 2 offers compelling performance-per-dollar and can be a respected alternative to Nvidia's offerings. While Nvidia A100 remains a powerhouse, Gaudi 2 provides better value in many scenarios.
Memory Bandwidth: Gaudi 2 achieves higher memory bandwidth utilization than H100 and A100.
In summary, Intel Gaudi 2 offers compelling performance-per-dollar and can be a respected alternative to Nvidia's offerings. While Nvidia A100 remains a powerhouse, Gaudi 2 provides better value in many scenarios.
Advertisements