Huawei presents its vision for the future of data storage at the Innovative Data Infrastructure Forum 2024 held in Berlin, Germany. The Chinese giant is committed to an AI Ready Data infrastructure with high performance, reliable, scalable, sustainable and that allows data fabric.
Dr. Peter Zhou, vice president of Huawei and president of data storage product line, highlighted the need to transform traditional data storage to meet the demands of the AI era. According to Zhou, data consumption is skyrocketing and current storage technologies are not up to par to deliver what is expected of them.
Huawei proposes a series of solutions to address these challenges, aligned with the general industry trend. towards AI and data analysis. As businesses collect and analyze more data, they need more powerful and scalable data storage solutions.
Current issues that need to be addressed
Since the nineties, the Enterprise storage applications have been evolving from simple hosting, databases and virtualization, to Big Data and high performance analytics (HPDA), which has already led to a shift in DAS, SAN and NAS storage technologies to have the ability to handle unstructured data . Now, generative AI is the next step, with models that increasingly make use of the GPU, creating new challenges and problems for training algorithms.
The long process of repeatedly writing checkpoint data and resuming training generates downtime on the GPU, with AI model performance drops of up to 50%. This implies problems in the efficiency of the data center, that is, unproductive energy consumption. It is estimated that by 2026 the energy consumption of data centers globally will be 2.3x times higher than in 2022, and more than half of this total consumption will be caused by the processing of AI algorithms.
Huawei’s AI Ready Data infrastructure brings the solution
Las keys to Huawei’s proposal for infrastructure are:
- Ultra performance– Increase the performance of today’s data storage by a factor of 1 to 10, with PB/s bandwidth and 100 million IOPS. Something fundamental, since AI needs to be fed with a large amount of data, and the latency of access to this data is vital.
- Data resilience: high reliability of 99.9999% and ransomware detection with an efficiency of 99.99%, thus avoiding reliability or service availability problems.
- New data paradigm: Support for multidimensional data, fast data recovery, and hallucination removal in large AI models.
- Scalability– Expansions will be allowed at the EB capacity level and support for multiple GPUs, DPUs or NPUs.
- Sustainability– Exceptional storage power efficiency (less than 1 W/TB) and storage density (greater than 1 PB/U).
- Data structure– Improvements in global data visibility and management, with 10x more efficient data mobility.
These innovations have materialized in the launch of the OceanStor A800, a high-performance storage system that offers 30% more AI model utilization, four times more bandwidth, and eight times more IOPS than competitive solutions. Huawei is also driving sustainable development with new storage media, such as high-capacity SSDs that provide 10 times more capacity with the same disk size and reduce power consumption by 88%.
Additionally, to be AI-ready, businesses need a robust and scalable data infrastructure. Huawei offers a comprehensive solution that includes the Omni-Dataverse global file system, which makes data assets visible, manageable and mobile across regions.