Balancing sustainability in AI training: Addressing compute resources and environmental impact

Compute resources and storage needs: AI training involves processing vast amounts of data, which requires significant compute resources and storage capacity. For example, the Meta Llama 3.1 model has 405 billion parameters and requires 213GB of storage. The storage needs for AI training are substantial, as large datasets must be stored and accessed quickly to train models effectively. 

Environmental impact and storage efficiency: The environmental impact of AI training is partly due to the energy consumption of data centres, which house the storage infrastructure. Efficient storage solutions can help reduce the overall energy consumption and carbon footprint. For instance, the Meta Llama 3.2 Vision model required 2.02 million hours of training and emitted 584 tonnes of CO2 equivalent (tCO2e). Optimising storage efficiency can contribute to more sustainable AI training practices. 

Optimising AI models and storage requirements: Reducing the size of AI models while maintaining their performance can decrease storage requirements. The Meta Llama 3.2 model was optimised to 3 billion parameters, reducing its size to just 2GB. Similarly, the lightweight version of the Meta Llama 3.2 Vision model has 11 billion parameters and requires only 8GB of storage. These optimisations help lower the storage demands for AI training. 

AI Inferencing and memory needs: AI inferencing, which involves running trained models to make predictions or decisions, also requires substantial memory and storage. For example, running a model with 7 billion parameters requires approximately 8GB of DRAM. Optimising AI models for inferencing, such as through model quantisation, can help reduce memory and storage requirements. 

Data centre architecture: Data centres are being rearchitected to handle neural computational workloads more efficiently, balancing higher compute density with lower power consumption. This includes optimising storage infrastructure to support the high data throughput and low latency required for AI training and inferencing. 

Increased storage for AI-denerated data: AI generates large amounts of data, which must be stored and managed efficiently. For example, generative AI models like large language models (LLMs) and diffusion models produce significant amounts of output data. This drives the need for high-capacity, high-performance storage solutions to handle the increased data volume.