Express Computer
Home  »  News  »  OVHcloud introduces 7B AI model designed to run on lower hardware requirements

OVHcloud introduces 7B AI model designed to run on lower hardware requirements

0 8

OVHcloud has introduced Olala mini 7A1B, a 7-billion-parameter large language model (LLM) designed to reduce computing and memory requirements by activating only a portion of its parameters for each request.

The model uses a hybrid architecture combining Differential-TPA and Mamba-3 attention mechanisms, alongside a mixture-of-experts (MoE) design. According to OVHcloud, Olala mini has one billion active parameters distributed across 256 experts per layer, allowing it to process requests without activating the entire model.

The company claims the model can run on a GPU with as little as 8GB of memory, delivering 160 tokens per second on an NVIDIA RTX A4500. It can also run on a standard CPU, with OVHcloud reporting speeds of up to 50 tokens per second. The model currently supports a context window of 32,000 tokens.

The model is scheduled to be released on Hugging Face under the Apache 2.0 licence, with its weights, pre-trained base model, technical report and AI Act documentation.

Smaller models and inference efficiency

The architecture is intended to reduce the computing resources required to run AI inference. In a mixture-of-experts model, multiple specialised components, or experts, are available within the model, but only selected experts are activated for a particular input.

This allows a model to retain a larger overall parameter count without necessarily requiring the same computational resources as a dense model with all parameters active for every request.

Reducing hardware requirements could make it easier for businesses and developers to deploy AI models on their own infrastructure, including systems with limited GPU memory. However, actual performance will depend on workload, hardware configuration, model quantisation and the complexity of requests.

OVHcloud said search engine Qwant has already adopted Olala mini for its flash-answer feature. The company claims the deployment has reduced response times by 20% and cut operating costs to one-third of those associated with the previously used model.

Public computing resources used for training

Olala mini was trained using three European public supercomputers – Jupiter, MeluXina and Jean Zay – equipped with NVIDIA GH200, A100 and H100 GPUs.

OVHcloud said the training received public computing support equivalent to €7 million in computing hours. The initiative also contributed to the creation of the company’s Agentic Lab, OVHai LLM, which will work on developing further models.

The model’s release is part of OVHcloud’s efforts to build its own AI capabilities rather than depend entirely on models developed by external providers.

Further models planned for enterprise workloads

OVHcloud plans to release Olala version 1.1 by the end of 2026, with an expanded context window of 128,000 tokens and improvements in mathematical reasoning.

The company is also planning Olala 2.0, which is expected to exceed 100 billion parameters and include models designed for enterprise agentic AI, cloud DevOps and cybersecurity.

For enterprise AI, the company plans to use the models in OVHai Workspace, including an encrypted version designed to run locally on workstations. Its DevOps models are expected to target developers, architects and operations teams, while its cybersecurity models will cover defensive and offensive applications.

OVHcloud CEO Octave Klaba said the company aims to develop models for its own requirements and those of customers, adding that the cost of training LLMs has declined significantly.

Leave A Reply

Your email address will not be published.