„Bloom“ – another new AI language model

„Bloom“ – another new AI language model

San Francisco, 7/13/2022

On the website of the AI startup Hugging Face there is a reference to „Bloom“. This is an open source language model that can be tried for free without having to download it.

https://huggingface.co/bigscience/bloom

With 176 billion parameters, Bloom is similar in size to OpenAI’s GPT-3 model. More parameters generally means a more effective and sophisticated model. To protect against misuse, users must legally agree not to use Bloom for malicious purposes.

The open-access project is a collaboration between Hugging Face and two French supercomputing groups. Together, they formed BigScience, an international research group composed of more than 1,000 collaborators.

The predictive text model can write and summarize text, create poems and recipes, output code and more. It supports 13 programming languages and 46 languages – the first LLM of this scale to be multilingual. BigScience received $7 million worth of computing time on the Jean Zay supercomputer in France.

Unlike other large language models, Bloom was provided along with the code and datasets used to train it. This is part of an effort to further democratize large language models that generate and respond to text and allow researchers to reduce their potential damage.

Background: Earlier this year, Meta gave researchers free access to OPT-175B, a language model with 175 billion parameters, as well as the pre-trained models and code to train and deploy them. The smaller counterpart, OPT-66B, was also released for unrestricted download, meaning no access request is required.