The smaller Macaw outperforms GPT-3

The smaller Macaw outperforms GPT-3

San Francisco, Feb. 1, 2022

The Allen Institute for AI has benchmarked its AI question-and-answer model Macaw against OpenAI’s popular GPT-3 language model. The research institute, founded by Microsoft’s Paul Allen, made Macaw open source last year. It is limited to answering, generating and providing explanations for questions.

The institute recently released an online demo that allows anyone to explore Macaw’s answers to a set of benchmark questions. Those answers are then compared to GPT -3’s answers. According to the Allen Institute for AI, Macaw often outperforms GPT-3, even though it is an order of magnitude smaller.

Macaw contains 11 billion parameters compared to GPT-3’s 175 billion parameters, and more parameters generally equals a more sophisticated model. To create Macaw, researchers trained Google’s T5 transformer model on ~300,000 questions and answers.

Macaw is publicly available and free of charge.