Search results
Chinchilla is a family of large language models developed by the research team at DeepMind, presented in March 2022. It is named " chinchilla " because it is a further development over a previous model family named Gopher.
Apr 11, 2022 · The star of the new paper is Chinchilla, a 70B-parameter model 4 times smaller than the previous leader in language AI, Gopher (also built by DeepMind), but trained on 4 times more data. Researchers found that Chinchilla “uniformly and significantly” outperforms Gopher, GPT-3, Jurassic-1, and Megatron-Turing NLG across a large set of ...
DeepMind's Chinchilla AI is an AI-powered large language model that has outperformed existing models like GPT-3 and Gopher on an array of tasks.
Mar 29, 2022 · We test this hypothesis by training a predicted compute-optimal model, Chinchilla, that uses the same compute budget as Gopher but with 70B parameters and 4$\times$ more more data. Chinchilla uniformly and significantly outperforms Gopher (280B), GPT-3 (175B), Jurassic-1 (178B), and Megatron-Turing NLG (530B) on a large range of downstream ...
Apr 9, 2022 · Chinchilla is a massive language released by DeepMind as part of a recent paper that focuses on scaling large language models in a compute-optimal manner. It outperforms recent models like GPT-3,...
Mar 28, 2022 · Chinchilla is a 70B parameters model trained as a compute-optimal model with 1.4 trillion tokens. Findings suggest that these types of models are trained optimally by equally scaling both model size and training tokens. It uses the same compute budget as Gopher but with 4x more training data.
Feb 15, 2023 · Chinchilla is an autoregressive decoder-only language model. Trains on a similar dataset as Gopher. Use SentencePiece. Has 70B parameters, 80 layers, 64 heads, 128 (key/value of each head), 8192 hidden dimension, batch size (starts at 1.5M, then double to 3M midway through training). Evaluation.
Oct 21, 2023 · So What Exactly is Chinchilla AI? Chinchilla is a deep learning model trained by DeepMind to understand and generate human language. Specifically, it employs something called transformer neural networks to process massive amounts of text data.
Apr 12, 2022 · by Kartik Wali. Researchers at DeepMind have proposed a new predicted compute-optimal model called Chinchilla that uses the same compute budget as Gopher but with 70 billion parameters and 4 times more data.
Jan 14, 2023 · Chinchilla outperforms Gopher, GPT-3, Jurassic-1, and Megatron-Turing NLG on a range of downstream evaluation tasks.
