Meta withdraws its Galactica science model after three days
Meta AI released Galactica on 15 November 2022, a language model trained on scientific text and intended to help write papers and summaries, with a public demo. Users showed it producing fluent but false claims and invented citations, and Meta took the demo down on 17 November.
Why it mattered The withdrawal became a standard example of fluent output being read as reliable, and of what a model trained on scientific prose does not thereby know about science.
Galactica was trained on scientific papers, reference material, and other technical text, and was offered as an aid to writing papers, summaries, and encyclopedia entries. A public demo let anyone type a prompt and read what came back.
Researchers began posting the results within hours. The model wrote confident scientific prose that was wrong, produced articles on subjects it had invented, and attached citations to papers that did not exist. The failures were hard to spot precisely because the register was right: the output looked like a paper in every respect except being true.
Meta took the demo offline roughly three days after putting it up. The paper describing the model stayed on arXiv.