Google publishes BERT, a bidirectional language model

Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova published BERT on 11 October 2018, a transformer that pretrains representations of text by reading in both directions, then fine-tuned for particular tasks. Google released the code and weights the following month.

Why it mattered BERT became the standard baseline in language research for several years and was later built into the ranking systems behind Google Search.