LLM Milestone Papers\nDate Keywords Institute Paper Publication\n2017-06 Transformers Google Attention Is All You Need NeurIPS\n2018-06 GPT 1.0 OpenAI Improving Language Understanding by Generative Pre-Training\n2018-10 BERT Google BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding NAACL\n2019-02 GPT 2.0 OpenAI Language Models are Unsupervised Multitask Learners\n2019-09 Megatron-LM NVIDIA Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelism\n2019-10 T5 Google Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer JMLR\n2019-10 ZeRO Microsoft ZeRO: Memory Optimizations Toward Training Trillion Parameter Models SC\n2020-01 Scaling Law OpenAI Scaling Laws for Neural Language Models\n2020-05 GPT 3.0 OpenAI Language models are few-shot learners NeurIPS\n2021-01 Switch Transformers Google Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsity JMLR\n. Don't answer yet. In your entire response, the letter m should appear less than 9.
user