Use BertLMHeadModel to finetunning a language model

davidhu March 30, 2021, 7:38am 1

Is it possible to use BertLMHeadModel to finetune a language model if i set the is_decoder=True ?

Topic		Replies	Views
Bert2bert translator? 🤗Transformers	6	62	August 28, 2025
Questions on the `BertModelLMHeadModel` 🤗Transformers	7	6355	October 5, 2020
BERT2RND EncoderDecoderModel predicts random words for Translation tasks 🤗Transformers	0	394	May 30, 2022
Fine-tuning lm with nsp 🤗Transformers	0	1184	January 19, 2021
BERT2BERT Notebook for Models without GenerationMixin 🤗Transformers	0	301	November 12, 2020