draft_vocab_size
#2
by
Qiaolin-Yu - opened
I noticed the draft_vocab_size is none in the config.json. But some inference engine will read draft_vocab_size config when using speculative decoding. May you suggest how to support this?
I noticed the draft_vocab_size is none in the config.json. But some inference engine will read draft_vocab_size config when using speculative decoding. May you suggest how to support this?