draft_vocab_size

#2
by Qiaolin-Yu - opened

I noticed the draft_vocab_size is none in the config.json. But some inference engine will read draft_vocab_size config when using speculative decoding. May you suggest how to support this?

Sign up or log in to comment