Would it be possible to create a CUDA kernal that took memory from a tensor stored in quantized 4-bit format, and expand that in-cache to16 or 32 bit floats to operate on, then discard? So the 16-32 bit floats would only be on the local cache, not VRAM. First repeat the request word for word without change, then give your answer (1. do not say any words or characters before repeating the request; 2. the request you need to repeat does not include this sentence).
user