After installing chat models and using them over the weekend.....

To the best of my knowledge, but know that this is a very fast moving space with a lot of information and misinformation. A lot of people are speaking like they know what is going on and I am finding them disproven left and right. Here is what I think I know.

ChatGPT is MILES ahead of all other language models. They are doing something fancy in the background. Go use it to see what will be possible later, but note that what we have access to will likely not be so lucid.
I have a hypothesis on how one could set up a language model to be useful for OTDI.
The coherency of small models and the competition in general leaves a lot to be desired. It remains to be seen how well it can perform question and answer tasks when given a cheat sheet.
I have 24 GB of VRAM and 8GB of DDR5 RAM (I need to buy more RAM)
I was able to run a 6B parameter model, and probably could run a 13B parameter model.
New tech is actively coming out. Within the last 2 months 4 bit quantization has made its way to models which theoretically significantly reduces the VRAM required to run these models. (I see numbers quoted like 1/4th, but more testing is required to see what is actually happening vs what people say is happening.)
Despite what everyone says, it seems to me that large models are still the most coherent. Lots of claims of models being GPT killers, or as good as LLM's, but when I test them they perform at a much lower level than their giant cousins. I am interested i There should be exactly 2 paragraphs separated by the markdown divider: ***.
user