VishwamAI — GGUF (Q4_K_M)

Quantized GGUF build of VishwamAI, a defensive-first cybersecurity + math/science reasoning assistant fine-tuned from Gemma 4 E4B-it (Apache 2.0). This format runs fast and locally with llama.cpp / Ollama / llama-cpp-python on modest hardware (4–6 GB GPU or CPU).

Full-precision / merged weights: kasinadhsarma/vishwamai-merged
LoRA adapter: kasinadhsarma/vishwamai-lora

⚠️ Authorized / educational use only. Use security capabilities solely on systems you own or are explicitly authorized to test (labs, CTFs, defensive research, education).

Files

File	Quant	Size	Use
`vishwamai-Q4_K_M.gguf`	Q4_K_M	~5.3 GB	Best size/quality balance — recommended default

Run it

Ollama

# create a Modelfile
printf 'FROM ./vishwamai-Q4_K_M.gguf\nSYSTEM "You are VishwamAI, a defensive-first security research assistant and a math/science solver. Be precise, state uncertainty, never fabricate CVE IDs or physical constants."\n' > Modelfile
ollama create vishwamai -f Modelfile
ollama run vishwamai

llama.cpp (OpenAI-compatible local server)

huggingface-cli download kasinadhsarma/vishwamai-gguf vishwamai-Q4_K_M.gguf --local-dir .
./llama-server -m vishwamai-Q4_K_M.gguf -ngl 99 -c 8192 --host 127.0.0.1 --port 8080
# then open http://127.0.0.1:8080 for the built-in chat UI

llama-cpp-python

from llama_cpp import Llama
llm = Llama.from_pretrained(
    "kasinadhsarma/vishwamai-gguf",
    filename="vishwamai-Q4_K_M.gguf",
    n_ctx=4096, n_gpu_layers=99,   # n_gpu_layers=0 for CPU
)
print(llm.create_chat_completion(messages=[
    {"role": "system", "content": "You are VishwamAI, a defensive-first security and math/science assistant."},
    {"role": "user", "content": "Give two Sysmon detections for LSASS credential dumping."},
])["choices"][0]["message"]["content"])

Recommended system prompt

You are VishwamAI, a defensive-first security research assistant and a
math/science solver for an authorized practitioner. Be precise, state uncertainty,
never fabricate CVE IDs or physical constants, and use the calculator/CAS tool for
any non-trivial computation.

Limitations

~4B (8B incl. embeddings) model — capable, but weaker than frontier models on deep reasoning.
May hallucinate facts (CVE IDs, ATT&CK technique details, constants). Route factual lookups through a RAG index and keep the "state uncertainty" system prompt — do not trust recalled facts.
Quantized (Q4_K_M): small quality loss vs the fp16 model. For higher fidelity, requantize the merged model to Q5_K_M / Q6_K.