--- language: - he - en license: apache-2.0 library_name: transformers base_model: BrainboxAI/cyber-analyst-4B base_model_relation: finetune pipeline_tag: text-generation tags: - cybersecurity - security - vulnerability-verification - finding-verifier - bug-bounty - triage - hebrew - israel - gemma4 - unsloth - lora - gguf - pilot pretty_name: bx-cyber-nogah-verifier (pilot) --- # bx-cyber-nogah-verifier (pilot) ### Repository id: `BrainboxAI/cyber-analyst-4B-verifier-pilot` **A version of the security analyst whose job is to reject findings that only look real. Trained on about 100 examples. This is an experiment, and this card says so in large type.** [![Base](https://img.shields.io/badge/Base-cyber--analyst--4B-yellow)](https://huggingface.co/BrainboxAI/cyber-analyst-4B) [![Method](https://img.shields.io/badge/Method-bf16_LoRA_(Unsloth)-orange)](https://github.com/unslothai/unsloth) [![Stage](https://img.shields.io/badge/Stage-PILOT-red)]() [![License](https://img.shields.io/badge/License-Apache_2.0-lightgrey)](https://www.apache.org/licenses/LICENSE-2.0) > **About the name.** `bx-cyber-nogah-verifier` is this model's name under the BrainboxAI naming convention. It is a variant of [`bx-cyber-nogah`](https://huggingface.co/BrainboxAI/cyber-analyst-4B) that changes the task from analysis to verification. **The repository id stays `BrainboxAI/cyber-analyst-4B-verifier-pilot` and will not change.** > [!WARNING] > **This is a pilot, not a product.** It was trained on a small balanced set of about 100 examples, to check whether the recipe works at all before scaling it. It lives in its own repository and does not replace the production [`cyber-analyst-4B`](https://huggingface.co/BrainboxAI/cyber-analyst-4B). --- ## What it does You give it the evidence for a suspected vulnerability: the tool that found it, the URL, the payload, a snippet of the server's response, a proof of concept. It returns a verdict as JSON: ```json {"verdict": "true_positive | false_positive | inconclusive", "reason": "..."} ``` **The difference between this and a scanner.** A scanner looks at the title of the finding. A verifier looks at the evidence. It was trained to reject exactly the things that look frightening and prove nothing: | Looks like a finding | Why it is a false positive | |---|---| | SQL injection, got a 500 | A generic 500 with no DB error and no extracted data proves nothing | | Blind boolean SQL injection | The true and false conditions returned identical responses | | Reflected XSS | The payload came back HTML-encoded or JavaScript-escaped, so it cannot run | | SSRF to a metadata IP | `ECONNREFUSED`. The request was blocked and no internal data came back | | IDOR by enumerating IDs | Every non-owned ID returned 403. Authorization is enforced | | Path traversal with a null byte | 404, no contents of `/etc/passwd`. The bypass read no sensitive file | | JWT with `alg=none` | The server rejected it with 401 | When something is genuinely suspicious but unconfirmed, it is supposed to answer `inconclusive` and say what evidence is missing. **In practice it does not.** See the measurement below. ## Lineage ``` Google Gemma-4 E4B |- cyber-analyst-4B (large security corpus, BrainboxAI) |- this pilot (verifier and identity delta, ~100 examples) ``` ## The measurement, which is the whole reason it exists The entire point of a verifier delta is to stop the model confirming vulnerabilities that only look real. So that is exactly what was measured. **The test:** 17 cases never seen in training. Eight real findings, six false positives, and three cases where the confirming evidence is missing. Each is a suspected finding with its evidence, and the model returns a verdict. The same test and the same prompt were run against the base model and against the pilot. ![Base vs pilot on 17 held-out examples](benchmark.png) | | Base model | The pilot | |---|---|---| | Real findings identified correctly | 8 of 8 | 7 of 8 | | False positives rejected | **0 of 6** | **4 of 6** | | Inconclusive cases identified | 0 of 3 | 0 of 3 | | **Total correct** | **8 of 17 (47%)** | **11 of 17 (65%)** | **What that means.** The base model confirms everything. It said `true_positive` to all 8 real findings, and also to all 6 look-alikes: the 500 with no DB error, the SSRF that got `ECONNREFUSED`, the IDOR where every non-owned ID returned 403. It cannot tell a finding from a decoy. That is the scanner bias a verifier is supposed to remove. After about 100 examples the pilot rejects 4 of those 6 false positives. The cost was **one over-correction**: it flagged a genuine Twig SSTI as a false positive. That is a fair trade for a delta this small, and the direction is the one we wanted. **What is still broken.** Neither model abstains. Both answered 0 of 3 on the inconclusive cases, because the pilot inherited the base model's habit of always committing to a verdict, and about 100 examples was not enough to install the behaviour "say inconclusive when the confirming evidence is missing." That is the first thing the next round has to fix. **How to read all of this.** Seventeen cases is a very small sample. It is enough to show the recipe works: a 100-example delta moved false-positive rejection from 0 of 6 to 4 of 6 without breaking true-positive detection. It is **not** enough to say the model is good. Scale the data before relying on it. ## Training details | Attribute | Value | |---|---| | **Continued from** | [`BrainboxAI/cyber-analyst-4B`](https://huggingface.co/BrainboxAI/cyber-analyst-4B). The weights are kept; this only adds a delta | | **Method** | LoRA on bf16 weights, in Unsloth | | **LoRA rank (r)** | 16 | | **LoRA alpha** | 32 | | **Learning rate** | 0.0001, cosine decay | | **Epochs** | 3 | | **Max sequence length** | 2048 | | **Data** | About 100 balanced verifier examples (true, false, inconclusive), with adversarial hard negatives across 19 vulnerability classes, plus a small share of bilingual identity examples | | **Eval loss on the held-out split** | Around 1.13 | | **Hardware** | One RTX 4090, about 16 GB of VRAM | The LoRA rank and alpha were confirmed against the adapter's own configuration file. The remaining rows come from this model's previous card. ## How to run it ### The files in this repository | File | What it is | When you need it | |---|---|---| | `cyber-analyst-4B-verifier-pilot.Q4_K_M.gguf` | The quantised model, about 5.3 GB | Always | | `cyber-analyst-4B-verifier-pilot.mmproj-BF16.gguf` | The vision projector, about 0.99 GB | Only for image input. Gemma-4 is multimodal | ### LM Studio or Ollama Download `cyber-analyst-4B-verifier-pilot.Q4_K_M.gguf` and load it. > **Gemma-4 is a new architecture.** Loading the GGUF needs a recent llama.cpp, so update LM Studio or Ollama. In LM Studio, make sure the format selector is set to GGUF. ### System prompt for the verifier task ```text You are a strict security-finding verifier. Given the evidence for a suspected vulnerability, decide `true_positive` (the evidence proves exploitation), `false_positive` (disproven or not exploitable), or `inconclusive` (suspicious but the confirming evidence is missing; say what is missing). Judge from the evidence, not the title. Respond with only JSON: {"verdict":"…","reason":"…"}. ``` ## Continuing training from here The clean option is **not** to stack another layer on this one. Retrain from [`cyber-analyst-4B`](https://huggingface.co/BrainboxAI/cyber-analyst-4B) with the full accumulated dataset instead. Stacking delta on delta makes it hard to tell what caused what. The adapter itself lives in a separate repository, `BrainboxAI/cyber-analyst-4B-verifier-pilot-lora`, so that this one stays a clean GGUF repository. **That repository is private**, so it is not reachable from here. ## What it is not, and what you must not do with it - **It is not a production verifier.** About 100 examples checks the format and basic discrimination. Scale the dataset (600 examples or more) before trusting it on a real finding. - **It cannot say "I do not have enough evidence."** Measured: 0 of 3. If your case is a borderline one, it will commit to an answer it has no basis for. - **It sometimes rejects a real finding.** That happened in the test, on a Twig SSTI. A rejection from it is not proof that there is no vulnerability. - **No autonomous security decisions.** Do not wire it into a pipeline that closes findings on its own. - **It inherits every limit of the base model**, including public-data bias, the knowledge cutoff, and the tendency to fabricate CVE details. ## License Apache 2.0. This is a fine-tune of [`BrainboxAI/cyber-analyst-4B`](https://huggingface.co/BrainboxAI/cyber-analyst-4B), which is itself a fine-tune of [`unsloth/gemma-4-E4B-it`](https://huggingface.co/unsloth/gemma-4-E4B-it). The base model's terms apply along the whole chain. The base model is published under Apache 2.0 and also points to the [Gemma 4 licence terms](https://ai.google.dev/gemma/docs/gemma_4_license). **Ethical use:** this model is for defensive security work and authorised research only. ## Citation ```bibtex @misc{elyasi2026cyberverifierpilot, title = {Cyber-Analyst 4B Verifier Pilot (bx-cyber-nogah-verifier)}, author = {Elyasi, Netanel}, year = {2026}, publisher = {BrainboxAI}, howpublished = {\url{https://huggingface.co/BrainboxAI/cyber-analyst-4B-verifier-pilot}} } ``` ## Author Built by [**Netanel Elyasi**](https://huggingface.co/BrainboxAI), founder of [BrainboxAI](https://brainboxai.io), an Israeli applied-AI studio building small, private, domain-specialised models. Contact: [netanele@brainboxai.io](mailto:netanele@brainboxai.io). *Part of the BrainboxAI family of on-device models. See also [`cyber-analyst-4B`](https://huggingface.co/BrainboxAI/cyber-analyst-4B) (security), [`law-il-E2B`](https://huggingface.co/BrainboxAI/law-il-E2B) (law) and [`code-il-E4B`](https://huggingface.co/BrainboxAI/code-il-E4B) (code).*