AMOS-2.8 Decision Model (LoRA Adapter)

AMOS-2.8 is the official release of the AMOS decision-model family (internal codename amos-3.0-gen-v16): a generative decision model built on Qwen3-8B, served through the systemone structured decision protocol (choice / score / noul primitives) — a single forward pass directly outputs a normalized probability distribution: fully structured output (100% parseable), calibrated probabilities, low latency.

This repository contains the LoRA adapter weights; load them on top of the Qwen/Qwen3-8B base model (see Usage below).

Model Summary

Item	Value
Base model	Qwen/Qwen3-8B (bf16)
Fine-tuning	QLoRA (lora r=32, α=64, nf4-double quantization during training)
Training data	50,567 rows of curriculum-based decision tasks (v15 incremental curriculum + v16 targeted enhancement)
Training config	1 epoch, lr 5e-05, completed 2026-10-06
Output	systemone primitives: choice / score / noul probability distributions
Adapter size	≈349 MB (adapter_model.safetensors)
Inference latency	≈0.9–1.2 s warm (public end-to-end, incl. gateway)

Performance

Board 1: 500-question fresh randomized blind test (amos-2.8 / jev-1.13 / laya same board)

Independently generated banks, rule-based gold scoring, zero text-level overlap with training data. laya (convaiinnovations/laya-multilingual) was invoked zero-shot on the exact same board with identical request construction (2026-10-09).

Scope	amos-2.8	jev-1.13 (commercial API)	laya (zero-shot)
Raw accuracy	96.6% (483/500, 0 API errors; four-round mean 97.45% ±0.6pp)	96.4% (482/500)	42.8% (214/500)
Same-day live invocations (two rounds)	97.8%	97.8% (tied)	— (single run)
Tier-A pure decision tasks	98.6%	97.9%	52.1% (146/280)
Strict OOD (task-domain overlap removed)	96.05%	95.81%	37.0% (159/430)
Latency p50 / p95	≈950–1200 ms (public e2e)	1161.9 / 2811.2 ms	20.7 / 21.8 ms (local GPU direct)

Board 2: 6,000-question three-model same board (ext6kb, seed=2027)

Scope	amos family*	jev-1.13	laya (zero-shot)
Main tasks (15,000 questions)	86.77%	72.28%	45.07%
Out-of-scope attributes (48,000 questions)	96.23%	86.87%	43.89%
Latency p50 / p95 (ms)	358.8 / 1184.3	1155.0 / 2804.7	24.1 / 28.2
API errors	0	0	0
Rank	1st	2nd	3rd

* ext6kb was run with amos-2.6 as the family reference; amos-2.8 was not run on this board. laya's only absolute advantage is raw latency.

Per-Category Accuracy (Board 1, R3: amos-2.8 vs jev-1.13 vs laya)

12 of 17 categories at 100% for amos-2.8. amos-2.8 leads on NLI (+13.3pp), GSM8K word problems (+12.5pp), length band (+20pp); jev leads on character counting and deduplication; laya leads on no category (its best is topic at 100%, tied).

OOD disclosure (read before citing): question texts are fresh (0/500 text-level overlap with training data; sha1 exclusion: 5,475 full / 11,143 stem hashes blocked), but this board is NOT a domain-disjoint OOD benchmark. Sentiment and topic (70 questions) share their task domain with the training curriculum (3,646 sentiment / 250 topic Chinese samples) — their 100% scores are in-domain results. The remaining 15 categories have no same-template training samples, yet they all still fall within the decision-task family the model was designed for. Generalization to task types entirely absent from training has NOT been verified by this evaluation — all 17 categories are in-scope task types; open-set generalization to genuinely novel task domains requires a dedicated benchmark and remains untested. Under the strict scope (the 70 domain-overlapping questions removed), amos-2.8 scores 96.05% vs jev-1.13 95.81%; this number measures within-family domain overlap removal, not open-set generalization.

Category	Benchmark	n	amos-2.8	jev-1.13	laya (zero-shot)
extraction	Extraction	30	100.0%	100.0%	83.3%
knowledge	MMLU	30	100.0%	100.0%	26.7%
math_word	GSM8K	40	100.0%	87.5%	22.5%
length_band	Attr	10	100.0%	80.0%	20.0%
moderation	Toxicity	30	100.0%	100.0%	36.7%
sentiment	SST	40	100.0%	100.0%	62.5%
topic	AG News	30	100.0%	100.0%	100.0%
commonsense	HellaSwag	20	100.0%	100.0%	30.0%
reasoning	ARC	30	100.0%	100.0%	23.3%
pii	PII	30	100.0%	100.0%	50.0%
case_form	Attr	20	100.0%	100.0%	25.0%
lang_id	LangID	30	100.0%	100.0%	16.7%
math_basic	Arithmetic	60	98.3%	96.7%	41.7%
percent	Percent	20	95.0%	100.0%	15.0%
nli	NLI	30	93.3%	80.0%	56.7%
duplicate	Dedup	30	86.7%	100.0%	36.7%
instruction	Counting	20	55.0%	85.0%	50.0%

• Question banks are programmatically constructed from benchmark templates (SST / AG News / NLI / MMLU / HellaSwag / ARC / GSM8K, etc.).
• Full three-model comparison (amos-2.8 vs jev-1.13 vs laya): see reports/amos-2.8-vs-jev-1.13-vs-laya.md in this repository.

Repository Files

├── adapter_config.json          # PEFT/LoRA config (r=32, α=64, nf4-double)
├── adapter_model.safetensors    # LoRA adapter weights (≈349 MB)
├── chat_template.jinja          # Qwen3 chat template
├── tokenizer_config.json
├── tokenizer.json               # Qwen3 tokenizer
├── README.md
└── reports/                     # External benchmark reports (English)

Usage

This repository is a PEFT LoRA adapter; load the Qwen3-8B base first, then attach the adapter:

# pip install torch transformers peft safetensors
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

base = AutoModelForCausalLM.from_pretrained(
    "Qwen/Qwen3-8B", torch_dtype=torch.bfloat16, device_map="auto"
)
model = PeftModel.from_pretrained(base, "/path/to/amos-2.8")
tokenizer = AutoTokenizer.from_pretrained("/path/to/amos-2.8")

Production deployments use the systemone protocol endpoint (POST /v1/systemone). Request example:

{
  "protocol": "choice",
  "state": {"user_id": "demo"},
  "questions": {
    "q1": {
      "type": "choice",
      "query": "User message: is this SMS a scam? 'Congratulations, you won 50,000. Pay a 500 fee to claim.'",
      "options": ["Scam", "Legit notice", "Marketing", "Undecidable"]
    }
  }
}

The response returns, for each question, the chosen option index and the normalized probability distribution.

Intended Use

• High-throughput text adjudication: intent classification, risk grading, sentiment polarity, language ID, structured attribute extraction, moderation
• Scenarios that need probabilistic output for downstream threshold decisions / human-review triage
• As a discriminative decision layer combined with LLMs (LLM generates, AMOS adjudicates)

Limitations and Known Weaknesses

• Decision-task accuracy is ~96–98%; a small fraction of high-confidence errors remains (confidence ≥ 0.9 errors ≈ 0.4%)
• Exact arithmetic / character counting (generative benchmarks) are not strengths
• Specialized domains (medical, legal clauses) require domain fine-tuning
• The adapter was trained under nf4 quantization; when merging back into the bf16 base, run a calibration check

Citation

AMOS-2.8 Decision Model (amos-3.0-gen-v16), 2026.
Qwen/Qwen3-8B base · systemone three-primitive decision interface · QLoRA curriculum learning