thejarx-mv commited on
Commit
91a7840
·
1 Parent(s): 92f01be

Add model card

Browse files
Files changed (4) hide show
  1. .gitattributes +2 -0
  2. README.md +389 -3
  3. assets/benchmarks.png +3 -0
  4. assets/performance.png +3 -0
.gitattributes CHANGED
@@ -33,3 +33,5 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
 
 
 
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
36
+ assets/benchmarks.png filter=lfs diff=lfs merge=lfs -text
37
+ assets/performance.png filter=lfs diff=lfs merge=lfs -text
README.md CHANGED
@@ -1,3 +1,389 @@
1
- ---
2
- license: apache-2.0
3
- ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: apache-2.0
3
+ language:
4
+ - en
5
+ base_model:
6
+ - nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-BF16
7
+ ---
8
+
9
+ # Pulsar 16B
10
+ <div align="center">
11
+
12
+ [![License](https://img.shields.io/badge/License-Apache_2.0-blue.svg)](https://opensource.org/licenses/Apache-2.0)
13
+ [![HuggingFace](https://img.shields.io/badge/🤗-Model_Hub-yellow.svg)](TODO_PULSAR_HF_URL)
14
+ [![Discord](https://img.shields.io/badge/Discord-Community-5865F2?logo=discord&logoColor=white)](https://discord.gg/cGas9uStqp)
15
+
16
+ Powered by CompactifAI
17
+
18
+ **Optimized for Fast and Efficient Inference** · **Reduced Memory Footprint**
19
+
20
+ </div>
21
+
22
+ ---
23
+
24
+ ## Table of Contents
25
+
26
+ - [Model Overview](#model-overview)
27
+ - [Key Characteristics](#key-characteristics)
28
+ - [Quick Start](#quick-start)
29
+ - [Reasoning Control](#thinking-reasoning-control)
30
+ - [Tool Calling](#tool-calling)
31
+ - [Training & Fine-Tuning](#training--fine-tuning)
32
+ - [Evaluation & Benchmarks](#evaluation--benchmarks)
33
+ - [Languages](#languages)
34
+ - [Safety & Limitations](#safety--limitations)
35
+ - [Model Information](#model-information)
36
+ - [Citation](#citation)
37
+
38
+ ---
39
+
40
+ ## Model Overview
41
+
42
+ **Pulsar 16B A3B BF16** is a **model based on [NVIDIA-Nemotron-3-Nano-30B-A3B-BF16](https://huggingface.co/nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-BF16)**, developed by **Multiverse Computing**. The original model is a **~31.6B parameter**, part of the Nemotron model family. It supports **long-context inference up to 1M tokens** and is designed for general-purpose language modeling tasks.
43
+
44
+ This version applies **model compression techniques** to significantly reduce parameter count and deployment requirements while maintaining compatibility with the Nemotron Hybrid Mamba2-Transformer with MoE architecture. The resulting model achieves **50% compression**, reducing the parameter count to **16.15B parameters** and lowering memory requirements.
45
+
46
+ ---
47
+
48
+ ## Key Characteristics
49
+
50
+ | Characteristic | Description |
51
+ |-----------------------|-------------|
52
+ | Base model | [nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-BF16](https://huggingface.co/nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-BF16). **31.6B** total parameters, **3.6B** activated per forward pass (11.34% activation ratio). [NVIDIA Open Model License](https://www.nvidia.com/en-us/agreements/enterprise-software/nvidia-open-model-license/). |
53
+ | Pulsar (this model) | **16.15B** total parameters, **3.1B** activated per forward pass (19.28% activation ratio) after CompactifAI compression. |
54
+ | 📐 **Architecture** | Hybrid Mamba2-Transformer with MoE (same family as the base checkpoint). |
55
+ | 🛠️ **Tool calling** | Yes. Same tool-call structure and format as [Nemotron-3-Nano-30B-A3B-BF16](https://huggingface.co/nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-BF16). See [Tool Calling](#tool-calling). |
56
+ | 🗜️ **Compression** | CompactifAI (proprietary compression technology) |
57
+ | Primary language | English |
58
+ ---
59
+ ## Quick Start
60
+ This model can be loaded with the **Transformers** API. Use `trust_remote_code=True`. Recommended approach: `AutoModelForCausalLM` with `apply_chat_template`. This configuration has been tested with Transformers 4.57.6.
61
+
62
+ ```python
63
+ import torch
64
+ from transformers import AutoTokenizer, AutoModelForCausalLM
65
+
66
+ model_id = "MultiverseComputingCAI/Pulsar-16B-A3B-BF16"
67
+
68
+ tokenizer = AutoTokenizer.from_pretrained(
69
+ model_id,
70
+ trust_remote_code=True
71
+ )
72
+
73
+ model = AutoModelForCausalLM.from_pretrained(
74
+ model_id,
75
+ device_map="cuda" if torch.cuda.is_available() else "auto",
76
+ torch_dtype=torch.bfloat16,
77
+ trust_remote_code=True,
78
+ )
79
+ messages = [
80
+ {"role": "user", "content": "Write a haiku about GPUs"},
81
+ ]
82
+
83
+ tokenized_chat = tokenizer.apply_chat_template(
84
+ messages,
85
+ tokenize=True,
86
+ add_generation_prompt=True,
87
+ return_tensors="pt"
88
+ ).to(model.device)
89
+
90
+ outputs = model.generate(
91
+ tokenized_chat,
92
+ max_new_tokens=1024,
93
+ temperature=1.0,
94
+ top_p=1.0,
95
+ eos_token_id=tokenizer.eos_token_id
96
+ )
97
+ print(tokenizer.decode(outputs[0]))
98
+ ```
99
+ Alternatively you can use the `pipeline` API with `trust_remote_code=True`; the pipeline returns the full conversation structure, so extract the assistant message from `outputs[0]["generated_text"]` as needed.
100
+
101
+ ### vLLM Serving
102
+
103
+ #### Installation
104
+
105
+ ```bash
106
+ pip install -U "vllm>=0.12.0"
107
+ ```
108
+
109
+ #### Reasoning parser (NVIDIA)
110
+
111
+ Pulsar uses the same Nemotron v3 reasoning tags as the base model. NVIDIA provides the vLLM plugin as [`nano_v3_reasoning_parser.py`](https://huggingface.co/nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-BF16/blob/main/nano_v3_reasoning_parser.py) on the base Hugging Face repo (not specific to Pulsar). Direct download:
112
+
113
+ ```bash
114
+ wget https://huggingface.co/nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-BF16/resolve/main/nano_v3_reasoning_parser.py
115
+ ```
116
+
117
+ You can keep any local filename; the `vllm serve` flags below assume the file is in the current directory as `nano_v3_reasoning_parser.py`. If you mirror an identical copy under the Pulsar model repo, use that URL instead.
118
+
119
+ #### Serve
120
+
121
+ ```bash
122
+ vllm serve MultiverseComputingCAI/Pulsar-16B-A3B-BF16 \
123
+ --served-model-name model \
124
+ --max-num-seqs 8 \
125
+ --tensor-parallel-size 1 \
126
+ --port 8000 \
127
+ --trust-remote-code \
128
+ --enable-auto-tool-choice \
129
+ --tool-call-parser qwen3_coder \
130
+ --reasoning-parser-plugin nano_v3_reasoning_parser.py \
131
+ --reasoning-parser nano_v3
132
+ ```
133
+
134
+ > **Note:** The NeMo container `nvcr.io/nvidia/nemo:25.11.nemotron_3_nano` comes with `mamba_ssm` and `causal-conv1d` pre-installed.
135
+
136
+ ---
137
+
138
+ ## Thinking (Reasoning) Control
139
+
140
+ Pulsar supports a **hybrid reasoning mode**: the model can either think step-by-step before answering (reasoning mode) or reply directly (non-reasoning mode). The behaviour is controlled via the `enable_thinking` flag in the chat template.
141
+
142
+ > This section provides a brief overview of reasoning control in Pulsar. For comprehensive details please see the official Nemotron-3 Nano-30B model card at: https://build.nvidia.com/nvidia/nemotron-3-nano-30b-a3b/modelcard
143
+
144
+
145
+ ---
146
+
147
+ ### Transformers API
148
+
149
+ Pass `enable_thinking` through `apply_chat_template`:
150
+
151
+ **Thinking ON (default)**
152
+ ```python
153
+ tokenized_chat = tokenizer.apply_chat_template(
154
+ messages,
155
+ tokenize=True,
156
+ add_generation_prompt=True,
157
+ return_tensors="pt",
158
+ enable_thinking=True, # default — can be omitted
159
+ )
160
+ ```
161
+
162
+ **Thinking OFF**
163
+ ```python
164
+ tokenized_chat = tokenizer.apply_chat_template(
165
+ messages,
166
+ tokenize=True,
167
+ add_generation_prompt=True,
168
+ return_tensors="pt",
169
+ enable_thinking=False,
170
+ )
171
+ ```
172
+
173
+ When thinking is ON the model opens a `<think>` block before the answer.
174
+
175
+ ```python
176
+ output = tokenizer.decode(outputs[0], skip_special_tokens=True)
177
+ # Split on </think> to separate reasoning from the final answer
178
+ if "</think>" in output:
179
+ reasoning, answer = output.split("</think>", 1)
180
+ reasoning = reasoning.replace("<think>", "").strip()
181
+ answer = answer.strip()
182
+ else:
183
+ answer = output
184
+ ```
185
+
186
+ ---
187
+
188
+ ### vLLM
189
+
190
+ #### Server-level default
191
+
192
+ Set the default for **all requests** at startup with `--default-chat-template-kwargs`.
193
+
194
+ > Requires recent versions of vLLM.
195
+
196
+ **Thinking OFF for all requests**
197
+ ```bash
198
+ vllm serve MultiverseComputingCAI/Pulsar-16B-A3B-BF16 \
199
+ --served-model-name model \
200
+ --reasoning-parser-plugin nano_v3_reasoning_parser.py \
201
+ --reasoning-parser nano_v3 \
202
+ --trust-request-chat-template \
203
+ --default-chat-template-kwargs '{"enable_thinking": false}' \
204
+ ...
205
+ ```
206
+
207
+ **Thinking ON for all requests (default if flag is omitted)**
208
+ ```bash
209
+ vllm serve MultiverseComputingCAI/Pulsar-16B-A3B-BF16 \
210
+ --served-model-name model \
211
+ --reasoning-parser-plugin nano_v3_reasoning_parser.py \
212
+ --reasoning-parser nano_v3 \
213
+ --trust-request-chat-template \
214
+ --default-chat-template-kwargs '{"enable_thinking": true}' \
215
+ ...
216
+ ```
217
+
218
+
219
+ ---
220
+
221
+ #### Per-request override
222
+
223
+ > **`--trust-request-chat-template` is required** to allow per-request overrides.
224
+
225
+ Individual requests can override the server default by passing `chat_template_kwargs` in the request body. This works regardless of the server-level default.
226
+
227
+ **Thinking ON/OFF for one request**
228
+ ```python
229
+ import requests
230
+
231
+ response = requests.post("http://localhost:8000/v1/chat/completions", json={
232
+ "model": "model",
233
+ "messages": [{"role": "user", "content": "Solve: x² - 5x + 6 = 0"}],
234
+ "max_tokens": 1024,
235
+ "temperature": 1.0,
236
+ "chat_template_kwargs": {"enable_thinking": True},
237
+ })
238
+ ```
239
+
240
+ ---
241
+
242
+ ## Tool Calling
243
+
244
+ Pulsar emits tool calls in the following format:
245
+
246
+ ```
247
+ <tool_call>
248
+ <function=get_weather>
249
+ <parameter=city>Paris</parameter>
250
+ <parameter=unit>celsius</parameter>
251
+ </function>
252
+ </tool_call>
253
+ ```
254
+
255
+ When serving (e.g with vLLM), you **must** use the `qwen3_coder` tool parser.
256
+
257
+ ```bash
258
+ vllm serve <model_path> \
259
+ --enable-auto-tool-choice \
260
+ --tool-call-parser qwen3_coder \
261
+ --trust-remote-code
262
+ ```
263
+
264
+ ## Training & Fine-Tuning
265
+
266
+ ### Base Model: NVIDIA-Nemotron-3-Nano-30B-A3B-BF16
267
+
268
+ The base model [nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-BF16](https://huggingface.co/nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-BF16) is a large language model (LLM) trained from scratch by NVIDIA, and designed as a unified model for both reasoning and non-reasoning tasks. The model's reasoning capabilities can be configured through a flag in the chat template. See the [original model card](https://huggingface.co/nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-BF16) for details.
269
+
270
+
271
+ ### CompactifAI Compression
272
+
273
+ CompactifAI was applied to produce a smaller, efficient model (16B parameters) while aiming to preserve reasoning and tool-use capabilities. Supervised Fine Tuning was applied for improving cabapilities.
274
+
275
+ ---
276
+
277
+ ## Evaluation & Benchmarks
278
+
279
+ ![Combined benchmark chart](assets/benchmarks.png)
280
+
281
+ | Benchmark | Nemotron-3-Nano-30B-A3B | Pulsar-16B | gpt-oss-20b |
282
+ |----------------------|--------------------------|-------------|---------------|
283
+ | MMLU-Pro | 78.30 | 74.78 | 74 |
284
+ | AIME 2025 | 87.29 | 87.22 | 72 |
285
+ | BFCL-v4 | 53.80 | 49.03 | 38.25 |
286
+ | LiveCodeBench | 67.40 | 65.45 | 64.34 |
287
+ | Tau2-bench (Telecom) | 40.94 | 31.29 | 58.77 |
288
+ | SciCode | 32.05 | 25.84 | 33.53 |
289
+ | AA-LCR | 34.00 | 29.33 | 37 |
290
+ | AA-Omni | 20.69 | 15.60 | 17.2 |
291
+ | IFBench | 71.97 | 70.79 | 56.67 |
292
+ | GPQA-D | 72.02 | 71.41 | 58.88 |
293
+ | HLE | 10.47 | 10.24 | 7 |
294
+
295
+ ### Performance
296
+ ![Performance results](assets/performance.png)
297
+ - **Framework:** [guidellm](https://github.com/vllm-project/guidellm)
298
+ - **Inference:** vLLM 0.18.0
299
+ - **Decode:** temperature=1.0, top_p=1.0
300
+ - **Phase duration:** Each phase lasts 3 minutes (excluding ramp-up and cool-down periods).
301
+ - **Workload shape:** Sweep across different levels of concurrency. 180s measure windows with ramp-up and cooldown excluded. 8k/16k workload as in the original model's card.
302
+ - **Estimated Peak Memory Usage:** KV cache utilization is monitored during the phase and we estimate memory usage as follows: `model_ weights_{gb} + kv_ cache_{usage_pct} × (nvml_used_{gb} − model_ weights_{gb})`
303
+
304
+ ### Evaluation Methodology
305
+
306
+ Benchmark scores were obtained with the following setups. Methodology varies by benchmark family.
307
+
308
+ #### MMLU-Pro, AIME25, GPQA:d, LiveCodeBench, IFBench, AA-LCR, SciCode
309
+
310
+ - **Framework**: [Nemo-Skills](https://github.com/NVIDIA-NeMo/Skills)
311
+ - **Inference**: vLLM 0.18.0
312
+ - **Decode**: temp 1.0, top_p 1.0
313
+
314
+ #### BFCL v4
315
+
316
+ - **Framework**: [EvalScope](https://github.com/EvalScope/EvalScope) 1.4.1
317
+ - **Inference**: vLLM 0.18.0
318
+ - **Decode**: temp 0.6, top_p 0.95, parallel_tool_calls true
319
+
320
+ #### Tau2-bench (Telecom)
321
+
322
+ - **Framework**: [EvalScope](https://github.com/EvalScope/EvalScope) 1.4.1
323
+ - **Inference**: vLLM 0.18.0
324
+ - **Decode (agent)**: temp 0.6, top_p 0.95
325
+ - **Decode (judge / user sim)**: temp 0.7, top_p 0.8, top_k 20, min_p 0
326
+ - **Repro**: telecom subset; max steps 100; 3 runs; judge parser hermes
327
+
328
+ ---
329
+
330
+ ## Languages
331
+
332
+ - **Primary language**: English
333
+ - **Other languages**: Spanish
334
+
335
+ Trained mainly on English with added Spanish. No systematic evaluation for languages outside English and Spanish.
336
+
337
+
338
+
339
+ ## Safety & Limitations
340
+
341
+ ### Known Limitations
342
+
343
+ - English-centric training data (inherited from base model).
344
+ - Tool calling depends on correct schema and tool design; exact parity with the original model is not guaranteed.
345
+ - Compression may affect some behaviors; evaluate for your use case.
346
+
347
+ ### Recommendations
348
+
349
+ - Validate tool outputs before running them
350
+ - Human oversight for critical use
351
+ - Task-specific eval before production
352
+
353
+ ---
354
+
355
+ ## Model Information
356
+
357
+ | Field | Value |
358
+ |--------------|--------------------- |
359
+ | Model name | Pulsar 16B |
360
+ | Based on | [NVIDIA-Nemotron-3-Nano-30B-A3B-BF16](https://huggingface.co/nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-BF16) |
361
+ | Version | v1.5.0 |
362
+ | Release date | TBD |
363
+ | Developed by | Multiverse Computing |
364
+ | License | Apache 2.0 |
365
+ | Contact | [email protected] |
366
+
367
+ ---
368
+
369
+ ## Citation
370
+
371
+ If you use this model, please cite the base model and pulsar:
372
+
373
+ ```bibtex
374
+ @misc{nemotron3nanoTR,
375
+ title = {NVIDIA Nemotron 3 Nano Technical Report},
376
+ author = {{NVIDIA}},
377
+ year = {2025},
378
+ url = {https://research.nvidia.com/labs/nemotron/files/NVIDIA-Nemotron-3-Nano-Technical-Report.pdf}
379
+ }
380
+ @misc{pulsar16b,
381
+ title = {Pulsar 16B: Model developed from NVIDIA Nemotron-3-Nano-30B-A3B},
382
+ author = {Multiverse Computing},
383
+ year = {2026},
384
+ url = {https://huggingface.co/MultiverseComputingCAI/Pulsar-16B-A3B-BF16},
385
+ note = {Model developed based on nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-BF16 using CompactifAI technology}
386
+ }
387
+ ```
388
+
389
+ **Built by [Multiverse Computing](https://www.multiversecomputing.com)** · [Report an issue](TODO_PULSAR_HF_URL/discussions) · [Discord](https://discord.gg/cGas9uStqp)
assets/benchmarks.png ADDED

Git LFS Details

  • SHA256: b72715e2006ce58059cc344cae62e5bdc0424d419c8028fbd0447e84d779c230
  • Pointer size: 131 Bytes
  • Size of remote file: 199 kB
assets/performance.png ADDED

Git LFS Details

  • SHA256: a1cbad5bc4016ec6cded114e62ebfa61804672aeab60011341de60ab3dd3ff4f
  • Pointer size: 131 Bytes
  • Size of remote file: 262 kB