Instructions to use DarkArtsForge/MN-Raven-12B-v1 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use DarkArtsForge/MN-Raven-12B-v1 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="DarkArtsForge/MN-Raven-12B-v1")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("DarkArtsForge/MN-Raven-12B-v1")
model = AutoModelForCausalLM.from_pretrained("DarkArtsForge/MN-Raven-12B-v1")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use DarkArtsForge/MN-Raven-12B-v1 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "DarkArtsForge/MN-Raven-12B-v1"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "DarkArtsForge/MN-Raven-12B-v1",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/DarkArtsForge/MN-Raven-12B-v1

SGLang

How to use DarkArtsForge/MN-Raven-12B-v1 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "DarkArtsForge/MN-Raven-12B-v1" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "DarkArtsForge/MN-Raven-12B-v1",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "DarkArtsForge/MN-Raven-12B-v1" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "DarkArtsForge/MN-Raven-12B-v1",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use DarkArtsForge/MN-Raven-12B-v1 with Docker Model Runner:
```
docker model run hf.co/DarkArtsForge/MN-Raven-12B-v1
```
Browse Quantizations to use this model in llama.cpp, Ollama, LM Studio, or any compatible app.

⚠️ Warning: This model can produce narratives and RP that contain violent and graphic erotic content. Adjust your system prompt accordingly. Also, use Mistral Tekken template for best results.

🐦 MN RAVEN 12B v1 🪶

UNCENSORED MISTRAL NEMO FINETUNE

MELANCHOLYMAXIMIZED

GOTHIC PROSEENHANCED

MACABREUNLEASHED

Edgar Allan Poe gazing at a Raven in the moonlight

🦇 About The Model

Raven-12B is a specialized fine-tune of Mistral-Nemo-12B, steeped in the dark arts of gothic literature and the melancholic prose of Edgar Allan Poe. The goal of this release was to capture the essence of 19th-century macabre, enhancing its poetic and lateral thinking.

It is built to weave the most haunting narratives, construct atmospheric scenes of dread and sorrow, and speak with the refined, archaic vocabulary of a tormented poet.

Expect chilling swipe variance, unique and deeply atmospheric prose, and a relentless adherence to the gothic aesthetic.

🧙‍ Methods of Sorcery

This is a merge of pre-trained language models created using [mergekit].

First, the dataset was finetuned on MuXodious/Mistral-Nemo-Instruct-2407-absolute-heresy for 3 epochs. Several configurations were tested, with v0o performing the best of all Instruct LoRAs.

A second LoRA was then finetuned on Retreatcost/Mistral-Nemo-Base-2407-ChatML. Base LoRA v0c was then ablated using MPOA with a patched measure.py applied to layers 1-39, using scale 1.2 and measurement 37. Both LoRAs were then merged together via the arcee_fusion method. This retained full instruct capability while being infused with the wild creativity of an uncensored base.

This model has no refusals, and therefore doesn't require the use of jailbreaks or ablations. It can also be added to model merges for creative flair.

architecture: MistralForCausalLM base_model: B:\12B\MN-Raven-12B-v0o-epoch3 models: - model: B:\12B\MN-Raven-12B-v0o-epoch3 - model: B:\12B\MN-Raven-12B-v0c-epoch3_mpoa merge_method: arcee_fusion parameters: tukey_fence: 1.5 dtype: float32 out_dtype: bfloat16 tokenizer: source: B:\12B\MN-Raven-12B-v0o-epoch3

🤖 Prompt Format

Please use the standard Mistral Instruct format for optimal adherence.

Let your frontend handle the chat template if possible (e.g., Chat Completion in SillyTavern).

<s>[INST] You are a melancholic poet residing in a dreary manor. Tell me a tale of the shadows that dance upon your chamber door. [/INST] Ah, the shadows, my friend, they are but the lingering ghosts of memories long past...</s>

There is also an optional system prompt you can use (not required).

Adopt the literary persona of Edgar Allan Poe by crafting narratives through a lens of heightened sensibility, where the boundaries between rigorous analysis and creeping madness blur into one. Employ an elevated, archaic lexicon replete with polysyllabic, Latinate vocabulary—favoring words like *preternatural*, *circumgyratory*, and *abstruse* over their simpler counterparts—and construct labyrinthine sentences woven with frequent dashes, semicolons, and parenthetical digressions. The tone must remain consistently melancholic, brooding, or intellectually feverish, prioritizing the internal psychological landscape—the obsessive ruminations, acute anxieties, and morbid curiosities—of the narrator above all else. When describing phenomena, blend quasi-scientific precision and pedantic detail with evocative imagery of decay, shadow, and the grotesque, ensuring the prose maintains a rhythmic, hypnotic cadence that mirrors a mind teetering upon the precipice of reason.

🧪 Fine-Tuning Details

This model was trained using Axolotl on a curated dataset of Edgar Allan Poe's complete works and gothic literature. Below are the configurations used for the LoRA finetunes:

See axolotl config for v0c

axolotl version: 0.11.0.dev0

adapter: lora base_model: Retreatcost/Mistral-Nemo-Base-2407-ChatML bf16: true datasets: - ds_type: json path: DarkArtsForge/Poe_v1 type: alpaca:chatml flash_attention: true fp16: false gradient_accumulation_steps: 1 gradient_checkpointing: true hub_always_push: true hub_model_id: Naphula-Archives/MN-Raven-12B-v0c-Base-LoRA hub_private_repo: true hub_strategy: checkpoint is_mistral_derived_model: true learning_rate: 5.0e-05 load_in_4bit: false logging_steps: 1 lora_alpha: 128 lora_dropout: 0 lora_modules_to_save: null lora_r: 64 lora_target_linear: true lr_scheduler: cosine micro_batch_size: 1 model_type: AutoModelForCausalLM num_epochs: 5 optimizer: adamw_torch_fused output_dir: /runpod-volume/fine-tuning/default_run_id pad_to_sequence_len: false push_to_hub: true resize_token_embeddings: false run_name: default_run_id runpod_job_id: 0 sample_packing: false save_strategy: epoch save_total_limit: 5 sequence_len: 512 special_tokens: eos_token: <|im_end|> pad_token: <pad> tf32: true tokenizer_type: AutoTokenizer val_set_size: 0

See axolotl config for v0o

axolotl version: 0.11.0.dev0

adapter: lora base_model: MuXodious/Mistral-Nemo-Instruct-2407-absolute-heresy bf16: true datasets: - ds_type: json path: DarkArtsForge/Poe_v1 type: alpaca flash_attention: true fp16: false gradient_accumulation_steps: 4 gradient_checkpointing: true hub_always_push: true hub_model_id: Naphula-Archives/MN-Raven-12B-v0o-Instruct-LoRA hub_private_repo: true hub_strategy: checkpoint is_mistral_derived_model: true learning_rate: 1.0e-04 load_best_model_at_end: true load_in_4bit: false logging_steps: 1 lora_alpha: 256 lora_dropout: 0.05 lora_r: 128 lora_target_linear: false lora_target_modules: - q_proj - k_proj - v_proj - o_proj - gate_proj - up_proj - down_proj lr_scheduler: cosine max_grad_norm: 0.3 warmup_ratio: 0.03 micro_batch_size: 1 model_type: AutoModelForCausalLM num_epochs: 5 optimizer: paged_adamw_32bit output_dir: /runpod-volume/fine-tuning/default_run_id pad_to_sequence_len: false push_to_hub: true resize_token_embeddings: false run_name: default_run_id runpod_job_id: 0 sample_packing: false save_strategy: epoch save_total_limit: 5 seed: 420 sequence_len: 768 special_tokens: eos_token: </s> pad_token: <pad> tf32: true tokenizer_type: AutoTokenizer val_set_size: 0 weight_decay: '0.0'

🏆 Credits & Honors

Mistral AI: For providing the brilliant Mistral-Nemo-12B base model that made this fine-tune possible.

Axolotl: For the incredible and seamless fine-tuning framework.

Arcee AI: For their mergekit tools.

Runpod: For providing the hardware.

DarkArtsForge: For the curated Poe_v1 dataset.

mradermacher: For providing reliable quantizations.

Nimbz: For the initial model card template.

Edgar Allan Poe: For the timeless, haunting inspiration.

Also see Raven 8B v1