ALLaM-Thinking: Arabic Large Language Model with Enhanced Reasoning Capabilities

Overview

ALLaM-Thinking is an advanced Arabic Large Language Model specifically optimized for reasoning and mathematical problem-solving tasks. This model builds on state-of-the-art language model architecture and has been fine-tuned using the Unsloth library for improved performance and efficiency.

Key Features

• Arabic-First Design: Built from the ground up to excel at understanding and generating high-quality Arabic text
• Enhanced Reasoning: Specialized in step-by-step problem solving, particularly for mathematical questions
• Optimized Performance: Accelerated using Unsloth for faster inference and reduced computational requirements
• GRPO Implementation: Utilizes Group Relative Policy Optimization for improved alignment

Usage Example

from transformers import AutoTokenizer
from vllm import LLM, SamplingParams

# Load the tokenizer
tokenizer = AutoTokenizer.from_pretrained("almaghrabima/ALLaM-Thinking")

# Initialize the model using vLLM
# Note: You should only initialize the model once, using vLLM directly
model = LLM(model="almaghrabima/ALLaM-Thinking")

# Format the prompt using chat template
text = tokenizer.apply_chat_template([
    {"role": "user", "content": "في فريق مكون من 15 لاعباً، 40% منهم يسجلون الأهداف. إذا سجل كل لاعب من اللاعبين الذين يسجلون الأهداف في المتوسط 5 أهداف خلال الموسم، فكم عدد الأهداف الكلي التي سجلها اللاعبون الذين يسجلون الأهداف؟"}
], tokenize=False, add_generation_prompt=True)

# Configure sampling parameters
sampling_params = SamplingParams(
    temperature=0.8,
    top_p=0.95,
    max_tokens=1024,
)

# Generate response
outputs = model.generate([text], sampling_params)
output = outputs[0].outputs[0].text
print(output)

Answer

 أولاً، دعنا نجد عدد اللاعبين الذين يسجلون الأهداف.

40% من 15 لاعباً يساوي:

0.40 * 15 = 6 لاعبين

الآن، إذا كان كل لاعب من هؤلاء اللاعبين الستة يسجل في المتوسط 5 أهداف خلال الموسم، فإن إجمالي عدد الأهداف التي سجلها اللاعبون الذين يسجلون الأهداف سيكون:

6 لاعبين * 5 أهداف لكل لاعب = 30 هدفاً

لذلك، سجل اللاعبون الذين يسجلون الأهداف مجموع 30 هدفاً خلال الموسم. 

Unsloth Optimization

This model has been optimized using Unsloth, which provides significant speedups for training and inference.

Training Details

ALLaM-Thinking was trained using a combination of techniques:

• Base architecture fine-tuned on diverse Arabic datasets
• GRPO (Group Relative Policy Optimization) for better alignment
• Specialized training on mathematical reasoning and step-by-step problem-solving

Performance

ALLaM-Thinking demonstrates strong capabilities in:

• Mathematical problem-solving with step-by-step reasoning
• Logical analysis and deduction
• Maintaining coherence in long-form responses
• Domain-specific reasoning in technical fields

Limitations

• Model outputs should always be verified by human experts, especially for critical applications
• May occasionally produce incorrect mathematical reasoning despite the step-by-step approach
• Limited context window compared to some larger models
• Performance may vary based on query complexity and domain specificity

Citation

If you use ALLaM-Thinking in your research or applications, please cite:

@misc{almaghrabima2025allam,
  author = {Mohammed Al-Maghrabi Research},
  title = {ALLaM-Thinking: Arabic Large Language Model with Enhanced Reasoning Capabilities},
  year = {2025},
  publisher = {Hugging Face},
  howpublished = {\url{https://huggingface.co/almaghrabima/ALLaM-Thinking}}
}

License

This model is released under the Apache 2.0 License.