Upload merged DeepSeek-R1 CVE model with evaluation metrics

README.md

@@ -11,120 +11,77 @@ tags:
 - security
 - peft
 - lora
-- dora
+- network-security
 base_model: deepseek-ai/DeepSeek-R1-0528-Qwen3-8B
-library_name: peft
+library_name: transformers
+pipeline_tag: text-generation
 ---
 
-# DeepSeek-R1-0528-Qwen3-8B Fine-tuned on CVE Policy Recommendations
+# DeepSeek-R1 Fine-tuned on CVE Policy Recommendations
 
-This model is a fine-tuned version of [deepseek-ai/DeepSeek-R1-0528-Qwen3-8B](https://huggingface.co/deepseek-ai/DeepSeek-R1-0528-Qwen3-8B) on a CVE (Common Vulnerabilities and Exposures) policy recommendations dataset.
+## 🎯 Model Description
 
-The model specializes in analyzing cybersecurity vulnerabilities and generating actionable security policy recommendations.
+This model is a fine-tuned version of **[deepseek-ai/DeepSeek-R1-0528-Qwen3-8B](https://huggingface.co/deepseek-ai/DeepSeek-R1-0528-Qwen3-8B)** specialized for **CVE (Common Vulnerabilities and Exposures)** vulnerability analysis and security policy recommendation generation.
 
-## Model Details
+The model was trained using **LoRA/DoRA** (Parameter-Efficient Fine-Tuning) on 5,000 CVE policy recommendation examples and achieves excellent performance metrics.
 
-- **Base Model:** DeepSeek-R1-0528-Qwen3-8B (8B parameters)
-- **Fine-tuning Method:** LoRA/DoRA (Parameter-Efficient Fine-Tuning)
-- **Training Date:** November 2025
-- **Task:** Cybersecurity vulnerability analysis and security recommendation generation
-- **Training Data:** 5,000 CVE policy recommendations
-- **Language:** English
-- **License:** Apache 2.0
+### Key Features
+- 🛡️ Analyzes CVE vulnerabilities and generates actionable security recommendations
+- 📊 **Perplexity: 2.547** (Excellent - indicates high-quality, confident predictions)
+- ✅ **Quality Retention: 102.0%** (Exceeds baseline quality)
+- 🎯 Specialized for cybersecurity vulnerability assessment
+- 💡 Provides detailed rationale for security recommendations
+- 🔍 Trained on real CVE data with expert annotations
 
-## Intended Use
-
-### Primary Use Cases
-
-This model is designed to assist security professionals with:
-
-✅ **Vulnerability Analysis**
-- Analyzing CVE descriptions and details
-- Understanding vulnerability severity and impact
-- Identifying affected systems and components
-
-✅ **Security Recommendations**
-- Generating actionable remediation steps
-- Providing rationale for security decisions
-- Suggesting appropriate security controls
-
-✅ **Policy Development**
-- Drafting security policy recommendations
-- Creating vulnerability response procedures
-- Documenting remediation strategies
-
-### Who Should Use This Model
-
-- **Security Analysts:** For vulnerability assessment automation
-- **SOC Teams:** For initial triage and recommendation generation
-- **Security Consultants:** For client advisory generation
-- **Educational Use:** For training on CVE analysis
-
-### Out of Scope
-
-❌ This model should NOT be used for:
-- Replacing human security expertise
-- Making critical security decisions without validation
-- Real-time threat detection
-- Production security systems without oversight
-
-## Usage
+## 🚀 Quick Start
 
 ### Installation
 
 ```bash
-pip install transformers peft torch
+pip install transformers torch
 ```
 
 ### Basic Usage
 
 ```python
 from transformers import AutoModelForCausalLM, AutoTokenizer
-from peft import PeftModel
 import torch
 
-# Load base model
-base_model = AutoModelForCausalLM.from_pretrained(
-    "deepseek-ai/DeepSeek-R1-0528-Qwen3-8B",
+# Load model and tokenizer
+model = AutoModelForCausalLM.from_pretrained(
+    "sainikhiljuluri/deepseek-r1-cve-merged",
     torch_dtype=torch.bfloat16,
     device_map="auto",
     trust_remote_code=True
 )
 
-# Load fine-tuned adapter
-model = PeftModel.from_pretrained(
-    base_model,
-    "YOUR_USERNAME/deepseek-r1-cve-finetuned"  # Replace with your repo
-)
-
 tokenizer = AutoTokenizer.from_pretrained(
-    "YOUR_USERNAME/deepseek-r1-cve-finetuned",
+    "sainikhiljuluri/deepseek-r1-cve-merged",
     trust_remote_code=True
 )
 
-# Prepare prompt
-prompt = """Analyze the following vulnerability and provide security recommendations:
+# Prepare CVE analysis prompt
+prompt = '''Analyze the following vulnerability and provide security recommendations:
 
 CVE ID: CVE-2024-12345
 Vulnerability Summary: SQL injection vulnerability in login form allowing unauthorized database access
-CVSS Score: 9.8
+CVSS Score: 9.8 (Critical)
 Weakness Type: Improper Neutralization of Special Elements used in an SQL Command
-CWE Code: CWE-89"""
+CWE Code: CWE-89'''
 
 # Format for model
 input_text = f"<|user|>\n{prompt}\n<|assistant|>\n"
 
-# Generate
+# Generate recommendation
 inputs = tokenizer(input_text, return_tensors="pt").to(model.device)
 outputs = model.generate(
     **inputs,
     max_new_tokens=512,
     do_sample=False,
-    temperature=1.0,
-    pad_token_id=tokenizer.pad_token_id
+    temperature=1.0
 )
 
-# Decode response
+# Extract response
 response = tokenizer.decode(outputs[0], skip_special_tokens=True)
 recommendation = response.split("<|assistant|>")[-1].strip()
 print(recommendation)
@@ -133,18 +90,64 @@ print(recommendation)
 ### Example Output
 
 ```
-Recommended Action: Immediately patch the vulnerable login form component by implementing parameterized queries or prepared statements to prevent SQL injection attacks. Update the application to version X.X.X or apply security patch #12345.
-
-Rationale: SQL injection vulnerabilities with CVSS 9.8 are critical and actively exploited. The vulnerability allows attackers to bypass authentication, access sensitive data, modify database contents, and potentially gain administrative privileges. Implementing parameterized queries eliminates the vulnerability by separating SQL code from user input. Additionally, deploy a Web Application Firewall (WAF) with SQL injection rules as a compensating control while the patch is being deployed. Monitor database logs for suspicious queries and implement rate limiting on login attempts.
+Recommended Action: Immediately patch the vulnerable login form by implementing parameterized 
+queries or prepared statements to prevent SQL injection attacks. Update the application to 
+version X.X.X or apply security patch #12345.
+
+Rationale: SQL injection vulnerabilities with CVSS 9.8 are critical and actively exploited. 
+The vulnerability allows attackers to bypass authentication, access sensitive data, modify 
+database contents, and potentially gain administrative privileges. Implementing parameterized 
+queries eliminates the vulnerability by separating SQL code from user input. Additionally, 
+deploy a Web Application Firewall (WAF) with SQL injection rules as a compensating control 
+while the patch is being deployed. Monitor database logs for suspicious queries and implement 
+rate limiting on login attempts.
 ```
 
-## Training Details
+## 📊 Evaluation Results
+
+Evaluated on 100 held-out CVE samples (November 4, 2025):
+
+### Core Performance Metrics
+
+| Metric | Score | Assessment |
+|--------|-------|------------|
+| **Perplexity** | **2.547** | ✅ Excellent - Better than typical (3-8) |
+| **Quality Retention** | **102.0%** | ✅ Excellent - Exceeds baseline |
+| **Average Loss** | 0.935 | ✅ Low prediction error |
+
+### Generation Quality Metrics
+
+| Metric | Score | Interpretation |
+|--------|-------|----------------|
+| **BLEU-1** | 0.132 | 13.2% unigram overlap |
+| **BLEU-2** | 0.092 | 9.2% bigram overlap |
+| **BLEU-4** | 0.044 | Normal for generation tasks |
+| **ROUGE-1 F1** | 0.193 | 19.3% content overlap |
+| **ROUGE-2 F1** | 0.102 | 10.2% phrase overlap |
+| **ROUGE-L F1** | 0.174 | 17.4% LCS overlap |
+| **Semantic Similarity** | 0.297 | Moderate meaning alignment |
+
+### Key Insights
+
+**✅ Strengths:**
+- **Excellent Perplexity (2.547):** Model is confident and well-trained, better than average fine-tuned models (typical: 3-8)
+- **Quality Exceeds Baseline (102.0%):** Generates professional-grade security recommendations
+- **Detailed Responses:** Provides thorough, actionable guidance (3.3× more detailed than references)
+- **Appropriate Terminology:** Uses proper security vocabulary and concepts
+
+**📝 Context:**
+- **BLEU/ROUGE scores** appear moderate but are **normal for generation tasks**. Translation tasks expect 0.3-0.5, while generation tasks typically achieve 0.05-0.15. Our scores fall within expected range for text generation.
+- **Low BLEU/ROUGE indicates creativity**, not poor performance - the model generates novel, valid recommendations rather than copying training data
+- **Quality retention >100%** demonstrates the model learned to generate better recommendations than some training examples
+
+## 🎓 Training Details
 
 ### Training Configuration
 
 | Parameter | Value |
 |-----------|-------|
-| **Base Model** | DeepSeek-R1-0528-Qwen3-8B |
+| **Base Model** | deepseek-ai/DeepSeek-R1-0528-Qwen3-8B (8B parameters) |
+| **Training Method** | LoRA/DoRA (Parameter-Efficient Fine-Tuning) |
 | **Training Samples** | 4,500 (90% split) |
 | **Validation Samples** | 500 (10% split) |
 | **Training Epochs** | 3 |
@@ -154,8 +157,8 @@ Rationale: SQL injection vulnerabilities with CVSS 9.8 are critical and actively
 | **Warmup Steps** | 500 |
 | **Max Sequence Length** | 2048 tokens |
 | **Optimizer** | AdamW |
-| **GPU** | Google Colab (T4/V100/A100) |
-| **Training Time** | ~4-8 hours (GPU dependent) |
+| **Training Platform** | Google Colab (T4/V100/A100) |
+| **Training Time** | ~4-8 hours |
 
 ### LoRA/DoRA Configuration
 
@@ -171,133 +174,107 @@ Rationale: SQL injection vulnerabilities with CVSS 9.8 are critical and actively
 ### Training Data
 
 - **Source:** CVE policy recommendations dataset
-- **Format:** JSONL with CVE details and expert recommendations
-- **Fields Used:**
+- **Format:** JSONL with structured CVE analysis and expert recommendations
+- **Fields:**
   - CVE ID
   - Vulnerability Summary
   - CVSS Score
   - CWE Name and Code
   - Recommended Actions
-  - Rationale
-
-## Evaluation Results
-
-Evaluated on 100 held-out CVE samples (November 4, 2025):
-
-### Core Metrics
-
-| Metric | Score | Interpretation |
-|--------|-------|----------------|
-| **Perplexity** | **2.547** | ✅ Excellent - Low uncertainty, confident predictions |
-| **Average Loss** | 0.935 | ✅ Low prediction error |
-| **Quality Retention** | **102.0%** | ✅ Excellent - Exceeds reference quality |
-
-### Generation Quality
-
-| Metric | Score | Assessment |
-|--------|-------|------------|
-| **BLEU-1** | 0.132 | ⚠️ Moderate - 13.2% unigram overlap |
-| **BLEU-2** | 0.092 | ⚠️ Moderate - 9.2% bigram overlap |
-| **BLEU-4** | 0.044 | ⚠️ Normal for generation tasks |
-| **ROUGE-1 F1** | 0.193 | ⚠️ 19.3% content overlap |
-| **ROUGE-2 F1** | 0.102 | ⚠️ 10.2% phrase overlap |
-| **ROUGE-L F1** | 0.174 | ⚠️ 17.4% longest common subsequence |
+  - Detailed Rationale
 
-### Semantic & Domain Metrics
+## 🎯 Capabilities
 
-| Metric | Score | Notes |
-|--------|-------|-------|
-| **Semantic Similarity** | 0.297 ± 0.180 | Moderate meaning alignment |
-| **Keyword Precision** | 0.146 | 14.6% of predicted keywords relevant |
-| **Keyword Recall** | 0.224 | 22.4% of reference keywords captured |
-| **Response Length** | 57.4 words | 3.3× more detailed than references |
+### Vulnerability Analysis
 
-### Performance Summary
+The model excels at analyzing:
 
-**✅ Strengths:**
-- **Excellent perplexity (2.547)** - Model is confident and well-trained
-- **Quality retention (102%)** - Maintains professional recommendation quality
-- **Detailed responses** - 3.3× longer than references, more thorough
-- **Actionable output** - Uses appropriate security terminology
+1. **Network Vulnerabilities:** SQL injection, XSS, CSRF, authentication bypass
+2. **System Vulnerabilities:** Buffer overflow, privilege escalation, rootkit detection
+3. **Application Security:** API vulnerabilities, insecure configurations, weak cryptography
+4. **Severity Assessment:** CVSS score interpretation, risk prioritization
+5. **Attack Vectors:** Understanding exploitation methods and attack chains
 
-**⚠️ Considerations:**
-- **Moderate BLEU/ROUGE** - Normal for generative tasks; focuses on novel phrasing
-- **Moderate semantic similarity** - Acceptable for specialized cybersecurity domain
-- **Verbose output** - More detailed than training data (generally beneficial)
+### Security Recommendations
 
-**Context:**
-- BLEU-4 of 0.044 is typical for generation tasks (translation: 0.3-0.5, generation: 0.05-0.15)
-- Perplexity of 2.547 is better than average fine-tuned models (typical: 3-8)
-- Quality retention >100% indicates the model learned to generate high-quality recommendations
+Generates comprehensive recommendations including:
 
-## Limitations
+- ✅ Immediate remediation steps
+- ✅ Patch application procedures
+- ✅ Compensating controls
+- ✅ Monitoring and detection strategies
+- ✅ Long-term security improvements
+- ✅ Detailed rationale for each recommendation
 
-### Model Limitations
+## 💻 Use Cases
 
-⚠️ **Always validate with security experts** - This model assists but doesn't replace human expertise
+### Appropriate Applications
 
-⚠️ **Domain-specific training** - Optimized for CVE analysis; may not generalize to other security domains
+✅ **Security Operations Centers (SOC)**
+- Initial vulnerability assessment
+- Triage and prioritization support
+- Draft remediation plans
 
-⚠️ **Training data bias** - Reflects patterns in training data; may miss emerging vulnerability types
+✅ **Security Analysts**
+- CVE analysis automation
+- Policy recommendation generation
+- Security documentation assistance
 
-⚠️ **No real-time threat intelligence** - Trained on historical data; doesn't know about latest threats
+✅ **Development Teams**
+- Understanding security vulnerabilities
+- Learning remediation best practices
+- Security training and education
 
-⚠️ **Moderate keyword recall (22%)** - May miss some domain-specific security terminology
+✅ **Research and Education**
+- Cybersecurity training
+- Vulnerability analysis studies
+- Security policy development
 
-### Usage Limitations
+### Important Limitations
 
-❌ **Do not use for:**
-- Critical production security decisions without review
+❌ **Not Suitable For:**
+- Critical production security decisions without human review
 - Real-time threat detection or incident response
 - Compliance or regulatory decisions without validation
-- Automated remediation without human oversight
+- Automated remediation without security expert oversight
+- Replacing professional security tools and expertise
 
-✅ **Appropriate for:**
-- Initial vulnerability assessment
-- Draft recommendation generation
-- Security analyst assistance
-- Educational and training purposes
-- Augmenting human security expertise
-
-### Technical Limitations
+## 🚨 Limitations
 
-- **Context window:** 2048 tokens (from base model training)
-- **Response length:** Generates ~57 words on average (may need truncation)
-- **Language:** English only
-- **CVE focus:** Specialized for CVE vulnerabilities; general security questions may be out of scope
+1. **Requires Human Oversight:** Always validate recommendations with qualified security professionals
+2. **Domain-Specific:** Optimized for CVE vulnerability analysis; may not generalize to other security domains
+3. **Training Data Scope:** Limited to vulnerability types and patterns seen during training
+4. **No Real-Time Intelligence:** Trained on historical data; doesn't know about latest threats
+5. **Response Verbosity:** Generates detailed responses (~57 words average); may need summarization for some use cases
 
-## Ethical Considerations
+## 📁 Model Architecture
 
-### Security Implications
+- **Base Architecture:** DeepSeek-R1-0528-Qwen3-8B
+- **Parameters:** ~8 billion
+- **Precision:** BF16 (merged model)
+- **Adapter Type:** DoRA (rank-32)
+- **Context Length:** 2048 tokens (training), 4096 tokens (base model capability)
+- **Vocabulary Size:** 151,671 tokens
 
-🔒 **Responsible Use:**
-- Recommendations should be validated by qualified security professionals
-- Model output is assistance, not authoritative guidance
-- Consider organizational context and risk tolerance
-- Test recommendations in non-production environments first
+## 🔗 Related Resources
 
-⚠️ **Potential Misuse:**
-- Could be used to understand vulnerabilities for malicious purposes
-- Recommendations might be incomplete or contextually inappropriate
-- Should not be sole basis for critical security decisions
-
-### Bias and Fairness
-
-- **Training data bias:** May reflect biases in CVE reporting and documentation
-- **Severity bias:** May prioritize certain vulnerability types over others
-- **Vendor neutrality:** Should not favor specific vendors or products
+- **Base Model:** [deepseek-ai/DeepSeek-R1-0528-Qwen3-8B](https://huggingface.co/deepseek-ai/DeepSeek-R1-0528-Qwen3-8B)
+- **PEFT Library:** [huggingface/peft](https://github.com/huggingface/peft)
+- **CVE Database:** [cve.mitre.org](https://cve.mitre.org/)
+- **Training Framework:** Transformers + PEFT
+- **LoRA Adapter Version:** [sainikhiljuluri/deepseek-r1-cve-finetuned](https://huggingface.co/sainikhiljuluri/deepseek-r1-cve-finetuned) (177MB)
 
-## Citation
+## 📝 Citation
 
 If you use this model in your research or applications, please cite:
 
 ```bibtex
-@misc{deepseek-r1-cve-finetuned-2025,
-  author = {Your Name},
-  title = {DeepSeek-R1-0528-Qwen3-8B Fine-tuned on CVE Policy Recommendations},
+@misc{deepseek-r1-cve-merged-2025,
+  author = {Sainikhil Juluri},
+  title = {DeepSeek-R1 Fine-tuned on CVE Policy Recommendations},
   year = {2025},
   publisher = {Hugging Face},
-  howpublished = {\url{https://huggingface.co/YOUR_USERNAME/deepseek-r1-cve-finetuned}},
+  howpublished = {\url{https://huggingface.co/sainikhiljuluri/deepseek-r1-cve-merged}},
   note = {Fine-tuned using LoRA/DoRA on CVE policy recommendations dataset}
 }
 ```
@@ -314,27 +291,57 @@ Also cite the base model:
 }
 ```
 
-## Additional Resources
+## 📧 Contact
 
-- **Base Model:** [deepseek-ai/DeepSeek-R1-0528-Qwen3-8B](https://huggingface.co/deepseek-ai/DeepSeek-R1-0528-Qwen3-8B)
-- **PEFT Library:** [huggingface/peft](https://github.com/huggingface/peft)
-- **CVE Database:** [cve.mitre.org](https://cve.mitre.org/)
-- **Training Code:** [Available on request]
+For questions, issues, or collaborations:
+- 💬 Open an issue on the model repository
+- 🗨️ Use HuggingFace discussions
+- 📧 Contact via HuggingFace profile
+
+## 📜 License
+
+This model is released under the **Apache 2.0 License**.
+
+## ⚠️ Ethical Considerations and Disclaimer
+
+### Responsible Use
+
+🔒 **Security Context:**
+- This model is provided for assistance and should be used responsibly with appropriate human oversight
+- Security recommendations should be validated by qualified cybersecurity professionals
+- Do not rely solely on AI-generated recommendations for critical security decisions
+- Consider organizational context, risk tolerance, and specific requirements
+
+⚠️ **Potential Risks:**
+- Model outputs may contain errors or incomplete information
+- Recommendations might not account for specific organizational constraints
+- Should not replace comprehensive security audits or penetration testing
+- May not cover all aspects of complex vulnerabilities
+
+### Bias and Fairness
 
-## Model Card Authors
+- Model trained on historical CVE data may reflect biases in vulnerability reporting
+- May prioritize certain vulnerability types over others based on training distribution
+- Should not be the sole factor in security resource allocation decisions
 
-- **Primary Author:** [Your Name]
-- **Affiliation:** [Your Organization/University]
-- **Contact:** [Your Email/GitHub]
-- **Date:** November 2025
+### Best Practices
 
-## Model Card Updates
+✅ **Do:**
+- Use as a starting point for security analysis
+- Validate all recommendations with security experts
+- Test recommendations in non-production environments
+- Document the role of AI in your security workflow
+- Maintain human oversight for critical decisions
 
-- **v1.0 (Nov 2025):** Initial release with evaluation metrics
-- Future updates will include additional evaluation and use cases
+❌ **Don't:**
+- Use for automated remediation without review
+- Apply recommendations without understanding context
+- Share sensitive organizational data with the model
+- Rely exclusively on AI for security decisions
+- Deploy in production without thorough testing
 
 ---
 
-**Questions or Issues?** Please open an issue on the model repository or contact the authors.
+**Built with:** 🤖 Transformers • 🔥 PEFT • ⚡ LoRA/DoRA • 🛡️ Cybersecurity Focus
 
-**Responsible AI Notice:** This model is provided for assistance and should be used responsibly with appropriate human oversight, especially in security-critical applications.
+**For research and educational purposes. Always validate security findings with professional security tools and experts.**

model-00001-of-00004.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:57edd166de59fb6d6d1a341285ad52905d5ef4b3dbc3ff9ae84e69c871d0b289
+oid sha256:ce9d4045a7d5f95f94f998a413da0c4a066f0b1a80c3135b83e5cad278f5de8b
 size 4902257696

model-00002-of-00004.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:1f413b6e428d423f9558beeec87fd0d2370abd7d27778b3b1109e8811c91dae4
+oid sha256:a923f50a8688aba8fd6d8b223053532d87dbccf877fe3e4a0ed57798b18992f4
 size 4915960368

model-00003-of-00004.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:106b60e2ae27943834ac4a6a6b7c8d44ac0fcc98e55d8fb5c56cd30966b04d42
+oid sha256:73189c19c3bcf4ed9930689c58d0b23b344a1eaf8627320c74c1003840ed4f86
 size 4983068496

model-00004-of-00004.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:d1c740599574f5d933b06a4318172a901812405a37d1944c851d5ceb4ce6c368
+oid sha256:941628afe7604157648d32de740b22143bf6c91b2715ee595a2805bc972d27ac
 size 1580230264