refactor(report): reflect distillation experiments data in report

This update reorders the models, adjusts average performance metrics, and revises recommendations based on the distillation experiments.

The aim is to provide an accurate and up-to-date reflection of our findings, guiding the audience towards the best models for their use cases.

REPORT.md

@@ -6,9 +6,9 @@ This report presents a comprehensive analysis of Model2Vec distillation experime
 
 ### Evaluated Models Overview
 
-**Simplified Distillation Models:** 13
+**Simplified Distillation Models:** 14
 **Peer Comparison Models:** 19
-**Total Models Analyzed:** 32
+**Total Models Analyzed:** 33
 
 ### Best Performing Simplified Model: code_model2vec_all_mpnet_base_v2
 
@@ -28,15 +28,16 @@ This report presents a comprehensive analysis of Model2Vec distillation experime
 | code_model2vec_all_MiniLM_L6_v2 | [sentence-transformers/all-MiniLM-L6-v2](https://huggingface.co/sentence-transformers/all-MiniLM-L6-v2) | 0.7385 | 0.7049 | 0.7910 | 🥈 2nd |
 | code_model2vec_jina_embeddings_v2_base_code | [jina-embeddings-v2-base-code](https://huggingface.co/jina-embeddings-v2-base-code) | 0.7381 | 0.6996 | 0.8130 | 🥉 3rd |
 | code_model2vec_paraphrase_MiniLM_L6_v2 | [sentence-transformers/paraphrase-MiniLM-L6-v2](https://huggingface.co/sentence-transformers/paraphrase-MiniLM-L6-v2) | 0.7013 | 0.6638 | 0.7665 | #4 |
-| code_model2vec_Reason_ModernColBERT | [lightonai/Reason-ModernColBERT](https://huggingface.co/lightonai/Reason-ModernColBERT) | 0.6598 | 0.6228 | 0.7260 | #5 |
-| code_model2vec_bge_m3 | [BAAI/bge-m3](https://huggingface.co/BAAI/bge-m3) | 0.4863 | 0.4439 | 0.5514 | #6 |
-| code_model2vec_jina_embeddings_v3 | [jinaai/jina-embeddings-v3](https://huggingface.co/jinaai/jina-embeddings-v3) | 0.4755 | 0.4416 | 0.5456 | #7 |
-| code_model2vec_nomic_embed_text_v2_moe | [nomic-ai/nomic-embed-text-v2-moe](https://huggingface.co/nomic-ai/nomic-embed-text-v2-moe) | 0.4532 | 0.4275 | 0.5094 | #8 |
-| code_model2vec_gte_Qwen2_1.5B_instruct | [Alibaba-NLP/gte-Qwen2-1.5B-instruct](https://huggingface.co/Alibaba-NLP/gte-Qwen2-1.5B-instruct) | 0.4238 | 0.3879 | 0.4719 | #9 |
-| code_model2vec_Qodo_Embed_1_1.5B | [Qodo/Qodo-Embed-1-1.5B](https://huggingface.co/Qodo/Qodo-Embed-1-1.5B) | 0.4101 | 0.3810 | 0.4532 | #10 |
-| code_model2vec_graphcodebert_base | [microsoft/codebert-base](https://huggingface.co/microsoft/codebert-base) | 0.3420 | 0.3140 | 0.3704 | #11 |
-| code_model2vec_Linq_Embed_Mistral | [Linq-AI-Research/Linq-Embed-Mistral](https://huggingface.co/Linq-AI-Research/Linq-Embed-Mistral) | 0.2868 | 0.2581 | 0.3412 | #12 |
-| code_model2vec_codebert_base | [microsoft/codebert-base](https://huggingface.co/microsoft/codebert-base) | 0.2779 | 0.2534 | 0.3136 | #13 |
+| code_model2vec_all_mpnet_base_v2_fine_tuned | [sentence-transformers/all-mpnet-base-v2](https://huggingface.co/sentence-transformers/all-mpnet-base-v2) | 0.6906 | 0.6372 | 0.7917 | #5 |
+| code_model2vec_Reason_ModernColBERT | [lightonai/Reason-ModernColBERT](https://huggingface.co/lightonai/Reason-ModernColBERT) | 0.6598 | 0.6228 | 0.7260 | #6 |
+| code_model2vec_bge_m3 | [BAAI/bge-m3](https://huggingface.co/BAAI/bge-m3) | 0.4863 | 0.4439 | 0.5514 | #7 |
+| code_model2vec_jina_embeddings_v3 | [jinaai/jina-embeddings-v3](https://huggingface.co/jinaai/jina-embeddings-v3) | 0.4755 | 0.4416 | 0.5456 | #8 |
+| code_model2vec_nomic_embed_text_v2_moe | [nomic-ai/nomic-embed-text-v2-moe](https://huggingface.co/nomic-ai/nomic-embed-text-v2-moe) | 0.4532 | 0.4275 | 0.5094 | #9 |
+| code_model2vec_gte_Qwen2_1.5B_instruct | [Alibaba-NLP/gte-Qwen2-1.5B-instruct](https://huggingface.co/Alibaba-NLP/gte-Qwen2-1.5B-instruct) | 0.4238 | 0.3879 | 0.4719 | #10 |
+| code_model2vec_Qodo_Embed_1_1.5B | [Qodo/Qodo-Embed-1-1.5B](https://huggingface.co/Qodo/Qodo-Embed-1-1.5B) | 0.4101 | 0.3810 | 0.4532 | #11 |
+| code_model2vec_graphcodebert_base | [microsoft/codebert-base](https://huggingface.co/microsoft/codebert-base) | 0.3420 | 0.3140 | 0.3704 | #12 |
+| code_model2vec_Linq_Embed_Mistral | [Linq-AI-Research/Linq-Embed-Mistral](https://huggingface.co/Linq-AI-Research/Linq-Embed-Mistral) | 0.2868 | 0.2581 | 0.3412 | #13 |
+| code_model2vec_codebert_base | [microsoft/codebert-base](https://huggingface.co/microsoft/codebert-base) | 0.2779 | 0.2534 | 0.3136 | #14 |
 
 
 ### 📊 Model Specifications Analysis
@@ -49,6 +50,7 @@ Our distilled models exhibit consistent architectural characteristics across dif
 | all_MiniLM_L6_v2 | 29,525 | 7.6M | 256 | 14.4MB |
 | jina_embeddings_v2_base_code | 61,053 | 15.6M | 256 | 29.8MB |
 | paraphrase_MiniLM_L6_v2 | 29,525 | 7.6M | 256 | 14.4MB |
+| all_mpnet_base_v2_fine_tuned | 77,316 | 19.8M | 256 | 75.5MB |
 | Reason_ModernColBERT | 50,254 | 12.9M | 256 | 24.5MB |
 | bge_m3 | 249,999 | 64.0M | 256 | 122.1MB |
 | jina_embeddings_v3 | 249,999 | 64.0M | 256 | 122.1MB |
@@ -67,9 +69,9 @@ Our distilled models exhibit consistent architectural characteristics across dif
 #### Key Insights from Model Specifications:
 
 
-- **Vocabulary Consistency**: All models use vocabulary sizes ranging from 29,525 to 249,999 tokens (avg: 106,592)
-- **Parameter Efficiency**: Models range from 7.6M to 64.0M parameters (avg: 27.3M)
-- **Storage Efficiency**: Disk usage ranges from 14.4MB to 122.1MB (avg: 52.0MB)
+- **Vocabulary Consistency**: All models use vocabulary sizes ranging from 29,525 to 249,999 tokens (avg: 104,501)
+- **Parameter Efficiency**: Models range from 7.6M to 64.0M parameters (avg: 26.8M)
+- **Storage Efficiency**: Disk usage ranges from 14.4MB to 122.1MB (avg: 53.7MB)
 - **Embedding Dimensions**: Consistent 256 dimensions across all models (optimized for efficiency)
 
 
@@ -79,7 +81,7 @@ Our distilled models exhibit consistent architectural characteristics across dif
 - **Best Teacher Model**: code_model2vec_all_mpnet_base_v2 (NDCG@10: 0.7387)
 - **Least Effective Teacher**: code_model2vec_codebert_base (NDCG@10: 0.2779)
 - **Performance Range**: 62.4% difference between best and worst
-- **Average Performance**: 0.5178 NDCG@10
+- **Average Performance**: 0.5302 NDCG@10
 
 
 ## 🎯 Language Performance Radar Charts
@@ -108,6 +110,10 @@ Our distilled models exhibit consistent architectural characteristics across dif
 
 ![code_model2vec_paraphrase_MiniLM_L6_v2 Radar Chart](analysis_charts/radar_code_model2vec_paraphrase_MiniLM_L6_v2.png)
 
+#### code_model2vec_all_mpnet_base_v2_fine_tuned (Teacher: [sentence-transformers/all-mpnet-base-v2](https://huggingface.co/sentence-transformers/all-mpnet-base-v2)) - NDCG@10: 0.6906
+
+![code_model2vec_all_mpnet_base_v2_fine_tuned Radar Chart](analysis_charts/radar_code_model2vec_all_mpnet_base_v2_fine_tuned.png)
+
 #### code_model2vec_Reason_ModernColBERT (Teacher: [lightonai/Reason-ModernColBERT](https://huggingface.co/lightonai/Reason-ModernColBERT)) - NDCG@10: 0.6598
 
 ![code_model2vec_Reason_ModernColBERT Radar Chart](analysis_charts/radar_code_model2vec_Reason_ModernColBERT.png)
@@ -174,20 +180,21 @@ Our distilled models exhibit consistent architectural characteristics across dif
 | 16 | code_model2vec_all_MiniLM_L6_v2 | **🔥 Simplified Distillation** | 0.7385 | 0.7049 | 0.7910 |
 | 17 | code_model2vec_jina_embeddings_v2_base_code | **🔥 Simplified Distillation** | 0.7381 | 0.6996 | 0.8130 |
 | 18 | code_model2vec_paraphrase_MiniLM_L6_v2 | **🔥 Simplified Distillation** | 0.7013 | 0.6638 | 0.7665 |
-| 19 | code_model2vec_Reason_ModernColBERT | **🔥 Simplified Distillation** | 0.6598 | 0.6228 | 0.7260 |
-| 20 | potion-multilingual-128M | Model2Vec | 0.6124 | 0.5683 | 0.7017 |
-| 21 | huggingface/CodeBERTa-small-v1 | Code-Specific | 0.5903 | 0.5350 | 0.6779 |
-| 22 | Salesforce/codet5-base | Code-Specific | 0.4872 | 0.4500 | 0.5742 |
-| 23 | code_model2vec_bge_m3 | **🔥 Simplified Distillation** | 0.4863 | 0.4439 | 0.5514 |
-| 24 | code_model2vec_jina_embeddings_v3 | **🔥 Simplified Distillation** | 0.4755 | 0.4416 | 0.5456 |
-| 25 | code_model2vec_nomic_embed_text_v2_moe | **🔥 Simplified Distillation** | 0.4532 | 0.4275 | 0.5094 |
-| 26 | code_model2vec_gte_Qwen2_1.5B_instruct | **🔥 Simplified Distillation** | 0.4238 | 0.3879 | 0.4719 |
-| 27 | code_model2vec_Qodo_Embed_1_1.5B | **🔥 Simplified Distillation** | 0.4101 | 0.3810 | 0.4532 |
-| 28 | microsoft/graphcodebert-base | Code-Specific | 0.4039 | 0.3677 | 0.4650 |
-| 29 | code_model2vec_graphcodebert_base | **🔥 Simplified Distillation** | 0.3420 | 0.3140 | 0.3704 |
-| 30 | code_model2vec_Linq_Embed_Mistral | **🔥 Simplified Distillation** | 0.2868 | 0.2581 | 0.3412 |
-| 31 | code_model2vec_codebert_base | **🔥 Simplified Distillation** | 0.2779 | 0.2534 | 0.3136 |
-| 32 | microsoft/codebert-base | Code-Specific | 0.1051 | 0.1058 | 0.1105 |
+| 19 | code_model2vec_all_mpnet_base_v2_fine_tuned | **🎓 Fine-tuned Distillation** | 0.6906 | 0.6372 | 0.7917 |
+| 20 | code_model2vec_Reason_ModernColBERT | **🔥 Simplified Distillation** | 0.6598 | 0.6228 | 0.7260 |
+| 21 | potion-multilingual-128M | Model2Vec | 0.6124 | 0.5683 | 0.7017 |
+| 22 | huggingface/CodeBERTa-small-v1 | Code-Specific | 0.5903 | 0.5350 | 0.6779 |
+| 23 | Salesforce/codet5-base | Code-Specific | 0.4872 | 0.4500 | 0.5742 |
+| 24 | code_model2vec_bge_m3 | **🔥 Simplified Distillation** | 0.4863 | 0.4439 | 0.5514 |
+| 25 | code_model2vec_jina_embeddings_v3 | **🔥 Simplified Distillation** | 0.4755 | 0.4416 | 0.5456 |
+| 26 | code_model2vec_nomic_embed_text_v2_moe | **🔥 Simplified Distillation** | 0.4532 | 0.4275 | 0.5094 |
+| 27 | code_model2vec_gte_Qwen2_1.5B_instruct | **🔥 Simplified Distillation** | 0.4238 | 0.3879 | 0.4719 |
+| 28 | code_model2vec_Qodo_Embed_1_1.5B | **🔥 Simplified Distillation** | 0.4101 | 0.3810 | 0.4532 |
+| 29 | microsoft/graphcodebert-base | Code-Specific | 0.4039 | 0.3677 | 0.4650 |
+| 30 | code_model2vec_graphcodebert_base | **🔥 Simplified Distillation** | 0.3420 | 0.3140 | 0.3704 |
+| 31 | code_model2vec_Linq_Embed_Mistral | **🔥 Simplified Distillation** | 0.2868 | 0.2581 | 0.3412 |
+| 32 | code_model2vec_codebert_base | **🔥 Simplified Distillation** | 0.2779 | 0.2534 | 0.3136 |
+| 33 | microsoft/codebert-base | Code-Specific | 0.1051 | 0.1058 | 0.1105 |
 
 
 ## 📈 Performance Analysis
@@ -236,12 +243,12 @@ Our distilled models exhibit consistent architectural characteristics across dif
 
 | Language | Best Model Performance | Average Performance | Language Difficulty |
 |----------|------------------------|--------------------|--------------------|
-| Go | 0.9780 | 0.6950 | Easy |
-| Java | 0.9921 | 0.6670 | Easy |
-| Javascript | 0.9550 | 0.5847 | Easy |
-| Php | 1.0000 | 0.6379 | Easy |
-| Python | 1.0000 | 0.8604 | Easy |
-| Ruby | 0.9493 | 0.6372 | Easy |
+| Go | 0.9780 | 0.6978 | Easy |
+| Java | 0.9921 | 0.6618 | Easy |
+| Javascript | 0.9550 | 0.5877 | Easy |
+| Php | 1.0000 | 0.6355 | Easy |
+| Python | 1.0000 | 0.8615 | Easy |
+| Ruby | 0.9493 | 0.6398 | Easy |
 
 
 ## 🎯 Conclusions and Recommendations
@@ -251,13 +258,13 @@ Our distilled models exhibit consistent architectural characteristics across dif
 Based on the evaluation results across all simplified distillation models:
 
 
-1. **Best Teacher Model**: sentence-transformers/all-mpnet-base-v2 (NDCG@10: 0.7387)
+1. **Best Teacher Model**: sentence-transformers/all-MiniLM-L6-v2 (NDCG@10: 0.7385)
 2. **Least Effective Teacher**: microsoft/codebert-base (NDCG@10: 0.2779)
 3. **Teacher Model Impact**: Choice of teacher model affects performance by 62.4%
 
 ### Recommendations
 
-- **For Production**: Use sentence-transformers/all-mpnet-base-v2 as teacher model for best performance
+- **For Production**: Use sentence-transformers/all-MiniLM-L6-v2 as teacher model for best performance
 - **For Efficiency**: Model2Vec distillation provides significant size reduction with competitive performance
 - **For Code Tasks**: Specialized models consistently outperform general-purpose models
 
@@ -295,5 +302,5 @@ Based on the evaluation results across all simplified distillation models:
 
 ---
 
-*Report generated on 2025-05-31 11:39:39 using automated analysis pipeline.*
+*Report generated on 2025-05-31 16:36:16 using automated analysis pipeline.*
 *For questions about methodology or results, please refer to the CodeSearchNet documentation.*