Update README.md
#1
by yagilb - opened
- .DS_Store +0 -0
- .gitattributes +0 -6
- README.md +9 -53
- assets/.DS_Store +0 -0
- assets/images_example/example_mistral_common_1.png +0 -3
- assets/images_example/example_mistral_common_2.png +0 -3
- assets/images_example/example_mistral_common_3.png +0 -3
- assets/images_example/example_mistral_common_res_1.png +0 -3
- assets/images_example/example_mistral_common_res_2.png +0 -3
- assets/images_example/example_mistral_common_res_3.png +0 -3
- assets/tuto_open_hands/.DS_Store +0 -0
- params.json +1 -2
.DS_Store
ADDED
|
Binary file (6.15 kB). View file
|
|
|
.gitattributes
CHANGED
|
@@ -40,9 +40,3 @@ assets/tuto_open_hands/agent_working.png filter=lfs diff=lfs merge=lfs -text
|
|
| 40 |
assets/tuto_open_hands/build_app.png filter=lfs diff=lfs merge=lfs -text
|
| 41 |
assets/tuto_open_hands/agent_prompting.png filter=lfs diff=lfs merge=lfs -text
|
| 42 |
assets/tuto_open_hands/app_ui.png filter=lfs diff=lfs merge=lfs -text
|
| 43 |
-
assets/images_example/example_mistral_common_1.png filter=lfs diff=lfs merge=lfs -text
|
| 44 |
-
assets/images_example/example_mistral_common_2.png filter=lfs diff=lfs merge=lfs -text
|
| 45 |
-
assets/images_example/example_mistral_common_3.png filter=lfs diff=lfs merge=lfs -text
|
| 46 |
-
assets/images_example/example_mistral_common_res_1.png filter=lfs diff=lfs merge=lfs -text
|
| 47 |
-
assets/images_example/example_mistral_common_res_2.png filter=lfs diff=lfs merge=lfs -text
|
| 48 |
-
assets/images_example/example_mistral_common_res_3.png filter=lfs diff=lfs merge=lfs -text
|
|
|
|
| 40 |
assets/tuto_open_hands/build_app.png filter=lfs diff=lfs merge=lfs -text
|
| 41 |
assets/tuto_open_hands/agent_prompting.png filter=lfs diff=lfs merge=lfs -text
|
| 42 |
assets/tuto_open_hands/app_ui.png filter=lfs diff=lfs merge=lfs -text
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
README.md
CHANGED
|
@@ -1,5 +1,4 @@
|
|
| 1 |
---
|
| 2 |
-
library_name: vllm
|
| 3 |
language:
|
| 4 |
- en
|
| 5 |
- fr
|
|
@@ -26,17 +25,17 @@ language:
|
|
| 26 |
- hi
|
| 27 |
- bn
|
| 28 |
license: apache-2.0
|
|
|
|
| 29 |
inference: false
|
| 30 |
base_model:
|
| 31 |
-
- mistralai/
|
| 32 |
extra_gated_description: >-
|
| 33 |
If you want to learn more about how we process your personal data, please read
|
| 34 |
our <a href="https://mistral.ai/terms/">Privacy Policy</a>.
|
| 35 |
-
|
| 36 |
-
- mistral-common
|
| 37 |
---
|
| 38 |
|
| 39 |
-
#
|
| 40 |
|
| 41 |
Devstral is an agentic LLM for software engineering tasks built under a collaboration between [Mistral AI](https://mistral.ai/) and [All Hands AI](https://www.all-hands.dev/) 🙌. Devstral excels at using tools to explore codebases, editing multiple files and power software engineering agents. The model achieves remarkable performance on SWE-bench which positionates it as the #1 open source model on this [benchmark](#benchmark-results).
|
| 42 |
|
|
@@ -108,15 +107,14 @@ The model can also be deployed with the following libraries:
|
|
| 108 |
- [`mistral-inference`](https://github.com/mistralai/mistral-inference): See [here](#mistral-inference)
|
| 109 |
- [`transformers`](https://github.com/huggingface/transformers): See [here](#transformers)
|
| 110 |
- [`LMStudio`](https://lmstudio.ai/): See [here](#lmstudio)
|
| 111 |
-
- [`llama.cpp`](https://github.com/ggml-org/llama.cpp): See [here](#llama.cpp)
|
| 112 |
- [`ollama`](https://github.com/ollama/ollama): See [here](#ollama)
|
| 113 |
|
| 114 |
|
| 115 |
### OpenHands (recommended)
|
| 116 |
|
| 117 |
-
#### Launch a server to deploy Devstral
|
| 118 |
|
| 119 |
-
Make sure you launched an OpenAI-compatible server such as vLLM or Ollama as described above. Then, you can use OpenHands to interact with `Devstral
|
| 120 |
|
| 121 |
In the case of the tutorial we spineed up a vLLM server running the command:
|
| 122 |
```bash
|
|
@@ -323,6 +321,7 @@ from mistral_common.protocol.instruct.messages import (
|
|
| 323 |
)
|
| 324 |
from mistral_common.protocol.instruct.request import ChatCompletionRequest
|
| 325 |
from mistral_common.tokens.tokenizers.mistral import MistralTokenizer
|
|
|
|
| 326 |
from huggingface_hub import hf_hub_download
|
| 327 |
from transformers import AutoModelForCausalLM
|
| 328 |
|
|
@@ -373,7 +372,7 @@ You can serve the model locally with [LMStudio](https://lmstudio.ai/).
|
|
| 373 |
* Download [LM Studio](https://lmstudio.ai/) and install it
|
| 374 |
* Install `lms cli ~/.lmstudio/bin/lms bootstrap`
|
| 375 |
* In a bash terminal, run `lms import devstralQ4_K_M.gguf` in the directory where you've downloaded the model checkpoint (e.g. `mistralai/Devstral-Small-2505_gguf`)
|
| 376 |
-
* Open the LMStudio application, click the terminal icon to get into the developer tab. Click select a model to load and select Devstral Q4 K M. Toggle the status button to start the model, in setting
|
| 377 |
* On the right tab, you will see an API identifier which should be devstralq4_k_m and an api address under API Usage. Keep note of this address, we will use it in the next step.
|
| 378 |
|
| 379 |
Launch Openhands
|
|
@@ -395,23 +394,6 @@ docker run -it --rm --pull=always \
|
|
| 395 |
Click “see advanced setting” on the second line.
|
| 396 |
In the new tab, toggle advanced to on. Set the custom model to be mistral/devstralq4_k_m and Base URL the api address we get from the last step in LM Studio. Set API Key to dummy. Click save changes.
|
| 397 |
|
| 398 |
-
### llama.cpp
|
| 399 |
-
|
| 400 |
-
Download the weights from huggingface:
|
| 401 |
-
|
| 402 |
-
```
|
| 403 |
-
pip install -U "huggingface_hub[cli]"
|
| 404 |
-
huggingface-cli download \
|
| 405 |
-
"mistralai/Devstral-Small-2505_gguf" \
|
| 406 |
-
--include "devstralQ4_K_M.gguf" \
|
| 407 |
-
--local-dir "mistralai/Devstral-Small-2505_gguf/"
|
| 408 |
-
```
|
| 409 |
-
|
| 410 |
-
Then run Devstral using the llama.cpp CLI.
|
| 411 |
-
|
| 412 |
-
```bash
|
| 413 |
-
./llama-cli -m Devstral-Small-2505_gguf/devstralQ4_K_M.gguf -cnv
|
| 414 |
-
```
|
| 415 |
|
| 416 |
### Ollama
|
| 417 |
|
|
@@ -419,30 +401,4 @@ You can run Devstral using the [Ollama](https://ollama.ai/) CLI.
|
|
| 419 |
|
| 420 |
```bash
|
| 421 |
ollama run devstral
|
| 422 |
-
```
|
| 423 |
-
|
| 424 |
-
### Example: Understanding Test Coverage of Mistral Common
|
| 425 |
-
|
| 426 |
-
We can start the OpenHands scaffold and link it to a repo to analyze test coverage and identify badly covered files.
|
| 427 |
-
Here we start with our public `mistral-common` repo.
|
| 428 |
-
|
| 429 |
-
|
| 430 |
-
After the repo is mounted in the workspace, we give the following instruction
|
| 431 |
-
```
|
| 432 |
-
Check the test coverage of the repo and then create a visualization of test coverage. Try plotting a few different types of graphs and save them to a png.
|
| 433 |
-
```
|
| 434 |
-
The agent will first browse the code base to check test configuration and structure.
|
| 435 |
-
|
| 436 |
-

|
| 437 |
-
|
| 438 |
-
Then it sets up the testing dependencies and launches the coverage test:
|
| 439 |
-
|
| 440 |
-

|
| 441 |
-
|
| 442 |
-
Finally, the agent writes necessary code to visualize the coverage.
|
| 443 |
-

|
| 444 |
-
|
| 445 |
-
At the end of the run, the following plots are produced:
|
| 446 |
-

|
| 447 |
-

|
| 448 |
-

|
|
|
|
| 1 |
---
|
|
|
|
| 2 |
language:
|
| 3 |
- en
|
| 4 |
- fr
|
|
|
|
| 25 |
- hi
|
| 26 |
- bn
|
| 27 |
license: apache-2.0
|
| 28 |
+
library_name: vllm
|
| 29 |
inference: false
|
| 30 |
base_model:
|
| 31 |
+
- mistralai/Devstrall-Small-2505
|
| 32 |
extra_gated_description: >-
|
| 33 |
If you want to learn more about how we process your personal data, please read
|
| 34 |
our <a href="https://mistral.ai/terms/">Privacy Policy</a>.
|
| 35 |
+
pipeline_tag: text2text-generation
|
|
|
|
| 36 |
---
|
| 37 |
|
| 38 |
+
# Devstrall-Small-2505
|
| 39 |
|
| 40 |
Devstral is an agentic LLM for software engineering tasks built under a collaboration between [Mistral AI](https://mistral.ai/) and [All Hands AI](https://www.all-hands.dev/) 🙌. Devstral excels at using tools to explore codebases, editing multiple files and power software engineering agents. The model achieves remarkable performance on SWE-bench which positionates it as the #1 open source model on this [benchmark](#benchmark-results).
|
| 41 |
|
|
|
|
| 107 |
- [`mistral-inference`](https://github.com/mistralai/mistral-inference): See [here](#mistral-inference)
|
| 108 |
- [`transformers`](https://github.com/huggingface/transformers): See [here](#transformers)
|
| 109 |
- [`LMStudio`](https://lmstudio.ai/): See [here](#lmstudio)
|
|
|
|
| 110 |
- [`ollama`](https://github.com/ollama/ollama): See [here](#ollama)
|
| 111 |
|
| 112 |
|
| 113 |
### OpenHands (recommended)
|
| 114 |
|
| 115 |
+
#### Launch a server to deploy Devstral-Small-2505
|
| 116 |
|
| 117 |
+
Make sure you launched an OpenAI-compatible server such as vLLM or Ollama as described above. Then, you can use OpenHands to interact with `Devstral-Small-2505`.
|
| 118 |
|
| 119 |
In the case of the tutorial we spineed up a vLLM server running the command:
|
| 120 |
```bash
|
|
|
|
| 321 |
)
|
| 322 |
from mistral_common.protocol.instruct.request import ChatCompletionRequest
|
| 323 |
from mistral_common.tokens.tokenizers.mistral import MistralTokenizer
|
| 324 |
+
from mistral_common.tokens.tokenizers.tekken import SpecialTokenPolicy
|
| 325 |
from huggingface_hub import hf_hub_download
|
| 326 |
from transformers import AutoModelForCausalLM
|
| 327 |
|
|
|
|
| 372 |
* Download [LM Studio](https://lmstudio.ai/) and install it
|
| 373 |
* Install `lms cli ~/.lmstudio/bin/lms bootstrap`
|
| 374 |
* In a bash terminal, run `lms import devstralQ4_K_M.gguf` in the directory where you've downloaded the model checkpoint (e.g. `mistralai/Devstral-Small-2505_gguf`)
|
| 375 |
+
* Open the LMStudio application, click the terminal icon to get into the developer tab. Click select a model to load and select Devstral Q4 K M. Toggle the status button to start the model, in setting oggle Serve on Local Network to be on.
|
| 376 |
* On the right tab, you will see an API identifier which should be devstralq4_k_m and an api address under API Usage. Keep note of this address, we will use it in the next step.
|
| 377 |
|
| 378 |
Launch Openhands
|
|
|
|
| 394 |
Click “see advanced setting” on the second line.
|
| 395 |
In the new tab, toggle advanced to on. Set the custom model to be mistral/devstralq4_k_m and Base URL the api address we get from the last step in LM Studio. Set API Key to dummy. Click save changes.
|
| 396 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 397 |
|
| 398 |
### Ollama
|
| 399 |
|
|
|
|
| 401 |
|
| 402 |
```bash
|
| 403 |
ollama run devstral
|
| 404 |
+
```
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
assets/.DS_Store
ADDED
|
Binary file (6.15 kB). View file
|
|
|
assets/images_example/example_mistral_common_1.png
DELETED
Git LFS Details
|
assets/images_example/example_mistral_common_2.png
DELETED
Git LFS Details
|
assets/images_example/example_mistral_common_3.png
DELETED
Git LFS Details
|
assets/images_example/example_mistral_common_res_1.png
DELETED
Git LFS Details
|
assets/images_example/example_mistral_common_res_2.png
DELETED
Git LFS Details
|
assets/images_example/example_mistral_common_res_3.png
DELETED
Git LFS Details
|
assets/tuto_open_hands/.DS_Store
ADDED
|
Binary file (6.15 kB). View file
|
|
|
params.json
CHANGED
|
@@ -7,7 +7,6 @@
|
|
| 7 |
"n_kv_heads": 8,
|
| 8 |
"rope_theta": 1000000000.0,
|
| 9 |
"norm_eps": 1e-05,
|
| 10 |
-
"vocab_size": 131072
|
| 11 |
-
"max_position_embeddings": 131072
|
| 12 |
}
|
| 13 |
|
|
|
|
| 7 |
"n_kv_heads": 8,
|
| 8 |
"rope_theta": 1000000000.0,
|
| 9 |
"norm_eps": 1e-05,
|
| 10 |
+
"vocab_size": 131072
|
|
|
|
| 11 |
}
|
| 12 |
|