Text Generation
Transformers
Safetensors
English
echo
text-generation-inference
conversational
custom_code
🇪🇺 Region: EU
Instructions to use ethicalabs/Echo-DSRN-114M with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use ethicalabs/Echo-DSRN-114M with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="ethicalabs/Echo-DSRN-114M", trust_remote_code=True) messages = [ {"role": "user", "content": "Who are you?"}, ] pipe(messages)# pip install -U transformers accelerate # Load model directly from transformers import AutoModelForCausalLM model = AutoModelForCausalLM.from_pretrained("ethicalabs/Echo-DSRN-114M", trust_remote_code=True, device_map="auto") - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use ethicalabs/Echo-DSRN-114M with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "ethicalabs/Echo-DSRN-114M" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "ethicalabs/Echo-DSRN-114M", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker
docker model run hf.co/ethicalabs/Echo-DSRN-114M
- SGLang
How to use ethicalabs/Echo-DSRN-114M with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "ethicalabs/Echo-DSRN-114M" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "ethicalabs/Echo-DSRN-114M", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "ethicalabs/Echo-DSRN-114M" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "ethicalabs/Echo-DSRN-114M", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }' - Docker Model Runner
How to use ethicalabs/Echo-DSRN-114M with Docker Model Runner:
docker model run hf.co/ethicalabs/Echo-DSRN-114M
Download configuration_echo.py from ethicalabs/Echo-DSRN-114M: direct link, hf CLI and curl.
- Browser
- Download file 2.09 kB
-
https://huggingface.co/ethicalabs/Echo-DSRN-114M/resolve/main/configuration_echo.py
- Command line
-
hf download hf://ethicalabs/Echo-DSRN-114M/configuration_echo.py
-
curl -L -o configuration_echo.py https://huggingface.co/ethicalabs/Echo-DSRN-114M/resolve/main/configuration_echo.py
2.09 kB
| from transformers import PretrainedConfig | |
| class EchoConfig(PretrainedConfig): | |
| model_type = "echo" | |
| def __init__( | |
| self, | |
| vocab_size=49152, | |
| embed_dim=768, | |
| num_layers=4, | |
| num_heads=4, | |
| mlp_ratio=4, | |
| gate_bias_init=0.0, | |
| use_hybrid_attention=True, | |
| use_rmsnorm=True, | |
| **kwargs, | |
| ): | |
| # Synchronize hidden_size and embed_dim | |
| hidden_size = kwargs.pop("hidden_size", embed_dim) | |
| if embed_dim != hidden_size: | |
| # Prefer larger if both are non-standard | |
| major_dim = max(embed_dim, hidden_size) | |
| embed_dim = hidden_size = major_dim | |
| self.vocab_size = vocab_size | |
| self.embed_dim = embed_dim | |
| self.hidden_size = hidden_size | |
| self.num_layers = num_layers | |
| self.num_heads = num_heads | |
| self.mlp_ratio = mlp_ratio | |
| self.gate_bias_init = gate_bias_init | |
| self.use_hybrid_attention = use_hybrid_attention | |
| self.use_rmsnorm = use_rmsnorm | |
| # Standard HF aliases | |
| self.num_hidden_layers = num_layers | |
| self.num_attention_heads = num_heads | |
| # TGI/HF AutoMap support | |
| self.auto_map = { | |
| "AutoConfig": "configuration_echo.EchoConfig", | |
| "AutoModel": "modeling_echo.EchoModel", | |
| "AutoModelForCausalLM": "modeling_echo.EchoForCausalLM", | |
| } | |
| # vLLM Advanced Parallelism Plans | |
| self.base_model_tp_plan = { | |
| "model.embedding": "rowwise", | |
| "lm_head": "colwise", | |
| "model.blocks.*.attn.qkv_proj": "colwise", | |
| "model.blocks.*.attn.out_proj": "rowwise", | |
| "model.blocks.*.mlp_up": "colwise", | |
| "model.blocks.*.mlp_down": "rowwise", | |
| "model.blocks.*.linear_gate": "colwise", | |
| "model.blocks.*.linear_memory": "colwise", | |
| "model.blocks.*.linear_read": "rowwise", | |
| } | |
| self.base_model_pp_plan = { | |
| "blocks": (["x", "state_prev"], ["x", "h_new_full"]) # Inputs # Outputs | |
| } | |
| super().__init__(**kwargs) | |