Skip to content

📦 Installation

Requires Python 3.10 and a CUDA GPU (CPU works for compression, just slower).

1. Clone the repository

git clone https://github.com/oteomamo/SALT.git
cd SALT

2. Create the environment

With conda:

conda env create -f environment.yml
conda activate salt

Or with venv:

python3.10 -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt

3. Install SALT in editable mode

pip install -e .

This also installs the four console commands: salt (one-shot compression, see Usage), saltChat (interactive chat, see Chatbot mode), saltServe (persistent serving, see Serving) and salt-mcp (the MCP server, see MCP server).

4. Authenticate with Hugging Face - the eval model (meta-llama/Llama-3.1-8B-Instruct) is gated:

hf auth login

Or skip the CLI and export the token directly: export HF_TOKEN=hf_....

5. (Optional) vLLM backend. eval.py defaults to vLLM, saltChat --backend vllm uses it for prefix caching, and saltServe launches a persistent model server with it. Install it into the salt env:

pip install "vllm==0.11.0" "prometheus-fastapi-instrumentator>=8.0.1"

The second pin keeps the server's routes healthy next to newer fastapi releases. Skip this and run eval.py --backend hf for a portable run that needs no vLLM. saltChat already defaults to its HF backend. saltServe can also run a vLLM installed in a separate environment through --vllm-bin.

6. (Optional) MCP server. salt-mcp puts SALT behind the Model Context Protocol, so an editor or an agent runtime can compress text and keep conversation memory through it. Install the extra:

pip install "salt[mcp]"

Then point a client at the salt-mcp command, see MCP server. It needs no GPU.

bash scripts/setup_env.sh does steps 2–3 in one shot (add WITH_VLLM=1 to include vLLM).