Deploy faster-whisper on a VPS

faster-whisper is a CTranslate2-based reimplementation of OpenAI's Whisper speech-to-text model. It runs well on CPU-only VPS instances, which makes it a good fit for most RamNode plans (GPU is optional and only helps with larger models or high throughput).

1. Choose and size the VPS

Use case	Recommended RamNode plan specs
Occasional transcription, `small`/`base` model	2 vCPU, 4 GB RAM
Regular use, `medium` model	4 vCPU, 8 GB RAM
Heavy use / `large-v3`, low latency	8 vCPU, 16 GB RAM (or a GPU instance if available)

faster-whisper is much lighter on RAM than the original openai-whisper package because CTranslate2 uses int8/float16 quantization, but disk space for model weights still matters — budget 1–3 GB per model size you plan to keep cached.

Recommended OS: Ubuntu 24.04 LTS.

2. Initial server setup

SSH in as root (or a sudo user) and do baseline hardening/updates first:

shell

apt update && apt -y upgrade
apt -y install build-essential python3 python3-venv python3-pip ffmpeg git ufw ca-certificates

# Firewall: allow SSH, and whatever port you'll expose the API on (8000 used below)
ufw allow OpenSSH
ufw allow 8000/tcp
ufw enable

ffmpeg is required — faster-whisper (via CTranslate2/PyAV) uses it for audio decoding.

Create a dedicated, non-root service user:

shell

adduser --system --group --home /opt/faster-whisper whisper
mkdir -p /opt/faster-whisper
chown whisper:whisper /opt/faster-whisper

3. Install faster-whisper in a virtualenv

shell

su - whisper -s /bin/bash
cd /opt/faster-whisper
python3 -m venv venv
source venv/bin/activate
pip install --upgrade pip
pip install faster-whisper fastapi "uvicorn[standard]" python-multipart

Quick sanity test from a Python shell:

shell

from faster_whisper import WhisperModel

model = WhisperModel("small", device="cpu", compute_type="int8")
segments, info = model.transcribe("some_audio.wav", beam_size=5)

print(f"Detected language: {info.language}")
for seg in segments:
    print(f"[{seg.start:.2f}s -> {seg.end:.2f}s] {seg.text}")

The first run downloads the model from Hugging Face into ~/.cache/huggingface; subsequent runs use the local cache.

compute_type="int8" is the right default for CPU-only VPS instances — best speed/memory tradeoff.
Model sizes, smallest to largest: tiny, base, small, medium, large-v2, large-v3. Start with small or medium and only move up if accuracy demands it.

4. Wrap it in a small HTTP API

Create /opt/faster-whisper/server.py:

shell

from fastapi import FastAPI, UploadFile, File
from faster_whisper import WhisperModel
import tempfile, os

MODEL_SIZE = os.environ.get("WHISPER_MODEL", "small")
DEVICE = os.environ.get("WHISPER_DEVICE", "cpu")
COMPUTE_TYPE = os.environ.get("WHISPER_COMPUTE_TYPE", "int8")

app = FastAPI()
model = WhisperModel(MODEL_SIZE, device=DEVICE, compute_type=COMPUTE_TYPE)

@app.post("/transcribe")
async def transcribe(file: UploadFile = File(...)):
    with tempfile.NamedTemporaryFile(suffix=".audio", delete=False) as tmp:
        tmp.write(await file.read())
        tmp_path = tmp.name

    try:
        segments, info = model.transcribe(tmp_path, beam_size=5)
        text = "".join(seg.text for seg in segments)
        return {"language": info.language, "text": text.strip()}
    finally:
        os.remove(tmp_path)

@app.get("/health")
async def health():
    return {"status": "ok", "model": MODEL_SIZE}

Test it manually before wiring up systemd:

shell

uvicorn server:app --host 127.0.0.1 --port 8000

5. Run it as a systemd service

Exit back to root and create /etc/systemd/system/faster-whisper.service:

shell

[Unit]
Description=faster-whisper transcription API
After=network.target

[Service]
Type=simple
User=whisper
Group=whisper
WorkingDirectory=/opt/faster-whisper
Environment=WHISPER_MODEL=small
Environment=WHISPER_DEVICE=cpu
Environment=WHISPER_COMPUTE_TYPE=int8
ExecStart=/opt/faster-whisper/venv/bin/uvicorn server:app --host 127.0.0.1 --port 8000
Restart=on-failure
RestartSec=5
LimitNOFILE=65535

[Install]
WantedBy=multi-user.target

Enable and start:

shell

systemctl daemon-reload
systemctl enable --now faster-whisper
systemctl status faster-whisper

6. Put nginx in front (recommended)

Bind uvicorn to 127.0.0.1 only (as above) and reverse-proxy through nginx so you get TLS and can control access:

shell

server {
    listen 80;
    server_name whisper.example.com;

    location / {
        proxy_pass http://127.0.0.1:8000;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        client_max_body_size 100M;   # allow large audio uploads
    }
}

Then get a certificate:

shell

apt -y install certbot python3-certbot-nginx
certbot --nginx -d whisper.example.com

Close the raw port in ufw once nginx is fronting it:

shell

ufw delete allow 8000/tcp
ufw allow "Nginx Full"

7. Operational notes

Model caching: pre-download models at deploy time (run a one-off transcription) rather than on first real request, to avoid a cold-start timeout hitting a real user.
Concurrency: faster-whisper releases the GIL during inference but is still CPU-bound; for concurrent requests, run multiple uvicorn workers (--workers N) or put a queue in front, sized to vCPU count.
Monitoring: add an HTTP check against /health to Nagios/your monitoring stack.
Updates: pip install -U faster-whisper inside the venv, then systemctl restart faster-whisper.
Backups: the only state worth backing up is the Hugging Face model cache (re-downloadable) and your server.py config — no database involved.