Deploy SeaweedFS on a VPS

Slug: deploy-seaweedfs-distributed-storage-vps Difficulty: Intermediate to Advanced Estimated time: 60 minutes Tested on: Ubuntu 24.04 LTS, SeaweedFS 4.44

Introduction

SeaweedFS is a distributed storage system built around Facebook's Haystack design. It stores billions of small files with O(1) disk seeks, which is the opposite of what most object stores are tuned for. Where MinIO and Garage store one object per file, SeaweedFS packs objects into large volume files and keeps a compact needle index in memory. The result is very fast small-file access at a fraction of the metadata overhead.

On top of the blob store, SeaweedFS layers a Filer that provides directories, POSIX attributes, an S3 gateway, WebDAV, and FUSE mounts. You can run all of it on one VPS and split components onto separate machines later without changing the client-facing API.

Good fits for a RamNode VPS:

• Image, thumbnail, and avatar storage for a web application
• S3-compatible backend for a CMS or media pipeline
• Filer-backed WebDAV or FUSE mount shared across servers
• Cloud tiering, keeping hot data local and cold data on remote S3
• A storage layer that needs to scale horizontally later without migration

This guide deploys the four core components as separate systemd services, adds the admin UI, exposes S3 over TLS, and covers day-two operations.

Architecture

RamNode VPS sizing

Memory scales with file count, not total bytes. The volume server keeps roughly 20 bytes of index per file in RAM by default, so 10 million files is about 200 MB of index. Use -index=leveldb on the volume server if you need to trade a little latency for much lower memory use.

Prerequisites

• A RamNode KVM VPS running Ubuntu 24.04 LTS
• Root or sudo access
• A domain pointed at the VPS. This guide uses s3.example.com and swadmin.example.com
• Ports 80 and 443 open

Step 1: Prepare the server

sudo apt update && sudo apt upgrade -y
sudo apt install -y curl wget tar jq

Create a service user and the directory layout.

sudo useradd --system --no-create-home --shell /usr/sbin/nologin seaweedfs

sudo mkdir -p /var/lib/seaweedfs/{master,volume,filerldb2,admin}
sudo mkdir -p /etc/seaweedfs
sudo mkdir -p /var/log/seaweedfs

sudo chown -R seaweedfs:seaweedfs /var/lib/seaweedfs /var/log/seaweedfs
sudo chmod 750 /var/lib/seaweedfs

Raise the file descriptor limit. Volume servers hold many open files.

cat <<'EOF' | sudo tee /etc/security/limits.d/seaweedfs.conf
seaweedfs soft nofile 65536
seaweedfs hard nofile 262144
EOF

Step 2: Install the weed binary

SeaweedFS ships a single binary named weed. Download release 4.44.

cd /tmp
wget https://github.com/seaweedfs/seaweedfs/releases/download/4.44/linux_amd64.tar.gz
tar xzf linux_amd64.tar.gz
sudo install -m 755 weed /usr/local/bin/weed
rm -f weed linux_amd64.tar.gz

Verify.

weed version

If you plan to store more than 30 TB on a single volume server, use the linux_amd64_large_disk.tar.gz build instead. It uses 8-byte offsets rather than 4-byte, doubling index memory but removing the volume size ceiling. For ARM64 instances use linux_arm64.tar.gz.

Step 3: Start the master server

The master tracks topology and hands out file IDs. Create /etc/systemd/system/seaweedfs-master.service.

[Unit]
Description=SeaweedFS Master
After=network-online.target
Wants=network-online.target

[Service]
Type=simple
User=seaweedfs
Group=seaweedfs
ExecStart=/usr/local/bin/weed master \
  -ip=127.0.0.1 \
  -port=9333 \
  -mdir=/var/lib/seaweedfs/master \
  -defaultReplication=000 \
  -volumeSizeLimitMB=30000
Restart=on-failure
RestartSec=5
LimitNOFILE=65536
NoNewPrivileges=true
PrivateTmp=true
ProtectSystem=strict
ProtectHome=true
ReadWritePaths=/var/lib/seaweedfs /var/log/seaweedfs

[Install]
WantedBy=multi-user.target

-defaultReplication=000 means no replication, which is correct on a single node. The three digits mean copies on other data centers, other racks, and other servers respectively. On a two-node setup you would use 001.

Start it.

sudo systemctl daemon-reload
sudo systemctl enable --now seaweedfs-master
curl -s http://127.0.0.1:9333/cluster/status | jq

Step 4: Start the volume server

Create /etc/systemd/system/seaweedfs-volume.service.

[Unit]
Description=SeaweedFS Volume
After=seaweedfs-master.service
Requires=seaweedfs-master.service

[Service]
Type=simple
User=seaweedfs
Group=seaweedfs
ExecStart=/usr/local/bin/weed volume \
  -mserver=127.0.0.1:9333 \
  -ip=127.0.0.1 \
  -port=8080 \
  -dir=/var/lib/seaweedfs/volume \
  -max=0 \
  -dataCenter=dc1 \
  -rack=rack1 \
  -index=leveldb
Restart=on-failure
RestartSec=5
LimitNOFILE=262144
NoNewPrivileges=true
PrivateTmp=true
ProtectSystem=strict
ProtectHome=true
ReadWritePaths=/var/lib/seaweedfs /var/log/seaweedfs

[Install]
WantedBy=multi-user.target

-max=0 lets the volume server size itself against free disk space. -index=leveldb keeps the needle index on disk instead of in RAM, which matters on a 2 GB VPS holding millions of files. Drop it to -index=memory if you have RAM to spare and want lower read latency.

sudo systemctl daemon-reload
sudo systemctl enable --now seaweedfs-volume
curl -s http://127.0.0.1:9333/dir/status | jq '.Topology'

You should see the volume server registered with its free volume count.

Step 5: Configure and start the Filer

The Filer needs a metadata store. LevelDB on local disk is the default and is fine for a single node. Generate a scaffold config.

weed scaffold -config=filer | sudo tee /etc/seaweedfs/filer.toml > /dev/null
sudo chown seaweedfs:seaweedfs /etc/seaweedfs/filer.toml
sudo chmod 640 /etc/seaweedfs/filer.toml

Edit /etc/seaweedfs/filer.toml and make sure the leveldb2 block is the only enabled store.

[leveldb2]
enabled = true
dir = "/var/lib/seaweedfs/filerldb2"

If you expect heavy metadata churn or plan to run multiple Filers later, switch to PostgreSQL instead. Disable leveldb2 and enable:

[postgres]
enabled = true
hostname = "127.0.0.1"
port = 5432
username = "seaweedfs"
password = "REPLACE_ME"
database = "seaweedfs"
sslmode = "disable"
connection_max_idle = 5
connection_max_open = 50

Create /etc/systemd/system/seaweedfs-filer.service.

[Unit]
Description=SeaweedFS Filer
After=seaweedfs-volume.service
Requires=seaweedfs-master.service

[Service]
Type=simple
User=seaweedfs
Group=seaweedfs
Environment=WEED_DIR=/etc/seaweedfs
ExecStart=/usr/local/bin/weed -config_dir=/etc/seaweedfs filer \
  -master=127.0.0.1:9333 \
  -ip=127.0.0.1 \
  -port=8888
Restart=on-failure
RestartSec=5
LimitNOFILE=65536
NoNewPrivileges=true
PrivateTmp=true
ProtectSystem=strict
ProtectHome=true
ReadWritePaths=/var/lib/seaweedfs /var/log/seaweedfs

[Install]
WantedBy=multi-user.target

sudo systemctl daemon-reload
sudo systemctl enable --now seaweedfs-filer
curl -s http://127.0.0.1:8888/ | head

Step 6: Start the S3 gateway

The S3 gateway translates S3 API calls into Filer operations. Create /etc/systemd/system/seaweedfs-s3.service.

[Unit]
Description=SeaweedFS S3 Gateway
After=seaweedfs-filer.service
Requires=seaweedfs-filer.service

[Service]
Type=simple
User=seaweedfs
Group=seaweedfs
ExecStart=/usr/local/bin/weed -config_dir=/etc/seaweedfs s3 \
  -filer=127.0.0.1:8888 \
  -ip.bind=127.0.0.1 \
  -port=8333 \
  -allowEmptyFolder=false
Restart=on-failure
RestartSec=5
LimitNOFILE=65536
NoNewPrivileges=true
PrivateTmp=true
ProtectSystem=strict
ProtectHome=true
ReadWritePaths=/var/lib/seaweedfs /var/log/seaweedfs

[Install]
WantedBy=multi-user.target

sudo systemctl daemon-reload
sudo systemctl enable --now seaweedfs-s3

At this point the gateway allows anonymous access, because no identities are configured. Fix that in the next step before exposing anything publicly.

Step 7: Create S3 credentials

The modern approach stores identities in the Filer, so every S3 gateway connected to that Filer picks up changes automatically. Use weed shell.

sudo -u seaweedfs weed shell -master=127.0.0.1:9333

Inside the shell:

s3.configure -user=appuser \
  -access_key=AKIAEXAMPLEKEY \
  -secret_key=REPLACE_WITH_A_LONG_RANDOM_SECRET \
  -buckets=assets \
  -actions=Read,Write,List,Tagging \
  -apply

s3.bucket.create -name assets
s3.bucket.list
exit

Generate the secret properly rather than typing one:

openssl rand -base64 32

Available actions are Read, Write, List, Tagging, Admin, and ReadAcp/WriteAcp. Create one identity per application and scope it to specific buckets with -buckets=. Reserve Admin for a break-glass account.

The S3 gateway logs a line confirming it reloaded /etc/seaweedfs/iam/identity.json when the configuration changes. No restart needed.

Step 8: Start the admin UI

SeaweedFS 4.x ships a web administration interface covering cluster topology, volume management, S3 users and buckets, and maintenance tasks.

Create /etc/systemd/system/seaweedfs-admin.service.

[Unit]
Description=SeaweedFS Admin UI
After=seaweedfs-filer.service
Requires=seaweedfs-master.service

[Service]
Type=simple
User=seaweedfs
Group=seaweedfs
ExecStart=/usr/local/bin/weed admin \
  -port=23646 \
  -masters=127.0.0.1:9333 \
  -dataDir=/var/lib/seaweedfs/admin \
  -adminUser=admin \
  -adminPassword=REPLACE_WITH_A_STRONG_PASSWORD
Restart=on-failure
RestartSec=5
NoNewPrivileges=true
PrivateTmp=true
ProtectSystem=strict
ProtectHome=true
ReadWritePaths=/var/lib/seaweedfs

[Install]
WantedBy=multi-user.target

The admin service runs a gRPC listener for worker connections on the HTTP port plus 10000, so 33646 in this configuration. Leave that port closed to the internet.

Putting a password on a systemd unit file makes it readable by anyone who can read the unit. Tighten it:

sudo chmod 600 /etc/systemd/system/seaweedfs-admin.service
sudo systemctl daemon-reload
sudo systemctl enable --now seaweedfs-admin

Never run the admin UI without -adminUser and -adminPassword. It starts fine without them and logs a warning, which is exactly how open clusters end up on the internet.

Step 9: Firewall

Every SeaweedFS component in this guide binds to 127.0.0.1. Only the reverse proxy is public.

sudo ufw allow OpenSSH
sudo ufw allow 80/tcp
sudo ufw allow 443/tcp
sudo ufw --force enable
sudo ufw status verbose

Confirm nothing is listening on a public interface:

sudo ss -tlnp | grep weed

Every line should show 127.0.0.1.

Step 10: Reverse proxy with TLS

Install nginx and Certbot.

sudo apt install -y nginx certbot python3-certbot-nginx

Create /etc/nginx/sites-available/seaweedfs.

server {
    listen 80;
    server_name s3.example.com swadmin.example.com;
    return 301 https://$host$request_uri;
}

server {
    listen 443 ssl;
    http2 on;
    server_name s3.example.com;

    ssl_certificate     /etc/letsencrypt/live/s3.example.com/fullchain.pem;
    ssl_certificate_key /etc/letsencrypt/live/s3.example.com/privkey.pem;

    client_max_body_size 0;
    proxy_request_buffering off;
    proxy_buffering off;
    chunked_transfer_encoding off;

    location / {
        proxy_pass http://127.0.0.1:8333;
        proxy_http_version 1.1;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
        proxy_connect_timeout 300s;
        proxy_read_timeout 3600s;
        proxy_send_timeout 3600s;
    }
}

server {
    listen 443 ssl;
    http2 on;
    server_name swadmin.example.com;

    ssl_certificate     /etc/letsencrypt/live/swadmin.example.com/fullchain.pem;
    ssl_certificate_key /etc/letsencrypt/live/swadmin.example.com/privkey.pem;

    # Restrict the admin UI to known addresses
    allow 203.0.113.0/24;
    deny all;

    location / {
        proxy_pass http://127.0.0.1:23646;
        proxy_http_version 1.1;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection "upgrade";
    }
}

Enable it and issue certificates.

sudo ln -s /etc/nginx/sites-available/seaweedfs /etc/nginx/sites-enabled/
sudo rm -f /etc/nginx/sites-enabled/default
sudo nginx -t && sudo systemctl reload nginx
sudo certbot --nginx -d s3.example.com -d swadmin.example.com

client_max_body_size 0 and proxy_request_buffering off are mandatory. Without them nginx spools every upload to disk and caps object size at 1 MB.

Step 11: Test the S3 endpoint

sudo apt install -y awscli
aws configure --profile seaweed
# Access key: AKIAEXAMPLEKEY
# Secret key: the value from Step 7
# Region: us-east-1

echo "seaweed test" > test.txt
aws --profile seaweed --endpoint-url https://s3.example.com s3 cp test.txt s3://assets/
aws --profile seaweed --endpoint-url https://s3.example.com s3 ls s3://assets/
aws --profile seaweed --endpoint-url https://s3.example.com s3 rm s3://assets/test.txt

If newer AWS SDKs return checksum errors, add this to the profile in ~/.aws/config:

request_checksum_calculation = when_required
response_checksum_validation = when_required

Step 12: Mount the Filer (optional)

SeaweedFS can be mounted as a POSIX filesystem via FUSE, which is useful for legacy applications that expect a directory.

sudo mkdir -p /mnt/seaweed
sudo weed mount -filer=127.0.0.1:8888 -dir=/mnt/seaweed -filer.path=/buckets/assets

As a systemd unit:

[Unit]
Description=SeaweedFS FUSE mount
After=seaweedfs-filer.service
Requires=seaweedfs-filer.service

[Service]
Type=simple
ExecStart=/usr/local/bin/weed mount -filer=127.0.0.1:8888 -dir=/mnt/seaweed -filer.path=/buckets/assets -allowOthers=true
ExecStop=/bin/fusermount -u /mnt/seaweed
Restart=on-failure
RestartSec=10

[Install]
WantedBy=multi-user.target

FUSE mounts are convenient but slower than direct S3 or Filer HTTP calls. Use them for compatibility, not for hot paths.

Operations

Cluster health

curl -s http://127.0.0.1:9333/cluster/status | jq
curl -s http://127.0.0.1:9333/dir/status | jq '.Topology'
curl -s http://127.0.0.1:8080/status | jq

Maintenance via weed shell

sudo -u seaweedfs weed shell -master=127.0.0.1:9333

Useful commands:

volume.list                    # topology and per-volume detail
volume.fix.replication         # restore replica counts after adding nodes
volume.balance -force          # even out volumes across servers
volume.vacuum -garbageThreshold=0.3   # reclaim space from deleted files
fs.meta.save -o /var/lib/seaweedfs/filer-meta.bak   # back up filer metadata
fs.du /buckets/assets          # space usage by path
s3.bucket.list

volume.vacuum matters. SeaweedFS marks deletes rather than reclaiming immediately, so space is not returned until a vacuum runs. The master triggers this automatically at a 30 percent garbage threshold by default, but run it manually after a bulk delete.

Backups

Three things need backing up:

1. Filer metadata. Run fs.meta.save from the weed shell on a schedule and copy the output off the box. With PostgreSQL, back up the database instead.
2. Volume data. Use weed backup to replicate volumes to another host, or snapshot the underlying disk.
3. Configuration. /etc/seaweedfs/ and your systemd unit files.

An hourly metadata dump with a nightly volume sync is a reasonable baseline for a single-node deployment.

sudo -u seaweedfs weed backup -server=127.0.0.1:8080 -dir=/backup/seaweedfs -volumeId=1

Monitoring

Every component can expose Prometheus metrics with -metricsPort. Add it to the master and volume units:

-metricsPort=9324

Scrape those endpoints from Prometheus and use the Grafana dashboards published in the SeaweedFS repository. Watch free volume count on the master and disk utilization on the volume server. Running out of free volumes stops writes even when disk space remains.

Upgrading

sudo -u seaweedfs weed shell -master=127.0.0.1:9333 -c "fs.meta.save -o /var/lib/seaweedfs/pre-upgrade.bak"
sudo systemctl stop seaweedfs-s3 seaweedfs-filer seaweedfs-volume seaweedfs-master
# install the new binary
sudo systemctl start seaweedfs-master seaweedfs-volume seaweedfs-filer seaweedfs-s3

Stop in dependency order, top down, and start bottom up. SeaweedFS releases frequently, roughly weekly, so pin a version you have tested rather than tracking the newest tag.

Scaling out

Adding capacity means adding volume servers, not resizing one box.

On a second RamNode VPS:

weed volume \
  -mserver=<master-private-ip>:9333 \
  -ip=<node2-private-ip> \
  -port=8080 \
  -dir=/var/lib/seaweedfs/volume \
  -max=0 \
  -dataCenter=dc2 \
  -rack=rack1

Then raise replication so new writes land on both nodes:

# In weed shell
volume.configure.replication -replication=001 -collection=""
volume.fix.replication

Keep master gRPC (19333) and volume gRPC (18080) reachable between nodes only, never publicly. If the nodes are in different RamNode locations, run the inter-node traffic over WireGuard rather than the public interface.

For real high availability, run three masters with Raft leader election:

weed master -peers=10.0.0.1:9333,10.0.0.2:9333,10.0.0.3:9333 ...

Troubleshooting

Writes fail with "no free volumes left". The master has no volume server with capacity. Check curl http://127.0.0.1:9333/dir/status. Either add disk, add a volume server, or run volume.vacuum to reclaim deleted space.

Filer starts but every request 500s. The Filer cannot reach its metadata store. Check filer.toml for more than one enabled store block. Only one may be enabled at a time.

S3 gateway allows anonymous access. No identities are configured. Run s3.configure as in Step 7. An empty identity list means no authentication is enforced.

Uploads larger than 1 MB fail behind nginx. Body buffering again. client_max_body_size 0 plus proxy_request_buffering off.

Deleted files did not free disk space. Run volume.vacuum -garbageThreshold=0.3 in weed shell.

Volume server uses far more RAM than expected. It is using the in-memory needle index. Switch to -index=leveldb and restart.

Admin UI login rejects the correct password. Confirm no shell metacharacters in the password are being mangled by systemd. Quote the value or use a password limited to alphanumerics for the unit file.

SeaweedFS or Garage?

Both are S3-compatible and both run happily on a small VPS, but they optimize for different things.

• Choose SeaweedFS for very large file counts, POSIX or WebDAV access alongside S3, cloud tiering, or when you know you will scale horizontally.
• Choose Garage for a simpler single-binary deployment, geo-distributed replication across cheap nodes, and lower operational surface area.

SeaweedFS has more moving parts and more knobs. That is the cost of its flexibility.

Next steps

• Enable erasure coding on cold volumes to cut storage overhead
• Configure cloud tiering to push warm data to a remote S3 provider
• Add a second volume server and move to 001 replication
• Set up a WebDAV endpoint with weed webdav for desktop clients
• Point your CMS or application media library at the S3 endpoint