Configuration Reference

Complete configuration guide for S2 Lite

S2 Lite is configured through command-line arguments and environment variables. This guide covers all available configuration options.

Command-Line Arguments

Storage Options

--bucket · string

S3 bucket name for object storage backend. If not specified, uses in-memory storage (unless --local-root is set).

s2 lite --bucket my-s2-bucket

--local-root · string

Local filesystem directory for storage. Conflicts with --bucket.

s2 lite --local-root ./s2-data

Local filesystem mode does not provide the same durability guarantees as object storage.

--path · string · default: ""

Base path prefix within object storage. Allows multiple instances to share a bucket.

s2 lite --bucket my-bucket --path s2lite/prod

Network Options

--port · integer

Port to listen on. Defaults to 443 if TLS is enabled, otherwise 80.

s2 lite --port 8080

--no-cors · boolean · default: "false"

Disable permissive CORS headers. By default, S2 Lite allows browser-based clients from any origin.

s2 lite --no-cors

TLS Options

--tls-self · boolean · default: "false"

Use a self-signed certificate for TLS. Useful for development and testing.

s2 lite --tls-self

Clients will need to use --insecure or configure SSL verification to trust self-signed certificates.

--tls-cert · string

Path to TLS certificate file (e.g., cert.pem). Must be used with --tls-key.

s2 lite --tls-cert /path/to/cert.pem --tls-key /path/to/key.pem

--tls-key · string

Path to TLS private key file (e.g., key.pem). Must be used with --tls-cert.

Initialization Options

--init-file · string

Path to JSON file defining basins and streams to create at startup. Can also be set via S2LITE_INIT_FILE environment variable.

s2 lite --init-file resources.json

See Init File Format below for details.

Environment Variables

S2 Lite Configuration

S2LITE_INIT_FILE · string

Path to initialization file. Alternative to --init-file.

export S2LITE_INIT_FILE=resources.json
s2 lite

S2LITE_PIPELINE · boolean · default: "false"

Enable append pipelining for improved performance against high-latency object storage.

export S2LITE_PIPELINE=true
s2 lite

Pipelining is currently disabled by default for safety. It will be enabled by default in a future release after further testing.

When enabled, S2 Lite pipelines up to 25 MiB of appends.

AWS Configuration

AWS_ACCESS_KEY_ID · string

AWS access key ID for S3 authentication.

export AWS_ACCESS_KEY_ID=AKIAIOSFODNN7EXAMPLE

AWS_SECRET_ACCESS_KEY · string

AWS secret access key for S3 authentication.

export AWS_SECRET_ACCESS_KEY=wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY

AWS_REGION · string

AWS region for S3 bucket.

export AWS_REGION=us-east-1

AWS_ENDPOINT_URL_S3 · string

Custom S3 endpoint URL. Required for S3-compatible services like MinIO, Tigris, or Cloudflare R2.

# Tigris
export AWS_ENDPOINT_URL_S3=https://fly.storage.tigris.dev

# MinIO
export AWS_ENDPOINT_URL_S3=http://minio:9000

# Cloudflare R2
export AWS_ENDPOINT_URL_S3=https://ACCOUNT_ID.r2.cloudflarestorage.com

AWS_PROFILE · string

AWS profile name from ~/.aws/credentials.

export AWS_PROFILE=production

SlateDB Configuration

S2 Lite uses SlateDB as its storage engine. Configure SlateDB using SL8_ prefixed environment variables.

See the SlateDB Settings reference for all available options.

SL8_FLUSH_INTERVAL · duration

Interval at which to flush writes to object storage.

Defaults:

• 50ms for S3 bucket
• 5ms for in-memory or local filesystem

export SL8_FLUSH_INTERVAL=100ms

Lower values reduce latency but increase object storage API calls. Higher values improve throughput but increase latency.

SL8_MANIFEST_POLL_INTERVAL · duration

Interval at which to poll for manifest updates. Used for fencing out previous instances.

export SL8_MANIFEST_POLL_INTERVAL=1s

SL8_L0_COMPACTION_THRESHOLD · integer

Number of L0 SSTables before triggering compaction.

export SL8_L0_COMPACTION_THRESHOLD=8

Logging Configuration

RUST_LOG · string · default: "info"

Logging level configuration. Supports module-level filtering.

# Global info level
export RUST_LOG=info

# Debug level for s2-lite
export RUST_LOG=s2_lite=debug

# Mixed levels
export RUST_LOG=s2_lite=debug,slatedb=info,tower_http=debug

Available levels: error, warn, info, debug, trace

Init File Format

The init file is a JSON document that defines basins and streams to create at startup. It uses create-or-reconfigure semantics, making it safe for repeated restarts.

Schema

schema.json

{
  "basins": [
    {
      "name": "string",
      "config": {
        "create_stream_on_append": "boolean (optional)",
        "create_stream_on_read": "boolean (optional)",
        "default_stream_config": {
          "storage_class": "'standard' | 'express' (optional)",
          "retention_policy": "'infinite' | duration string (optional)",
          "timestamping": {
            "mode": "'client-prefer' | 'client-require' | 'arrival' (optional)",
            "uncapped": "boolean (optional)"
          },
          "delete_on_empty": {
            "min_age": "duration string (optional)"
          }
        }
      },
      "streams": [
        {
          "name": "string",
          "config": {
            "storage_class": "'standard' | 'express' (optional)",
            "retention_policy": "'infinite' | duration string (optional)",
            "timestamping": { /* same as above */ },
            "delete_on_empty": { /* same as above */ }
          }
        }
      ]
    }
  ]
}

Example

resources.json

{
  "basins": [
    {
      "name": "production",
      "config": {
        "create_stream_on_append": true,
        "create_stream_on_read": false,
        "default_stream_config": {
          "storage_class": "standard",
          "retention_policy": "7days",
          "timestamping": {
            "mode": "client-prefer",
            "uncapped": false
          },
          "delete_on_empty": {
            "min_age": "1day"
          }
        }
      },
      "streams": [
        {
          "name": "events",
          "config": {
            "retention_policy": "infinite"
          }
        },
        {
          "name": "logs",
          "config": {
            "retention_policy": "24hours",
            "storage_class": "express"
          }
        },
        {
          "name": "metrics",
          "config": {
            "retention_policy": "30days",
            "timestamping": {
              "mode": "arrival"
            }
          }
        }
      ]
    },
    {
      "name": "ephemeral",
      "config": {
        "create_stream_on_append": true,
        "default_stream_config": {
          "retention_policy": "1hour",
          "delete_on_empty": {
            "min_age": "5m"
          }
        }
      }
    }
  ]
}

Field Reference

Basin Config

create_stream_on_append · boolean

Automatically create streams on append if they don’t exist, using the default stream configuration.

create_stream_on_read · boolean

Automatically create streams on read if they don’t exist, using the default stream configuration.

default_stream_config · object

Default configuration for auto-created streams in this basin.

Stream Config

storage_class · string

Storage class for recent writes.

Options:

• standard - Standard S3 storage
• express - S3 Express One Zone (ultra-low latency)

"storage_class": "express"

retention_policy · string

Retention policy for the stream.

Options:

• infinite - Retain records indefinitely
• Duration string - Auto-trim older records (e.g., 7days, 24hours, 1week)

"retention_policy": "7days"

Duration formats:

• 1day, 7days, 30days
• 1hour, 24hours
• 1week, 2weeks
• 1min, 30mins

timestamping · object

Timestamping behavior for appends.

"timestamping": {
  "mode": "client-prefer",
  "uncapped": false
}

timestamping.mode · string

Timestamping mode.

Options:

• client-prefer - Use client timestamp if provided, otherwise arrival time
• client-require - Require client timestamp (reject if not provided)
• arrival - Always use server arrival time

timestamping.uncapped · boolean

Allow client-specified timestamps to exceed arrival time. If false, client timestamps are capped at arrival time.

delete_on_empty · object

Delete-on-empty configuration.

"delete_on_empty": {
  "min_age": "1day"
}

delete_on_empty.min_age · string

Minimum age before an empty stream can be deleted automatically. Set to 0 (or omit) to disable auto-deletion.

"min_age": "1day"

Performance Tuning

Flush Interval

The most important performance knob is SL8_FLUSH_INTERVAL:

• Lower values (10-50ms): Lower latency, higher API call costs
• Higher values (100-500ms): Higher throughput, lower costs, higher latency

# Low latency
export SL8_FLUSH_INTERVAL=10ms

# High throughput
export SL8_FLUSH_INTERVAL=500ms

Pipelining

Enable pipelining for better performance with high-latency object storage:

export S2LITE_PIPELINE=true

This allows up to 25 MiB of appends to be in-flight simultaneously.

Storage Class

Use S3 Express One Zone for ultra-low latency:

{
  "storage_class": "express"
}

S3 Express requires an Express One Zone bucket (ending in --x-s3).

Example Configurations

Development (In-Memory)

s2 lite --port 8080

Production (S3 with IAM)

export AWS_REGION=us-east-1
export SL8_FLUSH_INTERVAL=50ms
export S2LITE_PIPELINE=true

s2 lite \
  --bucket my-s2-bucket \
  --path production \
  --no-cors \
  --init-file /etc/s2-lite/resources.json

Tigris with Static Credentials

export AWS_ACCESS_KEY_ID=tid_xxx
export AWS_SECRET_ACCESS_KEY=tsec_xxx
export AWS_ENDPOINT_URL_S3=https://fly.storage.tigris.dev
export SL8_FLUSH_INTERVAL=30ms

s2 lite \
  --bucket my-tigris-bucket \
  --port 8080

Local Filesystem

export SL8_FLUSH_INTERVAL=5ms

s2 lite \
  --local-root ./s2-data \
  --port 8080

Next Steps

Deployment

Deploy S2 Lite to production

Monitoring

Set up monitoring and alerts