Skip to main content

External S3 (BYO)

Overview​

Ilum does not require a bundled object storage sub-chart. Any S3-compatible backend reachable from the cluster can serve as the data plane, including AWS S3, Wasabi, Backblaze B2, on-prem MinIO, Google Cloud Storage via the S3 interoperability layer, or any self-hosted S3 implementation.

This page explains the configuration shape. The integration is operator-driven: Ilum does not provision the buckets, manage the credentials, or expose a console for an external provider.

PropertyValue
LicenseProvider-dependent
Status in IlumSupported
Bundled chartNone
Bundled imageNone
Default in helm_aioDisabled (both rustfs.enabled and minio.enabled set to false)
Distributed modeProvider-dependent
Console UXProvider-hosted; not embedded in the Ilum UI
OIDC integrationProvider-dependent

When to use an external S3 backend​

  • Production deployments where object storage operations should be delegated to a managed service.
  • Multi-cluster setups where multiple Ilum releases share a single backing bucket layout.
  • Air-gapped or compliance-bound environments that already operate a central S3-compatible service.
  • Cost or scale profiles that exceed what the bundled providers offer in their current default configurations.

Configure Ilum against an external S3 backend​

Disable both bundled providers and supply the endpoint plus credentials through the chart's existing values:

helm upgrade ilum ilum/helm_aio \
  --set rustfs.enabled=false \
  --set minio.enabled=false \
  --set objectStorage.endpoint=https://s3.us-east-1.amazonaws.com \
  --set objectStorage.credentials.accessKey=<aws-access-key-id> \
  --set objectStorage.credentials.secretKey=<aws-secret-access-key>

The shared ilum-objectstorage-credentials Secret carries the credentials; every bundled consumer reads from it via the env keys documented in Rotate Object Storage Credentials.

When both rustfs.enabled and minio.enabled are false, the chart does not render the ilum-objectstorage Service alias. Consumers resolve the configured endpoint hostname directly.

Region-specific examples​

AWS S3​

objectStorage:
  endpoint: https://s3.us-east-1.amazonaws.com
  region: us-east-1
  pathStyle: false
  credentials:
    existingSecret: ilum-objectstorage-credentials

Wasabi (EU)​

objectStorage:
  endpoint: https://s3.eu-central-1.wasabisys.com
  region: eu-central-1
  pathStyle: true
  credentials:
    existingSecret: ilum-objectstorage-credentials

Backblaze B2​

objectStorage:
  endpoint: https://s3.us-west-002.backblazeb2.com
  region: us-west-002
  pathStyle: false
  credentials:
    existingSecret: ilum-objectstorage-credentials

Google Cloud Storage (S3 interop)​

objectStorage:
  endpoint: https://storage.googleapis.com
  region: auto
  pathStyle: false
  credentials:
    existingSecret: ilum-objectstorage-credentials

The S3 interop credentials are HMAC keys. Provision them with gcloud against a service account that has the Storage HMAC Key Admin role:

# Grant the role on the project (skip if the SA already has it).
gcloud projects add-iam-policy-binding "$PROJECT_ID" \
  --member="serviceAccount:$SA_EMAIL" \
  --role="roles/storage.hmacKeyAdmin"

# Create the HMAC key. Capture access_id and secret from the output.
gcloud storage hmac create "$SA_EMAIL"

Both the accessId and the secret from the output map onto the access-key and secret-key entries in ilum-objectstorage-credentials. The S3-compatible endpoint for GCS is https://storage.googleapis.com regardless of bucket location.

Bucket bootstrap​

The bundled init Jobs (init-rustfs-buckets, init-minio-policies) do not run against an external backend. The operator must create the default buckets manually before installing Ilum:

ilum-files
ilum-data
ilum-tables
ilum-mlflow
ilum-kestra
ilum-ducklake
ilum-langfuse

The bucket list is configurable via objectStorage.defaultBuckets. Bundled consumers expect bucket names from that list.

Known limitations​

  • No embedded console. The Object Storage view in the Ilum UI does not render for external backends. Operators access the provider's own console through the provider's domain.
  • No bucket lifecycle automation. The bundled migration Job (migration.minioToRustfs) runs against the in-cluster providers only. Migrations involving an external backend should be performed manually with mc or the provider's own tooling. See Migrate Between Providers for the procedure.
  • OIDC and IAM-mode credentials. The chart sources S3 credentials from the shared Secret. AWS IAM-role-based authentication (IRSA, instance profiles) requires a sub-chart override that bypasses the Secret-based credential flow.

Reference​

Last updated on