Choose an Object Storage Provider
Overview
Ilum runs entirely on S3-compatible object storage. The bundled
helm_aio chart supports two providers natively (RustFS and
MinIO) and integrates with any external S3 service that the cluster
can reach. Two further providers (Garage and SeaweedFS) are on
the roadmap and already accommodated by the provider registry. This
page documents the trade-offs and helps an operator decide which path
to take.
Comparison matrix
The matrix combines qualitative properties with quantitative numbers from the Ilum parity test suite (see Phase 8 parity report).
The quantitative columns (compatibility score, throughput, multipart,
predicate-pushdown reads) are populated from the Ilum parity report,
which is regenerated when the parity matrix is re-run against the
current release. Numbers shown as [pending Task #77] indicate values
that will be filled in once the parity matrix has been executed against
a release-candidate cluster.
| Property | RustFS | MinIO | AWS S3 | Wasabi | Backblaze B2 | Garage (planned) | SeaweedFS (planned) |
|---|---|---|---|---|---|---|---|
| License | Apache-2.0 | AGPL-3.0 (server) | Proprietary (AWS) | Proprietary (Wasabi) | Proprietary (Backblaze) | Apache-2.0 / AGPL-3.0 | Apache-2.0 |
| Status in Ilum | Default since 6.7.2-RC2 | Supported (opt-in) | BYO | BYO | BYO | Planned, registry-ready | Planned, registry-ready |
| Bundled chart | Yes (RustFS 0.0.99) | Yes (Bitnami 15.0.7) | No | No | No | No | No |
| Distributed mode | Standalone (alpha-safe default) | Native | Managed (AWS) | Managed (Wasabi) | Managed (Backblaze) | Native (planned) | Native (planned) |
| Console UX | Embedded same-origin iframe | Embedded /external/minio/ proxy | Provider-hosted (no embed) | Provider-hosted (no embed) | Provider-hosted (no embed) | Planned | Planned |
| OIDC integration | Operator-supplied via extraEnv | Hydra-preconfigured | Managed by AWS IAM | Wasabi IAM | Backblaze application keys | Not yet evaluated | Not yet evaluated |
| S3 surface compatibility (run-A) | [pending Task #77] | [pending Task #77] | n/a (reference) | n/a (BYO) | n/a (BYO) | n/a (planned) | n/a (planned) |
| Hadoop S3A throughput (run-B) | [pending Task #77] | [pending Task #77] | n/a (BYO) | n/a (BYO) | n/a (BYO) | n/a (planned) | n/a (planned) |
| Multipart upload (>100 MiB, run-B5) | [pending Task #77] | [pending Task #77] | Native | Native | Native | n/a (planned) | n/a (planned) |
| Iceberg + Nessie write/read (run-C) | [pending Task #77] | [pending Task #77] | Supported | Supported | Supported | n/a (planned) | n/a (planned) |
| Delta Lake write/optimize/vacuum (run-D) | [pending Task #77] | [pending Task #77] | Supported | Supported | Supported | n/a (planned) | n/a (planned) |
| Consumer smoke (MLflow, Airflow, Kestra, run-G) | [pending Task #77] | [pending Task #77] | Operator-tested | Operator-tested | Operator-tested | n/a (planned) | n/a (planned) |
Parity report source
The bundled parity test harness lives at helm/test/parity/ in the
Ilum monorepo. The run-all.sh orchestrator executes all sections
(A: S3 API surface; B: Hadoop S3A; C: Iceberg on Nessie; D: Delta Lake;
E: mc admin; F: smoke perf; G: bundled-consumer smokes; H: iframe
smoke) and writes the resulting fingerprints to
helm/test/parity/results/parity-summary.md. The numbers in the matrix
above are taken verbatim from that file. Operators that want to confirm
the numbers for their own cluster can re-run the matrix against any
kind or minikube install.
Decision tree
Three short branches cover the common deployment shapes.
"I want zero-ops bundled storage"
Use the default. RustFS is enabled out of the box in 6.7.2-RC2 and later.
helm install ilum ilum/helm_aio
If the RustFS alpha status is a concern for production, enable MinIO instead:
helm install ilum ilum/helm_aio \
--set rustfs.enabled=false \
--set minio.enabled=true
Either choice creates the ilum-objectstorage Service alias and wires
every bundled consumer through it.
"I already operate an S3-compatible backend"
Disable both bundled providers and point Ilum at the existing service.
helm install ilum ilum/helm_aio \
--set rustfs.enabled=false \
--set minio.enabled=false \
--set objectStorage.endpoint=https://s3.us-east-1.amazonaws.com
For provider-specific configuration recipes, refer to Provider Reference: External S3.
"I want to benchmark providers side by side"
Run two providers concurrently and route user traffic through the alias.
Set objectStorage.activeProvider explicitly when more than one provider
is enabled. The alias selector flips on a single helm upgrade --set.
helm install ilum ilum/helm_aio \
--set minio.enabled=true \
--set rustfs.enabled=true \
--set objectStorage.activeProvider=rustfs
To re-run the parity matrix against the running cluster:
bash helm/test/parity/run-all.sh all
Reversibility
Switching providers is a single helm upgrade --set away. The
ilum-objectstorage Service alias selector flips atomically; every
bundled consumer picks up the new endpoint on its next connection.
The data side is not reversible without a migration. Refer to Migrate Between Providers for the mirror procedure that runs before the alias flip.
Constraints to be aware of
- RustFS alpha distributed mode. The bundled chart configures standalone mode. Erasure coding and multi-replica distribution are available upstream but marked "under testing".
- MinIO console pin. The MinIO console binds itself to the
absolute URL configured by
MINIO_BROWSER_REDIRECT_URL. The bundled default pins it to/external/minio/. The Ilum UI's/external/object-storage/alias path redirects to that absolute URL when MinIO is the active provider. - No simultaneous writes across providers. The cutover model assumes a single active provider at any given time. Operators that want multi-region replication should configure it at the provider layer, not by enabling two Ilum providers concurrently.
Reference
- Provider reference: MinIO · RustFS · External S3 · Garage (planned) · SeaweedFS (planned)
- Migration: Migrate Between Providers
- Troubleshooting: Troubleshoot Object Storage
- Helm values: Object Storage Helm Values