Skip to main content

Add a New Object Storage Provider

Overview​

The objectStorage.providers registry in helm_aio accepts new providers without any chart-template change. Adding a backend that is not (yet) bundled is therefore a values-file edit. This page documents the three things a new provider needs and walks through the procedure for adding one to a running install.

The same procedure works for any S3-compatible backend, including the currently-planned Garage and SeaweedFS ahead of their official bundled support.

Use cases for this procedure
  • Bringing Garage, SeaweedFS, Ceph RGW, or another bundled-candidate provider in early.
  • Running a multi-provider benchmark on a single cluster, with objectStorage.activeProvider picking which one user traffic targets.
  • Migrating to a provider that Ilum does not yet bundle, ahead of native chart support.

Three things a new provider needs​

The ilum-objectstorage Service alias is a label-selector Service. For its endpoints to resolve, the new provider must satisfy three requirements:

  1. A reachable Service in the release namespace. The provider can either expose its own Service (preferred) or piggyback on the alias. The provider's own Service makes direct probing easier during testing.
  2. Pods carrying two labels. Every pod backing the new provider must carry app.kubernetes.io/name: <provider> and app.kubernetes.io/instance: <release>. The alias Service selector matches both labels.
  3. A registry entry under objectStorage.providers.<name>. The entry declares the provider's iframe path and console routing mode for the Ilum UI.

Step-by-step recipe​

The example below adds SeaweedFS to an existing install. The same shape applies to any provider.

Step 1. Deploy the provider's chart in the release namespace​

Install the provider's own Helm chart, or run a hand-rolled Deployment plus Service. The Deployment must label its pods with both app.kubernetes.io/name and app.kubernetes.io/instance:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: seaweedfs
  namespace: ilum
  labels:
    app.kubernetes.io/name: seaweedfs
    app.kubernetes.io/instance: ilum
spec:
  replicas: 1
  selector:
    matchLabels:
      app.kubernetes.io/name: seaweedfs
      app.kubernetes.io/instance: ilum
  template:
    metadata:
      labels:
        app.kubernetes.io/name: seaweedfs
        app.kubernetes.io/instance: ilum
    spec:
      containers:
        - name: server
          image: chrislusf/seaweedfs:latest
          args: ["server", "-s3"]
          ports:
            - {containerPort: 9000, name: s3-api}
            - {containerPort: 9001, name: s3-console}

The pod IP is what the ilum-objectstorage Service alias will route to once the registry is updated.

Step 2. Register the provider in the chart values​

Add the entry under objectStorage.providers.<name> and set the activeProvider override so the alias targets the new provider:

helm upgrade ilum ilum/helm_aio \
  --reuse-values \
  --set objectStorage.providers.seaweedfs.enabled=true \
  --set objectStorage.providers.seaweedfs.consolePath=/external/seaweed/ \
  --set objectStorage.providers.seaweedfs.consoleMode=nginx-rewrite \
  --set objectStorage.activeProvider=seaweedfs

Step 3. Verify the alias selector​

The alias annotation should reflect the new active provider, and its endpoints should resolve to the new provider's pod IP:

kubectl -n ilum get svc ilum-objectstorage -o jsonpath='{.metadata.annotations.ilum\.cloud/object-storage-active-provider}{"\n"}'
# Expected: seaweedfs

kubectl -n ilum get endpoints ilum-objectstorage
# Expected ENDPOINTS: <pod-ip>:9001,<pod-ip>:9000

Step 4. Verify the UI iframe path​

The ilum-ui ConfigMap should expose the new console path:

kubectl -n ilum get configmap ilum-ui -o jsonpath='{.data.ILUM_OBJECT_STORAGE_PATH}{"\n"}'
# Expected: /external/seaweed/

Hard-reload the Ilum UI in the browser. The Object Storage nav button should load the new provider's console inside the iframe.

consoleMode: same-origin vs nginx-rewrite​

The registry's consoleMode field controls how the Ilum UI's nginx reverse proxy renders /external/object-storage/ and the provider-specific path.

ModeWhen to use
same-originThe upstream console respects a configurable base path that matches consolePath (for example, a Next.js basePath baked into the build). The nginx proxy passes requests through unchanged. RustFS uses this mode after the Ilum patched-console image rebuilds with NEXT_PUBLIC_BASE_PATH=/rustfs/console.
nginx-rewriteThe upstream console pins itself to a single absolute URL via an internal redirect (such as MinIO's MINIO_BROWSER_REDIRECT_URL). The nginx proxy rewrites /external/<name>/ to / before forwarding so the upstream console responds, and redirects /external/object-storage/ to the provider-specific consolePath to avoid loops.

When unsure, start with nginx-rewrite. It is the safer default for consoles that have not been built with Ilum-aware base-path awareness.

What still needs hand-wiring​

The registry handles alias selection, the UI iframe path, and the /external/object-storage/ redirect. The following items remain the operator's responsibility for a non-bundled provider:

  • Bucket bootstrap. Run the equivalent of the bundled init-rustfs-buckets or init-minio-policies Job against the new provider. The bucket list is documented under objectStorage.defaultBuckets.
  • Credentials. Either share the ilum-objectstorage-credentials Secret (the simplest option) or override the provider's chart values to read from a provider-specific Secret.
  • OIDC integration. The bundled Hydra integration plugs into MinIO through MINIO_IDENTITY_OPENID_* env vars. New providers must supply their own OIDC adapter.
  • Bucket policies. The bundled chart applies two policies (readilum, readwriteilum) to MinIO and RustFS on first install. Other providers need an equivalent.
  • Console UX testing. The Ilum UI iframe wraps the provider's own console. Cross-origin asset URLs, auto-redirects, and form submissions vary across providers. Test the embedded experience before flipping the active provider in production.

Reference​

Last updated on