Getting Started with your own
Data Lakehouse
Quick and Easy
Generate License
Kickstart your journey with Ilum's user-friendly and completely free licensing system. Our intuitive setup ensures that you can generate and manage your license without any technical hurdles. A step-by-step guide makes the process easy, saving you time and effort so you can focus on what matters most.
Start for Free!Download
Ilum is a modular platform, designed to adapt to your specific needs. The default version installs without any additional parameters and already gives you a complete lakehouse: Apache Spark, Jupyter, a SQL Editor with Spark SQL and DuckDB, Hive Metastore and DuckLake catalogs, and data lineage. You can enhance it by enabling specific modules during installation, or later from the Modules page in the Ilum UI.
To make this process easier, we've provided a simple tool below (Option 3) to help you select the modules you want to include and generate the matching Helm command.
Please keep in mind that adding additional modules will increase the required resources, such as RAM, CPU and storage. 12GB of RAM is enough for the Minimal version and 18GB for the default setup. Every extra module adds to that.
To setup Ilum within a Kubernetes cluster, you must use Helm.
Check our ArtifactHUB, blog and documentation for detailed instructions.
Haven't got a Kubernetes cluster? Look into Minikube or Microk8s!
Kubernetes
Ilum supports all the k8s distribution or managed cloud solutions, you can test it locally on Linux, macOs or even Windows. Our containers are built to support both amd64 and arm architectures.
Below is an example with minikube which is great for testing, but the same thing will work with K3s, GKE, AKS, EKS
minikube start --cpus 12 --memory 18192 --addons metrics-server
Option 1. Default version (Recommended)
Spark, Jupyter, SQL Editor with Spark SQL and DuckDB, Hive Metastore and DuckLake catalogs, Data Lineage and MinIO object storage, all with a single command.
helm repo add ilum https://charts.ilum.cloud
helm repo update
helm install ilum ilum/ilum
Option 2. Ilum CLI
The CLI checks prerequisites, finds your cluster (or creates a local minikube) and installs Ilum in one step. Add modules later with ilum module enable.
curl -fsSL https://get.ilum.cloud/cli | bash
ilum quickstart
All necessary services should launch within a few minutes.
Access UI
Ilum is installed with a user-friendly UI that can be accessed through service or port forwarding. Use admin/admin as default credentials. You can change them during the deployment process or in user management section on the UI.
kubectl port-forward svc/ilum-ui 9777:9777
Visit http://localhost:9777 to reach ilum-ui or use this command: minikube service ilum-ui
Option 3. Choose Version:
Minimal
Default
Data Engineer
Data Analyst
Machine Learning
Custom
Base
Kubernetes
Container orchestration platform
SQL
SQL gateway for the SQL Editor and JDBC/ODBC clients
Yarn
Resource manager for Hadoop clusters
Engine
Apache Spark
Unified analytics engine for large-scale data processing
DuckDB
In-process OLAP engine built into the SQL Editor
Trino
Distributed SQL query engine for big data
Apache Flink (Enterprise Beta)
Low-latency stream processing engine
Apache DataFusion (Coming soon)
Extensible query execution framework
Dask (Coming soon)
Library for parallel computing in Python
Catalog
Hive Metastore
Default table catalog for Spark, Trino and SQL
DuckLake
DuckDB-native lakehouse catalog on object storage
Nessie
Versioned data lakehouse control system
Unity Catalog
Open catalog with unified governance for data and AI
Polaris (Coming soon)
Open-source catalog for Apache Iceberg
Storage
MinIO
S3-compatible object store
RustFS
High-performance S3-compatible object store
Ceph
Distributed storage system, deployed with Rook
SeaweedFS (Coming soon)
Scalable distributed file system
JuiceFS (Coming soon)
Distributed POSIX file system
S3
Scalable object storage service
GCS
Cloud-based object storage service
Azure Blob Storage
Scalable cloud object storage
HDFS
Distributed file system for big data
Data transformation and workflows
Apache Airflow
Data engineering pipelines
n8n
Workflow automation platform with native AI capabilities.
Kestra
Open Source Declarative Orchestration Platform.
Apache NiFi
Data integration and workflow automation
dbt
Data transformation and modeling tool
'/%3e%3cpath%20opacity='0.4'%20d='M258.487%207.45312H325.635L201.225%20245.72H134.077L258.487%207.45312Z'%20fill='url(%23paint1_linear_789_14)'/%3e%3cpath%20d='M258.507%207.45312H325.655V245.72H258.507V7.45312Z'%20fill='url(%23paint2_linear_789_14)'/%3e%3cdefs%3e%3clinearGradient%20id='paint0_linear_789_14'%20x1='-6.60498'%20y1='7.45307'%20x2='286.799'%20y2='36.3535'%20gradientUnits='userSpaceOnUse'%3e%3cstop%20stop-color='%23FFCC19'/%3e%3cstop%20offset='0.585938'%20stop-color='%232ECDF7'/%3e%3cstop%20offset='1'%20stop-color='%239E7BFF'/%3e%3c/linearGradient%3e%3clinearGradient%20id='paint1_linear_789_14'%20x1='124.203'%20y1='7.45307'%20x2='303.17'%20y2='18.1404'%20gradientUnits='userSpaceOnUse'%3e%3cstop%20stop-color='%23FFCC19'/%3e%3cstop%20offset='0.585938'%20stop-color='%232ECDF7'/%3e%3cstop%20offset='1'%20stop-color='%239E7BFF'/%3e%3c/linearGradient%3e%3clinearGradient%20id='paint2_linear_789_14'%20x1='-6.59917'%20y1='7.45302'%20x2='286.804'%20y2='36.3535'%20gradientUnits='userSpaceOnUse'%3e%3cstop%20stop-color='%23FFCC19'/%3e%3cstop%20offset='0.585938'%20stop-color='%232ECDF7'/%3e%3cstop%20offset='1'%20stop-color='%239E7BFF'/%3e%3c/linearGradient%3e%3c/defs%3e%3c/svg%3e)
Mage
Framework for transforming and integrating data
Dagster (Coming soon)
Modern orchestration platform
Data Governance
OpenLineage
Data lineage collection for analysis
OpenMetadata
Data discovery, governance and data catalog
Apache Atlas (Enterprise)
Data governance and metadata management
DataHub (Coming soon)
Metadata platform for data discovery, cataloging, and governance.
Notebooks
Jupyter
Interactive computing
Apache Zeppelin
Web-based notebook
JupyterHub (Enterprise)
A multi-user version of the jupyter
Data visualisation
Apache Superset
Data exploration and visualization platform
Tableau
A powerful, secure, and flexible analytics platform.
Power BI
A business analytics tool
Looker
A modern data platform.
QlikSense
An analytics platform.
Metabase (Coming soon)
An open-source analytics platform.
Redash (Coming soon)
A lightweight platform for querying and visualizing data.
ML & AI
MLflow
ML and GenAI made simple
LangFuse
Tracing, evaluation and prompt management for LLM apps
Streamlit
Turn data scripts into shareable web apps
Kubeflow (Coming soon)
Machine learning workflow orchestration platform
OLAP
ClickHouse
A fast open-source OLAP database management system
Apache Doris (Coming soon)
MPP database for real-time analytics.
Apache Pinot (Coming soon)
A real-time distributed OLAP datastore.
Apache Druid (Coming soon)
A high-performance real-time analytics database.
Security
Apache Ranger (Enterprise)
Monitor and manage data security across the platform
Integration
Starburst (Coming soon)
SQL query engine for data analytics
Databricks (Coming soon)
Unified analytics platform for big data
Observability
Kube Prometheus Stack
Complete monitoring solution including Prometheus and Grafana
Loki and Promtail
Manage your logs using Loki with Promtail agent
OpenTelemetry Tracing
Pipeline traces and per-job cost with OpenTelemetry and Tempo
Installation
Run your first job in just 2 minutes!
helm repo add ilum https://charts.ilum.cloud
helm repo update
helm install ilum ilum/ilum
All necessary services should launch within a few minutes.
Have some ideas how to improve Ilum?
We value your feedback and are always eager to improve. If you have suggestions that could enhance your experience or make navigating the product easier, please let us know. Join us in shaping the future of our platform! Your input matters.
Add feature request