Skip to main content

Monitoring

Ilum makes it easier to monitor your spark workloads in order to achieve better performance and tracking down problems.

Spark UI​

Every Spark job provides a web interface that you can use to monitor the status and resource consumption of your Spark job. This UI is accessible from Ilum's UI level on job details page.

Spark History Server​

To be able to access the UIs of completed Spark jobs, the ilum package includes a Spark History Server. The history server uses the default kubernetes storage configured with the ilum-core.kubernetes.* helm parameters to store spark events logs. It is very important to provide access to this storage for each Spark job cluster that Ilum uses. The Spark history server is by default bundled in ilum package.

You can disable spark history server using a helm upgrade command. For instance:

helm upgrade \
    --set ilum-core.historyServer.enabled=false \
    --reuse-values ilum ilum/ilum

You can access Spark History Server from Ilum UI or using the port-forward command:

kubectl port-forward svc/ilum-history-server 9666:9666

Prometheus​

One of more sophisticated and detailed ways to monitor spark workloads is to use prometheus. Please be aware, that Kube Prometheus Stack is not bundled in ilum package by default.

You can enable kube prometheus stack using a helm upgrade command. For instance:

helm upgrade \
    --set kube-prometheus-stack.enabled=true \
    --set ilum-core.job.prometheus.enabled=true \
    --reuse-values ilum ilum/ilum

With this configuration prometheus will scrape metrics from all Spark jobs run by Ilum. Kube Prometheus Stack includes Grafana, open source analytics and interactive visualization tool, which use metrics scraped by prometheus to feed dashboards with data that can give better overview of your spark jobs. Ilum provides default dashboards to monitor spark jobs and ilum pods. They can be found in Ilum folder of grafana's dashboards.

You can access Prometheus UI using the port-forward command:

kubectl port-forward svc/prometheus-operated 9090:9090

You can access Grafana UI using the port-forward command:

kubectl port-forward svc/ilum-grafana 8080:80