Kubeflow

MPI environment validation in Kubeflow Trainer and fixes to the Python SDK

Open source · Contributor · May 2026 — Present

Overview

Kubeflow Trainer runs distributed training and LLM fine-tuning on Kubernetes, and the Kubeflow SDK is the Python SDK for running AI workloads there. I have one merged pull request in Trainer and two open in the SDK.

Contributions

feat(runtimes): add validation for reserved MPI environment variables (merged May 2026)
fix(spark): use .svc short form for Spark Connect URL (open)
fix(trainer): use resource key as device for custom Kubernetes resources (open)

Details

The Trainer change rejects TrainJob environment entries that collide with the four OpenMPI variables the operator manages, mirroring the existing Torch check. A maintainer asked me to pick it up after an earlier pull request went stale, so I rebased it and resolved the conflicts.

In the SDK, one fix stops job reads from failing when a container requests a custom device plugin resource, and the other switches in-cluster Spark Connect URLs to the shorter .svc form.

All Kubeflow PRs