Current Tools¶

Kubeflow Pipelines¶

KFP provides higher-level abstractions for Argo Workflows to reduce repetition when defining machine learning tasks. KFP has abstractions for defining pipelines and reusable components which it can compile and execute as Argo Workflows.

The primary interface of KFP is the Python SDK, which allows you to define pipelines and reusable components with Python. KFP also provides a Web UI for managing and tracking experiments, pipeline definitions, and pipeline runs. Finally, KFP provides a REST API that allows programmatic access to the platform.

Kubeflow Notebooks¶

Kubeflow Notebooks can run any web-based tool, but comes with pre-built images for JupyterLab, RStudio, and Visual Studio Code.

Running development environments inside a Kubernetes cluster has several advantages:

• Remote Resources: Users can work directly on the cluster, rather than locally on their workstations.
• Standard Environments: Cluster admins can provide standard environment images for their organization, with required and approved packages pre-installed.
• Sharing & Access Control: Access is managed via role-based-access-control (RBAC), enabling easier notebook sharing and collaboration across the organization.

Kubeflow Katib¶

The key features of Katib are:

• Support for Multiple Techniques: Katib supports techniques like Hyperparameter Tuning, Early Stopping, and Neural Architecture Search.
• Support for ML Frameworks: Katib natively supports many ML frameworks like TensorFlow, PyTorch, XGBoost, and more.
• Kubernetes Native: Katib can manage training jobs on any Kubernetes Resource, and has out-of-the-box support for Kubeflow Training Operator, Argo Workflows, Tekton Pipelines, and more.

Kubeflow Training Operator¶

The core function of the Kubeflow Training Operator is to provide Kubernetes Custom Resources (CRDs) that define and monitor training jobs on Kubernetes.

Many popular ML frameworks have been integrated with the Training Operator, including:

• PyTorch
• TensorFlow
• XGBoost
• MPI

Kubeflow Volumes¶

Kubeflow TensorBoards¶