Introduction

What is Kubeflow

Kubeflow is a community and ecosystem of open-source projects to address each stage in the machine learning (ML) lifecycle with support for best-in-class open source tools and frameworks. Kubeflow makes AI/ML on Kubernetes simple, portable, and scalable.

Whether you’re a researcher, data scientist, ML engineer, or a team of developers, Kubeflow offers modular and scalable tools that cater to all aspects of the ML lifecycle: from building ML models to deploying them to production for AI applications.

What are Standalone Kubeflow Components

The Kubeflow ecosystem is composed of multiple open-source projects that address different aspects of the ML lifecycle. Many of these projects are designed to be usable both within the Kubeflow Platform and independently. These Kubeflow components can be installed standalone on a Kubernetes cluster. It provides flexibility to users who may not require the full Kubeflow Platform capabilities but wish to leverage specific ML functionalities such as model training or model serving.

What is Kubeflow Platform

The Kubeflow Platform refers to the full suite of Kubeflow components bundled together with additional integration and management tools. Using Kubeflow as a platform means deploying a comprehensive ML toolkit for the entire ML lifecycle.

In addition to the standalone Kubeflow components, the Kubeflow Platform includes

• Kubeflow Notebooks for interactive data exploration and model development.
• Central Dashboard for easy navigation and management with Kubeflow Profiles for access control.
• Additional tooling for data management (PVC Viewer), visualization (TensorBoards), and more.

The Kubeflow Platform can be installed via Packaged Distributions or Kubeflow Manifests.

Kubeflow Overview Diagram

The following diagram shows the main Kubeflow components to cover each stage of the ML lifecycle on top of Kubernetes.

Read the architecture overview to learn about the Kubeflow ecosystem and to see how Kubeflow components fit in ML lifecycle.

Kubeflow Video Introduction

Watch the following video which provides an introduction to Kubeflow.

The Kubeflow mission

Our goal is to make scaling machine learning (ML) models and deploying them to production as simple as possible, by letting Kubernetes do what it’s great at:

• Easy, repeatable, portable deployments on a diverse infrastructure (for example, experimenting on a laptop, then moving to an on-premises cluster or to the cloud)
• Deploying and managing loosely-coupled microservices
• Scaling based on demand

Because ML practitioners use a diverse set of tools, one of the key goals is to customize the stack based on user requirements (within reason) and let the system take care of the “boring stuff”. While we have started with a narrow set of technologies, we are working with many different projects to include additional tooling.

Ultimately, we want to have a set of simple manifests that give you an easy to use ML stack anywhere Kubernetes is already running, and that can self configure based on the cluster it deploys into.

History

Kubeflow started as an open sourcing of the way Google ran TensorFlow internally, based on a pipeline called TensorFlow Extended. It began as just a simpler way to run TensorFlow jobs on Kubernetes, but has since expanded to be a multi-architecture, multi-cloud framework for running end-to-end machine learning workflows.

The Kubeflow logo represents the letters K and F inside the heptagon of the Kubernetes logo, which represent two communities: Kubernetes (cloud-native) and flow (Machine Learning). In this context, flow is not only indicating TensorFlow, but also all ML frameworks which make use of Dataflow Graph as the normal form for model/algorithm implementation.

Roadmaps

To see what’s coming up in future versions of Kubeflow, refer to the Kubeflow roadmap.

The following components also have roadmaps:

• Kubeflow Pipelines
• KServe
• Katib
• Training Operator

Getting involved

There are many ways to contribute to Kubeflow, and we welcome contributions!

Read the contributor’s guide to get started on the code, and learn about the community on the community page.

Next Steps

• Follow the installation guide to deploy standalone Kubeflow components or Kubeflow Platform.