Python DocsR DocsAdmin Docs
Search…
Python Docs
Welcome to Metaflow for Python
Introduction
Why Metaflow
What is Metaflow
Release Notes
Roadmap
Contributing to Metaflow
Get in Touch
Getting Started
Installing Metaflow
Tutorials
Metaflow on AWS
Metaflow on AWS
Metaflow Sandbox
Deploying to AWS
Developing with Metaflow
Basics of Metaflow
Inspecting Flows and Results
Debugging with Metaflow
Scaling Out and Up
Loading and Storing Data
Managing External Libraries
Dealing with Failures
Organizing Results
Going to Production with Metaflow
Scheduling Metaflow Flows
Coordinating Larger Metaflow Projects
Internals of Metaflow
Technical Overview
Testing Philosophy
Powered By GitBook
Roadmap
Metaflow has been used in production at Netflix since early 2018. The core Metaflow was open-sourced in December 2019. Some features of Metaflow are not available in open-source yet but we may open-source them later if there is sufficient external interest. These features are listed below.
Please click the link and comment / thumbs-up the corresponding GitHub issue if you want to see the feature open-sourced.

Support for Kubernetes

Bring all of Metaflow's capabilities to the Kubernetes universe (Github issue)

Metaflow UI(s)

A variety of UI(s) for Metaflow - tracking flows, model monitoring, etc

Faster and more flexible dependency management solutions

Support for dependency management tools beyond conda and docker and address existing pain points (Github issue)

Flow Composition

Support composing Metaflow flow from other flows (Github issue)

MetaflowBot

A Slack bot for Metaflow. Use it to ask questions about past runs (Github issue)
Update - Metaflowbot is now available in Open Source!

Metaflow DataFrame

Support in-memory processing of large data sets (Github issue)​

More tutorials and recipes

Provide advanced tutorials and documentation highlighting non-trivial use-cases (Github issue)​

Support for hosting models as a micro-service

An easy-to-use Function-as-a-Service -style microservice hosting platform for artifacts (e.g. models) produced by Metaflow runs (Github issue)

Support for R Lang

​Metaflow in the R language. Provide an idiomatic R API which uses the Python library as the backend (Github issue)
Update - Metaflow-R is now available!

Support deployments to a production DAG scheduler

Netflix uses an internal DAG scheduler to orchestrate most modeling and ETL pipelines in production. Metaflow flows can be deployed to the production scheduler with a single command. A similar integration could be provided e.g. for AWS Step Functions (Github issue)
Update - Metaflow 2.1.0 introduced integration with AWS Step Functions.​
Introduction - Previous
Release Notes
Next - Introduction
Contributing to Metaflow
Last modified 2mo ago
Export as PDF
Copy link
Contents
Support for Kubernetes
Metaflow UI(s)
Faster and more flexible dependency management solutions
Flow Composition
MetaflowBot
Metaflow DataFrame
More tutorials and recipes
Support for hosting models as a micro-service
Support for R Lang
Support deployments to a production DAG scheduler