Skip to main content

Effortless Task Inspection with Default Cards

Metaflow comes with a built-in Default Card that shows all artifacts produced by a task. Let’s create a simple flow to test it.

from metaflow import FlowSpec, Parameter, step, card
from datetime import datetime

class DefaultCardFlow(FlowSpec):

    alpha = Parameter('alpha', default=0.5)

    @card
    @step
    def start(self):
        self.example_dict = {'first_key': list(range(10)),
                             'second_key': {'one', 'two'}}
        self.timestamp = datetime.utcnow()
        self.next(self.end)

    @step
    def end(self):
        pass

if __name__ == "__main__":
    DefaultCardFlow()

The only new feature introduced in this flow is the @card decorator which attaches a card to the start step. Since no arguments are given to the decorator, the Default Card is used. Save the example in defaultcard.py and execute the flow as usual:

python defaultcard.py run

After the run has finished, you can open a generated card on the command line:

python defaultcard.py card view start

The command will open the card in your local web browser. It will look like this:

The Default Card shows basic metadata about the task, parameters given for the flow, artifacts accessible in the task, as well as a visualization of the flow DAG. You can use this information to quickly observe and verify results of a task without making any changes in the code.

Visualizing Artifacts with the Default Card​

As shown in the screenshot above, the artifacts table shows all Metaflow artifacts related to the task. Large artifacts are truncated for display - you can access the originals using the Client API.

If an artifact contains an image or a dataframe, the artifact is visualized in a separate section in addition to its string representation. Take a look at the following example which contains an artifact, self.image storing an animated GIF and another artifact, a Pandas dataframe:

from metaflow import FlowSpec, Parameter, step
import requests, pandas, string

URL = "https://upload.wikimedia.org/wikipedia/commons/4/45/Blue_Marble_rotating.gif"

class FancyDefaultCardFlow(FlowSpec):

    image_url = Parameter('image_url', default=URL)

    @step
    def start(self):
        self.image = requests.get(self.image_url,
                                  headers={'user-agent': 'metaflow-example'}).content
        self.dataframe = pandas.DataFrame({'lowercase': list(string.ascii_lowercase),
                                           'uppercase': list(string.ascii_uppercase)})
        self.next(self.end)

    @step
    def end(self):
        pass

if __name__ == "__main__":
    FancyDefaultCardFlow()

To demonstrate how cards can be attached to runs on the fly, this example doesn’t include the @card decorator in the code. Save the code to fancydefaultcard.py and run it as follows:

python fancydefaultcard.py run --with card

Note that the example expects that you have the requests and pandas libraries installed. The --with card option attaches a @card decorator to every step without changes in the code. You can execute any existing flow --with card to inspect its results visually.

You can open the card as before:

python fancydefaultcard.py card view start

You will see additional sections in the card which visualize dataframe as a table and show the image stored in the image artifact.

Thanks to this feature, you can use any plotting library such as Matplotlib to create arbitrary visualizations in a Metaflow task, which are then shown in the Default Card automatically without you having to write a line of additional code. You can use this feature during development to quickly debug flows.

Cards Are Stored And Versioned Automatically​

A major benefit of @card is that reports produced by it are versioned and stored in the Metaflow datastore automatically, alongside their parent task. This way, you or your colleagues can easily access any historical card, e.g. a model scorecard associated with a particular version of the model.

You can access any historical card on the command line by using a run ID of a past run. For instance

python fancydefaultcard.py card view 1638257165470922/start

In the case of foreach, a single step can produce multiple tasks and cards. You can view an individual card by giving a full task ID (aka pathspec) corresponding to a task:

python fancydefaultcard.py card view 1638257165470922/start/1

You can see all available cards in the latest run with the “card list” command:

python fancydefaultcard.py card list

It is possible to produce multiple separate cards from a single task by adding multiple @card decorators in a step, which are all shown by “card list”. To make it easier to identify specific cards, you can also assign them a unique ID, as described in Multiple Cards In a Step.

Accessing Cards via an API​

Besides the command line interface, you can access and view cards programmatically through an API. This is particularly convenient, if you want to access cards in a Jupyter notebook.

Given a Task ID (a pathspec), or a Task object from the Client API, the get_cards function lists all cards of the task. You can try this in a notebook cell. Replace the Task ID with an actual ID from a previous run:

from metaflow.cards import get_cards
get_cards('CountryModelFlow/1641937201798104/train_country_model/2')

This will show the card in the output cell:

The get_cards function works well in conjunction with the Client API. For instance, you can use the Client API to search for a task with a specific artifact and view its card:

run = Run('CountryModelFlow/1641937201798104')
[brazil] = [task for task in run['train_country_model']
            if task['country'].data == 'Brazil']
get_cards(brazil)

Sharing Cards​

Since cards are self-contained HTML files, they can be easily shared and viewed by anyone without having to install additional software. To share a card, first save the desired card to a file:

python fancydefaultcard.py card get start mycard.html

Use the “card get” command to save the HTML without opening it in a browser. You can attach the resulting card file, here mycard.html, say, in an email or a Slack message. If you want to share reports automatically e.g. via email, you can use the get_cards API discussed above to obtain the HTML programmatically.

Some recipients may prefer a PDF file over HTML. In this case, you can simply choose “Print” in your browser followed by “Save as PDF”.