Adding tags and metadata to assets

Assets feature prominently in the Dagster UI. It is often helpful to attach information to assets to understand where they are stored, what they contain, and how they should be organized.

In Dagster, you can attach ownership information, organize your assets with tags, attach rich, complex information with metadata, and link your assets with their source code.

What you'll learn

• How to add owners to your assets
• How to use tags to organize assets
• How to attach complex information to an asset using metadata
• How to link your assets with their source code

---

Prerequisites

To follow the steps in this guide, you'll need:

• A basic understanding of Dagster and assets. See the Quick Start tutorial for an overview.

---

Adding owners to your assets

In a large organization, it's important to know who is responsible for a given data asset. With owners it's straightforward to add individuals and teams as owners for your asset:

Using owners

Loading...

owners must either be an email address, or a team name prefixed by team:.

With Dagster+ Pro, you can create asset-based alerts that will automatically notify an asset's owners when triggered. Refer to the Dagster+ alert documentation for more information.

Choosing between tags or metadata for custom information

In Dagster, you can attach custom information to assets in two ways: tags and metadata.

Tags are a simple way to organize assets in Dagster. You can attach several tags to an asset when it is defined, and they will appear in the UI. You can also use tags to search and filter for assets in the asset catalog. They are structured as key-value pairs of strings.

Here's an example of some tags one might apply to an asset:

{"domain": "marketing", "pii": "true"}

Metadata allows you to attach rich information to the asset, like a Markdown description, a table schema, or a time series. Metadata is more flexible than tags, as it can store more complex information. Metadata can be attached to an asset at definition time (i.e. when the code is first imported) or at runtime (every time an asset is materialized).

Here's an example of some metadata one might apply to an asset:

{
    "link_to_docs": MetadataValue.url("https://..."),
    "snippet": MetadataValue.md("# Embedded markdown\n..."),
    "file_size_kb": MetadataValue.int(1024),
}

Attaching tags to an asset

Like owners, just pass a dictionary of tags to the tags argument when defining an asset:

Using tags

Loading...

Keep in mind that tags must contain only strings as keys and values. Additionally, the Dagster UI will render tags with the empty string as a "label" rather than a key-value pair.

Attaching metadata to an asset at definition time

Attaching metadata at definition time is quite similar to how you attach tags.

Using metadata at definition time

Loading...

To learn more about the different types of metadata you can attach, see the MetadataValue API docs.

Some metadata keys will be given special treatment in the Dagster UI. See the Standard metadata types section for more information.

Attaching metadata to an asset at runtime

Metadata becomes very powerful when it is attached when an asset is materialized. This allows you to update metadata when information about an asset changes and track historical metadata such as execution time and row counts as a time series.

Using metadata at runtime

Loading...

Any numerical metadata will be treated as a time series in the Dagster UI.

Standard metadata types

Some metadata keys will be given special treatment in the Dagster UI.

Key	Description
dagster/uri	Type: str The URI for the asset, e.g. "s3://my_bucket/my_object"
dagster/column_schema	Type: TableSchema For an asset that's a table, the schema of the columns in the table. Refer to the Table and column metadata secton for details.
dagster/column_lineage	Type: TableColumnLineage For an asset that's a table, the lineage of column inputs to column outputs for the table. Refer to the Table and column metadata secton for details.
dagster/row_count	Type: int For an asset that's a table, the number of rows in the table. Refer to the Table metadata documentation for details.
dagster/partition_row_count	Type: int For a partition of an asset that's a table, the number of rows in the partition.
dagster/relation_identifier	Type: str A unique identifier for the table/view, typically fully qualified. For example, my_database.my_schema.my_table
dagster/code_references	Type: CodeReferencesMetadataValue A list of code references for the asset, such as file locations or references to Github URLs. Refer to the Linking your assets with their source code section for details. Should only be provided in definition-level metadata, not materialization metadata.

Table and column metadata

Two of the most powerful metadata types are TableSchema and TableColumnLineage. These metadata types allow stakeholders to view the schema of a table right within Dagster, and, in Dagster+, navigate the asset catalog via the column lineage.

Table schema metadata

Here's a quick example of how to attach this metadata at both definition time and runtime:

Table schema metadata

Loading...

Note that there are several data types and constraints available on TableColumn objects. Refer to the API documentation for more information.)

Column lineage metadata

Many integrations such as dbt automatically attach this metadata out-of-the-box.

Column lineage metadata is a powerful way to track how columns in a table are derived from other columns. Here is how you can manually attach this metadata:

Table column lineage metadata

Loading...

Dagster+ provides rich visualization and navigation of column lineage in the asset catalog. Refer to the Dagster+ documentation for more information.

Linking your assets with their source code

This feature is considered experimental and is under active development. This guide will be updated as we roll out new features.

Attaching code reference metadata to your Dagster asset definitions allows you to easily view those assets' source code from the Dagster UI both in local development and in production.

Many integrations such as dbt support this capability out of the box. Refer to the integration documentation for more information.

Attaching Python code references for local development

Dagster can automatically attach code references to your assets during local development with one line of code.

Local source code references

Loading...

Attaching custom code references for local development

You can manually add the dagster/code_references metadata to your asset definitions to attach custom code references. This is useful when your asset's source code is not primarily in Python (e.g. when parsing a YAML config or integrating with another language).

Custom local source code references

Loading...

Attaching code references in production (Dagster+)

Dagster+ can automatically annotate your assets with code references to source control such as GitHub or GitLab.

Production source code references (Dagster+)

Loading...

Attaching code references in production (OSS)

If you aren't using Dagster+, you can annotate your assets with code references to source control, but it requires a little more manual mapping.

Production source code references (OSS)

Loading...

link_code_references_to_git currently supports GitHub and GitLab repositories. It also supports customization of how file paths are mapped; see the AnchorBasedFilePathMapping API docs for more information.