Definitions now accepts Executor instances in its executor argument, not just ExecutorDefinitions.
@multi_asset_sensor now accepts a request_assets parameter, which allows it to directly request that assets be materialized, instead of requesting a run of a job.
Improved the performance of instantiating a Definitions when using large numbers of assets or many asset jobs.
The job passed to build_schedule_from_partitioned_job no longer needs to have a partitions_def directly assigned to it. Instead, Dagster will infer from the partitions from the assets it targets.
OpExecutionContext.asset_partition_keys_for_output no longer requires an argument to specify the default output.
The “Reload all” button on the Code Locations page in Dagit will now detect changes to a pyproject.toml file that were made while Dagit was running. Previously, Dagit needed to be restarted in order for such changes to be shown.
get_run_record_by_id has been added to DagsterInstance to provide easier access to RunRecord objects which expose the start_time and end_time of the run.
[dagit] In the “Materialize” modal, you can now choose to pass a range of asset partitions to a single run rather than launching a backfill.
[dagster-docker] Added a docker_container_op op and execute_docker_container_op helper function for running ops that launch arbitrary Docker containers. See the docs for more information.
[dagster-snowflake-pyspark] The Snowflake I/O manager now supports PySpark DataFrames.
[dagster-k8s] The Docker images include in the Dagster Helm chart are now built on the most recently released python:3.x-slim base image.
Previously, the build_asset_reconciliation_sensor could time out when evaluating ticks over large selections of assets, or assets with many partitions. A series of performance improvements should make this much less likely.
Fixed a bug that caused a failure when using run_request_for_partition in a sensor that targeted multiple jobs created via define_asset_job.
The cost of importing dagster has been reduced.
Issues preventing “re-execute from failure” from working correctly with dynamic graphs have been fixed.
[dagit] In Firefox, Dagit no longer truncates text unnecessarily in some cases.
[dagit] Dagit’s asset graph now allows you to click “Materialize” without rendering the graph if you have too many assets to display.
[dagit] Fixed a bug that stopped the backfill page from loading when assets that had previously been backfilled no longer had a PartitionsDefinition.
[dagster-k8s] Fixed an issue where k8s_job_op raised an Exception when running pods with multiple containers.
[dagster-airbyte] Loosened credentials masking for Airbyte managed ingestion, fixing the Hubspot source, thanks @joel-olazagasti!
[dagster-airbyte] When using managed ingestion, Airbyte now pulls all source types available to the instance rather than the workspace, thanks @emilija-omnisend!
[dagster-airbyte] Fixed an issue which arose when attaching freshness policies to Airbyte assets and using the multiprocessing executor.
[dagster-fivetran] Added the ability to force assets to be output for all specified Fivetran tables during a sync in the case that a sync’s API outputs are missing one or more tables.
The asset_keys and asset_selection parameters of the experimental @multi_asset_sensor decorator have been replaced with a monitored_assets parameter. This helps disambiguate them from the new request_assets parameter.
Large asset graphs can now be materialized in Dagit without needing to first enter an asset subset. Previously, if you wanted to materialize every asset in such a graph, you needed to first enter * as the asset selection before materializing the assets.
Added a pin of the sqlalchemy package to <2.0.0 due to a breaking change in that version.
Added a pin of the dbt-core package to <1.4.0 due to breaking changes in that release that affected the Dagster dbt integration. We plan to remove this pin in the next release.
Added a pin of the jupyter-client package to <8.0 due to an issue with the most recent release causing hangs while running dagstermill ops.
The nux section of dagster.yaml config has been fixed.
Changes when heartbeats occur in the daemon to avoid crashes in certain conditions.
Fixed an issue where passing a workspace file as an argument into the dagster dev command raised an error
[dagit] Fixes an issue with asset names being truncated by long asset descriptions in the asset catalog, making them impossible to click.
[dagit] The backfill page no longer fails to load if any of the asset backfills had assets that were partitioned at the time of the backfill but are no longer partitioned.
Added a new dagster dev command that can be used to run both Dagit and the Dagster daemon in the same process during local development. See the new Running Dagster Locally guide in the docs for more information.
Added instructions for installing the dagster package on M1 and M2 Macs that avoids installation errors when building the grpcio package. See the Installing Dagster guide in the docs for more information.
create_repository_using_definitions_args has been added for users to backport their repository definitions to the new Definitions API
When running Dagit on your local machine, a prompt will now appear that allows you to optionally enter an email address to receive Dagster security updates or subscribe to the Dagster newsletter. This prompt can be dismissed in the UI, or permanently disabled by adding the following to your dagster.yaml file:
nux:
enabled: false
The grpcio pin in Dagster to <1.48.1 has been restored for Python versions 3.10 and 3.11, due to upstream issues in the grpcio package causing hangs in Dagster.
[dagit] Improved query performance on Scheduled Runs page.
[dagit] The "Materialize" button now allows you to add tags to asset materialization runs. If your assets do not require config or partitions, you may need to shift-click "Materialize".
[dagit] The kind tags and logos shown on assets in the Asset Graph now appear in other parts of Dagit so it's easier to understand your assets.
[dagit] Selecting a materialization event on the Asset Details page now shows links to the upstream materialzations ("Source Data") that were incorporated into that version of the asset.
[dagit] Added or improved document (browser tab) titles throughout Dagit.
[dagster-snowflake] Snowflake resource and IO manager now support private key authentication with unencrypted keys
[dagster-dbt] The op created when using load_assets_from_dbt* is now configurable, allowing you to pass in specific parameters to the underlying dbt command (such as --full-refresh). To do so, supply config of the form {"ops": {"run_dbt_<abcde>": {"config": {"full_refresh": True}}}} when defining your asset job, or in Dagit.
For time-partitioned assets, the build_asset_reconciliation_sensor will now only materialize partitions from the past day. Previously, if a new asset was added with a large number of missing partitions, a run for each of the missing partitions would be launched immediately.
A variety of performance improvements to the build_asset_reconciliation_sensor, which should help significantly speed up sensor evaluation in cases where there is a large number of assets or partitions.
[dagit] Fixed broken code location names in the “Definitions” filter dialog accessible from the left navigation.
[dagit] The Backfills pages in Dagit no longer crash when your backfills span tens of thousands of partition keys
[dagit] The asset graph now links to the failed run, not the last successful materialization, when assets are shown in a "Failed" state.