Releases: dagster-io/dagster
Releases · dagster-io/dagster
1.10.20 (core) / 0.26.20 (libraries)
New
- The
@asset
decorator now supports ahooks
argument to allow for functions to be executed on asset success / failure (thanks @brunobbaraujo)! - The log message produced when an asset check is evaluated now includes its pass / fail state.
- The
@multi_asset_check
decorator now supports thepool
argument. - [dagster-dbt] The
DagsterDbtTranslator
class now has aget_asset_check_spec
method which can be overridden to customize theAssetCheckSpecs
that are produced for each individual dbt test.
Bugfixes
- Fixed an issue where setting the non-public
blocking
attribute on anAssetCheckSpec
would halt execution of the step as soon as any asset check failure was emitted, even if the step still had asset materializations or check evaluations to emit that were not downstream of the failed asset check. Now that this issue has been fixed, theblocking
attribute onAssetCheckSpec
has been made a public attribute. If you were making use of theblocking
attribute on AssetCheckSpec before it was public and relying on the previous behavior, you should exit from your asset evaluation function after emitting an AssetCheckFailure from within your multi-asset in order to halt further execution of the step. - Fixed a bug where
GraphDefinition.to_job()
would not work if an op had a custom IO manager key. - Fixed an issue that would cause
.allow()
and.ignore()
applications to not propagate through.since()
automation conditions. - Fixed a bug where assets with cross-location dependencies could sometimes be incorrectly reported as "Unsynced".
- Fixed an issue where
dagster dev
was sometimes failing to load code locations with a "Deadline Exceeded" error unless the--use-legacy-code-server-behavior
flag was set. - The backfill daemon can now be configured to use a threadpool executor via helm (thanks @HynekBlaha)!
- [dagster-gcp] Added a
google-cloud-bigquery>=1.28.3
pin to correctly reflect the lowest compatible version.
Breaking Changes
- [ui] Moved legacy Auto-materialize (global AMP) tab from Overview to Automations.
Dagster Plus
- Fixed an issue where certain rare network conditions could cause steps to hang while uploading compute logs after a step finished.
- [ui] For users with the new Observe UIs enabled, the Asset Health and Resources tabs are no longer shown on the Timeline page.
dg & Components (Preview)
- Fix a bug with
dg check yaml
where valid component type names were rejected if they were not registered (i.e. visible fromdg check components
). dg create-dagster
now warns when scaffolding a project or workspace if it is not the latest version.- The
project.registry_modules
configuration can now accept wildcards (e.g.foo_bar.components.*
). This will register any module matching the pattern withdg
. - The
env
YAML function now errors if the specified env var is unset. Default values can be provided as an additional argument:{{ env('MY_ENV_VAR', 'default') }}
- defs.yaml files can now specify a component in the module where it is defined, as opposed to just the module where it is exposed in the
dg
registry. - The
PipesSubprocessScriptCollectionComponent
has been removed. - Running dg commands such as
dg check defs
anddg dev
in a project folder that is part of the workspace will now only apply to that project, instead of every project in the workspace. - Scaffolded projects no longer contain a "components" directory or a Python
dagster_dg_cli.plugin
entry point. - Scaffolded components can now be placed anywhere within a project module hierarchy.
- The entry point group used by shared libraries exposing custom components to
dg
has been renamed fromdagster_dg_cli.plugin
todagster_dg_cli.registry_modules
(projects no longer need to define an entry point group at all). dg list plugin-modules
has been renamed todg list registry-modules
.dg list defs
now supports configuring output columns with the--columns/-c
option.- [dagster-airbyte] Introduced a
AirbyteCloudWorkspaceComponent
which can be used to pull in Airbyte Cloud connections into Dagster
1.10.19 (core) / 0.26.19 (libraries)
New
- The database table used by the
DbIOManager
is now configurable via"table"
output (asset) metadata key #30310 - Changed default settings for backfill daemon to
use_threads=True
,num_workers=4
. Thanks @HynekBlaha! - A new function
build_asset_check_context
can be used to build asset check contexts for direct invocation. - Changed
Definitions.get_all_asset_specs
to only return assets directly passed in as AssetSpecs or AssetsDefinitions. - Removed
selection
argument fromDefinitions.map_asset_specs
. If selection is needed use the newDefinitions.map_resolved_asset_specs
. Definitions.get_job_def
now warns when it finds an unresolved job or no job.- Changed
Definitions.get_assets_def
to return an AssetsDefinition without resolving if it was passed in directly. - [dagster-dbt]
build_schedule_from_dbt_selection
now supports aselector
argument, allowing you to use yaml-based selectors. - [dagster-k8s] Pods created by the Kubernetes run launcher and executor from Dagster Plus now include the
dagster/deployment-name
label. - [dagster-pipes] Pipes execution errors are no longer treated as framework errors, meaning they properly invoke RetryPolicies.
- [helm] Backfill daemon configuration now supported. Thanks @HynekBlaha!
- [ui] Show relative start time on runs in run timeline hover lists. #30327
Bugfixes
- [ui] Fixed live updating of asset materialization statuses in asset graph.
dg & Components (Preview)
- Running dg commands like
dg check defs
anddg dev
in a project folder that is part of the workspace will now only apply to that project, instead of every project in the workspace. dg list defs
now supports the--assets/-a
option, to show only assets matching the provided selection.dg list defs
now supports a--path
argument to subset the defs files shown.- The
create-dagster workspace
command now accepts the same required path argument as thecreate-dagster project
command, instead of defaulting to adagster-workspace
subfolder of the current working directory. - The entry point group used by shared libraries exposing custom components to dg has been renamed from
dagster_dg_cli.plugin
todagster_dg_cli.registry_modules
(projects no longer need to define an entry point group at all). dg list plugin-modules
has been renamed todg list registry-modules
.- Newly scaffolded projects no longer contain a "components" directory or a Python
dagster_dg_cli.plugin
entry point. - Newly scaffolded components can now be placed anywhere within a project module hierarchy.
Resolvable
subclasses can now use baredict
andlist
as field types.- Resolving a
Resolvable
subclass will no longer change empty strings toNone
. - Users can define multiple
@definitions
-decorated functions in a single module in thedefs
hierarchy and they are automatically merged and incorporated into the project. - Added
@component_instance
to replace@component
. This allows multiple component instances in a python file. - Fixed an issue where
dg
commands would sometimes output extradagster_telemetry_logger
lines to stdout at the end of commands. - Added
@template_var
as an alternative approach for defining variables in a templating context.
1.10.18 (core) / 0.26.18 (libraries)
New
BackfillPolicy
is now marked as generally available (GA).- Optimized the order of the
@asset
decorator overloads to make custom wrappers around the asset decorator easier. Thanks @jonathanunderwood! - [dagster-slack] Added
get_client()
to SlackResource.
Bugfixes
Definitions
andAssetDefinition
will now warn if they get differentAssetSpec
s with the same key. This will become an exception in 1.11.- Functions that load all definitions from modules like
load_definitions_from_modules
now handle duplicateAssetSpec
s. - Fixed typo in logging. Thanks @eli-b!
- [dagster-dbt] An issue occurring when using dbt selection arguments with a dbt project using saved queries and semantic models has been fixed.
- [dagster-fivetran] Fixed an issue with
load_assets_from_fivetran_instance
where assets whose asset keys have been customized using a Fivetran translator would lead to an exception.
Documentation
- Fixed grammar issues in GCP docs. Thanks @D1n0!
- Fixed missing docs for
required_resource_keys
in@sensor
. Thanks @seyf97!
Breaking Changes
Definitions
andAssetDefinition
will now warn if they get differentAssetSpec
s with the same key. This will become an exception in 1.11.
Dagster Plus
- [dagster-cloud-cli] Added a
--wait
to thedagster-cloud job launch
command that makes it wait until the launched run copmletes. Thanks @stevenayers! - [fix][dagster-cloud-cli] Fixed an issue where the
dagster-cloud
cli failed to deploy PEX projects on python 3.12 in certain environments without setuptools already installed.
dg & Components (Preview)
- The
dg
CLI is now installed in each project's Python environment instead of as a global tool. If you previously haddg
installed globally and are upgrading, first uninstall the global dg tool (uv tool uninstall dagster-dg
orpip uninstall dagster-dg
) and adddagster-dg-cli
as a dependency to each of your projects. Newly scaffolded projects will automatically include thedg
CLI going forward. - A new
create-dagster
CLI has been added for scaffolding projects instead of workspaces.uvx create-dagster project
has replaceddg scaffold project
, anduvx create-dagster workspace
has replaceddg scaffold workspace
. The commands take identical arguments. - Definitions and component instances are now scaffolded with
dg scaffold defs <scaffolder>
instead ofdg scaffold <scaffolder>
. - The
component.yaml
file to specify a component instance is now called "defs.yaml". "component.yaml" will remain supported for several weeks, but is deprecated. - The
lib
folder in newly scaffolded projects has been renamed to "components". dg scaffold component-type
has been renamed todg scaffold component
.dg list plugins
has been renamed todg list plugin-modules
and now outputs only plugin module names, not plugin objects.dg list component
now lists component types instead of instances.- Exports from
dagster.components
are now available in the top-leveldagster
module. - Added
@component_instance
to replace@component
, which allows multiple component instances in a python file. - If you type a partial component name for
dg scaffold defs <component>
, you will now be prompted with possible matches. - When component classnames are unique, you can now use just the classname as an alias for the fully qualified name when running
dg scaffold defs <component>
. dg launch
now supports passing config files through--config/-c
.- Added
Component.from_attributes_dict
andComponent.from_yaml_path
to help with testing. - Added
dagster.components.testing.component_defs
utility to help with testing components. - Scaffolded schedules and sensors in
dg
are now loadable by default. dg scaffold defs inline-component
can now be used to create inline components and corresponding instances.dg list defs
now outputs resources.- [fix] The
dagster-dbt
cli flag--components
flag now correctly findsDbtProjectComponent
again. - [dagster-fivetran] Added a
FivetranAccountComponent
which can be used to pull in Fivetran connections into Dagster. - [dagster-dlt] The DltLoadCollectionComponent scaffolder no longer attempts to automatically construct loads for the given source and destination type.
- Fixed an issue where components failed to load when using
load_assets_from_airbyte_instance
or other APIs that return aCacheableAssetsDefinition
.
1.10.17 (core) / 0.26.17 (libraries)
Bugfixes
- Fixed an issue where an error was displayed in the UI while viewing run logs
- [dagster-dbt] Fixed an issue occurring when using dbt selection arguments with a dbt project using semantic models.
1.10.16 (core) / 0.26.16 (libraries)
New
typing_extensions
is now pinned to>=4.11.0
instead of>=4.10.0
.- [ui] Viewing an automation condition evaluation now automatically expands the set of applicable sub-conditions.
- [ui] Added the ability to navigate from an automation condition evaluation to upstream automation condition evaluations.
- [ui] Added an asset graph node facet for viewing automation conditions and the most recent evaluation. This can be enabled in the user settings via a feature flag (
Enable faceted asset nodes
). - [ui] A new experimental integrations marketplace tab is now available and can be enabled in your user settings via a feature flag (
Display integrations marketplace
). It provides easy access to the gallery of dagster-supported plugins.
Bugfixes
- Fixed an issue with the
ExternalNotebookDataRequest
GRPC call that would allow it to access files outside of the current directory. - Fixed an issue that would cause
op_tags
set on@observable_source_asset
s to be dropped from the underlying step context object when executed. - Fixed an issue where
dagster dev
would sometimes raise a gRPC error when loading several code locations at once. - Fixed an issue where setting an environment variable to the string "false", "0" or "None" for a dagster config field using a
BoolSource
would evaluate to True. - Fixed an issue where specifying
executable_path
in a workspace.yaml file to run code locations in a different virtual environment would not correctly inherit the PATH of that virtual environment in the code location. - [dagster-dbt] Fixed an issue causing dbt CLI invocation to fail when materializing assets when
OpExecutionContext
was used as the type hint for the context. - [dagster-deltalake] Corrected the
timeout
property data type inClientConfig
to be str instead of int (thanks, @edsoncezar16!)
Documentation
- Added additional config fields to the
K8sRunLauncher
example (thanks, @nishan-soni!) - Corrected broken links on the automation landing page (thanks, @briandailey!)
Dagster Plus
- [ui] Alert policy event tags no longer appear red and yellow outside of the policy notification history.
dg & Components (Preview)
- Added suggestions to component model error messages when using built-in models for common classes such as
AssetKey
andAssetSpec
. dg list env
now displays whether env vars are configured in each Dagster Plus scope.- Introduced
Resolver.passthrough()
to avoid processing fields on a component model. ResolvedAssetKey
is now exported fromdagster.components
.dg init
has been removed.dg scaffold project
anddg scaffold workspace
should be used instead.- Fixed an issue where
dg dev
failed with a temporarily file permissions error when running on Windows. Thanks @polivbr!
1.10.15 (core) / 0.26.15 (libraries)
New
- Added a config section to
dagster.yaml
to enable submitting backfill runs in a threadpool. - Expanded definition time validation for partition mappings to avoid runtime errors querying asset status.
- [ui][beta] You can now re-execute a run that targeted a multi-asset from the point of asset failure instead of step failure, meaning only assets that failed or were skipped will be re-executed. To enable this option, turn on the
Enable retries from asset failure
feature flag in your user settings. - [ui] Made it easier to select and copy image names for code locations.
- [ui] Added asset lineage navigation within the automation condition evaluation tree.
- [ui] Viewing an evaluation tick now auto-expands the set of applicable automation conditions.
- [ui] Added an asset graph node facet for viewing automation conditions and the most recent evaluation in the global asset graph.
- [dagster-fivetran] The
FivetranWorkspace
resource is now marked as generally available (GA).
Bugfixes
- Changed asset wipes to also wipe associated asset check evaluations.
- [dagster-fivetran] Fixed an issue causing the Fivetran integration to fail when the schema config does not exist for a connector.
Documentation
- Fixed a broken link in the airflow migration docs. Thanks @jjyeo!
- Updated example snippet to include a missing type hint. Thanks @agrueneberg!
Deprecations
- [dagster-fivetran] The
FivetranResource
resource is now deprecated. Use the newFivetranWorkspace
resource instead.
dg & Components (Preview)
- Changed
Scaffolder.scaffold
to have the params object as an attribute of theScaffoldRequest
object instead of a dictionary. This is a breaking change for those who have implemented a custom scaffolder. - Added support for scaffolding resources via
dg scaffold dagster.resources path/to/resources.py
. - Added support for the usage of
@definitions
in thedefs
hierarchy. - Dagster components now include code references by default. When viewing an asset emitted by a component in the asset catalog, this will allow you to jump to the backing
component.yaml
file in your editor. - [dagster-dbt]
DbtProjectComponent
fields now properly evaluate templates. - [dagster-sling] Updated the SlingReplicationCollectionComponent from using the
asset_attributes
parameter totranslation
, in order to match our other integration components. - Fixed an issue where
dg dev
failed with a temporary file permissions error when running on Windows. Thanks @polivbr!
1.10.14 (core) / 0.26.14 (libraries)
New
- [dagster-tableau] Refined Tableau integration for API 3.25 or greater.
- [dagster-tableau] Data sources with extracts can now be materialized in Tableau assets created with
build_tableau_materializable_assets_definition
. - [ui] Added kinds tag for treasuredata.
- [ui] Add Supabase kind icon.
Bugfixes
- Fixed a bug which could cause an error when calling
MultiPartitionsDefinition.has_partition_key()
on invalid keys. - Fixed a bug where the default multiprocess executor would fail runs where the child process for a step crashed, even if a retry policy resulted in a successful retry of that crashed step.
- Fixed a bug with
AutomationCondition.initial_evaluation
which could cause it to returnFalse
for an asset that went from having a condition, to having no condition at all, back to having the original condition again. - [ui] Fixed an issue which could cause the "Target" field of AutomationConditionSensorDefinitions to render incorrectly when exactly one asset check was defined in a code location.
- [dagster-dbt] Fix
DagsterDbtTranslatorSettings.enable_source_tests_as_checks
returning duplicate asset checks.
Documentation
- Added a sample Dagster+ ECS CloudFormation template which incorporates private subnets.
- Fixed incorrect storage values in the Fargate task section of the AWS deployment guide, thanks @alexpotv!
- Updated log stream docs, thanks @jjyeo!
- Fixed broken code in the configurable resources guide, thanks @nightscape!
Deprecations
dagster.InitResourceContext.dagster_run
has been deprecated in favor ofInitResourceContext.run
.
dg & Components (Preview)
- [dagster-k8s] PipesK8sComponent has been added.
- Dagster components no longer change the working directory while they are being loaded. This allows components to store relative paths and ensure that they will still work when accessed outside of the component loading codepath. This change may affect user-defined components that depend on
Path.cwd()
oros.getcwd()
. Instead, you should use a path relative to the current source file when loading paths in a component, using thecontext.resolve_source_relative_path
method (seeresolve_dbt_project
inDbtProjectComponent
for an example). - Added
dagster.job
scaffolder. - [dagster-dbt] The
DbtProjectComponent
now has atranslation_settings
argument for adjustingDagsterDbtTranslatorSettings
. - [dagster-dbt][fix]
DbtProjectComponent
fields now properly evaluate templates. - Fixed docstring for
load_defs
entrypoint, thanks @mattgiles!
1.10.13 (core) / 0.26.13 (libraries)
New
- If an unselected asset check is executed during a run, the system will now warn instead of throwing a hard error.
- When evaluating
AutomationCondition.any_deps_match
orAutomationCondition.all_dep_match
with an allow / ignore specified, an error will no longer be produced if the provided asset selection references an asset key that does not exist. - Added the ability to restrict the list of ports that
dagster dev
is allowed to use to open subprocesses when running on Windows, by setting theDAGSTER_PORT_RANGE
env var to a string of the form<start>=<end>
- for example "20000-30000". - [dagster-aws] The S3 sensor's
get_objects
now returns an empty list if no new files can be found since thesince_last_modified
parameter. Thanks @bartcode! - [dagster-dbt]
@dbt_assets
andbuild_dbt_manifest_asset_selection
now support aselector
argument, allowing you to use yaml-based selectors. - [dagster-dbt] you can now enable dbt source tests as asset checks by using the flag
DagsterDbtTranslatorSettings.enable_source_tests_as_checks
. - [dagster-k8s] improved run monitoring when running with increased backoff limits. Thanks @adam-bloom!
Bugfixes
- Fixed a bug with
AutomationCondition.initial_evaluation
which could cause it to returnFalse
for an asset that went from having a condition, to having no condition at all, back to having the original condition again. - Fixed a bug that would cause the
AutomationCondition.any_deps_updated()
condition to evaluate toFalse
when evaluated on a self-dependency. - Fixed a bug in the
quickstart_aws
example. Thanks @Thenkei! - [ui] Fixed navigation between asset tabs by no longer preserving query parameters from one tab to the next.
- [ui] Fixed an issue where the asset graph looked like it was still loading when it wasn't.
Documentation
- Added Scala Spark / Dagster Pipes guide.
dg & Components (Preview)
- [dg] Error message when an invalid configuration file is detected is now shorter and more clear.
- [components] Descriptions and examples have been restored for core models such as
ResolvedAssetSpec
.description
andexamples
arguments have been added toResolver
for documenting fields on non-pydantic model basedResolvable
classes.
1.10.12 (core) / 0.26.12 (libraries)
New
- [ui] Removed the
By partition
grouping view for recent events for assets that do not have a definition in the workspace. - [ui] The asset graph now displays asset health information when the new Observe UI feature flag is enabled.
- [ui] A new feature flag allows you to customize the appearance of assets on the asset graph by enabling and disabling individual facets.
- [ui] Fixed a bug that prevented filtering asset events by type.
- [dagster-k8s] K8sPipeClient failures will now include the last 100 lines of the logs of the pod that failed, instead of the full log output.
Bugfixes
- Fixed an issue which caused multi assets that were automatically broken apart in some contexts to remain separated even in cases where this was not necessary to maintain execution dependencies.
- Fixed a bug that would cause multi-assets defined with
can_subset=True
to error when usingdagster-pipes
if not all outputs were emitted. - [ui] Fixed an issue where the asset graph looked like it was still loading when it wasn't.
Documentation
dg & Components (Preview)
dg
will now fail with an error message if it's version is below the minimum supported version for the version ofdagster
in your environment.dg
now checks for new versions and prompts the user to update. The check runs automatically atdg
startup once per day.dg
will now emit a warning prompting the user to reinstall the package if it detects an entry point in project metadata that does not show up when running indg list plugins
.- Added
dagster.multi_asset
scaffolder - Added
dagster.asset_check
scaffolder - Fixed a bug where
dg list defs
would crash if anything was written to stdout while loading a project's definitions. - The default location for the
dg
user config file on Unix has been moved from~/.dg.toml
to~/.config/dg.toml
.~/.config
can be overridden by setting$XDG_CONFIG_HOME
. - Cache deserialization errors are now ignored. Previously, when the
dg
contents of the cache were in an outdated format,dg
could crash. Now it will just rebuild the cache. - The Components ETL Pipeline Tutorial now supports users of both
pip
anduv
. - The
dg
CLI will now emit a warning if you are using "active" mode for your project python environment, there is a virtual environment at<project_root>/.venv
, and the activated venv is not<project_root>/.venv
- A new
dg
settingcli.suppress_warnings
is available. This takes a list of warning types to suppress. - Added a warning message to inform users they need to install their project package when skipping automatic environment setup.
- Changed configuration of project Python environments. The
tool.dg.project.python_environment
previously accepted a string,"active"
or"persistent_uv"
. Now it accepts a table with one of two keys:{active = true}
: equivalent of previous"active"
{uv_managed = true}
: equivalent of previous"persistent_uv"
- Changed the default python environment for newly scaffolded projects to
tool.dg.project.python_environment
to{active = true}
. This means by default, no virtual environment oruv.lock
will be created when scaffolding a new project (viadg init
ordg scaffold project
). You can pass--python-environment uv_managed
for the old behavior. - Removed the
--skip-venv
flag ondg scaffold project
anddg init
. - The
dagster_components
package has been merged intodagster
and use of thedagster-components
package has been deprecated.dagster-components
will remain as a stub package for the next few weeks, but code should be updated to import fromdagster.components
instead ofdagster_components
. - [dagster-dbt] the
dagster-dbt project prepare-and-package
cli now supports--components
for handlingDbtProjectComponent
- [dagster-dbt]
DbtProjectComponent
has been reworked, changing both the python api and the yaml schema.dbt
has been replaced withproject
with a slightly different schema, andasset_attributes
withtranslation
.
1.10.11 (core) / 0.26.11 (libraries)
New
- [ui] Runs launched from the Dagster UI get a
dagster/from_ui = true
tag, making it easy to filter for them. - [ui] The run page now shows the number of log levels selected as well as the number of log levels available.
- Added
AirflowFilter
API for use withdagster-airlift
, allows you to filter down the set of dags retrieved up front for perf improvements.
Bugfixes
- [ui] Fixed a bug that prevented filtering asset events by type.
- Fixed a bug that would cause multi-assets defined with
can_subset=True
to error when usingdagster-pipes
if not all outputs were emitted. - [dagster-dbt] the
state_path
argument toDbtCliResource
now resolves relative to the project directory as documented. - [dagster-k8s] Made reliability improvements to PipesK8sClient log streaming when transient networking errors occur. The default behavior of the PipesK8sClient is now to reconnect to the stream of logs every hour (this value can be overridden by setting the
DAGSTER_PIPES_K8S_CONSUME_POD_LOGS_REQUEST_TIMEOUT
environment variable) and to retry up to 5 times if an error occurs while streaming logs from the launched Kubernetes pod (this value can be overridden by setting theDAGSTER_PIPES_K8S_CONSUME_POD_LOGS_RETRIES
environment variable.) - [dagster-dbt] Fixed a bug where dbt jobs would fail due to unparseable logs causing errors in
DbtCliInvocation.stream_raw_events
. (Thanks @ross-whatnot!)
Dagster Plus
- [ui] It is now possible to roll back a code location to a version that had previously been in an errored state.
dg & Components (Preview)
- The
dg
CLI will now emit a warning if you are using "active" mode for your project python environment, there is a virtual environment at<project_root>/.venv
, and the activated venv is not<project_root>/.venv
- A new
dg
settingcli.suppress_warnings
is now available. This takes a list of warning types to suppress. - Changed configuration of project Python environments. The
tool.dg.project.python_environment
previously accepted a string,"active"
or"persistent_uv"
. Now it accepts a table with one of three keys: -{active = true}
: equivalent of previous"active"
-{uv_managed = true}
: equivalent of previous"persistent_uv"
- Changed the default python environment for newly scaffolded projects to
tool.dg.project.python_environment
to{active = true}
. This means by default, no virtual environment oruv.lock
will be created when scaffolding a new project (viadg init
ordg scaffold project
). You can pass--python-environment uv_managed
for the old behavior. - Removed the
--skip-venv
flag ondg scaffold project
anddg init
. - Fixed a bug where new projects scaffolded with
dg scaffold project
were lackingdagster
as a dependency. - The "Creating a library of components" guide has been replaced by a new and more general "Creating a
dg
plugin" guide. - The arguments/options of the
dg init
command have changed. You may pass.
as an argument to initialize a project/workspace in the CWD. Seedg init --help
for more details. - The
dagster-components
package (which was converted to a stub package in the last release) is no longer being published. Alldagster-components
functionality is now part ofdagster
. - Projects should now expose custom component types under the
dagster_dg.plugin
entry point group instead ofdagster_dg.library
.dagster_dg.library
support is being kept for now for backcompatibility, but will be dropped in a few weeks. - Fixed formatting of line added to
project/lib/__init__.py
when scaffolding a component type. - The
dg env list
command is nowdg list env
- The
dg plus env pull
command is nowdg plus pull env
. - The
dg list component-type
command has been removed. There is a newdg list plugins
with output that is a superset ofdg list component-type
. - The
dg
user config file on Unix is now looked for at~/.dg.toml
instead of~/dg.toml
. - [dagster-dbt]
DbtProjectComponent
has been reworked, changing both the python api and the yaml schema.dbt
has been replaced withproject
with a slightly different schema, andasset_attributes
withtranslation
.