v1.3.2

What's new

Fixed ✅

• Fix issues with gcloud auth in beaker executor.

Commits

2ad79f8 Fix issues with gcloud auth (#609)
4a1ebea Fix readthedocs config (#608)

v1.3.1

What's new

Fixed ✅

• Minor bugs in the GSWorkspace().

Changed ⚠️

• Added CLI-style execution functions for experiments defined in Python.
• Added display() to ExecutorOutput for producing a table that summarizes the run.

Commits

4c8ae5a CLI-style execution for Python-defined experiments (#600)
8bb3472 GS workspace bug fixes (#607)

v1.3.0

What's new

Added 🎉

• Added the Workspace.remove_step() method to safely remove steps.
• The GSWorkspace() can now be initialized with google cloud bucket subfolders.

Changed ⚠️

• The BeakerExecutor now uses the HEAD commit at the time the executor is instantiated to executor a step instead of the HEAD commit at the time the step is run.

Fixed ✅

• Removed unnecessary code coverage dev requirements.
• Fixed issue where new version of torch caused no LR schedulers to be registered.
• Updated pinned versions of jax, jaxlib, and flax.

Commits

ed72140 Prepare for release v1.3.0
5d776d5 Revert "Prepare for release v1.3.0"
8eec3df Revert "remove folder"
671525f remove folder
1a384f7 Prepare for release v1.3.0
56c1476 Use commit at time executor is instantiated (#605)
11b5229 'Remove step' feature for local workspaces (#588)
3857415 GS workspaces can be bucket subfolders (#604)
b955ef7 CI errors be gone (#601)
01077eb LR schedulers (#573)
3a19688 Bug fix in getting results from GS workspace (#582)
4c3edce style: migrate to ruff (#562)
416ffa6 Add CITATION.cff file (#572)
70f2681 Update torch requirement from <1.14,>=1.9 to >=1.9,<2.1 (#539)
1ee2c56 Update wandb requirement from <0.13.11,>=0.12 to >=0.12,<0.14.3 (#547)
fcf1010 Bump sentencepiece from 0.1.97 to 0.1.98 (#548)
7d04cde Bump mypy from 1.0.1 to 1.2.0 (#555)
fbb068b Bump allenai/beaker-run-action from 1.1 to 1.2 (#534)
a717d70 Bump black from 23.1.0 to 23.3.0 (#554)
42322bb fix readthedocs
825ec19 remove stray pkl file
3638cdc refactor: move packaging information to pyproject.toml (#549)
e86ad65 Bump black from 23.1.0 to 23.3.0 (#543)
69e7574 remove unnecessary code coverage deps (#550)

v1.2.1

What's new

Added 🎉

• Added the following workspace methods to support the Tango viz UI: Workspace.search_registered_runs(), Workspace.search_step_info(), Workspace.num_registered_runs(), and Workspace.num_steps().

Fixed ✅

• Fixes a bug where FromParams would fail to parse when an object takes a Step argument directly.
• Changed a name so we don't override the built-in name set.
• Fixed a bug that would cause O(n^2) memory consumption in dense step graphs.

Commits

258f440 Only one step object (#545)
07abee5 Don't override the name set (#544)
4784bab Return more info from Workspace.search_registered_runs() (#536)
3d2d890 Fix bug when a FromParams object takes a Step argument directly (#535)
38561d0 New paginated workspace search methods (#489)
7a25e3e Fix datasets typing issue (#531)
810b742 minor updates to CI (#529)

v1.2.0

What's new

Added 🎉

• You can now add arguments to steps without invalidating the cache. See Step.SKIP_DEFAULT_ARGUMENTS.
• Fixed integration status messages in tango info command.
• Added abstractions for RemoteClient, RemoteStepCache, and RemoteWorkspace.
• Added a GS integration that comes with GSWorkspace, a remote Workspace implementation that uses google cloud storage.
• You can now bind functional steps to the underlying Step instance with @step(bind=True), meaning the first argument to the function will be a Step.
• Added ShellStep for running arbitrary shell commands.
• Added @make_registrable decorator to make arbitrary functions registrable, to make it easier to refer to them in tango configurations.

Fixed ✅

• Jsonnet parsing is now much faster and works on Windows.
• Warnings about locks are now reliably printed every 30 seconds
• We now make sure Beaker jobs have the latest version of beaker-py, so that we're compatible with the latest API changes.
• Stopping early now works when the metric doesn't change at all.
• Fixed bug with FromParams which didn't handle variable length tuples correctly.

Changed ⚠️

• The default log level for Tango is now warning.
• You can specify multiple steps with -s from the tango run command.

Commits

985f6fa fix lint
f77c0e0 fix release_notes script
2c9456d Prepare for release v1.2.0
f1dc63d Update wandb requirement from <=0.13.5,>=0.12 to >=0.12,<0.13.11 (#523)
32598c4 Update rich requirement from <13.0,>=12.3 to >=12.3,<14.0 (#498)
35ca0f0 Bump actions/checkout from 1 to 3 (#524)
739e40c Various dependencies (#525)
49f3afc Fix bug with variable length tuples (#527)
28ea796 minor workspace fixes (#526)
379095d GCSWorkspace (#417)
c949416 Shell step + Registrable functions (#521)
308689b Allow specifying multiple steps with -s from tango run (#516)
9002169 Stop early same metric (#515)
4a21132 Add bind option to @step decorator (#512)
8a6775e Beaker-py upgrade in Beaker jobs (#509)
679700b Rjsonnet (#505)
3760419 Default log level warning (#508)
8d29321 Lock warning (#506)
521de99 Pin wandb requirement until fixed (#507)
6618a04 upgrades for beaker-py upgrades (#504)
34dbec4 Fix integration status messages in tango info command (#502)
fbb7581 quick fix for Beaker-py upgrade
a10fb3b clone to a src dir (#495)
65f699d Fix #483 (#484)
4a183de Fix bug with extra uncacheable dependencies (#480)
ac0a193 Skip default args (#481)

v1.1.0

What's new

Added 🎉

• Added gpu_type field to StepResources. The BeakerExecutor can use this to determine which clusters to a submit a step to.
• Added machine field to StepResources. You can set this to "local" when using the BeakerExecutor to force it to run the step locally.
• Added --ext-var argument to tango run for setting JSONNET external variables
  when loading the experiment config.
• Added @step() decorator to create Step classes from functions.
• Added the transformers::with_soft_prompt integration, to make soft-prompted prefix transformers easy.

Removed 👋

• Removed PyTorch Lightning integration.
• Removed tango server command and --serve/--no-serve option for tango run.
• Removed source_release.py, which was checked in by accident.

Fixed ✅

• Fixed issue where Executor parallelism option in a Tango settings file would be ignored.
• Fixed a bug where the unique ID of a step that depends on a key-value of the result of another step could change if the name of the other step changes.
• Fixed a bug where importing certain libraries (like torchmetrics) would mess with our exception handling because they set sys.excepthook for some reason. Now we always reset sys.excepthook after importing.
• The type hints for the flax trainer suggested that the training split is optional when in fact it's mandatory.
• Made BeakerWorkspace / BeakerStepLock more robust when a job is preempted.
• Minor performance improvements for the Beaker executor and workspace.

Commits

73bfa86 Soft prompts (#231)
79b7d01 Beaker integration perf improvements
d455541 Train is not optional (#474)
3eab580 Beaker integration perf improvements (#475)
241b4eb Remove step that was never supposed to be there (#478)
5f5ba41 Add @step() decorator (#476)
c0c4ae0 Make BeakerStepLock robust to preempted jobs
39d3d66 Remove tango server (#470)
fef9bba reset sys.excepthook after importing other modules, add --ext-var CLI option (#471)
ccebb4c Automatically remove ephemeral Beaker datasets
81d773e BeakerExecutor improvements, fix bug with StepIndexer (#469)
c05a80a Bump sphinx-copybutton from 0.5.0 to 0.5.1 (#467)
147e408 Remove PyTorch Lightning integration (#468)
cd4f626 Update more-itertools requirement from <9.0,>=8.0 to >=8.0,<10.0 (#456)
d219dff Fix doc build failure
ee71c09 Fix bug with setting parallelism in settings file (#466)
1609eb4 Bump mypy from 0.982 to 0.991 (#465)

v1.0.2

What's new

Changed ⚠️

• BeakerScheduler can now return a list of clusters.

Commits

64215d8 Bump mypy from 0.982 to 0.990 (#463)
acc1082 Update torch requirement from <1.13,>=1.9 to >=1.9,<1.14 (#460)
570d24e Use new constraints field for cluster assignment (#462)
5afd89f Set Beaker client User-Agent header to Tango v*.. (#459)
9d9628f Fix progress logging statement from BeakerExecutor (#458)

v1.0.1

What's new

Fixed ✅

• LightningTrainStep now can take a Lazy model object which results in a gauranteed deterministic hash.
• Fixed issue where remote Workspace implementations like WandbWorkspace and BeakerWorkspace would use the same local cache regardless of the W&B / Beaker workspace
  being used.
• Fixed bug with TorchEvalStep when constructing callbacks.
• Fixed some import error issues caused when an integration is not installed.
• Fix incorrect reporting of final results in MulticoreExecutor.

Changed ⚠️

• Wandb step cache retries api call in case of timeout
• beaker-py >= 1.11 required.

Commits

26e6416 Bump sphinx from 5.2.3 to 5.3.0 (#455)
c10cec5 Retry wandb call (#450)
f950b12 Use separate local cache dirs for each workspace (#451)
4c70161 Uncacheable step failure should be reported (#453)
9476942 Bump black from 22.8.0 to 22.10.0 (#445)
821c0fc Bump furo from 2022.9.15 to 2022.9.29 (#435)
8d0b146 Allow Lazy model input to LightningTrainStep (#448)
12eec56 Fix import issues when missing integrations (#447)
d16997c Fix bug with TorchEvalStep (#442)
9a5f792 Bump beaker-py to version >=1.11 (#443)
f5c2e52 Add to FAQ (#440)

v1.0.0

This is the first stable release of AI2 Tango, the culmination of over a year's work and nearly 1000 commits!

We've been working on this project quietly for a while on the AllenNLP team, and it's being used daily by researchers here. So we're excited to officially announce it now that we're happy with the API 🎉

What's new since the last release

Added 🎉

• Added step_extra_dependencies input field to Step class that can be used to force a dependency on another step even if the current step doesn't directly depend on the output of the other step. See #418 for more context.

Changed ⚠️

• beaker-py >= 1.10 required.

Fixed ✅

• Long log lines will be soft-wrapped to ensure that links are clickable.
• Fixed a bug where some workspaces could be left in a bad state if a step's Format failed to serialize the step's result in Workspace.step_finished().
• Sometimes functions and methods end up as arguments to steps, which means we have to hash them. Instead of taking
  a hash of the function, we now take a hash of the function's module and name.
• Fixed a bug with the Beaker executor where it would hang at the end of a run if a step failed that is a dependency of another step.
• Fixed tests to work with new version of transformers.
• Fixed Executor.execute_sub_graph_for_step() to be able to run the step's dependencies in parallel.

Commits

a48a825 Only install gh in entrypoint if needed (#439)
69b948d Improve error handling for step state edge case (#429)
39b3132 Fix Executor.execute_sub_graph_for_step() (#438)
408944e Loosen range on some dependencies (#437)
76cd46d Follow up fix for #401 (#434)
04e7963 Fix bug with Beaker executor (#430)
fb077e6 Bump fairscale from 0.4.9 to 0.4.11 (#432)
25f7373 Bump furo from 2022.6.21 to 2022.9.15 (#410)
f11925c Bump mypy from 0.971 to 0.982 (#433)
4845317 Bump sphinx from 5.1.1 to 5.2.3 (#431)
572f83b Bump myst-parser from 0.18.0 to 0.18.1 (#426)
e1fc2d3 Add step_extra_dependencies option to Step class (#419)
19927d3 Don't pin protobuf anymore (#428)
3dbe8c7 We need click 8 to work. (#427)
cbdbe68 Hashing functions (#424)
747469f Mark Format serialization failures as step failure (#421)
b1d6431 Ensure base path for included modules kept in sys.path (#406)
e1a1cd1 Minor improvements to BeakerExecutor internals (#416)
d3e1891 Ensure log lines get soft-wrapped so links are always clickable (#415)

v0.14.0

What's new

Added 🎉

• Adds a function to modify a Hugging Face transformer with IA3 adaptors
• Added a BeakerScheduler registrable class, specified as the argument scheduler to BeakerExecutor, which controls the resources assigned to steps ran on Beaker.
  Users can implement their own BeakerScheduler subclasses to customize the resource assignment behavior.

Changed ⚠️

• In the tango run command, --no-server is now the default. Use --server to start the server.

Fixed ✅

• Made BeakerExecutor more robust to connection, timeout, SSL, and other recoverable HTTP errors.
• Made the BeakerStepLock more robust, and as a result BeakerWorkspace is more
  robust and should require less manual intervention for locks in a bad state.
• Fixed a bug with the internal scheduling logic of the BeakerExecutor which
  could delay submitting some steps in parallel.
• Fixed a bug where creating a StepInfo object from params might result in unnecessary imports.
• Fixed a bug where canceling the Beaker executor might not work properly.
• Fixed a bug where the trainer trains too much when train_epochs is set and you're using gradient accumulation.
• Fixed how the results of uncacheable steps are displayed by tango run.
• Beaker executor won't run duplicate cacheable steps at the same time.

Commits

0828adc BeakerExecutor won't run duplicate cacheable steps (#414)
7382019 IA3 adaptors (#403)
d498cf7 Hot fix to final output
bff9ebf Add warning when steps can't be run yet, bug fixes (#408)
c72552e Don't start server by default (#409)
d34fe09 Added BeakerScheduler class for handling resource assignment (#407)
5dcbb56 Gradient accumulation and train_epochs (#402)
d27bbef Make BeakerStepLock more robust (#401)
cd9b5fd Fix bug with StepInfo.from_params(), canceling BeakerExecutor, reserve "ref" name (#400)
6ff6b9e Fix bug with scheduling logic (#399)
15196f2 Deterministic hashing for tensors (#398)
230d78e Make BeakerExecutor more robust to all recoverable errors types (connection, HTTP, SSL, timeout, etc) (#397)
5f63a27 Bump fairscale from 0.4.8 to 0.4.9 (#391)