Skip to content

Conversation

@yngve-sk
Copy link
Contributor

@yngve-sk yngve-sk commented Jan 13, 2026

Overview of changes

  • Remove REFCASE, TIME_MAP, and HISTORY_OBSERVATION from ERT
  • Add tool to migrate older ERT configs so that information from REFCASE and TIME_MAP is embedded into the obs config
  • GENERAL_OBSERVATION can now only be used with RESTART, usage of DATE, HOURS, DAYS can be migrated if there is a REFCASE or TIME_MAP in the ERT config.
  • SUMMARY_OBSERVATION can now only be used with DATE, usage of RESTART, HOURS, DAYS can be migrated if there is a REFCASE or TIME_MAP in the ERT config.

Nice to know:

  • Tests that were previously testing now deprecated behaviors have been kept by shimming the obs config migration into the test.
  • Removed test case and logic for asserting that a SUMMARY_OBSERVATION's date matches up with the refcase / time map (this will be hard to pick up in migration, so suggest just dropping it)

Issue
Resolves #12619

Copy link

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR introduces a migration tool to convert deprecated HISTORY_OBSERVATION to SUMMARY_OBSERVATION and remove dependencies on TIME_MAP and REFCASE from observation configurations. The tool aims to help users migrate away from deprecated features as part of issue #12619.

Changes:

  • Adds new ert_config_migrations.py module with observation migration logic
  • Adds convert_observations CLI command to __main__.py

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 18 comments.

File Description
src/ert/config/ert_config_migrations.py New module implementing observation migration logic including text editing, observation conversion, and configuration parsing
src/ert/main.py Adds CLI subcommand convert_observations to invoke the migration tool

@SAKavli SAKavli force-pushed the 26.01.remove-history-obs-with-utility branch 2 times, most recently from cd74709 to d486212 Compare January 13, 2026 12:22
@yngve-sk yngve-sk force-pushed the 26.01.remove-history-obs-with-utility branch 7 times, most recently from b71c665 to 5acb06d Compare January 14, 2026 13:04
@yngve-sk yngve-sk changed the title Migrate timemap/refacse depdendent observations Migrate timemap/refcase depdendent observations Jan 14, 2026
@yngve-sk yngve-sk force-pushed the 26.01.remove-history-obs-with-utility branch 7 times, most recently from 5ec12c5 to 2a46e17 Compare January 15, 2026 06:58
@codecov-commenter
Copy link

codecov-commenter commented Jan 15, 2026

Codecov Report

❌ Patch coverage is 94.64668% with 25 lines in your changes missing coverage. Please review.
✅ Project coverage is 90.63%. Comparing base (f0d9de4) to head (9641e9d).
✅ All tests successful. No failed tests found.

Files with missing lines Patch % Lines
src/ert/config/observation_config_migrations.py 94.08% 23 Missing ⚠️
src/ert/config/_observations.py 93.10% 2 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main   #12629      +/-   ##
==========================================
- Coverage   90.68%   90.63%   -0.05%     
==========================================
  Files         429      430       +1     
  Lines       29808    30068     +260     
==========================================
+ Hits        27030    27253     +223     
- Misses       2778     2815      +37     
Flag Coverage Δ
cli-tests 37.33% <30.83%> (-0.23%) ⬇️
gui-tests 68.70% <32.54%> (-0.66%) ⬇️
performance-and-unit-tests 74.05% <92.93%> (+0.12%) ⬆️
test 37.59% <3.21%> (-0.48%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@codspeed-hq
Copy link

codspeed-hq bot commented Jan 15, 2026

Merging this PR will not alter performance

✅ 22 untouched benchmarks


Comparing yngve-sk:26.01.remove-history-obs-with-utility (90dd989) with main (0ac0ce1)

Open in CodSpeed

@yngve-sk yngve-sk force-pushed the 26.01.remove-history-obs-with-utility branch 3 times, most recently from 900c0e0 to 8bdce57 Compare January 16, 2026 07:27
@yngve-sk yngve-sk force-pushed the 26.01.remove-history-obs-with-utility branch from 8bdce57 to e81be7b Compare January 16, 2026 07:29
@oyvindeide oyvindeide requested a review from Copilot January 18, 2026 20:39
@yngve-sk yngve-sk force-pushed the 26.01.remove-history-obs-with-utility branch 5 times, most recently from 2cc6431 to 05a7c56 Compare January 19, 2026 11:13
@berland berland changed the title Migrate timemap/refcase depdendent observations Migrate timemap/refcase dependent observations Jan 19, 2026
@yngve-sk yngve-sk force-pushed the 26.01.remove-history-obs-with-utility branch from 47fa2f0 to 071d13f Compare January 19, 2026 12:15
declaration = (
f"SUMMARY_OBSERVATION "
f"{obs_row['observation_key']} {{\n"
f" VALUE = {obs_row['observations']};\n"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it is better to add :g to the value and error, to avoid an absolute value of 0.01 getting formatted as 0.099999123456 (it will prefer exponential format sometimes, but that is maybe fine enough here)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It gives some discrepancy in test_that_history_observations_values_are_fetched_from_refcase, not sure anymore... Removed the commit temporarily to just have other tests pass, can add it back if we conclude it should be there.

@yngve-sk yngve-sk force-pushed the 26.01.remove-history-obs-with-utility branch 4 times, most recently from f3a4b9d to 1835d98 Compare January 19, 2026 13:31
@berland
Copy link
Contributor

berland commented Jan 19, 2026

The documentation on HISTORY_OBSERVATION may also be deleted in this PR, as this keyword no longer works.

@yngve-sk yngve-sk force-pushed the 26.01.remove-history-obs-with-utility branch 2 times, most recently from 8ac79ce to c4ebe63 Compare January 19, 2026 13:52
Avoid error from overlapping obs types
Avoid error from overlapping obs names across different response types
History observations and general observations already allow for keys being duplicate downstream, in the observation dataset. Thus, it does not make sense to allow a history obs to create 200 duplicate summary obs keys, but for the equivalent to be disallowed if explicitly done in the ert obs config.
@yngve-sk yngve-sk force-pushed the 26.01.remove-history-obs-with-utility branch from b216a30 to 7561a15 Compare January 19, 2026 14:03
@yngve-sk yngve-sk force-pushed the 26.01.remove-history-obs-with-utility branch 2 times, most recently from 8bd1b85 to 8a34a0b Compare January 19, 2026 14:20
@yngve-sk yngve-sk self-assigned this Jan 19, 2026
@yngve-sk yngve-sk added the release-notes:breaking-change Automatically categorise as breaking change in release notes label Jan 19, 2026
@oyvindeide
Copy link
Collaborator

Can do if that is most ideal? Was not sure whether we should do an inline edit to the ert config and not only the obs config. In the case that someone wants to add a new history obs, and then migrate it, removal of time map / refcase would block that.

Think that should be covered by tests, and it should be removed from snake_oil as that no longer has HISTORY_OBSERVATION.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

release-notes:breaking-change Automatically categorise as breaking change in release notes

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Create conversion tool for HISTORY_OBSERVATION to SUMMARY_OBSERVATION

5 participants