GEP 7 and updates to GEPs 1-5 necessitated by GEP 6

[hmgaudecker]

What problem do you want to solve?

• Add a GEP for the revamped interface
• Update earlier GEPs to reflect the changes that have become necessary after GEP 6 (since our documentation is small, it does not make sense to keep outdated things around).
• Add the finalised schema from Validate params files #880 as an appendix to GEP 3

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 77.57%. Comparing base (8267dbd) to head (3525917).

Additional details and impacted files

@@                        Coverage Diff                        @@
##           collect-components-of-namespaces     #855   +/-   ##
=================================================================
  Coverage                             77.57%   77.57%           
=================================================================
  Files                                   175      175           
  Lines                                  7563     7563           
=================================================================
  Hits                                   5867     5867           
  Misses                                 1696     1696           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[ChristianZimpelmann]

Overall, it looks cool, but I feel like the entry barrier is still quite high for beginners. I made some comments (especially on aspects that I beginners might find complicated)

[ChristianZimpelmann]

The dag does not fit on the page and it is unclear how to scroll to the right.

[hmgaudecker]

I know I know... You can scroll by clicking at the bottom of the graph and dragging the pointer.

[hmgaudecker]

Overall, it looks cool, but I feel like the entry barrier is still quite high for beginners. I made some comments (especially on aspects that I beginners might find complicated)

Thanks! Implemented all of those except for one (no default for main_target(s), it is too important to understand conceptually that you have to request a particular target). Any further concrete suggestions of lowering entry barriers are very welcome!

[ChristianZimpelmann]

Some other observations from playing around in the notebook:

result = main(
    date_str="2025-01-01",
    main_target=MainTarget.results.df_with_mapper,
)

Leads to

ValueError: The following arguments to `main` are missing for computing the desired output:

[
    "('input_data', 'flat')",
]

"flat" seems wrong here.

result = main(
    date_str="2025-01-01",
    main_target=MainTarget.specialized_environment.tax_transfer_dag,
)

Leads to

ValueError: The following data columns are missing.

[ ....

Probably better to respond that the argument input_data is missing completely

• Why does the original_policy_environment obtained from MainTarget.policy_environment contain keys like anzahl_erwachsene_hhorp_id`? It is not clear to me why these are part of the policy environment?

[hmgaudecker]

Some other observations from playing around in the notebook:

Thanks!!!

result = main(
    date_str="2025-01-01",
    main_target=MainTarget.results.df_with_mapper,
)

Leads to

ValueError: The following arguments to `main` are missing for computing the desired output:

[
    "('input_data', 'flat')",
]

"flat" seems wrong here.

It is not wrong, but the message should be improved -- see #1005.

result = main(
    date_str="2025-01-01",
    main_target=MainTarget.specialized_environment.tax_transfer_dag,
)

Leads to

ValueError: The following data columns are missing.

[ ....

Probably better to respond that the argument input_data is missing completely

Agreed, see #1006

• Why does the original_policy_environment obtained from MainTarget.policy_environment contain keys like anzahl_erwachsene_hh orp_id? It is not clear to me why these are part of the policy environment?

As has always been the case, it includes all functions operating on data that are around, like anzahl_erwachsene_hh. In addition, we now have possible input columns, too (essentially a dynamic version of TYPES_INPUT_VARIABLES). Ofc, "policy environment" is too narrow a term for some of these elements, but that has always been the case and I don't have a good term in store to improve upon it. Suggestions welcome, ofc!