Endogenize investments

[hmgaudecker]

Endogeneity of investments means that concurrent states influence the choice of investments. Skillmodels currently has no way of including that.

In this PR, we achieve it this by modelling this through transition functions. When investment factors are present, each period is split into two sub-periods. The first sub-period is for states, which can influence investments in the second sub-period.

• Update model processing, allowing for an entry is_investment for factors. Add periods_raw (in the original data) in addition to periods (the doubled-up version used internally by skillmodels)
• Stack data, appropriately including measurements in sub-periods
• Add constraints so that factors are just carried along through sub-periods where state/investment is not relevant

Limitations:

• There can be no overlap between measurements for states and for investments because to skillmodels, they live in disjunct periods
• Anchoring is not supported when investment factors are present
• Stages are not supported when investment factors are present (would require extra logic for the constraints, might e doable without much work)

[codecov]

Codecov Report

❌ Patch coverage is 89.22687% with 85 lines in your changes missing coverage. Please review.
✅ Project coverage is 91.03%. Comparing base (8417b4b) to head (cc710a6).

Files with missing lines	Patch %	Lines
src/skillmodels/correlation_heatmap.py	25.00%	24 Missing ⚠️
src/skillmodels/transition_functions.py	50.00%	13 Missing ⚠️
src/skillmodels/visualize_transition_equations.py	62.06%	11 Missing ⚠️
src/skillmodels/simulate_data.py	70.00%	9 Missing ⚠️
src/skillmodels/constraints.py	95.03%	7 Missing ⚠️
src/skillmodels/process_data.py	88.23%	6 Missing ⚠️
src/skillmodels/process_model.py	94.25%	5 Missing ⚠️
src/skillmodels/utilities.py	64.28%	5 Missing ⚠️
src/skillmodels/check_model.py	88.88%	3 Missing ⚠️
src/skillmodels/visualize_factor_distributions.py	81.81%	2 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main      #80      +/-   ##
==========================================
- Coverage   91.43%   91.03%   -0.40%     
==========================================
  Files          40       42       +2     
  Lines        3070     3581     +511     
==========================================
+ Hits         2807     3260     +453     
- Misses        263      321      +58     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[janosg]

Why are endogenous factors not compatible with anchoring? And why not with stages? Especially stages are so orthogonal to everything else (they are just fixed constraints) that I don't think it would be hard.

For example, if you had chosen to implement all the logic for endogenous factors by transforming the user written model dict, the new stages could have been:

new_stagemap = list(np.repeat(old_stagemap, 2))

[review-notebook-app]

Check out this pull request on 

See visual diffs & provide feedback on Jupyter Notebooks.

---

Powered by ReviewNB

[hmgaudecker]

Autoreview.