Skip to content

feat(as_of): set_input_sparse + transition_formula for sparse forward simulation#1368

Merged
benjello merged 8 commits intomasterfrom
feat/transition-formula
Mar 4, 2026
Merged

feat(as_of): set_input_sparse + transition_formula for sparse forward simulation#1368
benjello merged 8 commits intomasterfrom
feat/transition-formula

Conversation

@benjello
Copy link
Member

@benjello benjello commented Feb 28, 2026

Summary

  • Holder.set_input_sparse(period, idx, vals) — API directe pour passer le diff (idx, vals) sans tableau dense intermédiaire. Économise 2×O(N) par SET dans les boucles de simulation forward.

  • transition_formula — alternative à formula pour les variables as_of. La formule retourne (masque_bool, valeurs) au lieu d'un tableau complet de taille N, permettant un stockage sparse O(k) sans diff interne.

    class Salaire(Variable):
    as_of = "start"

    def transition_formula(person, period, parameters):
        eligible = person("anciennete", period) >= 3
        nouveau  = person("salaire", period.last_month)[eligible] * 1.02
        return eligible, nouveau  # masque + k valeurs (ou scalaire)

    def transition_formula_2024_01_01(person, period, parameters):
        eligible = person("anciennete", period) >= 2
        nouveau  = person("salaire", period.last_month)[eligible] * 1.03
        return eligible, nouveau

    Même dispatch par date que formula_YYYY_MM_DD. Mutuellement exclusif avec formula. Requiert as_of.

Fichiers modifiés

Fichier Changement
variables/config.py Constante TRANSITION_FORMULA_NAME_PREFIX
variables/variable.py Détection, validation, dispatch transition_formula*
simulations/simulation.py _calculate_transition + _run_transition_formula
holders/holder.py _set_as_of_sparse, set_input_sparse, tracking _as_of_transition_computed
tests/core/test_asof_variable.py 6 nouveaux tests set_input_sparse
tests/core/test_transition_formula.py 12 nouveaux tests (nouveau fichier)
benchmarks/test_bench_asof.py set_input_sparse dans le bench forward + TestSetInputSparseVsDense

Test plan

  • pytest tests/core/test_asof_variable.py — 24 tests (dont 6 nouveaux) ✅
  • pytest tests/core/test_transition_formula.py — 12 tests ✅
  • pytest tests/ -q --ignore=tests/web_api — 355 passed, 0 régression ✅

🤖 Generated with Claude Code

@benjello benjello mentioned this pull request Feb 28, 2026
Open
eraviart
eraviart approved these changes Mar 4, 2026
View reviewed changes
Copy link
Member

@eraviart eraviart left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

transition_formula is a big change, but I don't see how to avoid it

@benjello benjello force-pushed the feat/transition-formula branch 2 times, most recently from 479caec to e171424 Compare March 4, 2026 13:54
benjello and others added 8 commits March 4, 2026 14:56
@benjello @claude
feat(as_of): add set_input_sparse and transition_formula for sparse f…
41d4c88 
…orward simulation

## set_input_sparse (Holder API)

New public method `Holder.set_input_sparse(period, idx, vals)` backed by
`_set_as_of_sparse`. Lets callers provide the diff directly (idx, vals)
without building a full N-array, saving 2×O(N) per SET iteration when only
k << N individuals change.

## transition_formula (Variable API)

Alternative to `formula` for `as_of` variables. Instead of returning a full
N-array (which triggers O(N) diff in the holder), the formula returns
`(selector, vals)` describing only who changes and their new values:

    def transition_formula(person, period, parameters):
        eligible = person("anciennete", period) >= 3
        nouveau  = person("salaire", period.last_month)[eligible] * 1.02
        return eligible, nouveau   # bool mask + k values (or scalar)

Supports dated variants (transition_formula_2024_01_01) with the same
dispatch mechanism as formula_*. Mutually exclusive with formula.
Requires as_of to be set.

## Implementation

- variables/config.py: TRANSITION_FORMULA_NAME_PREFIX constant
- variables/variable.py: detect/validate/dispatch transition_formula*
- simulations/simulation.py: _calculate_transition + _run_transition_formula
- holders/holder.py: _as_of_transition_computed tracking set
- tests/core/test_asof_variable.py: 6 new set_input_sparse tests
- tests/core/test_transition_formula.py: 12 new tests
- benchmarks/test_bench_asof.py: set_input_sparse in forward sim bench
  + new TestSetInputSparseVsDense comparison class

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@benjello @claude
feat(as_of): replace single snapshot cursor with multi-snapshot LRU c…
b4309b2 
…ache

- Replace _as_of_snapshot (single tuple) with _as_of_snapshots (OrderedDict)
  keeping up to _as_of_max_snapshots entries (default 3)
- Add _cache_snapshot() helper with LRU eviction (popitem last=False)
- Retroactive SET evicts all snapshots at >= instant instead of just one
- Add Variable.snapshot_count attribute to configure per-variable
- Add MemoryConfig.asof_max_snapshots for simulation-level default
- Resolution order: Variable.snapshot_count > MemoryConfig > 3
- Add 7 tests covering defaults, priority chain, eviction, and correctness

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@benjello @claude
feat(as_of): add initial_formula + fix spiral detection for transitio…
a27e600 
…n_formula

initial_formula:
- Optional counterpart to transition_formula, called once when no base
  snapshot exists to seed the as_of variable's initial state
- Supports date dispatch (initial_formula_YYYY_MM_DD) like transition_formula
- Requires as_of; validated at instantiation time
- If no initial_formula and no set_input, raises ValueError with clear message
- Add INITIAL_FORMULA_NAME_PREFIX constant, set_initial_formulas(),
  parse_initial_formula_name(), get_initial_formula(), has_initial_formula

Tracer / cycle detection fix:
- Replace _check_for_cycle with _check_for_strict_cycle in _calculate_transition
- _check_for_strict_cycle raises CycleError only for same variable@same period
- Removes false SpiralError for legitimate temporal reads (e.g. period.last_month)
- Termination for transition_formula is guaranteed by _as_of_transition_computed

Tests (+8):
- initial_formula establishes base, seeds transition sequence, date dispatch
- initial_formula without as_of raises at instantiation
- no initial_formula + no set_input raises ValueError
- temporal recursion (P reads P-1) no longer falsely triggers SpiralError
- true same-period cycle is caught and state preserved

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@benjello @claude
feat(tracer): add formula_type to TraceNode for as_of visibility
eb85f0b 
- Add formula_type: str | None = None field to TraceNode dataclass
- Add record_formula_type(kind) to FullTracer; no-op on SimpleTracer
- _calculate_transition annotates the current node with "initial" or
  "transition" before running the respective formula
- Add formula_type property to TraceNode Protocol in types.py
- Add 3 tests: initial → "initial", transition → "transition",
  regular formula → None

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@benjello @claude
feat(tracer): add show_formula_type option to computation log
7932c7c 
- Add show_formula_type: bool = False to lines(), print_log(),
  _get_node_log(), and _print_line()
- When True, appends [initial] or [transition] tag to the line for
  as_of nodes; regular formula nodes (formula_type=None) are unaffected
- Add test verifying tags appear/disappear based on the flag

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@benjello @claude
bump: 44.4.1 → 44.5.0
2b16909 
Add CHANGELOG entry covering transition_formula, initial_formula,
multi-snapshot LRU cache, formula_type tracer field, show_formula_type
option, and spiral-detection fix for as_of variables.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@benjello @claude
fix(lint): fix flake8 errors in test and benchmark files
5bed0db 
- Remove unused imports (Holder, period) from test_transition_formula.py
- Rename ambiguous variable l → line in assertion
- Add # noqa: T201 to intentional print in benchmark
- Rename loop variable p → _p to signal it is unused
- Remove unused months_periods variable in forward-simulation benchmark
- Fix stale _as_of_snapshot reference → _as_of_snapshots.clear()

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@benjello
style: black format benchmarks/test_bench_asof.py for CI lint
473ca77 
Made-with: Cursor
@benjello benjello force-pushed the feat/transition-formula branch from e171424 to 473ca77 Compare March 4, 2026 13:57
@benjello benjello merged commit 1a87a41 into master Mar 4, 2026
24 checks passed
@benjello benjello deleted the feat/transition-formula branch March 4, 2026 14:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@eraviart eraviart eraviart approved these changes

@guillett guillett Awaiting requested review from guillett

@benoit-cty benoit-cty Awaiting requested review from benoit-cty

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@benjello @eraviart