189 only use allowed values for imputation ratios

[AnneONS]

Pull Request submission

This branch fulfills the methods conditions:

• "Matched pairs must comprise of clean, not null, respondent non-zero data for both the target and predictive variables."
• (Matched pair : there is data for both periods within the imputation class.)

"respondent" data means that the following values only are allowed in the "status" column for calculating the imputation link:

"Clear", "Clear - overridden", "Clear - overridden SE"

To check for non zero and not null data the function flag_nulls_and_zeros() was created with a unit test.

This is called within loop_unique(), and the condition on statuses is applied here as well.

To check:

Maybe someone can check whether the condition on the statuses is addequately tested, it's included in the updated test for loop_unique(), but it's not tested in forward_imputation(), backwards_imputation() or runimputation()

Someone can add this in if they feel it would be helpful.

Closes or fixes

• Detail the ticket(s) you are closing with this PR
  Closes Changed instance dtype in toml to float64; in construction, zfill for… #189

Code

• Code runs The code runs on my machine and/or CDSW
• Conflicts resolved There are no conflicts (I have performed a rebase if necessary)
• Requirements My/our code functions according to the requirements of the ticket
• Dependencies I have updated the environment yaml so it includes any new libraries I have used
• Configuration file updated any high level parameters that the user may interact with have been put into the config file (and imported to the script)
• Clean Code
  • Code is as PEP 8 compliant as I can humanly make it
  • Code passess flake8 linting check
  • Code adheres to DRY
• Type hints All new functions have type hints

Documentation

Any new code includes all the following forms of documentation:

• Function Documentation Docstrings within the function(s')/methods have been created
  • Includes Args and returns for all major functions
  • The docstring details data types
• Updated Documentation: User and/or developer working doc has been updated

Data

• All data needed to run this script is available in Dev/Test
• All data is excluded from this pull request
• Secrets checker pre-commit passes

Testing

• Unit tests Unit tests have been created and are passing or a new ticket to create tests has been created

---

Peer Review Section

• All requirements install from (updated) environment.yaml
• Documentation has been created and is clear - check the working document
• Doctrings (Google format) have been created and accurately describe the function's functionality
• Unit tests pass, or if not present a new ticket to create tests has been created
• Code runs The code runs on reviewer's machine and/or CDSW

Final approval (post-review)

The author has responded to my review and made changes to my satisfaction.

• I recommend merging this request.

---

Review comments

Insert detailed comments here!

These might include, but not exclusively:

• bugs that need fixing (does it work as expected? and does it work with other code
  that it is likely to interact with?)
• alternative methods (could it be written more efficiently or with more clarity?)
• documentation improvements (does the documentation reflect how the code actually works?)
• additional tests that should be implemented (do the tests effectively assure that it
  works correctly?)
• code style improvements (could the code be written more clearly?)
• Do the changes represent a change in functionality so the version number should increase? Start a discussion if so.
• As a review you can generates the same outputs from running the code

Your suggestions should be tailored to the code that you are reviewing.
Be critical and clear, but not mean. Ask questions and set actions.

[github-actions]

Detailed Coverage Report

File	Stmts	Miss	Cover	Missing
src
__init__.py	0	0	100%
src/aggregation
__init__.py	0	0	100%
src/estimation
__init__.py	0	0	100%
src/imputation
__init__.py	0	0	100%
imputation.py	104	0	100%
src/outlier_detection
__init__.py	0	0	100%
src/outputs
__init__.py	0	0	100%
src/staging
__init__.py	0	0	100%
history_loader.py	32	2	93%	42, 54
spp_parser.py	14	0	100%
spp_snapshot_processing.py	33	0	100%
validation.py	80	12	85%	45–46, 69, 103, 106, 138, 174, 181, 212–213, 219–220
src/utils
__init__.py	0	0	100%
helpers.py	25	5	80%	14–15, 19–20, 22
local_file_mods.py	46	3	93%	116–118
TOTAL	334	22	93%

Summary of tests

Tests	Skipped	Failures	Errors	Time
40	0 💤	0 ❌	0 🔥	2.459s ⏱️

[okudum]

Code looks good and unit tests run.

[okudum]

Code looks good, I think, it is ready for merge

[jwestw]

The code looks good, but is it correct that the new function flag_nulls_and_zeros is not called?