Extended Parmest Capability for weighted SSE objective

[slilonfe5]

Fixes # .

Summary/Motivation:

Currently, the Parmest SSE objective does not support measurements in different units. This work adds a new capability (i.e., weighted SSE) to Parmest to handle measurements in different units.

Changes proposed in this PR:

• Added a new weighted SSE calculation
• Added a new covariance matrix calculation for the weighted SSE objective

Legal Acknowledgement

By contributing to this software project, I have read the contribution guide and agree to the following terms and conditions for my contribution:

1. I agree my contributions are submitted under the BSD license.
2. I represent I am authorized to make the contributions and grant the license. If my employer has rights to intellectual property that includes these contributions, I represent that I have received permission to make contributions and grant the required license on behalf of that employer.

[slilonfe5]

@adowling2 @djlaky

[adowling2]

@slilonfe5 Here is some quick feedback

compute_jacobian function

• Make this a private method by adding _ to the function name
• Add as an argument to the function relative_perturbation
• In the document string, explain this is using forward (?) finite difference
• Add as an argument the solver object. You can make the default Ipopt.

Feedback on the compute_FIM method:

• Add relative_tolerance and solver as arguments
• Also add a check that error_list must be the same length as y_hat_list
• Add a debugging step for the linear algebra error, compute the condition number of the Jacobian matrix and print it out
• Why would you ever get a linear algebra error for just matrix multiplication? Is this check even needed?

[slilonfe5]

@adowling2 @djlaky I also updated the calculation for the normal SSE such that we can use the user-supplied measurement error if defined; otherwise, we calculate the measurement error as usual.

[adowling2]

Nice progress. I think it is time to start writing tests for the new capabilities.

[adowling2]

@slilonfe5 Once you have the tests ready, tag us for feedback. Also, I think you can skip adding this to the depreciated class.

[adowling2]

Here is more feedback as you work on getting this ready for the Pyomo team to review.

[slilonfe5]

@adowling2 @djlaky I have created a separate method (cov_est) for computing the covariance matrix, supporting three calculation methods (jacobian, kaug, and reduced_hessian). I implemented these covariance calculation methods for both the SSE and SSE_weighted objectives. Lastly, as you suggested, I did not add the new capability to the deprecated interface.

I tested these with three examples (2 steady state and 1 dynamic), and all work well. I'm yet to write the test file for these.

[slilonfe5]

@adowling2 @djlaky @mrmundt @blnicho @jsiirola
I have implemented the comments on the parmest.py and test_parmest.py files. I moved the test from the previous test_new_parmest_capabilities.py file to the test_parmest.py file. In the test_parmest.py file, the Rooney-Biegler paper example is used to test the 'SSE' and 'SSE_weighted' objectives for all covariance calculation methods ('reduced_hessian', 'finite_difference', and 'automatic_differentiation_kaug'), as well as the two cases of measurement errors (user-supplied and not supplied by the user). To keep the test file short and clean, I didn't consider the reactor design example. Also, I removed the test for the deprecated interface since this will eventually be removed. The above tests passed without issues.

However, I encountered an issue with the test for other functions, such as bootstrap and likelihood ratio. I plan to discuss it today at the Pyomo Dev meeting.

Thanks.

[slilonfe5]

@adowling2 @djlaky @mrmundt @blnicho @jsiirola Like I mentioned yesterday, this PR is ready for final review. All the tests have passed. Also, based on yesterday's meeting, I will create a new PR to fix the bugs in the current parmest.py file and the examples that fail when running the test_examples.py and test_scenariocreator.py files. The new PR will have to go through before this PR goes through.

[adowling2]

After creating the new PR, please post a link here.

[slilonfe5]

@adowling2 @djlaky @mrmundt @blnicho @jsiirola Here is the link to the new PR on the bug fixes in the experiment_outputs and unknown_parameters checks in the current parmest.py file, and failing examples in the test_examples.py and test_scenariocreator.py files. #3635

[mrmundt]

This seems like a lot of comments, but there are a few repeats! This is looking pretty good so far; most of my questions/suggestions aren't "necessary" but will make the code cleaner.

[mrmundt]

This is a bit redundant. pyo.assert_optimal_termination will already raise a RuntimeError; why bother with raising a different one?

[slilonfe5]

@mrmundt The initial plan for raising a different one was to make the error message more clear. Do you suggest having only the code inside the try block (i.e., not do this in a try-except block)?

[mrmundt]

This is a bit redundant. pyo.assert_optimal_termination will already raise a RuntimeError; why bother with raising a different one?

[mrmundt]

This is a bit redundant. pyo.assert_optimal_termination will already raise a RuntimeError; why bother with raising a different one?

[slilonfe5]

@mrmundt Thank you for your feedback. I will work on them to make the code cleaner.