WIP: enabling hooks for KedroPipelineModel serving #608
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
Just wanted to hint/share what I did, I'll get to update docs and make it proper if we agree to go on with this. |
|
Also worth discussing how #580 (the hooks part) would relate to this PR. |
Updates the requirements on [scikit-learn](https://github.com/scikit-learn/scikit-learn) to permit the latest version. - [Release notes](https://github.com/scikit-learn/scikit-learn/releases) - [Commits](scikit-learn/scikit-learn@0.23.0...1.6.0) --- updated-dependencies: - dependency-name: scikit-learn dependency-type: direct:development ... Signed-off-by: dependabot[bot] <[email protected]>
…>=2.18 and ThreadRunner
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
I really wanted a functionality of pandera hook enabled in the model server, so I implemented what was neccesary to make it happen. This PR is not ready yet, but is a point for discussion on whether/how to implement this properly. Addresses #404
Development notes
Added optional hooks param to KedroPipelineModel, model factory and handling of that in mlflow_hook. Pickling hook classes as artifacts wasn't necessary, as they get saved alongside the pipeline pickle.
Only partial hook functionality was enabled - mostly before/after pipeline, node, dataset hooks - after context created and after catalog created were ommitted, as it would require compromises or mocking. This enables you to write your own hook that uses those endpoints to include it alongside the model in the server.
Tested it with my own project and KedroPandera hook - also copying metadata of catalog entries was necessary to make it work.
Checklist
CHANGELOG.mdfile. Please respect Keep a Changelog guidelines.Notice
I acknowledge and agree that, by checking this box and clicking "Submit Pull Request":
I submit this contribution under the Apache 2.0 license and represent that I am entitled to do so on behalf of myself, my employer, or relevant third parties, as applicable.
I certify that (a) this contribution is my original creation and / or (b) to the extent it is not my original creation, I am authorised to submit this contribution on behalf of the original creator(s) or their licensees.
I certify that the use of this contribution as authorised by the Apache 2.0 license does not violate the intellectual property rights of anyone else.