-
Notifications
You must be signed in to change notification settings - Fork 33
182 erroneous read only on server start #308
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
…ptyDB This test runs a query with 2 FindImage commands on a new/empty PMGD db instance and validates the response contains the excepted message `"No entities found"`. Signed-off-by: Steven Rojas <[email protected]>
…o avoid internal PMGD validation check exception A read-write validation check is performed internally by PMGD. When running a query node/edge transaction with a readOnly flag a validation check will trigger an exception. The exception message will be propagated to the query response. This change allows the validation to pass while technically the transaction is still a readOnly operation. The validation occurs in the PMGD codebase, updating it is not reasonable at this time. Signed-off-by: Steven Rojas <[email protected]>
bb72ec8 to
fdc6dcc
Compare
|
Target CPP Coverage: 71.4335% Target Python Coverage: 97.94% |
Signed-off-by: Steven Rojas <[email protected]>
|
Target CPP Coverage: 71.4335% Target Python Coverage: 97.94% |
|
Since the code coverage was reduced by more than 0.1% with these changes, the following screenshot summarizes the line numbers that tests are covering (new one on the left, original on the right): Here's some references to the code lines that are not been executed by the tests: Line 89 in 6ce7c3a
PMGDQueryHandler.cc (200-203): Line 200 in 6ce7c3a
QueryHandlerPMGD.cc (368-383): Line 368 in 6ce7c3a
As you can notice, all of the code that is no longer being executed by the tests is error handling code. This code was being executed before due to the exception thrown at https://github.com/omp87/pmgd/blob/f52a0b3c2c661bb07ae3218214557165f72a0b22/src/TransactionImpl.h#L189, which is exactly what this PR tries to fix, even though we don't have control over the pmgd codebase. Since the problem is fixed and this exception is never being reached, we would expect the error handling code to never run.
This is how the output would look like when this error comes up. |
Signed-off-by: Lacewell, Chaunte W <[email protected]>
* Change coverage reports to PR #308; Set gcovr to version of prev. scans; Signed-off-by: Lacewell, Chaunte W <[email protected]> * Upgrade gcovr (Changes coverage results) Signed-off-by: Lacewell, Chaunte W <[email protected]> * Update expected coverage Signed-off-by: Lacewell, Chaunte W <[email protected]> --------- Signed-off-by: Lacewell, Chaunte W <[email protected]>
Signed-off-by: cwlacewe <[email protected]> Co-authored-by: Rohit Verma <[email protected]> Co-authored-by: pre-commit-ci-lite[bot] <117423508+pre-commit-ci-lite[bot]@users.noreply.github.com> Co-authored-by: Chaunte W. Lacewell <[email protected]> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> * Updates for release (#279) * Updates for release Signed-off-by: Lacewell, Chaunte W <[email protected]> * Update setup script Signed-off-by: Lacewell, Chaunte W <[email protected]> * Update coverage from Orchestration PR Signed-off-by: Lacewell, Chaunte W <[email protected]> --------- Signed-off-by: Lacewell, Chaunte W <[email protected]> * Replace non-inclusive terminology (#284) * Update doxyfile comment, change name of main node detail to ControlPlaneNodeDetail Signed-off-by: Lacewell, Chaunte W <[email protected]> * Updated references of `master` node to `primary` or `control plane` Signed-off-by: Lacewell, Chaunte W <[email protected]> * disable to deactivate excluding external references Signed-off-by: Lacewell, Chaunte W <[email protected]> * dummy to placeholder Signed-off-by: Lacewell, Chaunte W <[email protected]> * kill to stop excluding commands Signed-off-by: cwlacewe <[email protected]> * Automated coverage update --------- Signed-off-by: Lacewell, Chaunte W <[email protected]> Signed-off-by: cwlacewe <[email protected]> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> * Neo4j configurable number of clients connection (#287) * Update Neo4j query handler to make number of client connections configurable Signed-off-by: Steven Rojas <[email protected]> * Update config keyword to `neo4j_conn_pool_sz` Signed-off-by: Steven Rojas <[email protected]> * Automated coverage update * Automated coverage update --------- Signed-off-by: Steven Rojas <[email protected]> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Chaunte W. Lacewell <[email protected]> * Bump flask from 3.1.0 to 3.1.1 in /.github (#288) Bumps [flask](https://github.com/pallets/flask) from 3.1.0 to 3.1.1. - [Release notes](https://github.com/pallets/flask/releases) - [Changelog](https://github.com/pallets/flask/blob/main/CHANGES.rst) - [Commits](pallets/flask@3.1.0...3.1.1) --- updated-dependencies: - dependency-name: flask dependency-version: 3.1.1 dependency-type: direct:production ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Improve documentation (#286) * Transfer wiki to mkdocs Signed-off-by: cwlacewe <[email protected]> * Add doc deploy to workflow Signed-off-by: cwlacewe <[email protected]> * Change trigger of update docs to push Signed-off-by: cwlacewe <[email protected]> * Remove doc artifact Signed-off-by: cwlacewe <[email protected]> * TESTING ONLY: Add push to branch as trigger Signed-off-by: cwlacewe <[email protected]> * mod update doc workflow, add dark/light mode to mkdocs.yml Signed-off-by: cwlacewe <[email protected]> * Automated coverage update * Change theme to readthedocs and add DeepWiki links Signed-off-by: cwlacewe <[email protected]> * Automated coverage update * Organize mkdocs and add req file Signed-off-by: cwlacewe <[email protected]> * Fix workflow Signed-off-by: cwlacewe <[email protected]> * Automated coverage update * Update guides Signed-off-by: cwlacewe <[email protected]> * Rename guide section Signed-off-by: cwlacewe <[email protected]> * Format updates 1 and re-structure files Signed-off-by: cwlacewe <[email protected]> * Format update Signed-off-by: cwlacewe <[email protected]> * Pull latest wiki updates Signed-off-by: cwlacewe <[email protected]> * Change references to wiki to site https://intellabs.github.io/vdms/; remove INSTALL.md and use version in docs/ Signed-off-by: cwlacewe <[email protected]> * Cleanup; Update trigger for pushes to develop and master Signed-off-by: cwlacewe <[email protected]> * Automated coverage update * Automated coverage update * Spelling fixes and add conditional statement to job pushing documentation (added test trigger for branch) Signed-off-by: Lacewell, Chaunte W <[email protected]> * fix workflow condition Signed-off-by: Lacewell, Chaunte W <[email protected]> * fix workflow command Signed-off-by: Lacewell, Chaunte W <[email protected]> * fix workflow command Signed-off-by: Lacewell, Chaunte W <[email protected]> * fix workflow command Signed-off-by: Lacewell, Chaunte W <[email protected]> * fix workflow command Signed-off-by: Lacewell, Chaunte W <[email protected]> * fix workflow command Signed-off-by: Lacewell, Chaunte W <[email protected]> * remove test text Signed-off-by: Lacewell, Chaunte W <[email protected]> * Remove test trigger Signed-off-by: Lacewell, Chaunte W <[email protected]> * Remove collections reference Signed-off-by: Lacewell, Chaunte W <[email protected]> * Removed *_index.md pages under commands to allow easier navigation with next and prev buttons: Fix incorrect link to Kubernetes guide Signed-off-by: Lacewell, Chaunte W <[email protected]> * Automated coverage update --------- Signed-off-by: cwlacewe <[email protected]> Signed-off-by: Lacewell, Chaunte W <[email protected]> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> * S3 Connection Logging: Copy-Paste errors (#290) * Update S3 connection error logs Signed-off-by: Steven Rojas <[email protected]> * Minor updates to error logs Signed-off-by: Steven Rojas <[email protected]> * Automated coverage update --------- Signed-off-by: Steven Rojas <[email protected]> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> * Integrating shared connection logic for PMGD * Automated coverage update * Automated coverage update * initial implementation of filter library API and memory management for filters (#292) * initial implementation of filter library API and memory management for filters * refactor intial implementation into classes and derived classes * [pre-commit.ci lite] apply automatic fixes * Implement add functionality for filter with Cuckoo path logic and displacement * [pre-commit.ci lite] apply automatic fixes * complete Filter implementation * [pre-commit.ci lite] apply automatic fixes * add unit tests * [pre-commit.ci lite] apply automatic fixes * adding more unit tests * [pre-commit.ci lite] apply automatic fixes * make signature longer for collisions and increase coverage for tests * [pre-commit.ci lite] apply automatic fixes * Automated coverage update --------- Co-authored-by: pre-commit-ci-lite[bot] <117423508+pre-commit-ci-lite[bot]@users.noreply.github.com> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Chaunte W. Lacewell <[email protected]> * add test cases for collection manager (#304) * add test cases for collection manager * [pre-commit.ci lite] apply automatic fixes * Automated coverage update --------- Co-authored-by: pre-commit-ci-lite[bot] <117423508+pre-commit-ci-lite[bot]@users.noreply.github.com> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> * 297 transaction succeeds despite exception (#302) * Include query operation failure tests Signed-off-by: Steven Rojas <[email protected]> * Check for query errors while performing operations Signed-off-by: Steven Rojas <[email protected]> * Check for query errors while performing Video operations Signed-off-by: Steven Rojas <[email protected]> * Update Image and Video tests to expect exception on invalid operations Signed-off-by: Steven Rojas <[email protected]> * Automated coverage update --------- Signed-off-by: Steven Rojas <[email protected]> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> * 295 addget fast filters integration (#305) * Add and find filter stubs * filter stubs updating * Initial add filter implentation completed, WiP * Fixed missing defintion in filter.h, initial compilation of add get and list complete for filters. Moving to testing... * Tests, WiP, currently working, but need more cases * Basic test integration * [pre-commit.ci lite] apply automatic fixes * Additional test, removal of debug output * Fixing merge issue in cmakelist, test tweak addition, reduce noisiness slightly * Automated coverage update * Automated coverage update * Update include/vcl/Filter.h Co-authored-by: Chaunte W. Lacewell <[email protected]> * Automated coverage update --------- Co-authored-by: kfadams <[email protected]> Co-authored-by: pre-commit-ci-lite[bot] <117423508+pre-commit-ci-lite[bot]@users.noreply.github.com> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Chaunte W. Lacewell <[email protected]> * Create intel_security_scan.yaml * 196 objects not cleaned up on transaction failure (#306) * Include S3 transaction rollback test This test counts the number of objects stored in S3, then executes a query that would cause a storage leak and finally makes sure the number of objects stored in S3 is still the same. Signed-off-by: Steven Rojas <[email protected]> * Implement S3 transaction rollback for PMGD handler In case of failure, the exception handler will make sure all added objects within the current query context are removed from S3. Signed-off-by: Steven Rojas <[email protected]> * Implement S3 transaction rollback for Neo4j handler In case of failure, all added objects within the current query context are removed from S3. Signed-off-by: Steven Rojas <[email protected]> * Automated coverage update --------- Signed-off-by: Steven Rojas <[email protected]> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> * 182 erroneous read only on server start (#308) * Update ImageTransactionRollback test and include new test FindImageEmptyDB This test runs a query with 2 FindImage commands on a new/empty PMGD db instance and validates the response contains the excepted message `"No entities found"`. Signed-off-by: Steven Rojas <[email protected]> * Update QueryNode and QueryEdge protobuf construct handler read mode to avoid internal PMGD validation check exception A read-write validation check is performed internally by PMGD. When running a query node/edge transaction with a readOnly flag a validation check will trigger an exception. The exception message will be propagated to the query response. This change allows the validation to pass while technically the transaction is still a readOnly operation. The validation occurs in the PMGD codebase, updating it is not reasonable at this time. Signed-off-by: Steven Rojas <[email protected]> * Restore previous test changes Signed-off-by: Steven Rojas <[email protected]> --------- Signed-off-by: Steven Rojas <[email protected]> * Adding CPP CodeQL Scanning (#309) * Add files via upload * [pre-commit.ci lite] apply automatic fixes --------- Co-authored-by: pre-commit-ci-lite[bot] <117423508+pre-commit-ci-lite[bot]@users.noreply.github.com> * Override coverage (#310) * Change coverage reports to PR #308; Set gcovr to version of prev. scans; Signed-off-by: Lacewell, Chaunte W <[email protected]> * Upgrade gcovr (Changes coverage results) Signed-off-by: Lacewell, Chaunte W <[email protected]> * Update expected coverage Signed-off-by: Lacewell, Chaunte W <[email protected]> --------- Signed-off-by: Lacewell, Chaunte W <[email protected]> * Bump protobuf from 4.24.2 to 4.25.8 in /client/python (#299) * Bump protobuf from 4.24.2 to 4.25.8 in /client/python Bumps [protobuf](https://github.com/protocolbuffers/protobuf) from 4.24.2 to 4.25.8. - [Release notes](https://github.com/protocolbuffers/protobuf/releases) - [Changelog](https://github.com/protocolbuffers/protobuf/blob/main/protobuf_release.bzl) - [Commits](protocolbuffers/protobuf@v4.24.2...v4.25.8) --- updated-dependencies: - dependency-name: protobuf dependency-version: 4.25.8 dependency-type: direct:production ... Signed-off-by: dependabot[bot] <[email protected]> * Upgrade protobuf every else in code Signed-off-by: Lacewell, Chaunte W <[email protected]> * Automated coverage update --------- Signed-off-by: dependabot[bot] <[email protected]> Signed-off-by: Lacewell, Chaunte W <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Lacewell, Chaunte W <[email protected]> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> * Convert CodeQL Scan from Basic to Advanced (#311) * Update codeql_analysis.yml * [pre-commit.ci lite] apply automatic fixes * Automated coverage update --------- Co-authored-by: pre-commit-ci-lite[bot] <117423508+pre-commit-ci-lite[bot]@users.noreply.github.com> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> * Upgrade protobuf to 5.29.5 (#312) * Upgrade protobuf to 5.29.5 Signed-off-by: Lacewell, Chaunte W <[email protected]> * Change google test version (compatibility issue with v1.17.0) Signed-off-by: Lacewell, Chaunte W <[email protected]> * downgrade google test version to v1.13.0 for neo4j Signed-off-by: Lacewell, Chaunte W <[email protected]> * Revert to googletest commit 4c9a3bb62bf3ba1f1010bf96f9c8ed767b363774 Signed-off-by: Lacewell, Chaunte W <[email protected]> * Automated coverage update * Update googletest version to one used in protobuf v29.5 Signed-off-by: Lacewell, Chaunte W <[email protected]> * Automated coverage update --------- Signed-off-by: Lacewell, Chaunte W <[email protected]> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> * Release requirements (#315) * Update requirements.txt Signed-off-by: Lacewell, Chaunte W <[email protected]> * Update actions to latest versions Signed-off-by: Lacewell, Chaunte W <[email protected]> * Automated coverage update * Update UDF requirements.txt Signed-off-by: Lacewell, Chaunte W <[email protected]> * Automated coverage update --------- Signed-off-by: Lacewell, Chaunte W <[email protected]> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> * Update codeql_analysis.yml (#318) * Update codeql_analysis.yml Removing security and quality query from CodeQL to reduce the noise in the findings. * Automated coverage update --------- Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> --------- Signed-off-by: dependabot[bot] <[email protected]> Signed-off-by: cwlacewe <[email protected]> Signed-off-by: Lacewell, Chaunte W <[email protected]> Signed-off-by: Steven Rojas <[email protected]> Co-authored-by: Ragaad <[email protected]> Co-authored-by: Michael Beale <[email protected]> Co-authored-by: Ian Adams <[email protected]> Co-authored-by: sys_vdms <[email protected]> Co-authored-by: Rohit Verma <[email protected]> Co-authored-by: Sameh Gobriel <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Stewart Blacklock <[email protected]> Co-authored-by: Rohit Verma <[email protected]> Co-authored-by: s-gobriel <[email protected]> Co-authored-by: pre-commit-ci-lite[bot] <117423508+pre-commit-ci-lite[bot]@users.noreply.github.com> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: rolandoquesada <[email protected]> Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com> Co-authored-by: Rolando Quesada <[email protected]> Co-authored-by: 100rish <[email protected]> Co-authored-by: StevenRojasC <[email protected]> Co-authored-by: kfadams <[email protected]>


This PR fixes an unexpected query response containing an exception message when running a Query Edge/Node transaction on a new/empty PMGD db instance.
Closes #182
Closes #307