Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

unexpected_index_query is not in usable format #10922

Open
suchintakp5 opened this issue Feb 10, 2025 · 1 comment
Open

unexpected_index_query is not in usable format #10922

suchintakp5 opened this issue Feb 10, 2025 · 1 comment
Labels
feature-request feature request

Comments

@suchintakp5
Copy link

Describe the bug
unexpected_index_query returned by the validation result can not be used to get the failed rows unless formatted properly

To Reproduce
Run any expectation in COMPLETE mode to get the unexpected_index_query in the result

Expected behavior
unexpected_index_query returned by the validation result should be in a proper usable format

"unexpected_index_query": "df.filter(F.expr(NOT (ID Number IS NOT NULL)))" can be formatted and used as follows:
"unexpected_index_query": "df.filter(F.expr('NOT (``ID Number`` IS NOT NULL)'))"

Similarly, "unexpected_index_query": "df.filter(F.expr((Country IS NOT NULL) AND (NOT (Country IN (US, GB, Algeria)))))" can be formatted and used as follows:
"unexpected_index_query": "df.filter(F.expr('(Country IS NOT NULL) AND (NOT (Country IN (\"US\", \"GB\", \"Algeria\")))'))"

Environment (please complete the following information):

  • Great Expectations Version: [1.3.5]
  • Data Source: [PySpark DataFrame]
  • Cloud environment: [Azure Databricks]
@adeola-ak adeola-ak moved this from To Do to In progress in GX Core Issues Board Feb 10, 2025
@adeola-ak adeola-ak added the feature-request feature request label Feb 10, 2025
@adeola-ak
Copy link
Contributor

@suchintakp5 Thanks for reaching out! This has come up before, and it's on our radar for consideration and prioritization. I’ll share any updates as soon as I have more info. Thanks again

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
feature-request feature request
Projects
Status: In progress
Development

No branches or pull requests

2 participants