Skip to content

Add itables & .interactive attribute #80

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 10 commits into from
Jun 25, 2025
Merged

Add itables & .interactive attribute #80

merged 10 commits into from
Jun 25, 2025

Conversation

charles-turner-1
Copy link
Collaborator

@charles-turner-1 charles-turner-1 commented Jun 4, 2025

See also https://github.com/intake/intake-esm/pull/723/files.

  • Added a .interactive property, aiming to keep everything as consistent as possible.

  • Also fixed a bug where passing columns_with_iterables as a string wouldn't break anything at load time but would fail when we start searching.

@charles-turner-1 charles-turner-1 requested a review from Copilot June 5, 2025 04:58
Copy link

@Copilot Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR introduces support for interactive table displays via itables while ensuring that columns_with_iterables can accept both string and list inputs in the catalog and search functionalities. Key changes include:

  • Extending tests to cover a case where "columns_with_iterables" is provided as a string.
  • Updating type annotations for parameters in the constructor and search functions.
  • Adding an interactive property that leverages itables for displaying the catalog.

Reviewed Changes

Copilot reviewed 8 out of 8 changed files in this pull request and generated no comments.

File Description
tests/test_core.py Expanded test cases to validate both string and list inputs for kwargs.
src/intake_dataframe_catalog/core.py Updated constructor type hints and added an interactive property to display the catalog.
src/intake_dataframe_catalog/_search.py Refined type annotation for columns_with_iterables and added conversion for string inputs.
pyproject.toml; ci/environment-*.yml Added itables as a dependency for both development and production environments.
Comments suppressed due to low confidence (1)

src/intake_dataframe_catalog/core.py:598

  • The 'interactive' property returns the result of itables.show(self._df), which may produce side effects or unexpected results upon simple property access. Consider converting this property into a method (e.g., 'show_interactive') or update the return type annotation to clearly indicate its intent.
def interactive(self) -> None:

Copy link
Collaborator

@dougiesquire dougiesquire left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is really neat. To make it easily useable, I think we want to:

  • remove the yaml_column column from the interactive table
  • have the name_column column as the first column

Hopefully this isn't too hard?

@dougiesquire
Copy link
Collaborator

Also, in quickstart.ipynb, the interactive search gives me different answers than cat.search?

Screenshot 2025-06-06 at 3 57 26 pm

but

Screenshot 2025-06-06 at 3 57 54 pm

@charles-turner-1
Copy link
Collaborator Author

This is really neat. To make it easily useable, I think we want to:

  • remove the yaml_column column from the interactive table
  • have the name_column column as the first column

Hopefully this isn't too hard?

I think removing the yaml column is straightforward - trying to group by name might get a bit messier.

Also, in quickstart.ipynb, the interactive search gives me different answers than cat.search?

I think this might be a quirk of the column types, but good catch. I'll dig into it properly.

@dougiesquire
Copy link
Collaborator

trying to group by name might get a bit messier.

I don't even mean grouping by name. I just mean having it as the first column. Not critical though

@charles-turner-1
Copy link
Collaborator Author

charles-turner-1 commented Jun 6, 2025

Turned out to be straightforward to fix.

I don't even mean grouping by name. I just mean having it as the first column. Not critical though

This fire problem on the other hand, seems to be related to the following:
Screenshot 2025-06-06 at 5 14 58 pm

I assume these too-long lists are being truncated & then ignored for JavaScript reasons. It's straightforwardly fixable by just exploding the lists - see below.

Screenshot 2025-06-06 at 5 22 10 pm

I'm not sure I like the explode the lists solution all that much, but I also don't know how else we can solve it.

@charles-turner-1
Copy link
Collaborator Author

charles-turner-1 commented Jun 6, 2025

We can also add a little widget like this to the top of the table, that lets users click through to select whatever they're after:

Screenshot 2025-06-06 at 5 26 09 pm

And then interactively filter:

Screenshot 2025-06-06 at 5 27 51 pm

I assume the CSS is getting screwed up because I'm using a notebook in VS Code and it wouldn't chop the top if I opened a browser jupyterlab.

@charles-turner-1
Copy link
Collaborator Author

Updated this to check all iterable columns and 'jointly explode' them if they all contain lists which have the same length lists in each record - functionality contained in the MinimalExploder class.

Kind of hard to explain what this means with words, but the tests should hopefully make the intent clear.

@dougiesquire
Copy link
Collaborator

Looks really great @charles-turner-1 . However, I found the widget unhelpful, mostly because of how much it truncated things. This was in browser jupyterlab:

Screenshot 2025-06-23 at 4 52 25 pm

Could we just remove for now?

@charles-turner-1
Copy link
Collaborator Author

Without the widget, there's no way to search on multiple fields at once - I haven't given much thought as to whether this is something that users are going to want? Happy to remove it if it's probably not something users will want - otherwise I might see if the package exposes a way to fiddle with the CSS and fix the truncation.

@dougiesquire
Copy link
Collaborator

there's no way to search on multiple fields at once

Good point. That probably is something that users would fine useful....

@charles-turner-1
Copy link
Collaborator Author

Cool - I'll dig into the CSS, I think we should be able to force the widgets to be wider.

@charles-turner-1
Copy link
Collaborator Author

charles-turner-1 commented Jun 23, 2025

Could you give me a snippet of code that reproduces those truncated widgets @dougiesquire, as well as the resolution/orientation of the screen you're opening it on?

I've opened up an interactive catalog in an online Jupyter instance and it looks fine on my laptop screen (see below, 13 inch Macbook, 'more space' setting). This frontend-y stuff can be a bit of a nightmare to debug/reproduce/test.

Screenshot 2025-06-23 at 5 21 35 pm

@dougiesquire
Copy link
Collaborator

Could you give me a snippet of code that reproduces those truncated widgets @dougiesquire, as well as the resolution/orientation of the screen you're opening it on?

I was just running the example in the quickstart.ipynb. Screen is 2560x1440 in landscape. It looks great in your screen grab. Does it also look good for you on a monitor?

@charles-turner-1
Copy link
Collaborator Author

Cool, I'll have a poke at the quickstart and see if I can track down any issues. Don't have a monitor to hand, but when I was working on this at home I didn't see any issues. Maybe @marc-white could try if he's in the office?

@charles-turner-1
Copy link
Collaborator Author

I've had a look, and the quickstart guide renders fine on my Macbook screen:

  • on Brave
Screenshot 2025-06-24 at 9 26 13 am
  • and on Safari
Screenshot 2025-06-24 at 9 33 57 am

Are you using Safari or a Chromium based browser? Or something else?
Couple of things it might be worth trying:

  • Can you either clear the browser cache or run the notebook from a tab in incognito mode and see if it renders differently?
  • Have you got any kind of javascript blocker/ ad blocker / etc running? If so, can you disable that & see if it changes how it renders?

If none of those work, maybe we should add a note that the interactive viewer might have rendering issues, depending on browser related stuff.

@dougiesquire
Copy link
Collaborator

Are you using Safari or a Chromium based browser? Or something else?

I was using Firefox. And the output is still truncated in a private session without any extensions active...

But, things also look good to me with Safari, so shall we just add a note as you suggest and move on?

@charles-turner-1
Copy link
Collaborator Author

Interesting. Looks like firefox uses a different browser engine (not Webkit or Chromium), so must be something related to that.

I'll add a note here, and also on the intake catalog docs.

Copy link
Collaborator

@dougiesquire dougiesquire left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Look great - thanks @charles-turner-1. I think users are going to find this very useful

@charles-turner-1 charles-turner-1 merged commit 9ae8579 into main Jun 25, 2025
31 checks passed
@charles-turner-1 charles-turner-1 deleted the itables branch June 25, 2025 23:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants