Skip to content

change: improvement in parquetQuery #63

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 2 commits into
base: master
Choose a base branch
from

Conversation

brennoaf
Copy link

parallel fetch added + logic operators conditions remade for better performance -> return, break + clean
20% faster

Copy link
Collaborator

@platypii platypii left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall looks like a good change, thanks @brennoaf!

I just approved CI. Can you fix the lint and test errors?

@brennoaf
Copy link
Author

Overall looks like a good change, thanks @brennoaf!

I just approved CI. Can you fix the lint and test errors?

yes, i can fix

Copy link
Author

@brennoaf brennoaf left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

tested many times, for some reason, adding a log specifically in 'filter' improves code performance by 25%~.
I left it open for observation

Copy link
Collaborator

@platypii platypii left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Still some linting errors, please run npm run lint to find them

src/query.js Outdated

// logging specifically 'filter' here gave a 25% performance boost, lol
// maybe forces JIT to improve this part of the code
const forceJIT = () => {console.log(filter)}
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is not an acceptable change. Can't be doing console.log in a library. I'm open to suggestions but this feels like a brittle optimization that might not work in different environments? (browser, node, etc)

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I know, I'm investigating a similar and not so grotesque way. But it made me curious, I tested it via express + firefox and it really improved, I'll try to see another way and expand the tests

filteredResults.sort((a, b) => compare(a[orderBy], b[orderBy]))
}
return filteredResults.slice(rowStart, rowEnd)
} else if (orderBy) {
Copy link
Contributor

@severo severo Feb 24, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: this is a breaking change for the corner case where orderBy === '' ('' is a valid object key, but Boolean('') === false)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants