Skip to content

Implement concat_json to join JSON strings given by strings column #2457

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 46 commits into from
Oct 10, 2024
Merged

Implement concat_json to join JSON strings given by strings column #2457

merged 46 commits into from
Oct 10, 2024

Conversation

ttnghia
Copy link
Collaborator

@ttnghia ttnghia commented Sep 30, 2024

This adds concat_json to join multiple JSON strings given by strings column into one unified JSON string. Along with the concatenated data buffer, the result also contains a vector that indicates whether each row in the input is null or empty, and the delimiter character used for concatenation.

karthikeyann and others added 21 commits August 29, 2024 23:35
@karthikeyann
add concat_jsons
cd235e2
@ttnghia
Fix compile error
c1c022e 
Signed-off-by: Nghia Truong <[email protected]>
@ttnghia
Optimize stream sync
033f07e 
Signed-off-by: Nghia Truong <[email protected]>
@ttnghia
Fix interface
6d86005 
Signed-off-by: Nghia Truong <[email protected]>
@ttnghia
Add JNI binding
02c7636 
Signed-off-by: Nghia Truong <[email protected]>
@ttnghia
Merge branch 'branch-24.10' into concat_json
fbe9620
@ttnghia
Change delimiter Java type
ec2e877 
Signed-off-by: Nghia Truong <[email protected]>
@ttnghia
Fix null mask
2768577 
Signed-off-by: Nghia Truong <[email protected]>
@ttnghia
Return is_valid column
081a14e 
Signed-off-by: Nghia Truong <[email protected]>
@ttnghia
Separate input and output validity
c41b947 
Signed-off-by: Nghia Truong <[email protected]>
@ttnghia
Merge branch 'branch-24.10' into concat_json
ccb6b4c
@ttnghia
Make changes in cudf's Java code
ed1b87e
@ttnghia
Merge branch 'branch-24.10' into concat_json
0cbb8a4 
# Conflicts:
#	thirdparty/cudf
@ttnghia
Merge branch 'branch-24.10' into concat_json
13278f0
@ttnghia
Change delimiter type from byte to char, and rewrite docs
c67a195 
Signed-off-by: Nghia Truong <[email protected]>
@ttnghia
Restore source file
0e919d2 
Signed-off-by: Nghia Truong <[email protected]>
@ttnghia
Rename file
652e0ea 
Signed-off-by: Nghia Truong <[email protected]>
@ttnghia
Add new source file
8470ea1 
Signed-off-by: Nghia Truong <[email protected]>
@ttnghia
Add file to cmake
c19a3df 
Signed-off-by: Nghia Truong <[email protected]>
@ttnghia
Optimize implementation
91769c7 
Signed-off-by: Nghia Truong <[email protected]>
@ttnghia
Fix start character
45a929f 
Signed-off-by: Nghia Truong <[email protected]>
@ttnghia ttnghia requested a review from revans2 September 30, 2024 23:42
@ttnghia ttnghia self-assigned this Sep 30, 2024
@ttnghia
Copy link
Collaborator Author

ttnghia commented Sep 30, 2024

build

@ttnghia ttnghia mentioned this pull request Sep 30, 2024
Merged
@ttnghia
Copy link
Collaborator Author

ttnghia commented Oct 1, 2024

build

@ttnghia
Copy link
Collaborator Author

ttnghia commented Oct 1, 2024

build

@ttnghia
Copy link
Collaborator Author

ttnghia commented Oct 1, 2024

build

@ttnghia
Copy link
Collaborator Author

ttnghia commented Oct 1, 2024

build

ttnghia added 10 commits October 2, 2024 15:04
@ttnghia
Implement JSONUtils.makeStructs
a2c4e64 
Signed-off-by: Nghia Truong <[email protected]>
@ttnghia
Rename variables and update docs
fe1071c 
Signed-off-by: Nghia Truong <[email protected]>
@ttnghia
Merge branch 'branch-24.10' into concat_json
b0db299
@ttnghia
Misc
2b726f0 
Signed-off-by: Nghia Truong <[email protected]>
@ttnghia
Add stream sync and extract code into separate functions for profiling
a048801 
Signed-off-by: Nghia Truong <[email protected]>
@ttnghia
Revert "Add stream sync and extract code into separate functions for …
143afc4 
…profiling"

This reverts commit a048801.
@ttnghia
Reorganize code
f10e73a 
Signed-off-by: Nghia Truong <[email protected]>
@ttnghia
Revert "Reorganize code"
ddc383e 
This reverts commit f10e73a.
@ttnghia
Misc
aab5d51
@ttnghia
Use one warp per row to improve performance
bb6fceb 
Signed-off-by: Nghia Truong <[email protected]>
ttnghia added 3 commits October 3, 2024 16:13
@ttnghia
Optimize write
9af88ca 
Signed-off-by: Nghia Truong <[email protected]>
@ttnghia
Revert "Optimize write"
e982290 
This reverts commit 9af88ca.
@ttnghia
Reorganize code
1417301 
Signed-off-by: Nghia Truong <[email protected]>
@ttnghia ttnghia mentioned this pull request Oct 4, 2024
Closed
@ttnghia
Copy link
Collaborator Author

ttnghia commented Oct 7, 2024

build

@ttnghia ttnghia changed the base branch from branch-24.10 to branch-24.12 October 8, 2024 18:30
@ttnghia
Copy link
Collaborator Author

ttnghia commented Oct 8, 2024

build

revans2
revans2 approved these changes Oct 10, 2024
View reviewed changes
Copy link
Collaborator

@revans2 revans2 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The performance got a lot better on my tests 24% better than the current baseline and all JSON related tests passed.

I need to track down what happened with some regexp tests, but they appear unrelated.

@ttnghia ttnghia merged commit 10fbcff into NVIDIA:branch-24.12 Oct 10, 2024
3 checks passed
@ttnghia ttnghia deleted the concat_json branch October 10, 2024 22:38
@GregoryKimball GregoryKimball mentioned this pull request Jan 10, 2025
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@revans2 revans2 revans2 approved these changes

Assignees

@ttnghia ttnghia

Labels
feature request
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@ttnghia @revans2 @karthikeyann