Skip to content

Clickhouse complex types handling #7103

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
wants to merge 3 commits into
base: main
Choose a base branch
from
Open

Clickhouse complex types handling #7103

wants to merge 3 commits into from

Conversation

asmit27rai
Copy link

@asmit27rai asmit27rai commented May 6, 2025 •
edited
Loading

This change enables proper storage of complex attribute types (maps/arrays) in ClickHouse by:

  1. Serialization

    • Convert pcommon.Map and pcommon.Slice values to JSON strings during storage
    • Handle nested structures recursively
    • Store JSON-encoded values as string attributes

  2. Deserialization

    • Detect JSON-encoded values during retrieval
    • Convert JSON strings back to appropriate pcommon types
    • Maintain original type information through round-trip conversion

  3. Compatibility

    • Existing simple types (bool/int/float/string) remain unchanged
    • Backward compatible with previously stored data
    • Add error handling for malformed JSON data

  4. Implementation Details

    • Added JSON marshaling for complex types in to_dbmodel.go
    • Implemented JSON unmarshaling in from_dbmodel.go
    • Created recursive conversion helpers for pcommon values
    • Maintained existing column structure while enhancing type support

Fixes: #7031

asmit27rai added 2 commits May 6, 2025 22:30
@asmit27rai
feat(clickhouse): Add OTLP/JSON serialization for complex types
07e4726 
- Implement JSON serialization for pcommon.Map and pcommon.Slice values
  in ClickHouse storage model conversion
- Add JSON deserialization support when reading from ClickHouse
- Handle nested map/slice structures recursively
- Maintain backward compatibility with existing attribute storage format
- Add helper functions for OTLP<->JSON value conversion
- Update attribute handling in both to_dbmodel and from_dbmodel

Signed-off-by: asmit27rai <[email protected]>
@asmit27rai
Merge branch 'main' of github.com:jaegertracing/jaeger into clickhous…
c519295 
…e-complex-types-handling
@asmit27rai asmit27rai requested a review from a team as a code owner May 6, 2025 17:03
@asmit27rai asmit27rai requested a review from albertteoh May 6, 2025 17:03
@dosubot dosubot bot added the area/storage label May 6, 2025
@asmit27rai
Copy link
Author

@zhengkezhou1 Pls review this PR...

@github-actions GitHub Actions
Copy link

This pull request has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. You may re-open it if you need more time.

@github-actions github-actions bot added the stale The issue/PR has become stale and may be auto-closed label Jul 28, 2025
@github-actions github-actions bot removed the stale The issue/PR has become stale and may be auto-closed label Aug 4, 2025
@github-actions GitHub Actions
Copy link

github-actions bot commented Oct 6, 2025

This pull request has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. You may re-open it if you need more time.

@github-actions github-actions bot added the stale The issue/PR has become stale and may be auto-closed label Oct 6, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@albertteoh albertteoh Awaiting requested review from albertteoh albertteoh is a code owner automatically assigned from jaegertracing/jaeger-maintainers

At least 1 approving review is required to merge this pull request.

Assignees

No one assigned

Labels

area/storage stale The issue/PR has become stale and may be auto-closed

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

[Feature]: Support complex types (Slice, Map) in Clickhouse

1 participant

@asmit27rai