Skip to content

feat: convert dot keys to nested for json logs #645

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 3 commits into
base: main
Choose a base branch
from

Conversation

nityanandagohain
Copy link
Member

@nityanandagohain nityanandagohain commented Jul 8, 2025

goos: darwin
goarch: arm64
pkg: github.com/SigNoz/signoz-otel-collector/utils/flatten
cpu: Apple M3 Pro
BenchmarkConvertJSON/NotJSON-11         	18709833	        64.03 ns/op	      64 B/op	       3 allocs/op
BenchmarkConvertJSON/Small-11           	  811808	      1505 ns/op	    1824 B/op	      33 allocs/op
BenchmarkConvertJSON/Medium-11          	  176397	      6908 ns/op	    7855 B/op	     123 allocs/op
BenchmarkConvertJSON/Large-11           	   49009	     23244 ns/op	   26815 B/op	     363 allocs/op
BenchmarkConvertJSON/VeryLarge-11       	   23659	     50417 ns/op	   62628 B/op	     926 allocs/op
PASS
ok  	github.com/SigNoz/signoz-otel-collector/utils/flatten	8.542s

the performance seems to be very bad .

I also tried a concurrent version where different go routines traveses different keys, but it has more overhead and it was more slower.

goos: darwin
goarch: arm64
pkg: github.com/SigNoz/signoz-otel-collector/utils/flatten
cpu: Apple M3 Pro
BenchmarkLargeJSONComparison/LargeJSON/Hybrid-11  	   10268	     99065 ns/op	   67045 B/op	     961 allocs/op
BenchmarkLargeJSONComparison/LargeJSON/ForcedConcurrent-11         	   12386	     96850 ns/op	   66991 B/op	     961 allocs/op
BenchmarkLargeJSONComparison/LargeJSON/ForcedNonConcurrent-11      	   19263	     62176 ns/op	   57917 B/op	     876 allocs/op
BenchmarkLargeJSONComparison/VeryLargeJSON/Hybrid-11               	   23764	     50303 ns/op	   62632 B/op	     926 allocs/op
BenchmarkLargeJSONComparison/VeryLargeJSON/ForcedConcurrent-11     	   15360	     78215 ns/op	   74391 B/op	    1099 allocs/op
BenchmarkLargeJSONComparison/VeryLargeJSON/ForcedNonConcurrent-11  	   23336	     50773 ns/op	   60319 B/op	     925 allocs/op
PASS
ok  	github.com/SigNoz/signoz-otel-collector/utils/flatten	12.006s

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant