-
Notifications
You must be signed in to change notification settings - Fork 206
CLIENT-3439 Provide user-level metrics for applications using Aerospike Database (Metrics v2) #477
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: stage
Are you sure you want to change the base?
Conversation
fad6fef
to
1c58797
Compare
1c58797
to
f4276db
Compare
3cd8f5a
to
ae20651
Compare
d6806f2
to
05eff99
Compare
05eff99
to
5a2f73c
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks mostly good. There are a few issues that need addressing:
- Some closures are unnecessary and need removal.
- Some closures in the hot path look to be costly both in terms of GC pressure and performance and need evaluation.
- The amount of data received from the server is not recorded in the stats.
- Histogram merge seems to have been broken.
We had a good discussion with Mirza, and made a few improvements, the most important of which is that now the client can reliably return the amount of data received from the server. We also reduced the number of allocations where possible. |
I think we are good on the total bytes received. I have have performed the following operations
The output for the commands ran above in the same sequence are
|
The map will forever require heap allocations. The Iterator on the other hand can be optimized away to NOT require allocations. At the moment Go's compiler is not that sophisticated and still allocates memory for the iterator, but I expect that to be resolved in the future. Having said that, Iterator still performs significantly better and allocates less memory even today.
So the reason we dont have the data corruption issue with maps is because we are creating a copy. This is also the overhead. We most likely could fix the issue when using iterator with locks maybe, I have not really thought this through, but since number of namespaces is not unbound I don't think it make sense to use a generator here. There are swissmaps coming to golang in 1.24 as default implementation for map. |
Adding detailed metrics and resultcodes with namespaces to golang client.
Valided and optimized updateOrInsert calls. Below is a list of benchmarks as result of those optimizations.
=== RUN BenchmarkApplyDetailedMetricsBytesSentAndTransmission
BenchmarkApplyDetailedMetricsBytesSentAndTransmission
BenchmarkApplyDetailedMetricsBytesSentAndTransmission-14 8375888 131.0 ns/op 24 B/op 1 allocs/op
=== RUN BenchmarkApplyDetailedConnectionAq
BenchmarkApplyDetailedConnectionAq
BenchmarkApplyDetailedConnectionAq-14 11469990 103.1 ns/op 24 B/op 1 allocs/op
=== RUN BenchmarkApplyDetailedParsing
BenchmarkApplyDetailedParsing
BenchmarkApplyDetailedParsing-14 11457337 104.4 ns/op 24 B/op 1 allocs/op
=== RUN BenchmarkCommandMergeCommandResultCodeMetrics
BenchmarkCommandMergeCommandResultCodeMetrics
BenchmarkCommandMergeCommandResultCodeMetrics-14 3045164 401.4 ns/op 64 B/op 5 allocs/op
=== RUN BenchmarkCommandMergeDetailMetrics
BenchmarkCommandMergeDetailMetrics
BenchmarkCommandMergeDetailMetrics-14 5773930 202.5 ns/op 24 B/op 2 allocs/op
=== RUN BenchmarkConnectionAcWithMetrics
BenchmarkConnectionAcWithMetrics
BenchmarkConnectionAcWithMetrics-14 10457097 115.9 ns/op 24 B/op 1 allocs/op
=== RUN BenchmarkParseAcWithMetrics
BenchmarkParseAcWithMetrics
BenchmarkParseAcWithMetrics-14 10541271 114.7 ns/op 24 B/op 1 allocs/op
PASS
Benchmark results without inline optimizations, aka on metrics callection in the command.executeAt method. As example
=== RUN BenchmarkApplyDetailedMetricsBytesSentAndTransmission
BenchmarkApplyDetailedMetricsBytesSentAndTransmission
BenchmarkApplyDetailedMetricsBytesSentAndTransmission-14 8488948 132.0 ns/op 24 B/op 1 allocs/op
=== RUN BenchmarkApplyDetailedConnectionAq
BenchmarkApplyDetailedConnectionAq
BenchmarkApplyDetailedConnectionAq-14 11312936 107.7 ns/op 24 B/op 1 allocs/op
=== RUN BenchmarkApplyDetailedParsing
BenchmarkApplyDetailedParsing
BenchmarkApplyDetailedParsing-14 10988397 108.8 ns/op 24 B/op 1 allocs/op
=== RUN BenchmarkCommandMergeCommandResultCodeMetrics
BenchmarkCommandMergeCommandResultCodeMetrics
BenchmarkCommandMergeCommandResultCodeMetrics-14 2924882 412.2 ns/op 64 B/op 5 allocs/op
=== RUN BenchmarkCommandMergeDetailMetrics
BenchmarkCommandMergeDetailMetrics
BenchmarkCommandMergeDetailMetrics-14 5800849 207.9 ns/op 24 B/op 2 allocs/op
------------> benchmarks without inline optmizations
=== RUN BenchmarkConnectionAcWithMetrics
BenchmarkConnectionAcWithMetrics
BenchmarkConnectionAcWithMetrics-14 9772182 120.3 ns/op 24 B/op 1 allocs/op
=== RUN BenchmarkParseAcWithMetrics
BenchmarkParseAcWithMetrics
BenchmarkParseAcWithMetrics-14 10514532 115.9 ns/op 24 B/op 1 allocs/op
PASS
ok github.com/aerospike/aerospike-client-go/v8 9.834s
Filtered result snipped of Stats() snapshot: