tcurl benchmark mode: sending a request many times

[ShanniLi]

r @Raynos @kriskowal
cc @breerly

ref: #68

[kriskowal]

Please add subChannel as a nulled default property in the constructor.

[kriskowal]

I hope that these comments clarify the intent of the delegate object. The delegate stands in place of a callback as a collector of multiple errors or results instead of a single error or response. Totally a great tool for collecting results for the benchmark. There should just be one instead of two.

[kriskowal]

That concludes my review. Looking forward to seeing this in master.

[blampe]

What does the user experience look like?

[Raynos]

@blampe You can find the user experience documented here in the man page :)

[blampe]

I was using tcurl.py with a batch mode like this to benchmark Python initially, but it didn't give me the numbers I actually wanted. Knowing how long it takes to complete 1000 requests just tells you how long the longest request took, which can be anomalous.

I suggest something like --bench which just fires off as many requests as possible (with some ceiling, and maybe ramping up initially), and then continuously display how many responses were received in the last second. I think this is similar to @prashantv's benchmark for Go.

[Raynos]

@blampe

Very good point. hyperbahn ( https://github.com/uber/hyperbahn/blob/master/test/lib/time-series-cluster.js#L588-L620 ) has a results object including latencies and error rate.

I think the expected use case is to send n req/s and see what the latency is. You can then manually increase n until the p99 is too big.

That's why we have a --rate for sending N req/s and we have --time or --requests to say how long to benchmark it.

It should definitely stream results like telling us every M seconds what the success rate and latencies are.

[ShanniLi]

I have talked to @kriskowal, we agreed to use a separate delegate since the user case for benchmark is quite different.