[rfc] about using second or ms

[zhyncs]

I prefer to use ms and we can add an option for ms.

[CatherineSue]

is ms option mostly for TTFT and TPOT? or throughput as well?

[zhyncs]

latency

[CatherineSue]

It is definitely an option we can provide.

One small note is that when running across different concurrencies, it is easy for the latency to increase to tens of seconds. For instance, below is a plot for fixed-input-length 7800, output-length 200. Once concurrency reached 64, the latency will reach to 20 seconds.

[junliu-mde]

I also prefer ms.

One possible solution for this issue is to provide a logarithmic scale option when comparing very large concurrency values.

BTW in my experience, when dealing with large-scale concurrency comparisons, the focus is less on specific numbers and more on observing the overall trend.