-
Notifications
You must be signed in to change notification settings - Fork 223
Description
Description of the feature
Looking at the gettimeofday() implementation used by gramine, there are many similarities with an implementation we did for "native" sgx apps that we develop where I work. But, we also took some different approaches listed below.
I have working and highly tested code I can share with you, if you're interested in my implementation. It's currently written in c++ but if there's interest I will be happy to port it to c and add it to gramine.
- our implementation also uses RDTSC when it's available, and falls back to OCALL if there's no other choice.
- we don't rely on cpuid heuristics to read the clock frequency (gramine has bare-metal and hypervisor implementation, which is partial).
- instead, we calculate the clock frequency ourselves.
- for this we need two "ground truth" readings of timeval + tsc pairs, and we can calculate the clock frequency by counting "how many cycles passed in a given time".
- we don't need to "sleep" for this to work, we use consecutive calls to gettimeofday() and the time elapsed between them to make the calculation.
- the clock frequency is periodically refreshed and recalibrated (by performing 2 "extra" ocalls that are far enough apart), in order to keep close to the host's time.
- the code was written to give consistent time between all threads in a thread safe and lockless manner (using atomics)
- there is a "pair" of global ground-truth states which we switch between in round-robin
- a single thread gets to "update and advance" the state
This code was stressed and tested on azure, I know in the past gramine had some issues with the fast-path not working on azure?
Please let me know if this is interesting to you.
Why Gramine should implement it?
- do not rely on "fixed" clock_frequency of the cpu and different hypervisor implementations for reporting it in cpuid
- there are probably some targets that should support this fast-path gettimeofday() using rdtsc, but currently use the slow path
- this implementation is completely lockless - gramine currently uses a spin lock to update the ground truth.
Out implementation is measured to run in ~47 cycles (with -O3 compile flag) which is faster than the ~60 cycles it takes to run gettimeofday() natively on the host!
Also, as an unrelated note, we do handle getting the (deprecated) timezone struct when re\calibrating our ground-truth, just in case some old code uses this parameter. I saw an issue regarding this still open on gramine.