Skip to content

[PAL-SGX] alternative gettimeofday() implementation for gramine #1799

@jonathan-sha

Description

@jonathan-sha

Description of the feature

Looking at the gettimeofday() implementation used by gramine, there are many similarities with an implementation we did for "native" sgx apps that we develop where I work. But, we also took some different approaches listed below.

I have working and highly tested code I can share with you, if you're interested in my implementation. It's currently written in c++ but if there's interest I will be happy to port it to c and add it to gramine.

  • our implementation also uses RDTSC when it's available, and falls back to OCALL if there's no other choice.
  • we don't rely on cpuid heuristics to read the clock frequency (gramine has bare-metal and hypervisor implementation, which is partial).
  • instead, we calculate the clock frequency ourselves.
    • for this we need two "ground truth" readings of timeval + tsc pairs, and we can calculate the clock frequency by counting "how many cycles passed in a given time".
    • we don't need to "sleep" for this to work, we use consecutive calls to gettimeofday() and the time elapsed between them to make the calculation.
    • the clock frequency is periodically refreshed and recalibrated (by performing 2 "extra" ocalls that are far enough apart), in order to keep close to the host's time.
  • the code was written to give consistent time between all threads in a thread safe and lockless manner (using atomics)
    • there is a "pair" of global ground-truth states which we switch between in round-robin
    • a single thread gets to "update and advance" the state

This code was stressed and tested on azure, I know in the past gramine had some issues with the fast-path not working on azure?

Please let me know if this is interesting to you.

Why Gramine should implement it?

  1. do not rely on "fixed" clock_frequency of the cpu and different hypervisor implementations for reporting it in cpuid
  2. there are probably some targets that should support this fast-path gettimeofday() using rdtsc, but currently use the slow path
  3. this implementation is completely lockless - gramine currently uses a spin lock to update the ground truth.

Out implementation is measured to run in ~47 cycles (with -O3 compile flag) which is faster than the ~60 cycles it takes to run gettimeofday() natively on the host!

Also, as an unrelated note, we do handle getting the (deprecated) timezone struct when re\calibrating our ground-truth, just in case some old code uses this parameter. I saw an issue regarding this still open on gramine.

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions