python: Add conversions for Duration and Timestamp

[gasmith]

This change adds the following conversions into Duration and Timestamp for python:

• Duration.from_secs(secs: float)
• Duration.from_timedelta(td: datetime.timedelta)
• Timestamp.from_epoch_secs(timestamp: float)
• Timestamp.from_datetime(dt: datetime.datetime)

[linear]

FG-10472 Update types for Time & Duration (python)

[bryfox]

Might be worth putting this in one of the examples (live-vizualization?) but looks good to me.

[gasmith]

Might be worth putting this in one of the examples (live-vizualization?) but looks good to me.

Oh yeah, I meant to do that, but forgot.

[jtbandes]

Do the access specifiers here prevent the user from constructing a Timestamp/Duration out of raw values? If not, I wonder if we may need manual implementations of ==, because the type system doesn't actually prevent nsec from being outside the [0, 1e9) range so there may be multiple representations of the same value. (Unless we are declaring those "invalid"?)

[gasmith]

The only constructor we expose is the new method below. But, it's true that we're not using that opportunity for normalization as we should be.

[gasmith]

OK, now normalization is done eagerly when constructing Duration and Timestamp. Here's how things shook out:

• Rust:
  • Removed public visibility on sec and nsec fields.
  • Added new and new_checked constructors.
  • Added getters for sec and nsec values.
• Python:
  • Use new_checked constructor, and raise OverflowError when out of range.
  • Add getters for sec and nsec values.

[jtbandes]

just curious if there is a reason we defined these as __new__ instead of __init__? I suspect since this is a .pyi only there's effectively no difference, other than I feel like __new__ is more commonly customized in python

[gasmith]

Dunno, I'm just following the convention from schemas.pyi, for better or worse.

In python, __new__ is responsible for allocating an instance, and __init__ is responsible for initializing it (docs). When you're working in python, you typically implement __init__, except for unusual circumstances like metaclasses.

Since we're documenting the type interface, I'm not sure the distinction matters all that much.

ChatGPT says it's preferred to document the constructor as __new__ in a .pyi for an extension class, but I'm not very convinced by its line of argument.

I think as long as IDEs and LSPs understand that this is the constructor interface, it probably doesn't matter too much. FWIW, pyright seems to understand it as such.

[jtbandes]

Are we doing this stuff to avoid total_seconds intentionally? I see the docs for that method say:

Note that for very large time intervals (greater than 270 years on most platforms) this method will lose microsecond accuracy.

But they also say

For interval units other than seconds, use the division form directly (e.g. td / timedelta(microseconds=1)).

So I wonder if we could use this division feature to extract the total microseconds directly, rather than having to extract each of the components?

I think what you've done is probably fine, but the manual multiplication by 24*3600 just triggered my "dates are hard"-o-meter a little bit, so I was curious if there's a way to avoid it and make python do the hard work for us.

[jtbandes]

I see it's also documented that "assumes there are exactly 3600*24 seconds in every day" 😅
Might at least be worth some more comments on this stuff

[jtbandes]

It does look like perhaps / loses full precision while // is able to retain it:

>>> (datetime.fromtimestamp(300*365*24*60*60,UTC)+timedelta(microseconds=1)-datetime.fromtimestamp(0,UTC))/timedelta(microseconds=1)
9460800000000000.0
>>> (datetime.fromtimestamp(300*365*24*60*60,UTC)+timedelta(microseconds=1)-datetime.fromtimestamp(0,UTC))//timedelta(microseconds=1)
9460800000000001

[jtbandes]

ooo and there's also divmod 😄 this might be my favorite so far since it gives us directly the result we want

>>> divmod((datetime.fromtimestamp(300*365*24*60*60,UTC)+timedelta(microseconds=1)-datetime.fromtimestamp(0,UTC)), timedelta(seconds=1))
(9460800000, datetime.timedelta(microseconds=1))

[gasmith]

Are we doing this stuff to avoid total_seconds intentionally?

Yes. If the caller is trying to convert a datetime, which is already integral, there's no point in bouncing that over to floating point and losing precision along the way.

I see it's also documented that "assumes there are exactly 3600*24 seconds in every day"

Right. That's unix time for you. Every day is 86,400 seconds, no exceptions.

ooo and there's also divmod 😄 this might be my favorite so far since it gives us directly the result we want

It's possible, but it's a lot more fiddly work than what I currently have. We'd need to import divmod from builtins, construct the timedelta(seconds=1) divisor, call divmod, extract the (u32, PyAny) result, downcast the PyAny into a PyDelta, getattr microseconds and extract as u32, and finally multiply by 1000. Not sure I see the benefit.

[gasmith]

Maybe this is an argument for putting even more of the interface in pure python, rather than in rust. We could have a pure-python wrapper with various constructors around a more spartan rust class. Seems like a lot of work, though, to accommodate this one particular case.