dev call 20220505

Dev Call: 5 May 2022

Present

Antonio, Simon, Ronan, Du Toit, Gianluca, Stepan, Tim, Florian

Agenda

• HPy release 0.0.4

  • Debug Mode PR: https://github.com/hpyproject/hpy/pull/314
  • Improve Doc PR: https://github.com/hpyproject/hpy/pull/314

• NumPy Progress and Plans

  • We finished migration of init (of _multiarray_umath)
  • Ongoing work on ufunc, array object, etc.
  • Wiki with some migration notes: https://github.com/hpyproject/hpy/wiki/Numpy-HPy-migration-notes:-blockers-and-concerns

• Open HPy PRs

  • Capsule PR: https://github.com/hpyproject/hpy/pull/308
  • exact type check: https://github.com/hpyproject/hpy/pull/309
  • detect HPy_Close/return of context constants in debug: https://github.com/hpyproject/hpy/pull/303

Misc

• Helper archive distribution
• Come up with a policy for which CPython version we are going to support -- conclusion was that we can drop 3.6, 3.7 now, in the future we may follow Numpy approach to this
• Antonio was interested in the way how GraalPython implements NaN-tagging of HPy handles. GraalPython uses that to box 32-bit integers and doubles mainly to avoid upcalls into the Java runtime since they turn out to be expensive for small operations. This brought a great speedup (like 5x) for the ujson-hpy benchmarks. GraalPython's NaN-tagging implementation is available here: https://github.com/oracle/graalpython/blob/master/graalpython/com.oracle.graal.python/src/com/oracle/graal/python/builtins/objects/cext/hpy/GraalHPyBoxing.java

Numpy Progress

• Biggest blocker on CPython: meta classes for heap types

  • https://github.com/hpyproject/hpy/issues/296
  • Simon just asked Victor (via email) to upstream that.
  • We could implement a workaround by creating the heap type and immediately after the return of PyType_FromSpecWithBases we allocate the struct with the right size and memcpy-ing into it.
  • Pulling-in PyType_FromSpecAndBases into HPy is a mess. It uses a lot of internal functions.
  • Implementation on GraalPython is also not straight forward. CPython's impl _just_ uses the meta class' alloc function but those are not really exposed in GraalPython.
  • Implementation on PyPy is also not easy as far as Ronan remembers.
  • Antonio thinks that heap types are not ready on CPython. We should maybe just 'malloc' a static type and use that (i.e. fill all the members and call PyType_Ready on it).
  • Ronan is not sure if we can hide that we are using static types. But at least in HPy the type object should be opaque that it makes no difference.

• Numpy also uses slots for _type checks_

  • Matti says: Numpy will remove the type check via slot comparison in future versions.

• The GraalPython team has made a lot of changes to HPy to make the Numpy port work. We will publish the branch but a lot of that changes are more hacks (just to make progress) and need a proper solution. For example: HPyDict_GetItem: returns a new handle (and not a borrowed one as in CPython). But the actual problem was that it ignores the current exception context. So, the proper solution is to use HPy_GetItem and have a way to save and restore the exception state.

Cython

• Matti tries to spend some time on it.
• Based on Florian's changes, he tries to make the CI work. Current problem is a compilation error in HPyTuple_Pack when compiling with C++.

SciPy Conference (July 2022)

• Antonio and Simon submitted an HPy BoF session request to SciPy 2022 in July.

Pillow

• Du Toit and Gianluca currently in the literature review progress
• Next step is to write the thesis proposal