Migrating from PyOxidizer to Briefcase

[andrewsanchez]

Here are some initial notes regarding the possibility of using Briefcase to build Anki.
I will add more details later but wanted to share the draft below now just to start the discussion.

This is regarding the possibility of migrating Anki from PyOxidizer to Briefcase.

Anki Build System Notes

Current Build System

• Uses PyOxidizer (patched fork) for binary builds
• Final output is embedded executable calling aqt.run()

Build Process

• Entry point is ./ninja script
• Calls runner script which generates build.ninja file in the git-ignored ./out directory
• Generated ninja script handles the actual build
  • See an example here

Potential PyOxidizer Replacement

Briefcase Proof of Concept

• MVP would be binary that successfully launches Anki via aqt.run()
• Reference implementation: https://github.com/AnkiHubSoftware/anki-briefcase-test/blob/main/helloworld/src/helloworld/app.py
  • This is a work in progress that I have just started fiddling with

This currently fails like so:

Loaded Anki collection: <anki.collection.Collection object at 0x10257d0d0>
Card count: 0
---------------------------------------------------------------------------
Application quit abnormally!
Traceback (most recent call last):
  File "Hello World.app/Contents/Resources/app/helloworld/__main__.py", line 4, in <module>
    main()
  File "Hello World.app/Contents/Resources/app/helloworld/app.py", line 15, in main
    import aqt
  File "Hello World.app/Contents/Resources/app_packages/aqt/__init__.py", line 63, in <module>
    from aqt import gui_hooks
  File "Hello World.app/Contents/Resources/app_packages/aqt/gui_hooks.py", line 11, in <module>
    from _aqt.hooks import *
  File "Hello World.app/Contents/Resources/app_packages/_aqt/hooks.py", line 18, in <module>
    from aqt.qt import QDialog, QEvent, QMenu, QModelIndex, QWidget, QMimeData
  File "Hello World.app/Contents/Resources/app_packages/aqt/qt/__init__.py", line 22, in <module>
    from .qt6 import *
  File "Hello World.app/Contents/Resources/app_packages/aqt/qt/qt6.py", line 11, in <module>
    from PyQt6 import sip
ImportError: cannot import name 'sip' from 'PyQt6' (/Users/andrewsanchez/Projects/beeware/beeware-console-app/helloworld/build/helloworld/macos/app/Hello World.app/Contents/Resources/app_packages/PyQt6/__init__.py)

Thoughts

• pyinstaller needs hooks to perform Qt-specific bundling of support libraries, extra modules, etc. A brief look at Briefcase seems to indicate it has one for PySide, but not PyQt:
• This a reason why bundlers can be a pain. e.g., Use a module that’s not yet supported, or one that has had its files moved around in an update, and you can end up having to manually ensure the relevant modules have been included. It gets even more complicated when trying to code sign - PyQt’s files aren’t in the right layout for signing for example. In the pyoxidizer case, Anki found it easier to do the layout and bundle those files separately.
• The uv solution would include all Python modules available, instead of trying to pick out only the ones that are in use before zipping them up.

Alternative Considered

• UV: PyOxidizer has been abandoned and will need replacing ankitects/anki#3081 (comment)
• Damien's feedback: PyOxidizer has been abandoned and will need replacing ankitects/anki#3081 (comment)

Related Issues

• PyOxidizer replacement issue #3081
  • Note: Partially outdated due to Astral developments

Out of Scope

The initial goal is simply to use Briefcase for building Anki Desktop. After that, we could naturally consider:

• Mobile client builds (potential future benefit)
• Toga

[freakboy3742]

To clarify exactly what Briefcase is doing - it's not using "bundlers" or a hook or anything like that. It is essentially the "dumbest thing that could work". When it installs a package, it's literally running a pip install; when that code is used at runtime, it's setting a PYTHONPATH to ensure the local install is visible, but nothing else. As a result, there's no special handling required for any Python module - if that module can be pip installed, it should work without modification in a Briefcase-packaged app.

The PySide6 support that you've noted isn't doing anything to the operation of the app. It is entirely an initial template, setting up the PySide6 dependency, and injecting the "Hello World" example application. It changes nothing about how the app works at runtime.

To that end... your example project worked out of the box for me. A git clone of the provided repository, with briefcase run in the helloworld directory generated:

The signing process does reveal some problems - it generates a bunch of warnings that lead me to suspect that the app may not run on x86_64 - but it definitely starts on M1. The build warnings look resolvable - they're an issue with identifying content that is being distributed as a non-universal macOS framework. I've opened #2317 to track this.

The error message you've reported suggests that you might have had a partially complete build state - possibly due to a failed previous build. If you delete the entire build directory and re-run briefcase run, that should guarantee a clean build state, and you should get a working app. If you don't, delete the build directory, run briefcase run --log, provide the log file that is generated, and we can take a closer look.

[andrewsanchez]

Thanks, @freakboy3742! I was able to get this to run after clearing the build directory.

Here are some quick notes from a side convo with Damien:

Initial tests

• tested out Briefcase's built-in code signing, and it was semi-successful.
• It seemed to produce a validly-signed app, but the app failed to show the webview when launched.
• WebEngine requires special entitlements to be included when signed, and it doesn’t look like Briefcase is currently doing that, and I didn’t see an option to provide them yet.

Code signing step

• Briefcase signs every library in the bundle each time you package.
• Anki's current packaging process splits off Qt (the bulk of our libraries) into a separate package which is code-signed in advance, so it can just be dropped into place during packaging, which significantly speeds up the build process.

We'll spend a bit more time experimenting and report back soon!

[freakboy3742]

• WebEngine requires special entitlements to be included when signed, and it doesn’t look like Briefcase is currently doing that, and I didn’t see an option to provide them yet.

Any entitlement you need that Briefcase isn't providing can be added to your pyproject.toml using the entitlement configuration option in the macOS section of your pyproject.toml file. I'm not sure what entitlements specifically are needed for WebEngine, but adding the line to your pyproject.toml and rebuilding the app (i.e., purge the build folder and start from scratch) should be all you need to do.

• Anki's current packaging process splits off Qt (the bulk of our libraries) into a separate package which is code-signed in advance, so it can just be dropped into place during packaging, which significantly speeds up the build process.

That's not a workflow that we currently have support for; but if you can share details of what you're doing, it's not out of the question to add as a feature.