Add pyodide support for AstroPi visualisations #913
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
These files originate from the following repositories: - https://github.com/RaspberryPiFoundation/pygal.js - https://github.com/RaspberryPiFoundation/sense_hat - https://github.com/raspberryPiFoundation/turtle They were created as part of the python-execution-prototypes project: https://github.com/RaspberryPiFoundation/python-execution-prototypes
Previously, I thought we needed to set these HTTP headers for the web worker to enable SharedArrayBuffer usage, but we actually need to set them for requests to the top-level web page. Therefore, ensure the crossorigin attribute is added to script/link tags for cross-origin resources.
The worker and its dependencies (pygal / _internal_sense_hat) use ES6 so we need to make sure this is transpiled as part of the standard webpack build so that it works with all browsers.
…rams https://pyodide.org/en/stable/usage/api/js-api.html#pyodide.FS In the future, we could open this up to support other file types (not just .py, .txt and .csv) and we could allow users to create files with Python and then save the result to a component in the project, e.g. to display an image of the Mandelbrot set.
I originally tried implementing this with redux by dispatching an event containing the ‘content’ object but this isn’t always serialisable, e.g. for turtle it is a JavaScript Map object. Redux prints a warning if the object isn’t serialisable. Therefore, instead, use a channel pattern that allows the sub-component (VisualOutput) to subscribe to the handleVisual method calls and update its DOM.
Most of this code was copied from PythonRunner.js and then adapted to implement the handleInput function and set the stdinBuffer array. See this commit for more information: RaspberryPiFoundation/python-execution-prototypes@42a7001
Previously, if you ran the following program in the editor using Pyodide
there was no way to cleanly exit:
```python
import sys
for line in sys.stdin:
print(line)
```
This wasn’t an issue before because Skulpt only supported the input()
function which scans to the end of the line and does not look for EOF
when the standard input is closed.
To support this, add the ability to press control-d which sets the first
element in the stdinBuffer to -1. This signals to the worker that stdin
is closed, which it signals to Pyodide by returning 0 from read(buffer).
These keys are used for copying and undoing on Windows.
Jest doesn’t support Web Workers and was erroring because the importScripts function isn’t available. Best practice seems to be to mock the module and returned canned responses. Currently, we don’t have Jest tests for the experimental PyodideRunner.js so an empty implementation is sufficient but we could change this later. https://stackoverflow.com/questions/42567535#answer-62436219
Currently, the AstroPiModel component stores its state on the global Sk variable. We want to use the model for Pyodide as well which doesn’t have an Sk (skulpt) global so change the interface to take a config.
tuzz
force-pushed
the
pyodide-astro-pi
branch
from
a9ef3f6
to
134d8ef
Compare
tuzz
had a problem deploying
to
previews/pyodide-astro-pi
GitHub Actions
Failure
tuzz
added a commit
that referenced
this pull request
Feb 13, 2024
This pull request adds experimental support for running Python programs using Pyodide. The code is based on the [python-execution-prototypes](https://github.com/RaspberryPiFoundation/python-execution-prototypes) and the [technical analysis](https://docs.google.com/document/d/1mpKliJaqYYkPJR1hGLZxUDhNFOwgBm69nc70iiVj_3I/edit#heading=h.3orc9tdzyy4b) document. The related GitHub issue is #891. The PyodideRunner.jsx can be used instead of the current PythonRunner.jsx (Skulpt) by provided a `?pyodide=true` parameter in the URL as shown in the screenshot below: <img width="1439" alt="Screenshot 2024-01-30 at 11 39 01" src="https://github.com/RaspberryPiFoundation/editor-ui/assets/892251/a0185ec3-aaf5-40d4-a5e8-e9973fb1c98f"> ### Notes The PyodideRunner supports the following features: - Execution of Python programs - Importing of [most of](https://pyodide.org/en/stable/usage/wasm-constraints.html) Python's standard library - Importing of [many common](https://pyodide.org/en/stable/usage/packages-in-pyodide.html) Python packages - Importing of pure Python packages from PyPi - Stopping execution - Sending standard input to Python programs - Closing the standard input stream by pressing `Ctrl-d` - Visualising turtle graphics using the [modified turtle package](https://github.com/RaspberryPiFoundation/turtle) - Visualising pygal charts using the [modified pygal package](https://github.com/RaspberryPiFoundation/pygal.js) Due to time constraints, the following tasks remain outstanding: - Visualising sense_hat graphics using the [modified sense_hat package](https://github.com/RaspberryPiFoundation/sense_hat) - There is a work-in-progress pull request [here](#913) - Apply HTTP headers in a service worker (see #903) - Note that `crossorigin` will need to be added to cross-domain resources once the HTTP headers are added - Add unit/integration tests that run real Python programs through the PyodideRunner.jsx - Manual testing for backwards compatibility of existing Python programs, projects and the WebComponent - Implementing the p5 package to work with Pyodide - See https://github.com/RaspberryPiFoundation/python-execution-prototypes/issues/1 - Implementing a dedicated SQL editor for use in the Ada CS project - Note that Pyodide supports importing `sqlite3` so we can add a UI wrapper around it --------- Co-authored-by: Scott Adams <[email protected]>
loiswells97
temporarily deployed
to
previews/pyodide-astro-pi
GitHub Actions
Inactive
loiswells97
temporarily deployed
to
previews/pyodide-astro-pi
GitHub Actions
Inactive
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This is a work-in-progress pull request that adds support for AstroPi visualisations in PyodideRunner. We have already completed a large piece of work to migrate the _internal_sense_hat.js shim from a Skulpt module to a plain ES6 module. See the last two pages of the technical analysis document for how this was achieved. We now need to integrate this new ES6 module with the editor-ui project's
<AstroPiModel>component.To do so, we need to refactor it so that it doesn't use the
Sk.sense_hatglobal state and instead passessenseHatConfigproperties to the React component tree. I think we have successfully completed this refactor and made<AstroPiModel>work with the existing PythonRunner.jsx (Skulpt) but we now need to get it to work with Pyodide. The tests are also broken and need to be updated to provide thesenseHatConfig.