reusable building blocks?

[baptiste]

Hi,

I'm revisiting the idea of using typst rather than markdown for literate programming; I know it is out of scope for litedown, which is fair enough, but I wonder if you might be able to point to reusable building blocks in the package that could be the basis for a typst-centric fork.

Specifically, I envisage the following:

1. typst query to capture as JSON the nodes that need to be evaluated (with metadata for chunk options)
2. evaluate said code using whichever computational tool (though I'll stick with R for now), store artefacts (plots, data, etc.)
3. inject the output into the typst document
4. compile the intermediate typst document

1 and 4 seem relatively straight-forward, but 2 and 3 are still quite vague in my mind, and it feels like reinventing the wheel without all the hard-won insights gained from the past decades of Sweave, knitr, etc.

An alternative of course would be via pandoc and/or quarto and custom filters, but this introduces a markdown bottleneck and some quirks as they're not typst-native solutions.

With this in mind, would you have a schematic overview of steps 2 and 3 to share, that would be translatable to a different system? My naive picture of this is:

• each chunk has an identifying label
• running the code produces artefacts (plots, tables, values, etc.) and these objects are stored in an inventory
• (optional) the inventory also keeps a cache to speed up re-runs
• there are format-specific methods to output objects in the target format (typst code here), e.g wrapping an image, a table, a value with typst syntax
• the outputs are injected in the right place in the document (more of a typst-side problem I guess, if it exposes an AST)

[baptiste]

Toy example,

#let code(x, id: "a") = {
  [#metadata((code: x.text, id: id)) #label("r-chunk")]
}

With this code,

#code(`x = 1 + 1`, id: "a")

or another

#code(`y = 2 + 2`, id: "b")

#let output = {
  if "results" in sys.inputs.keys() {
    json(sys.inputs.results)
  } else { // default values
   (x: 3, y: 4,)
  }
}

`x` is #output.x

`y` is #output.y

typst query --pretty knit.typ "<r-chunk>"

returns the following JSON

[
  {
    "func": "metadata",
    "value": {
      "code": "x = 1 + 1",
      "id": "a"
    },
    "label": "<r-chunk>"
  },
  {
    "func": "metadata",
    "value": {
      "code": "y = 2 + 2",
      "id": "b"
    },
    "label": "<r-chunk>"
  }
]

Then the following R code may evaluate those chunks and return a JSON output,

library(jsonlite)
library(evaluate)
library(purrr)

inputs <- jsonlite::read_json("inputs.json")

e <- new.env()
walk(inputs, \(.x)  evaluate(.x[["value"]][["code"]], envir = e))
jsonlite::write_json(as.list(e), 'results.json', auto_unbox=TRUE)

which is passed to typst for compilation of the final document

typst compile knit.typ --input results="results.json"