Asynchronous Expressions and the Delayed Functions experiment

[bluelhf]

Problem

Currently, Skript expressions cannot perform asynchronous computation, because the API forces them to have immediate values. Historically, this has been circumvented with effect sections, where addons ask the user to specify a variable to which the result of the computation is eventually stored, as well as a section of code that will be executed when the result is available. Sadly, this even applies in the case of functions that return values, as the function call is an expression.

For example, the following code will fail to parse

function get_message() returns text:
  wait a tick
  return "Hello, world!"

on load:
  broadcast get_message()

because the broadcast effect needs to know the value of get_message() immediately. This is frustrating, as the expected behaviour is completely reasonable: the broadcast effect should wait until the get_message() function returns a value before executing.

Solution

This PR solves the problem by introducing a yield. Yields are exceptions (specifically, HandlerYieldExceptions) that may be thrown by code with a return handler to signal that the computation is not complete, and give the call stack above the yield an opportunity to register a callback to try again later when the entire yielding trigger has completed (which is called a "resolution" of the yield).

When a syntax element yields, the entire call stack unwinds frame by frame. At each step, the call stack frame will register itself to re-execute once the yield is resolved. When the yield is resolved by the yielding syntax element (for example at the end of some asynchronous computation), all registered callbacks are executed in order, starting from the deepest call stack frame and advancing up. The effect is that the callback for any frame on the stack is only executed once all of its descendants' callbacks have been resolved.

API

This PR introduces a new type of expression, AsyncExpression, as well as its companion class SimpleAsyncExpression. They are the asynchronous equivalents of Expression and SimpleExpression respectively. Asynchronous expressions are represented as having an internal, thread-specific 'context', which keeps track of the state of the asynchronous computation. Further details are in the relevant Javadocs. A simple asynchronous expression is shown in the test expression ExprStall:

It is also worthwhile to look at the implementation of SimpleAsyncExpression:

Failed resolutions and loops

A syntax element that yields, as well as the entire call stack above it, must be prepared to handle being called twice; once initially when running regular code, and again once the yield is resolved and registered callbacks are executed. A misbehaving frame on the call stack may, for example, fail to recognise that it has already executed and attempt to call the yielding element again. This will cause an infinite loop, as the third execution of the yielding element will now start a new computation. Such situations are detected automatically and while they are not rectifiable, they will be reported as errors in console.

Delayed Functions

Note

Since delayed functions are in the experimental phase, user code may only use them by including a using delayed functions statement in the script. For scripts that do not explicitly enable delayed functions, old behaviour is kept and delayed functions will fail to parse with the regular error.

This PR implements preliminary asynchronous computation in Skript in the form of a delayed functions experiment. Delayed functions may both contain the Delay effect and return values. When Delay is called from within a delayed function, it yields by throwing a HandlerYieldException. The call site for the function catches the yield, registers the function to run again once the yield is complete, and propagates the yield up. As the code after the delay effect in the function is called before the yield is resolved, by the time the call site re-executes the function, the entire function trigger will have completed and the function will have a return value. This value is then returned immediately without re-executing the function.

This allows for code like this:

using delayed functions

function nested_delayed_one() returns integer:
  wait a tick
  return nested_delayed_two()

function nested_delayed_two() returns integer:
  wait a tick
  return 2

on load:
  broadcast nested_delayed_one()

to work and broadcast 2 after two ticks.

Even when delayed functions are enabled, functions that do not return values will work in the usual fashion of having the call site continue execution while the function runs on the next tick.

Testing Completed

Manual testing in the form of many different scenarios involving nested functions, loops, sections, and such. Asynchronous expressions were mostly tested manually. Unit tests for delayed functions exist in StructFunction.sk. The unit tests ensure nested and repeated calls work, and that local variables are preserved properly. There are still places, such as SectionUtils.loadLinkedCode, that do not allow delays. This is because the parent section must explicitly support yields for this to work.

Supporting Information

I would appreciate it if someone with a deeper understanding of the Skript interpreter looked at this, and also encourage everyone to be adversarial and try to find yield loops as well as circumstances in which they are never caught or behave improperly.

---

Completes: none
Related: none
AI assistance: Junie wrote the initial documentation for HandlerYieldException.

[UnderscoreTud]

very nice pr! it might be worthwhile to allow the return handler to define whether delays are supported or not

[bluelhf]

very nice pr! it might be worthwhile to allow the return handler to define whether delays are supported or not

I wonder if this behaviour could be contained to the return handler in general, such that only return handlers can cause yields and are more aware of their call site...

I imagine something like Delay would obtain a Yield like returnHandler.yield(), throw it and then yield.resolve() after the delay...

Perhaps the handler could be made aware of the trigger item it 'returns to'? Such that a handler like ScriptFunction would know that, say, EffBroadcast called it, and could signal that in the yield so the call site knows for sure that the yield belongs to it (because the return trigger will be the current trigger on the call site)

Certainly interesting...

[bluelhf]

Scope expanded, expressions work now! Things like effect sections need explicit support.