External Interop Samples

[Aidan63]

I've taken the small SDL sample from the dispatch pr and put it here https://github.com/Aidan63/SDL_Hxcoro. This version uses SDL3 and the new hxcpp marshalling types so there's less untyped weirdness going on.

To get it working you'll need to download the SDL3 devel zip from https://github.com/libsdl-org/SDL/releases/tag/release-3.4.0 and change the SDL_DIST_DIR build.hxml define to point to that extracted zip and you should be good to go.

I've got an old Libuv hxcpp extern repo hanging around, might try and get another sample combining libuv and hxcoro.

[Simn]

That's great! I just tried it and it worked immediately, although on a subsequent compilation I suddenly had linker errors:

Link: Main-debug.exe
   Creating library Main-debug.lib and object Main-debug.exp
cbdc8cc3_HxCoro_hxcoro_Coro_yield.obj : error LNK2011: precompiled object not linked in; image may not run
  Hint on symbols that are defined and could potentially match:
    __@@_PchSym_@00@UtrgUhwoPscxlilUyrmUlyqUnhexBJGERwvyftUPPkxsUifmgrnvUscxkkOlyq@linke5508ab046c6300be06fba4771dd2db4
    __@@_PchSym_@00@UtrgUhwoPscxlilUyrmUlyqUnhexBJGERwvyftUPPkxsUszcvUscxkkOlyq@link54d81adaa1cc8ec81d2566e381596dd6
Main-debug.exe : fatal error LNK1120: 1 unresolved externals

It worked again after deleting my bin directory... don't know what that's about.

The Event part looks great now. If we find a cleaner way to handle the data1 thing we can turn this into a nice sample in our documentation.

[Simn]

Here's a version with the pausing scheduler:

import haxe.Int64;
import haxe.coro.schedulers.IScheduleObject;
import hxcoro.task.ICoroNode;
import hxcoro.CoroRun;
import hxcoro.Coro.*;
import hxcoro.dispatchers.IDispatcher;
import hxcoro.schedulers.EventLoopScheduler;
import SDL;

@:coroutine function coroEntry(node:ICoroNode) {
	trace("It works!");
	delay(1000);
	trace("It still works!");
}

private class SdlDispatcher implements IDispatcher {
	final id : Int;

	public function new(id) {
		this.id = id;
	}

	public function dispatch(obj:IScheduleObject) {
		final event = new Event();
		event.type = id;
		event.user.data1 = untyped __cpp__('{0}.mPtr', obj);

		SDL.pushEvent(event);
	}
}

class SdlEventLoopScheduler extends EventLoopScheduler {
	var paused:Bool;

	final eventId:Int;

	public function new(eventId:Int, dispatcher:IDispatcher) {
		super(dispatcher);
		this.eventId = eventId;
		paused = true;
	}

	function maybeUnpause() {
		if (paused) {
			trace("Unpausing scheduler");
			paused = false;
			final event = new Event();
			event.type = eventId;
			SDL.pushEvent(event);
		}
	}

	override function schedule(ms:Int64, func:()->Void) {
		maybeUnpause();
		return super.schedule(ms, func);
	}

	override function scheduleObject(obj:IScheduleObject) {
		maybeUnpause();
		return super.scheduleObject(obj);
	}

	override function run() {
		if (heap.isEmpty()) {
			paused = true;
			trace("Pausing scheduler");
			return;
		}

		super.run();
		final event = new Event();
		event.type = eventId;
		SDL.pushEvent(event);
	}
}

function main() {
	SDL.init(SDL.INIT_VIDEO);

	final window           = SDL.createWindow("SDL Test", 1280, 720, SDL.WINDOW_RESIZABLE);
	final eventIds         = SDL.registerEvents(2);
	final schedulerEventId = eventIds;
	final dispatchEventId  = eventIds + 1;

	final dispatcher = new SdlDispatcher(dispatchEventId);
	final scheduler  = new SdlEventLoopScheduler(schedulerEventId, dispatcher);

	function runCoro() {
		final coroTask = CoroRun.with(scheduler).create(coroEntry);
		coroTask.start();
	}

	var run = true;

	final event = new Event();
	while (run) {
		while (SDL.pollEvent(event)) {
			switch event.type {
				case SDL.EVENT_QUIT:
					run = false;
				case SDL.EVENT_MOUSEBUTTONDOWN:
					runCoro();
				case id if (id == schedulerEventId):
					scheduler.run();
				case id if (id == dispatchEventId):
					// TODO : Haxe needs a nice way to access the hxcpp object rooting API.
					final obj : IScheduleObject = untyped __cpp__('::Dynamic { static_cast<::hx::Object*>({0}) }', event.user.data1);

					obj.onSchedule();
				case _:
					//
			}
		}
	}

	SDL.destroyWindow(window);
	SDL.quit();
}

Needs static inline final EVENT_MOUSEBUTTONDOWN : Int = 0x401; on the SDL class and then runs the coro function on every mouse click.

[Aidan63]

I've also put together a example which works with libuv, it needs an old libuv extern lib I made https://github.com/Aidan63/hxcpp_luv_io.

import hxcoro.schedulers.EventLoopScheduler;
import cpp.luv.Timer;
import haxe.coro.schedulers.IScheduleObject;
import hxcoro.task.ICoroNode;
import hxcoro.CoroRun;
import hxcoro.Coro.*;
import hxcoro.dispatchers.IDispatcher;
import hxcoro.schedulers.EventLoopScheduler;
import cpp.luv.Luv;

@:coroutine function coroEntry(node:ICoroNode) {
	trace("It works!");
	delay(1000);
	trace("It still works!");
}

class LuvDispatcher implements IDispatcher {
	final loop : LuvLoop;

	public function new(loop) {
		this.loop = loop;
	}

	public function dispatch(obj:IScheduleObject) {
		Timer.run(loop, obj.onSchedule);
	}
}

function main() {
	final loop       = Luv.allocLoop();
	final dispatcher = new LuvDispatcher(loop);
	final scheduler  = new EventLoopScheduler(dispatcher);

	final coroTask = CoroRun.with(scheduler).create(coroEntry);
	coroTask.start();

	while (Luv.runLoop(loop, RunMode.NoWait)) {
		scheduler.run();
	}
}

The entire scheduler could probably be swapped out with a libuv one since it does support timed events, cancellation, etc.

[Simn]

Hmm, I won't pretend to really know anything about libuv, but I'm surprised that it works like that. How come the runLoop call returns true here? The coro code should have made it to the scheduler from the delay call, but nothing has been dispatched yet because we never ran scheduler.run(), so I'd expect this to just terminate.

Edit: I mean nothing would be dispatched here anyway because it's a 1-second delay, but even with a yield() we would never do anything with the libuv loop, I think.

[Aidan63]

Seems I'd forgotten how how runLoop works, it returns true if there are pending events, false is nothing else is expected. So that while loop should be inverted, and doing that makes it not work, you see "it works!" and then the program closes. So we'd need a full libuv scheduler for the libuv loop to stay alive.

[Simn]

I just tried this on eval and it seems to work:

import haxe.Int64;
import haxe.coro.schedulers.Scheduler;
import haxe.coro.schedulers.IScheduleObject;
import haxe.coro.schedulers.ISchedulerHandle;
import hxcoro.task.ICoroNode;
import eval.luv.Timer;
import eval.luv.Loop;

import hxcoro.CoroRun;
import hxcoro.Coro.*;

function noOpHandle() {}

class TimerCancellation implements ISchedulerHandle {
	final timer:Timer;
	public function new(timer:Timer) {
		this.timer = timer;
	}

	public function close() {
		timer.stop();
		timer.close(noOpHandle);
	}
}

class LuvScheduler extends Scheduler {
	final loop:Loop;

	public function new(loop:Loop) {
		super();
		this.loop = loop;
	}

	function schedule(ms:Int64, func:() -> Void) {
		final timer = Timer.init(loop).resolve();
		timer.start(func, Int64.toInt(ms));
		return new TimerCancellation(timer);
	}

	function scheduleObject(obj:IScheduleObject) {
		schedule(0, obj.onSchedule);
	}

	function now() {
		return haxe.Timer.milliseconds();
	}
}

function main() {
	final loop = Loop.defaultLoop();
	final scheduler = new LuvScheduler(loop);

	final coroTask = CoroRun.with(scheduler).create(node -> {
		final task1 = node.async(node -> {
			trace("Task 1 delaying");
			delay(1000);
			return "Task 1 done";
		});
		final task2 = node.async(node -> {
			trace("Task 2 delaying way too long");
			delay(1000000);
			trace("Task 2 done");
		});
		trace(task1.await());
		trace("Cancelling task 2");
		task2.cancel();
		trace("Done");
	});
	coroTask.start();

	while (loop.run(DEFAULT)) { }
}

I didn't know what to use for the now() function, but this does what I expect it to do. Even cancellation seems to just work. Only thing I'm unsure about is who is supposed to call close() on the created timers.

[Simn]

I guess since we have to allocate something for ISchedulerHandle anyway, we might as well encapsulate the timer itself:

class LuvTimerEvent implements ISchedulerHandle {
	static function noOpHandle() {}

	public final timer:Timer;
	final func:() -> Void;

	public function new(loop:Loop, func:() -> Void) {
		timer = Timer.init(loop).resolve();
		this.func = func;
	}

	public function run() {
		func();
		timer.close(noOpHandle);
	}

	public function close() {
		timer.stop();
		timer.close(noOpHandle);
	}
}

And then

	function schedule(ms:Int64, func:() -> Void) {
		final event = new LuvTimerEvent(loop, func);
		event.timer.start(event.run, Int64.toInt(ms));
		return event;
	}

That still means double-allocation because the event.run closure has to allocate as well, but I don't think that could be avoided.

[Aidan63]

I've updated the SDL example to use the new root api, it's now untyped free!

[Simn]

Out of curiosity: how is this handled with regards to the GC? Specifically, if we dispatch an obj which isn't referenced anywhere else, doesn't the GC need to know about it to keep it alive?

[Aidan63]

Thats what the RootHandle which calls the underlying hx::GCAddRoot takes care of. hx::GCAddRoot takes a hx::Object**, we have to allocate that double pointer, store an object in it, and then pass it to the root function. e.g. RootHandle::create is implemented as

inline cpp::marshal::RootHandle cpp::marshal::RootHandle::create(::Dynamic obj)
{
	hx::Object** root = new hx::Object* { obj.mPtr };

	hx::GCAddRoot(root);

	return RootHandle(root);
}

This root then acts as a reference and the GC will keep the object in the root alive until the root is removed. The double pointer is for the moving and generational GC, if the object is moved the GC is able to update the object stored in the root. Because the user explicitly allocates and deletes roots with non-gc memory it's safe to pass around to non haxe generated code.

[Simn]

Ah nice, that double pointer is quite clever for this purpose!

[Simn]

I wanted to make a similar SDL sample using hlsdl, but got confused by its API being different than the C++ one. I've asked Yuxiao about this and she told me that their usage of SDL is more like this:

while(isAlive) {
    EventLoop.loop Once
    poll SDL events until no events left
    call update() and gameplay stuff
    call present() for display
}

So in such a scenario we wouldn't be interacting with the SDL loop at all and instead work with the EventLoop. If I understand correctly, this means that hlsdl can only be used with EventLoop and requires a blocking busy-loop to drive everything. For hxcoro I suppose that means that we don't have to worry about it because the actual integration would happen via EventLoop, not through SDL.

It bothers me a little that this is so all-or-nothing, but if that's the intended usage then I suppose we don't have to think about it further. Maybe I just have to get over my dislike of haxe.EventLoop and accept it as our lord and savior for some integration scenarios.

[Aidan63]

I've just updated the cpp sdl example to the new dispatcher stuff.