Using Window functions across DLL boundaries

[stofffe]

Hello everyone! I am writing an engine with support for hot reloading using DLLs, but I am having problems with calling Window functions over the DLL boundary.

The main structure of my project is.

1. The core engine is a crate which creates the window and runs the event loop
2. The user application creates an app that implements a Callback trait which then is injected into the core engine.

/// User callbaks
pub trait Callbacks {
    /// Use a custom `ContextBuilder`
    fn init_ctx() -> ContextBuilder {
        ContextBuilder::new()
    }

    /// Called after context initilization and before game/update loop
    fn new(_ctx: &mut Context, cache: &mut AssetCache) -> Self;

    /// Called once per frame after update
    ///
    /// Return value determines wether to exit game or not
    ///
    /// Must submit at least one render pass, panics otherwise
    fn render(
        &mut self,
        _ctx: &mut Context,
        _cache: &mut AssetCache,
        _screen_view: &wgpu::TextureView,
    ) -> bool {
        false
    }

    /// Called after window resize
    fn resize(
        &mut self,
        _ctx: &mut Context,
        _cache: &mut AssetCache,
        _new_size: winit::dpi::PhysicalSize<u32>,
    ) {
    }

    fn window_event(&mut self, _ctx: &mut Context, _event: &winit::event::WindowEvent) {}
}

pub async fn run<C: Callbacks>() {
    C::init_ctx().init_logging();

    let event_loop = winit::event_loop::EventLoop::with_user_event()
        .build()
        .expect("could not create event loop");

    let mut app = App::Uninitialized::<C> {
        proxy: Some(event_loop.create_proxy()),
        builder: C::init_ctx(),
    };

    event_loop
        .run_app(&mut app)
        .expect("could not run event loop");
}

#[cfg(not(target_arch = "wasm32"))]
pub fn run_sync<C: Callbacks>() {
    pollster::block_on(run::<C>())
}

/// general engine state
///
/// can be both initialized and uninitialized
#[allow(clippy::large_enum_variant)]
enum App<C: Callbacks> {
    Uninitialized {
        proxy: Option<winit::event_loop::EventLoopProxy<Context>>,
        builder: ContextBuilder,
    },
    Initialized {
        ctx: Context,

        cache: AssetCache,

        #[cfg(not(feature = "hot_reload"))]
        callbacks: C,
        #[cfg(feature = "hot_reload")]
        callbacks: DllCallbacks<C>,
    },
}

impl<C: Callbacks> winit::application::ApplicationHandler<Context> for App<C> {
    fn user_event(&mut self, _event_loop: &winit::event_loop::ActiveEventLoop, mut ctx: Context) {
        // TODO: init here?
        let mut cache = AssetCache::new();

        #[cfg(not(feature = "hot_reload"))]
        let callbacks = C::new(&mut ctx, &mut cache);

        #[cfg(feature = "hot_reload")]
        let callbacks = DllCallbacks::new(&mut ctx, &mut cache);

        *self = App::Initialized {
            callbacks,
            ctx,
            cache,
        };
    }

    fn resumed(&mut self, event_loop: &winit::event_loop::ActiveEventLoop) {
        // Ignore if already initialized
        let App::Uninitialized { proxy, builder } = self else {
            return;
        };
        if proxy.is_none() {
            return;
        }

        // initialize context
        async fn init(
            window: winit::window::Window,
            builder: ContextBuilder,
            proxy: winit::event_loop::EventLoopProxy<Context>,
        ) {
            let input = input::InputContext::new();
            let time = time::TimeContext::default();
            let filesystem = filesystem::FileSystemContext::new();
            let audio = audio::AudioContext::new();
            let render = render::RenderContext::new(window, &builder).await;
            let random = random::RandomContext::new();

            let ctx = Context {
                input,
                time,
                filesystem,
                audio,
                render,
                random,

                #[cfg(feature = "hot_reload")]
                hot_reload: hot_reload::HotReloadContext::new(),
            };

            let sucess = proxy.send_event(ctx).is_ok();
            assert!(sucess, "could not send context event");
        }

        let proxy = proxy.take().unwrap();
        let mut builder = builder.clone();

        #[cfg(target_arch = "wasm32")]
        {
            use web_sys::wasm_bindgen::JsCast;
            use winit::platform::web::WindowAttributesExtWebSys;

            let win = web_sys::window().unwrap();
            let document = win.document().unwrap();
            let canvas = document.get_element_by_id("gbase").unwrap();
            // let html_canvas_element = canvas.unchecked_into();
            let canvas = document
                .get_element_by_id("gbase")
                .expect("could not find canvas")
                .dyn_into::<web_sys::HtmlCanvasElement>()
                .expect("element was not a canvas");
            let (width, height) = (canvas.width(), canvas.height());
            builder.window_attributes = builder
                .window_attributes
                .with_canvas(Some(canvas))
                .with_inner_size(winit::dpi::LogicalSize::new(width, height));
        }

        let window = event_loop
            .create_window(builder.window_attributes.clone())
            .expect("could not create window");

        #[cfg(target_arch = "wasm32")]
        wasm_bindgen_futures::spawn_local(init(window, builder, proxy));

        #[cfg(not(target_arch = "wasm32"))]
        pollster::block_on(init(window, builder, proxy));
    }

    fn about_to_wait(&mut self, _event_loop: &winit::event_loop::ActiveEventLoop) {
        let App::Initialized { ref mut ctx, .. } = self else {
            tracing::warn!("app not initialized while receiving about to wait event -> skipping");
            return;
        };

        ctx.render.window().request_redraw();
    }

    fn device_event(
        &mut self,
        _event_loop: &winit::event_loop::ActiveEventLoop,
        _device_id: event::DeviceId,
        event: event::DeviceEvent,
    ) {
        let App::Initialized { ref mut ctx, .. } = self else {
            tracing::warn!("app not initialized while receiving device event -> skipping");
            return;
        };

        #[allow(clippy::single_match)]
        match event {
            DeviceEvent::MouseMotion { delta } => {
                if ctx.input.mouse.mouse_on_screen() {
                    ctx.input.mouse.set_mouse_delta(delta);
                }
            }
            _ => {}
        }
    }

    fn window_event(
        &mut self,
        event_loop: &winit::event_loop::ActiveEventLoop,
        _window_id: winit::window::WindowId,
        event: event::WindowEvent,
    ) {
        let App::Initialized {
            ref mut ctx,
            callbacks,
            cache,
        } = self
        else {
            tracing::warn!("app not initialized while receiving window event -> skipping");
            return;
        };

        // TODO: temp for egui
        println!("WINDOW EVENT");
        callbacks.window_event(ctx, &event);

        match event {
            WindowEvent::RedrawRequested => {
                // hot reload
                #[cfg(feature = "hot_reload")]
                {
                    if ctx.hot_reload.should_reload() {
                        tracing::info!("Hot reload");
                        callbacks.hot_reload(ctx, cache);
                    }
                    if ctx.hot_reload.should_restart() {
                        tracing::info!("Hot restart");
                        callbacks.hot_restart(ctx, cache);
                    }
                }

                // update
                if update_and_render(ctx, cache, callbacks) {
                    event_loop.exit();
                }
            }
            WindowEvent::CloseRequested => event_loop.exit(),
            WindowEvent::Resized(new_size) => {
                ctx.render.resize_window(new_size);
                callbacks.resize(ctx, cache, new_size);
            }
            // Keyboard
            WindowEvent::KeyboardInput { event, .. } => {
                let (key, pressed) = (event.physical_key, event.state.is_pressed());
                match (key, pressed) {
                    (PhysicalKey::Code(code), true) => ctx.input.keyboard.set_key(code),
                    (PhysicalKey::Code(code), false) => ctx.input.keyboard.release_key(code),
                    (PhysicalKey::Unidentified(code), _) => {
                        tracing::error!("pressed/released unidentified key {:?}", code)
                    }
                };
            }
            WindowEvent::ModifiersChanged(modifiers) => {
                ctx.input.keyboard.modifiers_changed(modifiers)
            }
            // Mouse
            WindowEvent::MouseInput { state, button, .. } => {
                match state {
                    winit::event::ElementState::Pressed => ctx.input.mouse.press_button(button),
                    winit::event::ElementState::Released => ctx.input.mouse.release_button(button),
                };
            }
            WindowEvent::MouseWheel { delta, .. } => match delta {
                winit::event::MouseScrollDelta::LineDelta(x, y) => {
                    ctx.input.mouse.set_scroll_delta((x as f64, y as f64));
                }
                winit::event::MouseScrollDelta::PixelDelta(pos) => {
                    ctx.input.mouse.set_scroll_delta((pos).into());
                }
            },
            WindowEvent::CursorMoved { position, .. } => {
                ctx.input.mouse.set_pos(position.into());
            }
            WindowEvent::CursorEntered { .. } => {
                ctx.input.mouse.set_on_screen(true);
            }
            WindowEvent::CursorLeft { .. } => {
                ctx.input.mouse.set_on_screen(false);
            }
            _ => {}
        }
    }
}

type NewFunc<T> = fn(ctx: &mut crate::Context, cache: &mut crate::asset::AssetCache) -> T;
type RenderFunc<T> = fn(
    callbacks: &mut T,
    ctx: &mut crate::Context,
    cache: &mut crate::asset::AssetCache,
    screen_view: &wgpu::TextureView,
) -> bool;
type ResizeFunc<T> = fn(
    callbacks: &mut T,
    ctx: &mut crate::Context,
    cache: &mut crate::asset::AssetCache,
    new_size: winit::dpi::PhysicalSize<u32>,
);
type ReloadFunc<T> =
    fn(callbacks: &mut T, ctx: &mut crate::Context, cache: &mut crate::asset::AssetCache);
type WindowEventFunc<T> =
    fn(callbacks: &mut T, ctx: &mut crate::Context, event: &winit::event::WindowEvent);

pub struct DllApi<T> {
    new_callback: NewFunc<T>,
    render_callback: Option<RenderFunc<T>>,
    resize_callback: Option<ResizeFunc<T>>,
    reload_callback: Option<ReloadFunc<T>>,
    window_event_callback: Option<WindowEventFunc<T>>,
}

/// Wrapper for callbacks + dll
pub struct DllCallbacks<T> {
    pub callbacks: T,
    pub dll: DllApi<T>,
}

impl<T> crate::Callbacks for DllCallbacks<T> {
    // NOTE:
    // init_ctx should only be called at program startup and never through dll
    // so we never store it here and we panic if called on DllContext
    fn init_ctx() -> crate::ContextBuilder {
        panic!("init_ctx on DllCallbacks should never be called");
    }

    fn new(ctx: &mut crate::Context, cache: &mut crate::asset::AssetCache) -> Self {
        let dll = load_dll();

        let mut callbacks = (dll.new_callback)(ctx, cache);

        if let Some(hot_reload) = dll.reload_callback {
            hot_reload(&mut callbacks, ctx, cache);
        }

        Self { callbacks, dll }
    }

    fn render(
        &mut self,
        ctx: &mut crate::Context,
        cache: &mut crate::asset::AssetCache,
        screen_view: &wgpu::TextureView,
    ) -> bool {
        match self.dll.render_callback {
            Some(render) => render(&mut self.callbacks, ctx, cache, screen_view),
            None => false,
        }
    }

    fn resize(
        &mut self,
        ctx: &mut crate::Context,
        cache: &mut crate::asset::AssetCache,
        new_size: winit::dpi::PhysicalSize<u32>,
    ) {
        #[allow(clippy::single_match)]
        match self.dll.resize_callback {
            Some(resize) => resize(&mut self.callbacks, ctx, cache, new_size),
            None => {}
        }
    }

    fn window_event(&mut self, ctx: &mut crate::Context, event: &winit::event::WindowEvent) {
        #[allow(clippy::single_match)]
        match self.dll.window_event_callback {
            Some(window_event) => window_event(&mut self.callbacks, ctx, event),
            None => {}
        }
    }
}

impl<T> DllCallbacks<T> {
    /// reload dll file
    ///
    /// keep game state
    pub fn hot_reload(&mut self, ctx: &mut crate::Context, cache: &mut crate::asset::AssetCache) {
        self.dll = load_dll();

        if let Some(hot_reload) = self.dll.reload_callback {
            hot_reload(&mut self.callbacks, ctx, cache);
        }
    }

    /// reload dll file
    ///
    /// reset game state
    pub fn hot_restart(&mut self, ctx: &mut crate::Context, cache: &mut crate::asset::AssetCache) {
        self.hot_reload(ctx, cache);
        self.callbacks = (self.dll.new_callback)(ctx, cache);
    }
}

fn load_dll<T>() -> DllApi<T> {
    let lib = dlopen::symbor::Library::open(super::dllname()).unwrap();

    let new_callback = match unsafe { lib.symbol::<NewFunc<T>>("new") } {
        Ok(f) => *f,
        Err(err) => {
            tracing::error!("could not find function new");
            tracing::error!("TIP: make sure callbacks are defined in library and not main.rs");
            tracing::error!("TIP: make sure functions are marked with #[no_mangle]");
            panic!("{}", err);
        }
    };
    let render_callback = match unsafe { lib.symbol::<RenderFunc<T>>("render") } {
        Ok(f) => Some(*f),
        Err(err) => {
            tracing::warn!("could not find function render: {}", err);
            None
        }
    };
    let resize_callback = match unsafe { lib.symbol::<ResizeFunc<T>>("resize") } {
        Ok(f) => Some(*f),
        Err(err) => {
            tracing::warn!("could not find function resize: {}", err);
            None
        }
    };
    let reload_callback = match unsafe { lib.symbol::<ReloadFunc<T>>("hot_reload") } {
        Ok(f) => Some(*f),
        Err(err) => {
            tracing::warn!("could not find function hot_reload: {}", err);
            None
        }
    };
    let window_event_callback = match unsafe { lib.symbol::<WindowEventFunc<T>>("window_event") } {
        Ok(f) => Some(*f),
        Err(err) => {
            tracing::warn!("could not find function window_event: {}", err);
            None
        }
    };

    DllApi {
        new_callback,
        render_callback,
        resize_callback,
        reload_callback,
        window_event_callback,
    }
}

When running in release mode, the callback trait is created on the main thread. When running in hot_reload mode a wrapper is used that loads the callback code from a DLL. In release mode everything works as expected. However, in the hot_reload mode the Window functions are not found (on MACOS).

invalid message send to -[WinitApplicationDelegate frame]: method not found

I know window functions must be called from the main thread in MACOS. But how does it work across DLL boundaries, isnt it still being called from the main thread? Any help would be very appreciated.

One solution would be to cache the relevant data in the core engine. However, I want to use crates such as egui which need to call these window methods directly.

I have had problems with logging since its a static variable that gets different values for different compilation units, could this be similar to that problem?

[madsmtm]

I'll close this in favour of #4344