[N64] Visual issue on console is not present on emulator (modded GoldenEye 007)

[AXDOOMER]

Describe the bug

In my mod, there's a visual issue on console which isn't present in Ares. See below for real hardware and emulator screenshots.

To Reproduce
Steps to reproduce the behavior:

1. Build Ares from the current latest commit: c95ddfe

2. Patch GoldenEye (NTSC) with the xdelta. buggy-test-rom.xdelta.zip

3. Enter single player or multiplayer mode

Expected behavior

Emulator should render the visual issue. These are screenshots from real hardware.

Screenshots

Instead, there's no visual issue on the emulator.

Additional context

The bug is still being investigated to find the root cause and a fix. Unfortunately it slipped through while I was developing my mod because it didn't show in Ares, Cen64 and ParaLLEl N64.

I will post a message in the thread when it's found.

[AXDOOMER]

Cause of the bug:

This libultra code which sets VI features

void osViSetSpecialFeatures(u32 func) {
    register u32 saveMask;
    saveMask = __osDisableInt();
    if ((func & OS_VI_GAMMA_ON) != 0) {
        __osViNext->control |= VI_CTRL_GAMMA_ON;
    }
    if ((func & OS_VI_GAMMA_OFF) != 0) {
        __osViNext->control &= ~VI_CTRL_GAMMA_ON;
    }
    if ((func & OS_VI_GAMMA_DITHER_ON) != 0) {
        __osViNext->control |= VI_CTRL_GAMMA_DITHER_ON;
    }
    if ((func & OS_VI_GAMMA_DITHER_OFF) != 0) {
        __osViNext->control &= ~VI_CTRL_GAMMA_DITHER_ON;
    }
    if ((func & OS_VI_DIVOT_ON) != 0) {
        __osViNext->control |= VI_CTRL_DIVOT_ON;
    }
    if ((func & OS_VI_DIVOT_OFF) != 0) {
        __osViNext->control &= ~VI_CTRL_DIVOT_ON;
    }
    if ((func & OS_VI_DITHER_FILTER_ON) != 0) {
        __osViNext->control |= VI_CTRL_DITHER_FILTER_ON;
        __osViNext->control &= ~VI_CTRL_ANTIALIAS_MASK;
    }
    if ((func & OS_VI_DITHER_FILTER_OFF) != 0) {
        __osViNext->control &= ~VI_CTRL_DITHER_FILTER_ON;
        __osViNext->control |= __osViNext->modep->comRegs.ctrl & VI_CTRL_ANTIALIAS_MASK;
    }
    __osViNext->state |= VI_STATE_CONTROL;

    __osRestoreInt(saveMask);
}

Was replaced with this code:

void osViSetSpecialFeatures(u32 func) {
    register u32 saveMask;
    saveMask = __osDisableInt();

    __osViNext->control &= ~VI_CTRL_GAMMA_ON;
    
    __osViNext->control &= ~VI_CTRL_DIVOT_ON;
    __osViNext->control &= ~VI_CTRL_DITHER_FILTER_ON;
    __osViNext->control |= VI_CTRL_ANTIALIAS_MASK;

    __osViNext->state |= VI_STATE_CONTROL;

    __osRestoreInt(saveMask);
}

[rasky]

Can you dump the value of VI_CTRL after you configure it specially?
EDIT: better dump the hex value of all registers if possible

[rasky]

Libdragon has a VI validator. We can move that to Ares as well, so that homebrew mode can at least warn about known-bad VI configurations.

[AXDOOMER]

The control value for the version that's buggy on console is 0x3306. The vanilla version has control value 0x13016.

[rasky]

It's probably this bug:
https://github.com/DragonMinded/libdragon/blob/3d63ffa193ed2e1ef3e0d207a690e81639b5ba06/src/vi.c#L132-L137

Libdragon automatic workaround for that is setting XSCALE to 0x201:
https://github.com/DragonMinded/libdragon/blob/3d63ffa193ed2e1ef3e0d207a690e81639b5ba06/src/display.c#L358-L369