Improving decompilation for SC/MP language spec

[sigurasg]

I'm writing a language spec for the SC/MP processor for giggles. This processor is of mid-to late 70s vintage, and the instruction set has some interesting quirks.

There are two issues that make the decompilation awful at the moment:

1. Pointer registers roll over at 12 bits.
2. The XMPP instruction is used for call, jump and return, and the decompiler can't seem to figure out which is which.

Additionally the decompiler can't seem to figure out the value of pointer registers on function return, though this might be related to the other two.

The processor has 4 pointer registers; PC, P1, P2 and P3, all of which are effectively divided into 4 MSB of "page" and 12 LSB of "address". When e.g. PC is incremented, only the first 12 bits increment, but the top 4 bits are unchanged. The same happens with indexed and auto-indexed adressing. The KITBUG monitor program I've been looking at uses this wraparound to its advantage, by referencing globals and stack in RAM by wrapping up to 0x0FFF from the zero page with PC-relative addressing.
This means that every increment of e.g. P2 (often used as stack pointer) becomes something like P2 = (P2 & 0xF000) | ((P2 + 1) & 0x0FFF), which then bleeds through to the decompiled code. I wonder if using a segmentop user-defined PCode will help with this?

The other problem is the XMPP instruction, which exchanges PC with another pointer register. This is used both to call functions and to return from functions, and apparently it's even used as a jump. The decompiler can't seem to figure out which case applies when, which makes a total hash of non-leaf functions.
Even after I fix the XPPC P3 instruction flows by hand (to CALL, in this case), the decompiler seems unable to infer the value of the P3 register on return, so I get hash like this:

  PUTC('\r');
  uVar5 = 0x4b;
  (*(code *)(uVar4 & 0xf000 | uVar4 + 1 & 0xfff))(10);  ; PUTC('\n');
  (*(code *)(uVar5 & 0xf000 | uVar5 + 1 & 0xfff))(0x2d); ; PUTC('-');

from assembly like this:

                             **************************************************************
                             *                          FUNCTION                          *
                             **************************************************************
                             void default CMDLP(void)
             void              <VOID>         <RETURN>
                             CMDLP
            003a c4 f6           LDI        0xf6
            003c 32              XPAL       P2
            003d c4 0f           LDI        0xf
            003f 36              XPAH       P2
            0040 c4 01           LDI        0x1
            0042 37              XPAH       P3
            0043 c4 c4           LDI        0xc4
            0045 33              XPAL       P3
            0046 c4 0d           LDI        0xd
            0048 3f              XPPC       P3                                               void PUTC(char param_1)
; The value of P3 here is the address of the returning XPPC instruction.
; This is followed by a JMP GETC instruction, which allows "chaining"
; function calls without reloading P3 with the function address each time.
                             -- Flow Override: CALL (COMPUTED_CALL)
            0049 c4 0a           LDI        0xa
            004b 3f              XPPC       P3 
                             -- Flow Override: CALL (COMPUTED_CALL)
                             SUB_004c                                        XREF[1]:     004e(c)  
            004c c4 2d           LDI        0x2d
             assume P3 = 0x1c4
            004e 3f              XPPC       P3=>FUN_004f                                     undefined FUN_004f()
                             -- Flow Override: CALL (COMPUTED_CALL)

I've tried to mess with the .cspec to try and explain to the decompiler that P3 is always an output from each function,
but so far no joy.

Any ideas for how I can improve this, or what I'm doing wrong?