panic: Unhandled EL1 data abort: 0x35

[bsdjhb]

I had this panic on my Morello box today when I pressed a key to open the screen locker to check on a port build.

panic: Unhandled EL1 data abort: 0x35
cpuid = 2
time = 1743777843
KDB: stack backtrace:
db_trace_self() at db_trace_self
db_trace_self_wrapper() at db_trace_self_wrapper+0x3c
vpanic() at vpanic+0x190
panic() at panic+0x48
do_el1h_sync() at do_el1h_sync+0x1b8
handle_el1h_sync() at handle_el1h_sync+0x14
--- exception, esr 0x96000035
pmap_enter() at pmap_enter+0xe88
vm_fault() at vm_fault+0x1168
vm_fault_trap() at vm_fault_trap+0x78
data_abort() at data_abort+0x1d4
do_el0_sync() at do_el0_sync+0xa8
handle_el0_sync() at handle_el0_sync+0x34
--- exception, esr 0x9200006c

This particular DFSC for a data abort is unusual:

0b110101 IMPLEMENTATION DEFINED fault (Unsupported Exclusive or Atomic access).

And it occurred on a SWP instruction to update a PTE:

at /usr/home/john/work/git/cheribsd/sys/arm64/arm64/pmap.c:5612
5612                            orig_l3 = pmap_load_store(l3, new_l3);
[11:14](https://ctsrd.slack.com/archives/CSL02MW3Y/p1743779662852239)
(kgdb) x/i $pc
=> 0xffff00000092b688 <pmap_enter+3720>:        swp     x19, x21, [c28]
[11:14](https://ctsrd.slack.com/archives/CSL02MW3Y/p1743779685694069)
(kgdb) p $c28
$2 = () 0xffffa084b7bfdc58 [rwRW,0xffffa084b7bfd000-0xffffa084b7bfe000] (invalid)

The address of $c28 is a valid DMAP address:

(kgdb) set $va = 0xffffa084b7bfdc58
(kgdb) pmap_kextract $va
KVA 0xffffa084b7bfdc58 => 0x84b7bfdc58
(kgdb) p/x dmap_phys_max
$69 = 0x8800000000
(kgdb) p/x dmap_phys_base
$70 = 0x80000000

Yet, when I try to find the PTE for it (e.g. to check if it's somehow mapped with WC due to the page previously being used by DRM which is my current theory): the L0 entry for it is invalid:

(kgdb) p/x ($va >> $L0_SHIFT) & $L0_ADDR_MASK
$73 = 0x142
(kgdb) p kernel_pmap_store.pm_l0[0x142]
$74 = 0

[bsdjhb]

My best guess so far btw is that the DRM driver allocated a page of RAM and changed its attributes to VM_MEMATTR_WC. When the page was released by DRM, it wasn't reset back to VM_MEMATTR_WRITE_BACK, and that maybe the atomic op failed when it tried to update the PTE because of that. I can still try to find the vm_page_t to see if it has a memattr set.

[bsdjhb]

(kgdb) p/x $pa
$95 = 0x84b7bfdc58
(kgdb) p/x vm_phys_segs[8].start
$96 = 0x8080000000
(kgdb) p/x $pa - 0x8080000000
$97 = 0x437bfdc58
(kgdb) p/x $97 / 4096
$98 = 0x437bfd
(kgdb) p vm_phys_segs[8].first_page[0x437bfd]
$99 = {plinks = {q = {tqe_next = 0x0, tqe_prev = 0x0}, s = {ss = {
        sle_next = 0x0}}, memguard = {p = 0, v = 0}, uma = {slab = 0x0, 
      zone = 0x0}}, listq = {tqe_next = 0x0, tqe_prev = 0x0}, object = 0x0, 
  pindex = 0, phys_addr = 570018484224, md = {pv_list = {tqh_first = 0x0, 
      tqh_last = 0xffffa0875f4d2d90 [rwRW,0xffffa0872b900000-0xffffa0877fc80000] (invalid)}, pv_gen = 0, pv_memattr = 2 '\002'}, ref_count = 0, 
  busy_lock = 4294967294, a = {{flags = 0, queue = 255 '\377', 
      act_count = 0 '\000'}, _bits = 16711680}, order = 13 '\r', 
  pool = 1 '\001', flags = 0 '\000', oflags = 0 '\000', psind = 0 '\000', 
  segind = 8 '\b', valid = 0 '\000', dirty = 0 '\000'}

That looks to be WB:

#define	VM_MEMATTR_WRITE_BACK		2

[bsdjhb]

Apparently I need the VAs to be unsigned long for my gdb scripts, but with that fixed the level 2 page table page isn't in the dump so I can't verify what the PTE was for the actual access:

(kgdb) pmap_l0 0 $va $l0e
(kgdb) p/x $l0e
$157 = 0x30000000efbbf003
(kgdb) pmap_l0_to_l1 $l0e $va $l1e
(kgdb) p/x $l1e
$158 = 0x846b971003
(kgdb) pmap_l1_to_l2 $l1e $va $l2e
Cannot access memory at address 0xffffa083eb971de8