Skip to content

IPFW NAT rule causes kernel crash due to misaligned capability access. #2411

@qwattash

Description

@qwattash

Steps to reproduce (as far as I managed):

kenv net.inet.ip.fw.default_to_accept=1
kldload ipfw ipfw_nat

sysctl net.inet.ip.forwarding=1

# Create ipfw nat instance
ipfw nat 1 config if re0
ipfw add 101 nat 1 ip from 10.0.4.0/24 to any out via re0
# ipfw add 103 nat 1 ip from any to any in via re0
ifconfig tap create
ifconfig bridge0 addm tap0
ifconfig bridge0 inet 10.0.4.1/24

# Now boot a bhyve vm that uses tap0, configure address 10.0.4.4/24 and set default gateway to 10.0.4.1.
bhyve ...
ifconfig vtnet0 inet 10.0.4.4./24
route add default 10.0.4.1
ping 8.8.8.8   # crash

Panic stack trace:

 x0: 0xffffa08363dd6400 [rwRW,0xffffa08363dd6400-0xffffa08363dd6500]
  x1: 0x0000000000000001
  x2: 0xffff0000011af740 [rwRW,0xffff0000011af740-0xffff0000011dd580] (__start_set_vnet + 0x0)
  x3: 0xffffa08363dd6468 [rwRW,0xffffa08363dd6468-0xffffa08363dd6478]
  x4: 0xffff0001a3f438f4 [rxR,0xffff0001a3f37000-0xffff0001a3f95000] ($d.2 + 0x224)
  x5: 0x0000000000000001
  x6: 0x0000000000000000
  x7: 0xffff00004050b000 [rwRW,0xffff00004050b000-0xffff000040539000] ($d.2 + 0x3e5f7ff0)
  x8: 0x000000000000ffff
  x9: 0x0000000000000065
 x10: 0x0000000000000008
 x11: 0xffff0001a3f589a4 (ipfw_chk + 0x272f)
 x12: 0x0000000000000000
 x13: 0x0000000000000054
 x14: 0x0000000000000001
 x15: 0x0000000000000000
 x16: 0xffff0001a3f93540 [rxR,0xffff0001a3f37000-0xffff0001a3f95000] (__stop_set_pcpu + 0xb50)
 x17: 0xffff00000090a8e9 [rxR,0x0000000000000000-0xffffffffffffffff] (sentry) (strncmp + 0x0)
 x18: 0xffff000132a9c320 [rwxRW,0xffff000132a98000-0xffff000132a9e000]
 x19: 0xffff000132a9c670 [rwxRW,0xffff000132a98000-0xffff000132a9e000]
 x20: 0x0000000000000008
 x21: 0xffffa08363dd6478 [rwRW,0xffffa08363dd6450-0xffffa08363dd6500]
 x22: 0x0000000000000001
 x23: 0xffff000132a9c7b0 [rwxRW,0xffff000132a9c7b0-0xffff000132a9c870]
 x24: 0xffff0000010a6610 [rwRW,0xffff0000010a6610-0xffff0000010a6620] (vnet_start + 0x0)
 x25: 0x0000000000000008
 x26: 0x0000000000000000
 x27: 0xffffa08363dd6400 [rwRW,0xffffa08363dd6400-0xffffa08363dd6500]
 x28: 0xffff00004050b850 [rwRW,0xffff00004050b850-0xffff00004050ba60] ($d.2 + 0x3e5f8840)
 x29: 0xffff000132a9c690 [rwxRW,0xffff000132a98000-0xffff000132a9e000]
 ddc: 0x0000000000000000
  sp: 0xffff000132a9c320 [rwxRW,0xffff000132a98000-0xffff000132a9e000]
  lr: 0xffff0001a3f583e1 [rxR,0xffff0001a3f37000-0xffff0001a3f95000] (sentry) (ipfw_chk + 0x216c)
 elr: 0xffff0001a3f59990 [rxR,0xffff0001a3f37000-0xffff0001a3f95000] (ipfw_chk + 0x371b)
spsr: 0x0000000084400009
 far: 0xffffa08363dd6488
 esr: 0x0000000096000021
panic: Misaligned access from kernel space!
cpuid = 3
time = 1748264552
KDB: stack backtrace:
db_trace_self() at db_trace_self
db_trace_self_wrapper() at db_trace_self_wrapper+0x3c
vpanic() at vpanic+0x190
panic() at panic+0x48
align_abort() at align_abort+0x100
handle_el1h_sync() at handle_el1h_sync+0x14
--- exception, esr 0x96000021
ipfw_chk() at ipfw_chk+0x371c
ipfw_check_packet() at ipfw_check_packet+0x120
pfil_mbuf_out() at pfil_mbuf_out+0x64
ip_tryforward() at ip_tryforward+0x360
ip_input() at ip_input+0x448
netisr_dispatch_src() at netisr_dispatch_src+0x158
ether_demux() at ether_demux+0x1a4
ether_nh_input() at ether_nh_input+0x3fc
netisr_dispatch_src() at netisr_dispatch_src+0x158
ether_input() at ether_input+0xac
tunwrite() at tunwrite+0x5ec
devfs_write_f() at devfs_write_f+0x114
dofilewrite() at dofilewrite+0x80
kern_writev() at kern_writev+0x64
sys_writev() at sys_writev+0x48
do_el0_sync() at do_el0_sync+0x5c0
handle_el0_sync() at handle_el0_sync+0x34
--- exception, esr 0x56000000
Uptime: 51m21s

I managed to track down the invalid memory access to the capability load here

t = ((ipfw_insn_nat *)cmd)->nat;

Although I did not manage to dig into how the micro-instructions are allocated in ipfw.
cc @markjdb @brooksdavis

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions