PC just freezes after starting ksm_um.exe

[no-realm]

Type of this issue (please specify)

• This is a bug in the upstream tree as-is unmodified. (only added debug print messages)

System information

1. CPU: Intel i7 3770K (Codename: Ivy Bridge)
2. Kernel: NT Kernel (ntkrnlmp.exe)
3. Kernel version: 10.0.14393 Build number: ?
4. OS: Windows 10 x64

Build Configuration

• ENABLE_DBGPRINT
• ENABLE_FILEPRINT
• EPAGE_HOOK
• INTROSPECT_ENGINE
• PMEM_SANDBOX
• ENABLE_RESUBV

I have to mention that this also happens when I just enable EPAGE_HOOK

Issue description

My issues is, that after I start the user mode application (ksm_um.exe), my PC just freezes.
I have waited up to about 10 minutes without anything happening. No crash, nothing.
The last log entry in the log file is right before the DPC call. After that, nothing gets through.
When I tested other hypervisors, I always got some kind of feedback (good or bad) like a BSOD which would help to track down the issue.

[asamy]

Works fine for me with same build configuration. I have Windows 10 build 15063.296 however, CPU configuration shouldn't really be an issue.

The initialization of EPAGE_HOOK should never cause this issue as it's just a spin lock init, plus an identity hash table initialization, are you calling ksm_hook_epage ?

[no-realm]

No, like I mentioned. It freezes my PC right after executing the user mode application.
I also just tried it in a VM and it did not have any issues there :/

Maybe I haven't expressed myself clearly enough: The driver itself (sc start) loads just fine, but after the UM application sends the 'subvert' command, the pc freezes.

[no-realm]

The annoying thing is that I don't even have a dump file or something similar to look though.
I will try limiting my CPUs and/or RAM amount. Maybe that will help.

[asamy]

Yeah, I understand. I am saying since it's only initialization of epage hook that is the suspect then it shouldn't really cause the issue. If it really is epage that's causing the issue, then can you modify do_ept_violation (vcpu.c), to look like this:

static bool do_ept_violation(struct ept_ve_around *ve)
{
	struct vcpu *vcpu = ve->vcpu;
	struct ept *ept = &vcpu->ept;
	struct ksm *k = vcpu_to_ksm(vcpu);
	struct ve_except_info *info = ve->info;

	if ((info->exit & EPT_VE_RWX) == 0) {	/* no access  */
		if (!ept_alloc_page(EPT4(ept, info->eptp),
				    EPT_ACCESS_ALL, info->gpa, info->gpa))
			return false;

		return true;
	}

	KSM_PANIC(EPT_BUGCHECK_CODE, EPT_UNHANDLED_VIOLATION, info->exit, info->gpa);
	return false;
}

And let me know if a bugcheck occurs then? Also, since you enabled fileprint, post the log too.

[no-realm]

I don't think it has anything to do with epage hook.
I didn't try without it, since I didn't even come to a point where that would be relevant.
But as you suggested, I tried the modification you posted, but my PC still just froze.
No BSOD/Bugcheck.

The log isn't very eventful, but here it is:

ksm: CPU 1: check_dynamic_pgtables: PXE: FFFF8BC5E2F17000 PPE FFFF8BC5E2E00000 PDE FFFF8BC5C0000000 PTE FFFF8B8000000000
ksm: CPU 4: check_dynamic_pgtables: Addr 0x22DC7F3C 0x22C00F3C
ksm: CPU 4: DriverEntry: We're mapped at FFFFF800A13C0000 (size: 65536 bytes (64 KB), on 16 pages)
ksm: CPU 4: ksm_init: EPT/VPID caps: 0x00000F0106114141
ksm: CPU 4: ksm_init: 8 physical memory ranges
ksm: CPU 0: ksm_init: Range: 0x0000000000001000 -> 0x000000000009F000
ksm: CPU 0: ksm_init: Range: 0x0000000000100000 -> 0x0000000020000000
ksm: CPU 0: ksm_init: Range: 0x0000000020200000 -> 0x0000000040004000
ksm: CPU 4: ksm_init: Range: 0x0000000040005000 -> 0x00000000C8F1C000
ksm: CPU 4: ksm_init: Range: 0x00000000C9848000 -> 0x00000000C98D4000
ksm: CPU 4: ksm_init: Range: 0x00000000CA177000 -> 0x00000000CA178000
ksm: CPU 4: ksm_init: Range: 0x00000000CA1BB000 -> 0x00000000CABF6000
ksm: CPU 2: ksm_init: Range: 0x00000000CAFF2000 -> 0x00000000CB000000
ksm: CPU 0: DriverEntry: ready
ksm: CPU 6: DriverEntry: ret: 0x00000000
ksm: CPU 0: DriverDispatch: open from ksm_um.exe
ksm: CPU 3: DriverDispatch: ksm_um.exe: IOCTL: 0x8008E008 of length: 0

Edit:
Like I mentioned above, I tested it with limited ram and/or cpus, and the results are a bit surprising.
The cpu count doesn't seem to influence whether it freezes or not, but when I limited my ram amount to 4GB, it didn't freeze...
After a couple minutes, a BSOD popped up though. The same as mentioned in #19 but without sandboxing any process. DRIVER_IRQL_NOT_LESS_OR_EQUAL

[asamy]

Weird about the memory issue, 8 physical memory ranges shouldn't be too much for it to handle. I don't know what the closest possible cause is, if it'd be too much for NonPagedPool then it'd have crashed, I think.
Can you upload the minidump, ksm.sys, ksm.pdb and ntoskrnl.exe?

[no-realm]

You mean the minidump for the 4GB RAM crash?
Edit: I am trying to reproduce the bsod crash right now, but it hasn't crashed for now.
Edit2: It's getting even weirder now. Just now my system just restarted. No BSOD or minidump.
Edit3: This time it just froze about one minute after successfully subverting the cpus. No crash here too.

[no-realm]

Now I finally got a crash and a dump file that can be examined.
Here are the relevant files:
files.zip
I have included the ksm log, minidump, ksm.sys, ksm.pdb and ntoskrnl.exe.

Edit: This here may actually be the right kernel file. At least WinDbg says it is the mapped one.
ntkrnlmp.zip

[asamy]

So seems to be an LIDT problem:

6: kd> u 0xffff9c0d`23a020bc
ffff9c0d`23a020bc 0f23f8          mov     dr7,rax
ffff9c0d`23a020bf 0f019d78040000  lidt    tbyte ptr [rbp+478h]  # faulting PC
ffff9c0d`23a020c6 ebe3            jmp     ffff9c0d`23a020ab
ffff9c0d`23a020c8 33c0            xor     eax,eax
ffff9c0d`23a020ca 41bb20000000    mov     r11d,20h
ffff9c0d`23a020d0 41898424e0050000 mov     dword ptr [r12+5E0h],eax
ffff9c0d`23a020d8 41f78424d006000000000040 test dword ptr [r12+6D0h],40000000h
ffff9c0d`23a020e4 48bee8b4c891583fa0a3 mov rsi,0A3A03F5891C8B4E8h

This happens when KSM fails to read from guest VA and therefore injects a page fault, I think it might be a fault on how KSM reads from guest virtual memory. I will take a look at this again in a few minutes but in the meantime... can you try without commit c956379?

[no-realm]

So, I have tried it without that commit and got a crash.
Here are the files:
ksm.zip
NT kernel should be the same.

[asamy]

Commit 172ca1f should do it.

[no-realm]

ok, I will try

[no-realm]

So, I have tried the latest commit but had no luck until now.
I can't get Windows to crash, it just freezes, so there is no dump for now..

[asamy]

Does it freeze when coming back from a sleep state? I wonder if you can let a VM freeze while having a debugger attached then break when it hangs to see where it's hanging at.

[no-realm]

I would have to set up a new VM since I somehow corrupted my old one when experimenting with hypervisors.
I will report when I have tested it in a VM. I would have to either use the release build or use a static c-runtime for the debug build to run it in the VM though.

And I also don't really know how to debug a VM :/