Once you are 2+ levels deep into a tree, it requires multiple ESC to get back to editing.
With the default config and leader set to SPC, if you just presss SPC once, qf is already selected, so it takes two presses to return.
In VSpaceCode, for example, you can quit from any level quickly with one ESC.