Deep recursion in resolve_unit_addrs_overlap_walk

[MikailBag]

Hi.

As far as I understand, in response to #137 a preprocessing step was added which identifies nested ranges and ensures that they are correctly symbolized.

However, this processing uses recursion and it turns out that in some cases this recursion can be deep. E.g. I have a binary (unfortunately, I'm not able to show it) where more than 2000 recursive calls to the resolve_unit_addrs_overlap_walk happen. And when library initialization happens in a some limited-stack environment, such as a fiber, this is enough to overflow stack.

I looked at the code a bit and it seems that recursion can be avoided without sacrificing readability (option 1: maintain a stack of current enclosing entries, option 2: maintain a vector of size count which maps each entry to the index of the immediately enclosing one).

I think this situation is worth improving because:

1. Documentation does not seem to mention how much stack you need to have to initialize libbacktrace.
2. Even disregarding fibers, this problem has chance to trigger on native thread too. I.e. if binaries with 2000 nested ranges are possible, it seems possible that on larger binaries even more ranges can be nested too.

If you are OK, I'll try to make a patch as well.

[ianlancetaylor]

Thanks for the note. If you could provide even an artificial test case, that would help.

If you want to send a patch that would be great.

Thanks.

[MikailBag]

If you could provide even an artificial test case

Unfortunately, I have no idea how to make one (i.e. I have only one reproducer binary, but I can't share it because it's closed source). I'll try to get help from my colleagues but I can't guarantee anything, sorry.

If you want to send a patch that would be great.

I've created #156 . I probably messed up the formatting (will look into it tomorrow), other than that PR should be ready for review.