Python Script Debugging

[bdemick]

I'm having a heck of a time being able to get the Eclipse PyDev debugger to catch any kind of breakpoint in a Ghidra Python script, both on macOS and Linux (Ubuntu 20). I have attempted the workarounds in #2713 and #1707 with no luck. Both environments are running Ghidra 9.2.3, with the bundled GhidraDev and PyDev 8.1.0 plugins installed, following the process in GhidraDev_README.html. Eclipse seems to find the packaged Jython.

My process has been:

1. Install Ghidra
2. Install PyDev plugin
3. Install GhidraDev plugin
4. Launch Eclipse
5. Create new GhidraScript project
6. Import script to project
7. Open script file, set breakpoint
8. Click Debug->DebugAs->Ghidra, Ghidra launches
9. Add script folder to Ghidra's script manager
10. Run script, expecting breakpoint to be hit, to no avail

Is there a start-to-finish guide anywhere for configuring Eclipse to debug a simple Python script? I must be missing something incredibly simple in my configuration or process. In both environments, the script just completes.

[ryanmkurtz]

Hmmm, I just tested your setup and didn't have an issue. I did the following:

1. Installed Ghidra 9.2.3 from ghidra-sre.org
2. Installed Eclipse for Java 2021-03
3. Installed the GhidraDev 2.1.2 that was found in the Ghidra installation directory under Extensions/Eclipse/GhidraDev.
4. Installed PyDev 8.1 which I downloaded from SourceForge
5. Created new GhidraScript project using the GhidraDev wizard. In the wizard, I pointed it at my 9.2.3 installation, and clicked the + button on the Python page. It successfully found and added the jython interpreter that lives in the Ghidra installation.
6. I opened up python_basics.py from my GhidraScript project, and set a breakpoint.
7. I did Debug->DebugAs->Ghidra
8. In Ghidra's Script Manager, I ran python_basics.py.
9. Breakpoint hit in Eclipse

Can you try debugging a script that ships with Ghidra, like python_basics.py? Perhaps there is an issue with adding new script folders to the Script Manager...

[bdemick]

Definitely not (on Mac) able to even get the python_basics.py script breakpoint to hit, following the steps above. Oddly, sometimes when I create a new GhidraScript project, it fails at the Jython interpreter linking stage, and other times it seems to have no problem. Going to give it another shot on the Linux side. The only thing that I may have previously been missing was the "Link to an alternate location (Linked Folder)" step for bringing in existing Ghidra scripts, but the fact that I can't even get the packaged scripts to debug seems like a problem.

Any other ideas? I just repeated the setup again on a Linux VM with the Eclipse for Java 2021-03, and I'm still not catching a breakpoint on the packaged python_basics.py script. When setting up a new GhidraScript project on Linux, I got this popup from the PyDev plugin (OK'd the default settings):

Everything seems to work smoothly (setup, launching Ghidra from Eclipse, debug server seems to be running), except for the actual breakpoint.

[ryanmkurtz]

You are definitely using Ghidra 9.2.3 and the GhidraDev plugin that ships with it? The behavior you are describing would make perfect sense if you were using any version prior to 9.2.3.

[bdemick]

Yes, definitely 9.2.3, and definitely using the plugin from Ghidra/Extensions/Eclipse/GhidraDev.

[ryanmkurtz]

When you launch Ghidra using the Debug As run configuration, you should see PyDev starting up its Debug Server on port 5678. Do you see that?

[bdemick]

Yes, I am able to see this on both macOS and Linux, although no further output shows up indicating connections or anything like that.

[chiffreKing]

I'm having the same problem? is there a solution for this issue yet? workarounds in #2713 and #1707 won't work for me too.

[ryanmkurtz]

Sorry but I'm stumped...I can't reproduce it. In Ubuntu 21.04 I just installed a new Ghidra 10.0.1, Eclipse 2021-06, PyDev 8.3.0 (through Eclipse Marketplace), and GhidraDev 2.1.3 (from Ghidra 10.0.1 directory). I created a new Ghidra Scripting Project, pointing the project at Ghidra 10.0.1. I then expanded the Ghidra Python scripts project, opened up python_basics.py, set a breakpoint near the beginning, launched Ghidra via my new GhidraScripts Eclipse project with the Ghidra Debug run configuration, went into the Script Manager and ran python_basics.py. 10 seconds later or so, the breakpoint hit in Eclipse, and I was able to step:

[chiffreKing]

I got it to work now. I'm not quite sure where the mistake was.
Under windows I used ghidra 9.2.4, the given ghidradev and a pydev version. I don't know which pydev version it was cause i deleted it but it was probably pydev8.3.0. I used 'eclipse for c/c++ developers' maye there was the mistake but that wouldn't make sence with a working pydev.
the new setup:
ghidra 10.1, given ghidradev automattically installed it by ghidra in eclipse, newest 'eclipse for java developers' 2021-06 pydev8.3.0
I basically followed the ghidraDev readme.
Afterwards used my own .py script: set breakpoint -> debugAs Ghidra -> selected project in ghidra -> executed script in script manager -> after a while i was able to click on the buttons "step into" and the other ones. That didnt work before. So breakpoint was hit and step by step debugging is working.
Thanks for your help.

P.S.: One last question: I was told to use an older eclipse version because this one is having trouble with pydev and jython. But it all works. what is jython good for?

[ryanmkurtz]

I'm happy to hear you got it working. I have never run into issues with needing to use an older Eclipse. We used to recommend using an older PyDev but those issues have been fixed now.

[chiffreKing]

Alright, I was able to reproduce this.
When you start a debug session in eclipse with any .py script that is not for ghidra, it opens up the debug window where you can see the threads. ryanmkurtz in your last pic from above the debug window is on the left. This window is opened automatically when starting a debug session. If you close this window the debug session stops.

I don't know why but when I start a debugging session for a ghidra script it doesn't open up this window. It gets a connection to the port though and starts ghidra. Then I launch the script in ghidra and console tells me that there is a connection with the debugger. The script works too until it reaches a breackpoint. There it seems like it is waiting for the user to do the next "step into" but when I then look in eclipse I'm not able to click on the button "step into". When I go for "window"->"show view"->"debug" it opens up the debug window. When this window is open I can click on step into.

So what basically happens sometimes is that this debug window is not opened automatically again after I closed it ones.

[ryanmkurtz]

When you launch Ghidra in debug mode the first time GhidraDev suggests to Eclipse to switch to the debug perspective, which is what has the step buttons on it. You can switch to the debug perspective at any time in Eclipse with the little debug icon in the upper right toolbar. You don't have to be debugging to do this. Eclipse perspectives are just predefined layouts. If you reach a break point, switch to the debug perspective, and the step buttons are disabled, I would consider that a problem worth investigating.