Serializing Pcode From Decompiler Results

[aa-software2112]

Hello Ghidra Devs,

After construction an analysis graph that includes PCodeOpAST elements as nodes (really my own custom objects that wrap these Pcode elements), I want to save my graph so that when I rerun analysis, I can simply load the graph from a file as opposed to re-decompiling, and re-computing the (very large) graphs.

Is PCode serializable in the sense that I can dump it with the graph, or are the dependencies within Pcode (like Varnodes) so deep that it would be a foolish venture? In this sense, would it be smarter to simply extract the more fundamental contents from Pcode (e.g., operation address, operation name) necessary for analysis, and store that instead?

Thanks for the insights!

Anthony

[d-millar]

@aa-software2112 There’s an ExportPCodeToCTADL script in Ghidra/Features/DecompilerDependent that exports pretty much everything there is to know about pcode from a given program (or function) to a directory of “fact” files, which can be used to recreate almost everything (not automatically, but we do this for CTADL processing). Probably a good starting point.

[aa-software2112]

After checking that code, from my understanding, if I wanted to export a PCode graph, then re-load it back into Ghidra, I'd have to manually re-instantiate the pcode objects and associate them to their proper Varnodes based on the output file contents. Of course with this comes things like the HighFunction and BlockBasic, so it is a deeply nested affair. Seems this is the only option ? (I don't see an equivalent Import script, so I assume I'd have to develop that aspect on my own?)

I ask because I'd like to keep my analysis running within Ghidra (as opposed to offloaded in a DB that is hooked into by something like Datalog) because users may wish to use other Ghidra script constructs to check against the results of our analysis for further insights.

[d-millar]

@aa-software2112 Re other options, I think I'd have to understand your use case. What you've said is, I believe, correct. Bear in mind, the decompiler re-computes a function's decompiled form on-the-fly. It has no real permanent existence. This is by design, as, in many cases, you'd like the decompilation to benefit from the current state of whatever analyses the user is working on. So, you'll probably need to store any artifacts you'd like to preserve somewhere. This is basically what CTADL (https://github.com/sandialabs/ctadl) does - the use case there being source-to-sink-style taint analyses. BSim similarly uses a database to store per-function artifacts in a form that's useful for future processing.

If you're looking for something lighter weight, the GraphASTScript and GraphASTAndFlowScript in Ghidra/Features/Decompiler might also be useful as prototypes for exfil. However, as you've noted, there is currently no obvious ingest mechanism. I do believe, though, it would be fairly straightforward to export (and re-import) those graphs (or the internal non-script equivalents) via gson.

[d-millar]

Following up, it just occurred to me that @ghidravore (or someone) added an "Export Graph" service. I forget exactly how it works, but I believe it should work on the PCode Data Graph. It'll export to CSV, DIMACS, DOT, GML, JSON, Viso, a bunch of things if I recall. Should be in the Help under "Graph Export". (Will try to remember the specifics and report back....)

[d-millar]

Hmmm, someone else may have to weigh-in here....I can't figure out how to get the decompiler graphs to the Export Service from the GUI. May not be possible. That said, the internal code for exporting Block/Flow/Call graphs should be pretty repurposable for the decompiler graphs.

[astrelsky]

Hmmm, someone else may have to weigh-in here....I can't figure out how to get the decompiler graphs to the Export Service from the GUI. May not be possible. That said, the internal code for exporting Block/Flow/Call graphs should be pretty repurposable for the decompiler graphs.

Don't mind the screenshot of the edited screenshot.

[d-millar]

@astrelsky THANKS! (I did these in the wrong order with no results, i.e. did "Graph Data Flow", then switched to "Graph Export", and then either didn't get the "Export Graph" dialog or couldn't find it.)

[astrelsky]

@astrelsky THANKS! (I did these in the wrong order with no results, i.e. did "Graph Data Flow", then switched to "Graph Export", and then either didn't get the "Export Graph" dialog or couldn't find it.)

I initially put the screenshots in the wrong order and edited it later, not sure which version you saw. You need to set it to graph export and then when you pick graph control flow or data flow you'll get the export dialog.

I didn't check to see if there was any kind of export option on the resulting graph when not exported (displayed in code browser). If there isn't, there probably should be.

[d-millar]

Agreed re option to export a displayed graph.