Best practices for working on an offline network

[d-millar]

Am staring this discussion with an answer from @ghidra1 in the hopes of alleviating future issues for other folks, but, if you've tried this and have advice for others, please chime in....

The challenge is the following: let's say you have a shared Ghidra project under active development, but you need to migrate the project to another (disconnected) network for, say, a workshop and then migrate it back. What is the right way to do this? I.e. what should you do to minimize your pain for the migration back?

@ghidra1 suggests the following approach (assuming i am hearing him correctly):

1. Users on the original network (I'll call it net0) should perform checkouts from the original server (server0).
2. Server0 should be zipped up (i.e. zip server0.rep) and transferred to the new disconnected workshop network (net1).
3. Users on the original network amy continue to work but should NOT check anything in. (In fact, maybe you'll want to shut server0 down for the duration of the workshop.)
4. Server0 can be revived on net1 as the new server (server1) fot the workshop.
5. Users on net1 may checkout/checkin as needed.
6. At the end of the workshop, shut server1 back down, re-zip it, and transfer it back to net0.
7. Re-start server1 (now server0 again) on net0 and allow any net0 users to checkin changes made on net0 during the workshop (but not at the workshop).

Thoughts?

[ReversingWithMe]

I've been considering approaches to this network migration challenge for Ghidra projects, thoughts for discussion:

I've been thinking about this type of project primarily in terms of running the XML exporter and automating exports, then manually reviewing changes with python/UI wrapper. This approach aims to minimize requirements on the original network (net0) and allow work to continue uninterrupted there.

I've also considered utilizing Ghidra's versioning code in cases where the memory map structure remains unchanged. The versioning approach could help migrate new notes and allow us to selectively choose the better elements (comments, function names, types) from both networks. While XML-based selective merging could accomplish something similar, this remains conceptual as I don't currently have scripts to implement it, and I think XML parsing has more footguns and pain.

Part of the complexity with both versioning and XML approaches is how to automate the process across all programs in the project, handle folder recursion, and resolve conflicts at each step. and make it generally work beyond some specific case.

I've also wondered if combining XML selective merging with a provided memory map dictionary (this address range became that on net1) might work for mappings from net0-start to net1-final, which could then migrate symbols back to net0-final at those addresses.

The "no check-ins" requirement seems challenging and possibly the biggest issue with this. If memory maps can't be touched then I think that really kills any chance for workshops on non-PE/ELF/Macho.

As for scripting the label transfer, this might mean rolling net0 back to the fork point, merging changes in, and then porting the net0-final version if it contains fewer changes than the workshop version. So whatever is easier to pull out labels from, whichever made more progress/changes is the one that gets auto-applied. Also thinking about this problem on a binary-by-binary basis instead of total project wide might relax some constraints particularly for shorter/smaller workshop settings, especially for some of the programs that weren't modified during the workshop period.

I'm thinking still trying to go through the pain of scripting over trying to work in the git branching style as I think that can fail more incrementally.

[astrelsky]

Honest question, I feel like I'm just nitpicking on the example here, but why would you want to keep any changes you made during a workshop? I'm not sure why you would demo a work in progress during a workshop either.

I have never used a shared project so unfortunately I can't provide any useful input.

[mumbel]

why would you throw away RE?

You have your developer network where users would run ghidra and have your ghidra server. someone else might host you for a workshop to collaborate on something that requires a large effort to RE, this is not your developer network and you cannot just talk to your developer network. You would like to take back all your RE back to your developer network when the collaboration effort is done.

@d-millar

While it would be nice for net0/net1 users to continue working as if they will sync up later, actions requiring exclusive checkout seems like a really bad situation with this setup. For sure would need to know before hand how and what might require exclusive and plan for it. Besides just changing things like memory maps/permissions/additional code/sections, or other similar things; would your workshop be during a ghidra release, would the goal be to use that upgrade and does that upgrade require exclusive checkout?

net1 users on server1 have to hijack net0 users removing net0 users ability to push to server0 when net1 users come back to server0. net0 users just risk it and RE while net1 users are doing server1 things and hope they don't lose everything in their last checkout