Skip to content

Conversation

@ZackAttack614
Copy link
Collaborator

This is the first phase in migrating the HEXRD library to the Core structure. Here, we reach the end of the second part of Slide 2 in the HEXRD Migration State documentation, following guidance from the good work on sorting the HEXRD functions done by the community's subcommittees.

The result of this effort categorizes the HEXRD codebase into 5 useful high-level workflows:

  1. core
  2. hed
  3. hedm
  4. laue
  5. powder

This reorganization makes it easier for new users of HEXRD to dive into their particular workflow, without having to search through the entire codebase for necessary functions. It also strives to make development contributions easier, by focusing the contributor on just the area that matters for their work.

While we believe we have done our best to work in good faith to adhere to the documented categorization, the development community expects that there will be necessary conversation with the community to make sure everything is in its rightful place before this can be merged.

As well, ensuring that old scripts and workflows that HEXRD users have will still work in the new architecture is presently difficult, considering the limited documented scripts the developers have access to. To that end, before a final merge into master can be conducted, the community will need to provide more examples of how HEXRD is used in real-world scenarios.

kpwelsh and others added 16 commits March 31, 2025 11:20
Fix duplicate imports

update cli build entry point

fix this relative import that didn't seem to get patched.

All other similar "form . import X" statements are fine.

patch extension installation

Fix some ast.Name references to hexrd modules

Updated documentation that references old modules
Forgot to add the "hexrd." when I moved this.

Replace "/" with "." in C extension path

Fix unpickling error in non-Windows env

Change to a human readable file_table instead of a pickle object.

Also addresses the unpickling issue across platforms.

mypy doesn't like the reassignment.

hand patches for the preprocess file updates.

+ slightly more robust module_map

Had to add to update module_map to support maintaining all imports.

put a TYPE_CHECKING guard around this, so we don't have a circular import

These files need to be copied across workflows as well to preserve imports and resolve circular dependencies.

Fix the import mapping to allow for unpickling use cases.

switch these things to absolute import.

Since we are changing the actual depth of these modules, it can cause issues due to the following:
* We need to register these modules with the old names as well so that comparisons work correctly
* Importlib seems to use the key in sys.modules to determine if a relative import goes "outside of the module"

Since we changed things like `hexrd.thing` -> `hexrd.core.thing`. If you think the module is @ `hexrd.thing` and you import `. . .other` it will look like you are escaping the package. (note, number of dots and effective depth depends on what `thing` actually is)

Move this __init__ + fix the file_table a bit
Also copied over the detector.py
@ZackAttack614 ZackAttack614 requested a review from psavery August 28, 2025 21:41
@codecov
Copy link

codecov bot commented Aug 28, 2025

Codecov Report

❌ Patch coverage is 49.12664% with 932 lines in your changes missing coverage. Please review.
✅ Project coverage is 55.97%. Comparing base (003da9d) to head (e09f301).
⚠️ Report is 23 commits behind head on master.

Files with missing lines Patch % Lines
hexrd/core/material/spacegroup.py 47.44% 103 Missing ⚠️
hexrd/hedm/fitting/calibration/grain.py 19.00% 81 Missing ⚠️
hexrd/core/fitting/peakfunctions.py 3.70% 78 Missing ⚠️
hexrd/core/material/mksupport.py 2.50% 78 Missing ⚠️
hexrd/hed/xrdutil/utils.py 39.83% 74 Missing ⚠️
hexrd/core/instrument/hedm_instrument.py 60.74% 53 Missing ⚠️
hexrd/core/fitting/fitpeak.py 11.53% 46 Missing ⚠️
hexrd/core/material/jcpds.py 2.50% 39 Missing ⚠️
hexrd/core/fitting/utils.py 17.39% 38 Missing ⚠️
hexrd/core/material/unitcell.py 62.00% 38 Missing ⚠️
... and 50 more
Additional details and impacted files
@@            Coverage Diff             @@
##           master     #836      +/-   ##
==========================================
+ Coverage   51.31%   55.97%   +4.66%     
==========================================
  Files         143      143              
  Lines       22545    21960     -585     
==========================================
+ Hits        11568    12292     +724     
+ Misses      10977     9668    -1309     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@ZackAttack614 ZackAttack614 self-assigned this Aug 29, 2025
Copy link
Collaborator

@psavery psavery left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.

There seem to be a few files where the history might have not been kept (it looks like at least GitHub thinks they are new files, although maybe not git itself). But those files are a small minority.

This also seems to be working fine with HEXRDGUI as things are. I think it is ready.

@psavery psavery merged commit e0e7938 into master Oct 1, 2025
7 checks passed
@psavery psavery deleted the master-reorg branch October 1, 2025 19:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants