CMSSW DQM bin-by-bin comparison failed with ROOT 6.36

[smuzaffar]

Check duplicate issues.

• Checked for duplicates

Description

Hi,
We noticed that after moving to ROOT 6.36 in CMSSW production IBs, our DQM bin-by-bin comparison test start failing with error [a]. This might be due to root's change #18735 .

[a]

PR file successfully opened
Traceback (most recent call last):
  File "/cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/49419/49598/CMSSW_16_0_X_2025-11-21-1100/bin/el8_amd64_gcc13/compareHistograms.py", line 345, in <module>
    create_dif(args.base_file, args.pr_file, args.pr_number, args.test_number, cmssw_version, args.num_processes, args.output_dir)
  File "/cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/49419/49598/CMSSW_16_0_X_2025-11-21-1100/bin/el8_amd64_gcc13/compareHistograms.py", line 80, in create_dif
    compare(shared_paths, pr_flat_dict, base_flat_dict, paths_to_save_in_pr, paths_to_save_in_base)
  File "/cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/49419/49598/CMSSW_16_0_X_2025-11-21-1100/bin/el8_amd64_gcc13/compareHistograms.py", line 182, in compare
    pr_array = np.array(pr_item)
  File "/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02916/el8_amd64_gcc13/lcg/root/6.36.05-302290958d85790e78b48654c2f216ba/lib/ROOT/_pythonization/_uhi.py", line 367, in _getitem
    uhi_index = _compute_common_index(self, index)
  File "/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02916/el8_amd64_gcc13/lcg/root/6.36.05-302290958d85790e78b48654c2f216ba/lib/ROOT/_pythonization/_uhi.py", line 223, in _compute_common_index
    return [_compute_uhi_index(self, idx, axis, include_flow_bins) for axis, idx in enumerate(index)]
  File "/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02916/el8_amd64_gcc13/lcg/root/6.36.05-302290958d85790e78b48654c2f216ba/lib/ROOT/_pythonization/_uhi.py", line 223, in <listcomp>
    return [_compute_uhi_index(self, idx, axis, include_flow_bins) for axis, idx in enumerate(index)]
  File "/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02916/el8_amd64_gcc13/lcg/root/6.36.05-302290958d85790e78b48654c2f216ba/lib/ROOT/_pythonization/_uhi.py", line 192, in _compute_uhi_index
    return _process_index_for_axis(self, index, axis)
  File "/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02916/el8_amd64_gcc13/lcg/root/6.36.05-302290958d85790e78b48654c2f216ba/lib/ROOT/_pythonization/_uhi.py", line 180, in _process_index_for_axis
    raise IndexError(f"Histogram index {index - 1} out of range for axis {axis}. Valid range: (0,{nbins})")
IndexError: Histogram index 30 out of range for axis 0. Valid range: (0,30)

Reproducer

Login to lxplus and run the following commands

cd /tmp/`whoami`
source /cvmfs/cms.cern.ch/cmsset_default.sh 
scram p CMSSW_16_0_X_2025-11-20-1100
cd CMSSW_16_0_X_2025-11-20-1100/
cmsenv
compareHistograms.py -b ~muzaffar/public/DQM_V0001_R000000001__Global__CMSSW_X_Y_Z__RECO.root -p ~muzaffar/public/DQM_V0001_R000000001__Global__CMSSW_X_Y_Z__RECO.root

The cmssw script which does the comparison is https://github.com/cms-sw/cmssw/blob/master/DQMServices/FileIO/scripts/compareHistograms.py

ROOT version

   ------------------------------------------------------------------
  | Welcome to ROOT 6.36.05                        https://root.cern |
  | (c) 1995-2025, The ROOT Team; conception: R. Brun, F. Rademakers |
  | Built for linuxx8664gcc on Nov 14 2025, 15:33:04                 |
  | From heads/master@tags/v6-36-05                                  |
  | With g++ (GCC) 13.4.0                                            |
  | Try '.help'/'.?', '.demo', '.license', '.credits', '.quit'/'.q'  |
   ------------------------------------------------------------------

Installation method

Build from sources using cmsdist

Operating system

AlmaLinux8/9, X86_64 and aarch64

Additional context

No response