Skip to content

Draft: Support parallel read #395

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 8 commits into
base: fix-unpickable-conf
Choose a base branch
from

Conversation

JasperCraeghs
Copy link
Member

@JasperCraeghs JasperCraeghs commented Jun 29, 2025

Closes #269

@JasperCraeghs JasperCraeghs changed the base branch from master to fix-unpickable-conf July 11, 2025 10:41
JasperCraeghs and others added 3 commits July 11, 2025 15:56
**Parallel Processing Fixes:**
- Fix missing attribute hyperlinks in parallel builds
- Fix missing attribute content rendering in worker processes
- Ensure attribute definitions are properly shared between worker processes
- Improve collection merging logic for robust parallel processing

**API Improvements:**
- Simplify add_attribute() API - no longer requires explicit collection parameter
- Items automatically get collection reference when added to collection
- Maintain backward compatibility with existing code
- Add comprehensive validation while improving user experience

**Test Suite Enhancements:**
- Add comprehensive parallel processing test suite
- Consolidate duplicate test files into unified test suite
- Add tests for attribute consistency between serial/parallel builds
- Improve test coverage for edge cases and error handling

- Enhanced `ParallelSafeTraceableCollection` for proper worker process handling
- Improved `begin_parallel_read()` to ensure all attribute definitions available
- Enhanced `merge_from()` logic for robust collection merging
- Modified `add_item()` to automatically set collection reference
- Updated multiple directive classes for consistent parallel behavior

- Resolves attribute hyperlink generation in parallel builds
- Resolves attribute content rendering issues
- Eliminates duplicate warnings in parallel processing
- Maintains data integrity across worker processes
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant