mapcache_seed: Refactor seeder for robust multiprocessing and shutdown #359
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This commit is a major overhaul of the mapcache_seed utility, addressing critical bugs in signal handling, multiprocessing, and thread synchronization that could cause deadlocks or unresponsive behavior.
The key changes are:
The Ctrl+C (SIGINT) handling has been completely rewritten to ensure a prompt and clean shutdown in all modes, fixing numerous deadlocks and race conditions.
For Multiprocessing:
pop_queue()
now checks asig_int_received
flag at the start to ensure child processes exit immediately after finishing their current task.push_queue()
's EINTR retry loop now also checks the signal flag, preventing the parent process from deadlocking on a full message queue after a signal has been received.For Multithreading:
apr_queue_interrupt_all()
to wake all blocked threads.pop_queue()
correctly handles the resulting APR_EOF to stop the worker.Graceful vs. Urgent Shutdown:
feed_worker()
that sends STOP commands is now wrapped in anif(!sig_int_received)
block. This ensures it only runs during a normal, graceful shutdown and is skipped on Ctrl+C, preventing a major deadlock.The -p mode has been significantly improved for stability and correctness.
log_thread_fn
andseed_worker
were updated to use this new mechanism.-p 1
Bugfix: A long-standing bug that caused the seeder to stall when using -p 1 has been fixed by changing all relevant logic guards fromnprocesses > 1
tonprocesses >= 1
.sig_int_received
is now correctly typed asvolatile sig_atomic_t
.memset()
is now used to initialize structs before use.-p
option has been clarified.Co-authored with gemini-2.5-pro