Skip to content

Filter active processes for the default user when running verdi process repair #6879

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

edan-bainglass
Copy link
Member

@edan-bainglass edan-bainglass commented May 17, 2025

This PR applies a filter on the active processes picked up by verdi process repair. The command takes the difference between the set of active processes and queued processes and adds the difference to the broker queue. However, in recent developments of a shared AiiDA setup, the database table of queued jobs (DbQueue) may hold jobs from different profiles (multiple profiles share the database). The broker should not pick up jobs from a profile different from the default user. Here, I apply this exact filter on active jobs (if process.user.is_default). This ensures that the broker does not pick up jobs from any other profile attached to the database.

I don't believe this breaks anything, as there should be no other user with active processes. Here, by active I mean jobs which the broker should correctly attempt to repair. I believe the concept of User in AiiDA was mostly to differentiate imported processes and data. But these processes should not be active (e.g. running, waiting). So the filter in the end makes no difference for single-profile-use database use cases.

@edan-bainglass edan-bainglass self-assigned this May 17, 2025
@edan-bainglass edan-bainglass changed the title Filter active processes to repair only for the default user Filter active processes for the default user when running verdi process repair May 17, 2025
@superstar54
Copy link
Member

I suggest adding the following cli options.

@options.ALL_USERS(help='Filter to repair processes for all users')
@options.USER(help='Filter to repair processes belonging to a specific user.')

And use the current user if neither is provided.

This will function the same as your current implementation, but also give flexibility for users, in the case the users create multiple "User" in the same profile (a scenario that is allowed, though not documented or encouraged).

@@ -525,7 +525,7 @@ def process_repair(manager, broker, dry_run):
"""
from aiida.engine.processes.control import get_active_processes, get_process_tasks, iterate_process_tasks

active_processes = get_active_processes(project='id')
active_processes = [process.pk for process in get_active_processes() if process.user.is_default]
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
active_processes = [process.pk for process in get_active_processes() if process.user.is_default]
active_processes = (process.pk for process in get_active_processes() if process.user.is_default)

the process will be evaluated and the list can be large, use iterator instead.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry, haven't had a chance to get back to this. The thing is, somehow in the tests, process is evaluated as an int. Unclear why. Also, maybe best to modify the active process query to optionally filter by the default user, so this list comprehension filtering could be avoided all together. Will think about this later this week.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants