Skip to content

[POC] AAP-40527 DAB tasking app #702

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 2 commits into
base: devel
Choose a base branch
from

Conversation

AlanCoding
Copy link
Member

@AlanCoding AlanCoding commented Feb 25, 2025

[AAP-39817] DAB app for background tasks via pg_notify

Description

This gives a higher-level interface to the dispatcher library, formerly the AWX dispatcher. As DAB exists to provide a framework for Django things, this splits out current practices for this in AWX to provide a framework for a background task service.

Apps, at their choosing, may use either dispatcher (lower-level) tasks, or durable tasks tracked by this app. By "lower-level", we mean, specifically, the psycopg client library for postgres. Since the dispatcher library only speaks psycopg, and not Django, this, predictably, contains a number of utils for translating between Django and psycopg language.

Also covers AAP-39484

Also covers AAP-40527 Now this is the primary issue

Investigation spike was AAP-39817

Type of Change

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • Documentation update
  • Test update
  • Refactoring (no functional changes)
  • Development environment change
  • Configuration change

Self-Review Checklist

  • I have performed a self-review of my code
  • I have added relevant comments to complex code sections
  • I have updated documentation where needed
  • I have considered the security impact of these changes
  • I have considered performance implications
  • I have thought about error handling and edge cases
  • I have tested the changes in my local environment

Testing Instructions

From my manual demo instructions, in a tab, run honcho start.

In another tab, you can run these snippets

from test_app.tasks import sleep
from ansible_base.task.tasks import manage_lost_tasks
from uuid import uuid4
from django.utils.timezone import now
import time

sleep.delay(seconds=60)

Task.objects.create(name='footask', state='running', wrapper_uuid=uuid4(), started_at=now())

time.sleep(1)

manage_lost_tasks(grace_period=0)
from test_app.tasks import *

hello_world.delay()
hello_world_other_queue.delay()

Steps to Test

  1. you can run the test_app content by honcho start
  2. /api/v1/trigger_task/ can trigger by the API

@AlanCoding AlanCoding changed the title AAP-39817 [Proof-of-Concept] DAB tasking app AAP-40527 [Proof-of-Concept] DAB tasking app Feb 27, 2025
@john-westcott-iv john-westcott-iv changed the title AAP-40527 [Proof-of-Concept] DAB tasking app [POC] AAP-40527 DAB tasking app Mar 4, 2025
Move psycopg param generator to DAB utility

Store standard type annotation

Make a solution to getting conninfo as a string

Add another test and wrap up linters

Init migration, connect things

Add task container

Demo via a test_app view

Forgot procfile

Forgot to include tasks

Change name to durable_task

Get subprocess test to run

Get multi-server working

Adopt new bind=True kwarg

Pass through args and kwargs

Write task fallback logic

Move config into its own module, pop inside utils method

Basics of multi-queue working
AlanCoding added a commit that referenced this pull request May 2, 2025
## Description
We have pushed out our objective merge date for DAB task in the general,
which is at #702, and
doing this creates a need to be able to feature-flag a release using
https://github.com/ansible/dispatcherd

Using dispatcherd by itself doesn't need most of the content from DAB
task, but it needs _some utility_ type methods. As we flip-flopped on
the exact order of operations for this, I had previously put up
#686, and closed it
_in favor of_ DAB task. Some content was added while I worked on DAB
task, so I created a new branch (this one) and put the updated content
there.

Why?

Merging this will allow the EDA-server and AWX work to de-duplicate code
that was copy+pasted there.


ansible/awx@devel...feature_dispatcher#diff-60fa214144754e9e3cf12cc5c7054d8b0ce04f43da33181c0fe9f0d927627038


ansible/eda-server@main...dispatcher-poc-v2#diff-ff14e1e048588b54cbac15edc61d4c830abc8a87cdae26b7fb603d52f3fe1960

Those were always intended to go live in DAB.

Next question - why can't we just put them in the dispatcherd repo?
Because _it does not have Django_. This is very important. I do intend
to write some docs in dispatcherd that will reference this content.

Ping @bzwei for review

## Type of Change
- [ ] Bug fix (non-breaking change which fixes an issue)
- [x] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to not work as expected)
- [ ] Documentation update
- [x] Test update
- [x] Refactoring (no functional changes)
- [ ] Development environment change
- [ ] Configuration change

## Self-Review Checklist
- [x] I have performed a self-review of my code
- [x] I have added relevant comments to complex code sections
- [ ] I have updated documentation where needed
- [ ] I have considered the security impact of these changes
- [ ] I have considered performance implications
- [ ] I have thought about error handling and edge cases
- [x] I have tested the changes in my local environment

## Testing Instructions
This probably needs to be tested with the respective EDA or AWX
branches. They have been working, this just consolidates these methods.

### Prerequisites
<!-- List any specific setup required -->

### Steps to Test
1. 
2. 
3. 

### Expected Results
<!-- Describe what should happen after following the steps -->

## Additional Context
<!-- Optional but helpful information -->

### Required Actions
- [ ] Requires documentation updates
- [x] Requires downstream repository changes
- [ ] Requires infrastructure/deployment changes
- [x] Requires coordination with other teams
- [ ] Blocked by PR/MR: #XXX

### Screenshots/Logs
<!-- Add if relevant to demonstrate the changes -->
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant