feat: add audit events plugin infrastructure (addresses #343)

[JPLachance]

Summary

This PR adds the foundational plugin infrastructure to enable streaming Access audit events to external systems (SIEM, logging platforms, analytics tools). This addresses the core need from #343 for exporting audit logs to destinations like S3, Splunk, Datadog, etc.

What's included

This PR provides the infrastructure layer that enables plugin developers to capture audit events. It does not include any specific destination plugins (S3, Kinesis, etc.) — those will come in follow-up PRs.

Core changes:

• Plugin system: New AuditEventsPluginSpec using Python pluggy framework
• Event envelope: AuditEventEnvelope dataclass with complete WHO/WHAT/WHEN/WHY context
• Event emission: All 18 state-changing operations now emit audit events after DB commit
• Error isolation: Plugin failures cannot crash Access operations (try/catch around all hook calls)
• Tests: Comprehensive test coverage for plugin infrastructure
• Documentation: Updated README with audit events plugin documentation

How it works

After any state-changing operation (approve access, add user to group, delete app, etc.):

1. Access commits the change to the database
2. An AuditEventEnvelope is created with full context
3. The pluggy hook calls all registered plugins
4. Plugins receive the event and can stream it wherever needed (S3, Splunk, etc.)

Plugins are discovered via setuptools entrypoints, following the same pattern as existing notification and conditional access plugins.

Event coverage

Events emitted for: access requests, role requests, apps, groups, tags, group memberships, role assignments (20 distinct event types covering all state changes)

Additional changes

• Import organization: Ran isort to alphabetize imports across all modified files for consistency
• Circular import fixes: Fixed circular import issues in api/operations/delete_user.py and constraint files that isort uncovered
• Schema bug fix: Fixed missing comma in api/views/schemas/core_schemas.py tuple (line 323)
• README typo: Fixed incorrect link reference in plugin documentation (requests.py → conditional_access.py)

Next steps

Follow-up PR will add an example S3 plugin implementation demonstrating how to use this infrastructure.

Testing

• ✅ All existing tests pass
• ✅ New tests verify plugin hook functionality
• ✅ Tests confirm system works with zero plugins installed
• ✅ Event envelope structure validated for all event types

Closes #343

[Copilot]

Pull request overview

This PR introduces the foundational plugin infrastructure for streaming Access audit events to external systems (SIEM, logging platforms, analytics tools). It establishes a pluggy-based plugin system with a comprehensive event envelope containing WHO/WHAT/WHEN/WHY context, and emits audit events after all state-changing operations complete.

Key changes:

• New AuditEventsPluginSpec and AuditEventEnvelope classes for the plugin system
• Audit event emission integrated into 18 state-changing operations across access requests, role requests, apps, groups, tags, and role assignments
• Comprehensive error isolation with try-catch blocks to prevent plugin failures from affecting core operations
• Import organization improvements and circular import fixes discovered during integration

Reviewed changes

Copilot reviewed 27 out of 27 changed files in this pull request and generated 3 comments.

Show a summary per file

File	Description
api/plugins/audit_events.py	Core plugin specification defining AuditEventsPluginSpec, AuditEventEnvelope dataclass, and hook initialization
api/plugins/init.py	Exports audit events plugin hooks and markers for external plugin implementations
tests/test_audit_events.py	Comprehensive tests for envelope structure, hook functionality, and plugin infrastructure
api/operations/approve_access_request.py	Emits access_approve audit event after access request approval
api/operations/reject_access_request.py	Emits access_reject audit event after access request rejection
api/operations/create_access_request.py	Emits access_create audit event after access request creation
api/operations/approve_role_request.py	Emits role_request_approve audit event after role request approval
api/operations/reject_role_request.py	Emits role_request_reject audit event after role request rejection
api/operations/create_role_request.py	Emits role_request_create audit event after role request creation
api/operations/modify_group_users.py	Emits 4 event types for member/owner additions and removals
api/operations/modify_role_groups.py	Emits 2 event types for role group assignments and unassignments
api/operations/create_group.py	Emits group_create audit event after group creation
api/operations/delete_group.py	Emits group_delete audit event after group deletion
api/operations/modify_group_tags.py	Emits tag_update audit event when group tags are modified
api/operations/create_app.py	Emits app_create audit event after app creation
api/operations/delete_app.py	Emits app_delete audit event after app deletion
api/operations/modify_app_tags.py	Emits tag_update audit event when app tags are modified
api/operations/create_tag.py	Emits tag_create audit event after tag creation
api/operations/delete_tag.py	Emits tag_delete audit event after tag deletion
api/operations/modify_group_type.py	Import reorganization only (isort)
api/operations/unmanage_group.py	Import reorganization and log message formatting improvement
api/operations/delete_user.py	Fixed circular import by using explicit import path
api/operations/constraints/check_for_self_add.py	Import reorganization (isort)
api/operations/constraints/check_for_reason.py	Import reorganization (isort)
api/operations/init.py	Alphabetized imports for consistency
api/views/schemas/core_schemas.py	Fixed missing comma in tuple definition (bug fix)
README.md	Fixed incorrect plugin filename reference from requests.py to conditional_access.py

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[JPLachance]

And yes, I tested it end to end:

• Created a plugin that sends audits to SQS, following the OCSF schemas
• SQS ← Logstash → OpenSearch
• OpenSearch Dashboards

This made it much easier to visualize what’s changing, when, why, and by whom.

[Copilot]

Pull request overview

Copilot reviewed 27 out of 27 changed files in this pull request and generated 2 comments.

Comments suppressed due to low confidence (1)

api/operations/approve_role_request.py:171

• This audit event is emitted before the role request approval side effects are applied (the calls to ModifyRoleGroups(...).execute() happen after this try/except block). If ModifyRoleGroups raises or rolls back, plugins will still have received a role_request_approve event for a change that did not actually succeed, which breaks the "after DB commit" guarantee documented in the hookspec and can produce incorrect downstream audit trails. Please move the audit-event emission to after the role-group modifications and final commit, so events are only sent once the approval has definitively succeeded.

        # Emit audit event to plugins (after DB commit)
        try:
            requester = db.session.get(OktaUser, self.role_request.requester_user_id)
            audit_hook = get_audit_events_hook()
            envelope = AuditEventEnvelope(
                id=uuid4(),
                event_type="role_request_approve",
                timestamp=datetime.now(timezone.utc),
                actor_id=self.approver_id or "system",
                actor_email=self.approver_email,
                target_type="role_request",
                target_id=str(self.role_request.id),
                target_name=f"Role request for {group.name}" if group else "Unknown group",
                action="approved",
                reason=self.approval_reason,
                payload={
                    "role_request_id": str(self.role_request.id),
                    "requester_user_id": self.role_request.requester_user_id,
                    "requester_email": requester.email if requester else None,
                    "requester_role_id": self.role_request.requester_role_id,
                    "requested_group_id": self.role_request.requested_group_id,
                    "requested_group_name": group.name if group else None,
                    "request_ownership": self.role_request.request_ownership,
                    "ending_at": self.ending_at.isoformat() if self.ending_at else None,
                },
                metadata={
                    "user_agent": request.headers.get("User-Agent") if context else None,
                    "ip_address": (
                        request.headers.get("X-Forwarded-For", request.headers.get("X-Real-IP", request.remote_addr))
                        if context
                        else None
                    ),
                },
            )
            audit_hook.audit_event_logged(envelope=envelope)
        except Exception as e:
            current_app.logger.error(f"Failed to emit audit event: {e}", exc_info=True)

        if self.role_request.request_ownership:
            ModifyRoleGroups(
                role_group=self.role_request.requester_role,
                groups_added_ended_at=self.ending_at,
                owner_groups_to_add=[self.role_request.requested_group_id],
                current_user_id=self.approver_id,
                created_reason=self.approval_reason,
                notify=self.notify,
            ).execute()
        else:
            ModifyRoleGroups(
                role_group=self.role_request.requester_role,
                groups_added_ended_at=self.ending_at,
                groups_to_add=[self.role_request.requested_group_id],
                current_user_id=self.approver_id,
                created_reason=self.approval_reason,
                notify=self.notify,
            ).execute()

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[JPLachance]

Hello @barborico,

Could you have a look at this proposed change?

It completes well your latest Groups Lifecycle plugin hook.

Thanks!