feat(gitlab): implemented keyset pagination for gitlab #8529

[moulivashisth]

⚠️ Pre Checklist

Please complete ALL items in this checklist, and remove before submitting

• I have read through the Contributing Documentation.
• I have added relevant tests.
• I have added relevant documentation.
• I will add labels to the PR, such as pr-type/bug-fix, pr-type/feature-development, etc.

Summary

Fix GitLab Users collection hitting offset pagination limits by adding keyset pagination.

This PR updates the GitLab CollectAccounts subtask to avoid max offset errors when collecting large user sets:

• Self-managed GitLab instances now use keyset pagination on /api/v4/users
  (pagination=keyset&order_by=id&sort=asc&per_page=N&id_after=<last_id>) and do not send page.
• gitlab.com / jihulab.com keep existing behavior on project member endpoints
  (/projects/:id/members[/all]) which typically remain under offset caps per project.
• Retains existing API-version fallback (/members/all vs /members/ for < v13.11).
• Response parser now tracks the last item’s id to advance the keyset cursor safely.
• No breaking changes to task wiring or raw table schema (gitlab_api_users).

Why: Some instances enforce strict offset caps (e.g., 50k), causing offset pagination is restricted errors when fetching Users. Keyset pagination removes the offset and enables full retrieval.

Risk/Compatibility:

• Backward compatible; only changes query parameters and cursor handling.
• If a project’s members list exceeds offset caps and the endpoint lacks keyset, users should collect site users (self-managed path) or shard by project/group—unchanged from current guidance.

Does this close any open issues?

Closes 8529 ([Bug][GitLab] Pagination not working Again)

Screenshots

N/A

Other Information

[klesh]

Is keyset and api/v4 available in Community Edition 11+ ?

[moulivashisth]

Thanks for bringing this up!
• Keyset pagination for /api/v4/users was introduced in 16.5, and 17.0+ requires it for large responses.

I’ve updated the code to gate keyset by server version (≥16.5) and fallback to offset otherwise. Project members endpoints keep their existing offset behavior.

This preserves compatibility with CE 11–16.4 while avoiding offset-cap failures on newer instances.

[klesh]

LGTM, thanks for your contribution.

[moulivashisth]

Thanks for the review.

I hit a typecheck error:
plugins/gitlab/tasks/account_collector.go:69:55: undefined: apiVersion

Fix: Declared and reused a single apiVersion variable sourced from the client.

[klesh]

Hi, you may run the following command to fix the golangci-lint error:

gofmt -s -w -l plugins/gitlab/tasks/account_collector.go

[github-actions]

This pull request has been automatically marked as stale because it has not had recent activity for 120 days. It will be closed in 7 days if no further activity occurs.

[github-actions]

This pull request has been closed because it has not had recent activity. You could reopen it if you try to continue your work, and anyone who are interested in it are encouraged to continue work on this pull request.