[Bug]: Job Schedulers Taking more space than Repeatable Jobs

[Haris-SP4RKy]

Version

v5.63.0

Platform

NodeJS

What happened?

🐞 Bug Report: Job Schedulers Taking More Space Than Repeatable Jobs

Summary: After migrating from Repeatable Jobs to the new Job Schedulers, we observed a significant increase in storage usage. Previously, with approximately 3 million repeatable jobs, the system maintained stable database storage usage. However, after upgrading to Job Schedulers, the storage footprint has roughly doubled, even though the number of scheduled jobs remains the same.

Environment:

Service: BullMQ (Job Scheduling System)
Previous Version: Using Repeatable Jobs
Current Version: Using Job Schedulers
Job Count: 3,000,000
Database: Redis

Issue Description:

Under the previous implementation using Repeatable Jobs, Redis memory usage remained within acceptable limits for 3M jobs.
After migrating to Job Schedulers, the same number of scheduled jobs now consumes approximately 2x more space.
This increase appears to stem from the additional metadata or data structure changes introduced in the Job Scheduler model.

Impact:

Redis storage costs and memory pressure have significantly increased.
Potential risk of reaching Redis memory limits sooner under the same load conditions.
Could affect job throughput or cause slower performance under high scheduling volumes.

Expected Behavior: Memory usage for Job Schedulers should be comparable to or optimized relative to Repeatable Jobs for the same workload.

Observed Behavior: Job Schedulers occupy roughly double the space previously used by Repeatable Jobs under identical job counts and configurations.

How to reproduce.

Steps to Reproduce:

1. Create 3 million repeatable jobs using the older Repeatable Jobs feature and measure Redis memory usage.
2. Migrate to Job Schedulers and recreate the same set of jobs.
3. Compare Redis memory usage before and after migration.

Relevant log output

Code of Conduct

• I agree to follow this project's Code of Conduct