Over 10k+ Dynamic Pages handling

[ratz6]

Summary

I have a next js app router configured on ec2 using pm2 cluster mode. I have an auto scaling setup having a core of 2 vCpus. My website has dynamic pages for a stock market application. So currently I have CDN on top of my elb for sometime to cache the html for a short amount of time. But it mostly skips CDN for now and goes to the machine which computes all data as SSR. All n/w calls are done on my website to handle seo and page awareness.

But the problem what I face is when I see a spike of 6k requests in 5 mins that’s approx 100rps. And my cpu of all the machines I have go up to 90%+.

I came across ISR recently. And generateStaticParam to generate build of certain paths at buildtime. I would want to know from the smart guys out there, how are you managing load and concurrent users ?

Will SSR fail here ? Will ISR come to rescue ? But even then computing 10k pages with each having 1sec time also is 10000secs which is just too much right ?

Also came across PPR but not sure if it’ll help with CPU for dynamic pages.

I’m just confused and looking for help, please let me know what you know.

Cheers

cc @icyJoseph @ijjk @timneutkens @sokra @huozhi @Timer @leerob

Additional information

https://www.reddit.com/r/nextjs/comments/1p33ozg/over_10k_dynamic_pages_handling/

Example

No response

[wwwdepot]

stock market app? ISR?

Traders want LIVE market data; they won’t wait for cache invalidation.

you need to re-evaluate Nextjs architecture is suitable for the job

[icyJoseph]

It was 1 second latency at p99.9 though - I didn't monitor CPU though that's true - focusing on the requests per second in this example - but I'd have expected the CPU /w that 512mb RAM to choke and drop to sub 50 as you are observing

Re run the test w/ the docker dashboard tools, and yeah it hits a peak, I guess, expected given that I am hammering that one CPU core, memory stays low though

[ratz6]

yes my machine also touches 100% CPU for a while with this setup, and CPU consumption over 80% for a long time is a cirtical condition for any service.
how do I go ahead with this ? Is there some issue in my machine ? my setup ?

how do people handle dynamic websites being seo friendly and real time data changes with high rps in next js ? any examples for the same ?

as CPU goes so high my website starts to hang here and latency increases

my test results >

TOTAL RESULTS 

    checks_total.......: 12002  99.355032/s
    checks_succeeded...: 99.98% 12000 out of 12002
    checks_failed......: 0.01%  2 out of 12002

    ✗ status is 200
      ↳  99% — ✓ 6000 / ✗ 1
    ✗ not server error
      ↳  99% — ✓ 6000 / ✗ 1

    HTTP
    http_req_duration..............: avg=353.13ms min=24.69ms med=106.62ms max=8.75s p(90)=1.02s p(95)=1.34s
      { expected_response:true }...: avg=353.17ms min=24.69ms med=106.62ms max=8.75s p(90)=1.02s p(95)=1.34s
    http_req_failed................: 0.01%  1 out of 6004
    http_reqs......................: 6004   49.70235/s

    EXECUTION
    iteration_duration.............: avg=356.05ms min=24.89ms med=106.99ms max=8.75s p(90)=1.03s p(95)=1.34s
    iterations.....................: 6001   49.677516/s
    vus............................: 3      min=2         max=110
    vus_max........................: 200    min=200       max=200

    NETWORK
    data_received..................: 81 MB  668 kB/s
    data_sent......................: 784 kB 6.5 kB/s




running (2m00.8s), 000/200 VUs, 6001 complete and 0 interrupted iterations
route53_dynamic ✓ [ 100% ] 000/200 VUs  2m0s  50.00 iters/s

[icyJoseph]

I guess for this use case, you are hitting the limits of the setup. Is it possible to test/verify by increasing CPUs? When I did go up to 4 CPUs, I noticed 2 were unused under this load.

Also, ran with:

➜ oha -z 45s -c 100 --urls-from-file source.txt

To hit different endpoints, and saw same results.

[ratz6]

When your CPU maxed out were you able to go to other pages in the app which are SSR'd ?
My pages were stuck when my CPU maxed out ?

also for 2 core machine, how much RPS did it req to touch 90%+ ?

So you suggest to keep increasing CPU to handle this ? is this a nextJs fundamental issue ? or a setup gap ? What is the ideal Next Js setup to handle scale ?
Will "cache components" help me out here ?

Also what if you add an API call in the page with a delay of 500ms let's say and it's cached for each ticker.. but every ticker will have a separate response.. then how is it affecting your machine ?

[icyJoseph]

Hi sorry for late answer, been busy with other items (looks like I'd need a couple more CPUs in my head right now 😅)

I ended up setting up pm2 and running my tests once again w/ 1 2 and 4 CPUs always with 512mb. This time also increasing to 100 concurrent workers, continuously sending requests to various /blog/[slug] variations. I was still able to reach home page while running the load test.

1 cpu:

Summary:
  Success rate:	100.00%
  Total:	45.0094 sec
  Slowest:	3.5046 sec
  Fastest:	0.5136 sec
  Average:	0.9976 sec
  Requests/sec:	101.4233

  Total data:	12.61 MiB
  Size/request:	2.89 KiB
  Size/sec:	286.95 KiB

2 cpu's:

Summary:
  Success rate:	100.00%
  Total:	45.0048 sec
  Slowest:	1.2697 sec
  Fastest:	0.1738 sec
  Average:	0.3119 sec
  Requests/sec:	321.8545

  Total data:	40.61 MiB
  Size/request:	2.89 KiB
  Size/sec:	924.06 KiB

and with 4 cpu's:

Summary:
  Success rate:	100.00%
  Total:	45003.5728 ms
  Slowest:	531.8039 ms
  Fastest:	56.4685 ms
  Average:	123.5731 ms
  Requests/sec:	810.3801

  Total data:	102.68 MiB
  Size/request:	2.89 KiB
  Size/sec:	2.28 MiB

I think the bottom line is that yeah, you have to horizontally scale these things. I am curious though, we do see a divergence, you have 2vCPUs and the 2 cpu's test I ran lands around 300-400 (on a good run) requests/sec, are there any other bottlenecks in your application?

A video from the 2 CPU run:

Screen.Recording.2025-11-26.at.21.37.18.mov

[Nguyen1976]

let's upvote my answer