[example] Add Instrumented Loading LRU cache example #66

marcdumais-work · 2025-05-06T20:05:28Z

What it does

This toy example was inspired by a real-life investigation, regarding a web-accessible application that had configurable caches, including a specific one whose performance came under scrutiny. That investigation produced some interesting traces, which we aim to approximately replicate with this example.

Related YouTube video:
https://www.youtube.com/watch?v=mSg8EO_FhuQ

Real life investigation - some info about that cache:

type: LRU (least recently used) - when the cache is full, the cache element that was the least recently used is evicted to make space for a new element to be inserted in its place.

configurable cache size: The number of elements it can hold is configurable by the app's admins

cache / cached data usage: The cache is used in a special way. Thought the cache-backed data can be accessed randomly, it's often accessed sequentially, from first element to the last, in order. The app is multi-threaded and often multiple reader threads would be reading the data through the cache, in parallel.

The app provides cache statistics, which seemed quite good (a high cache "hit" rate was reported).

Investigation:
The cache was configured to be slightly under-sized vs the data it was caching, which size (number of elements) was slowly but surely increasing. Given the good reported cache stats, was it an issue that it was a bit undersized (by ~15%)? Was the cache, as configured, potentially affecting performance?

Tracing to the rescue!
The cache and reader thread code were instrumented. From the generated traces, it was possible to distinguish cache hits vs misses. Capturing traces for a few scenarios, some surprising details started to emerge!

See the example's README for more details.

How to test

Follow the instructions in the example's README - try to visualize some of the captured traces. Comments welcome!

$ cd lrucacheexample
# Optionally edit pom.xml to change the simulation's parameters
$ mvn compile exec:exec
$ ../jsonify.py traces/trace.log traces/trace-cache.json
# Open `traces/trace-cache.json` in Trace Compass or using the `Trace Viewer for VSCode` extension

Here is what the generated traces look-like in Trace Compass:

Cache big enough to contain all elements:
Cache slightly undersized (fits 95% of elements):

Follow-ups

N/A

Review checklist

As an author, I have thoroughly tested my changes and carefully followed the instructions in this template

bhufmann

Would it be interesting to have some screenshots of the traces and show them in the readme? Interested people might not want to run it, but would like to see the results?

bhufmann · 2025-05-14T18:21:24Z

lrucacheexample/README.md

+
+#### Leads
+
+- How to explain the disproportional decrase in performance between the fully sized caches vs 95% sized results? It's a ~12x factor!


decase -> decrease

bhufmann · 2025-05-14T18:23:00Z

...acheexample/src/main/java/org/eclipse/tracecompass/traceeventlogger/lrucachedemo/Config.java

@@ -0,0 +1,96 @@
+package org.eclipse.tracecompass.traceeventlogger.lrucachedemo;


should it have a copyright header

bhufmann · 2025-05-14T18:23:16Z

...le/src/main/java/org/eclipse/tracecompass/traceeventlogger/lrucachedemo/LoadingLRUCache.java

@@ -0,0 +1,249 @@
+package org.eclipse.tracecompass.traceeventlogger.lrucachedemo;


should it have a copyright header

bhufmann · 2025-05-14T18:23:34Z

lrucacheexample/src/main/java/org/eclipse/tracecompass/traceeventlogger/lrucachedemo/Main.java

@@ -0,0 +1,240 @@
+package org.eclipse.tracecompass.traceeventlogger.lrucachedemo;


should it have a copyright header?

bhufmann

I think the lrucacheexample should be under a folder with name examples. Then we can add other examples when we have them.

marcdumais-work · 2025-05-15T14:33:45Z

I think the lrucacheexample should be under a folder with name examples. Then we can add other examples when we have them.

Done - will push new version momentarily

This toy example was inspired by a real-life investigation, regarding a web-accessible application that had configurable caches, including a specific one whose performance came under scrutiny. That investigation produces some interesting traces, which we aim to approximately replicate with this example. Real life investigation - some info about that cache: type: LRU (least recently used) - when the cache is full, the cache element that was the least recently used is evicted to make space for a new element to be inserted in its place. configurable cache size: The number of elements it can hold is configurable by the app's admins cache / cached data usage: The cache is used in a special way. Thought the cache-backed data can be accessed randomly, it's often accessed sequentially, from first element to the last, in order. The app is multi-threaded and often multiple reader threads would be reading the data through the cache, in parallel. The app provides cache statistics, which seemed quite good (a high cache "hit" rate was reported). Investigation: The cache was configured to be slightly under-sized vs the data it was caching, which size (number of elements) was slowly but surely increasing. Given the good reported cache stats, was it an issue that it was a bit undersized (by ~15%)? Was the cache, as configured, potentially affecting performance? Tracing to the rescue! The cache and reader thread code were instrumented. From the generated traces, it was possible to distinguish cache hits vs misses. Capturing traces for a few scenarios, some surprising details started to emerge! See the example's README for more details. Signed-off-by: Marc Dumais <[email protected]>

bhufmann

Looks good to me. Thanks!

marcdumais-work force-pushed the lru-cache-example branch 17 times, most recently from ab3bddd to 31b7bee Compare May 8, 2025 18:18

marcdumais-work changed the title ~~[example] Add Instrumented LRU cache example~~ [example] Add Instrumented Loading LRU cache example May 9, 2025

marcdumais-work force-pushed the lru-cache-example branch 11 times, most recently from 578c452 to b3b2fb4 Compare May 13, 2025 19:31

marcdumais-work marked this pull request as ready for review May 14, 2025 17:20

marcdumais-work force-pushed the lru-cache-example branch 2 times, most recently from a54961a to 9e3dad2 Compare May 14, 2025 18:02

bhufmann requested changes May 14, 2025

View reviewed changes

marcdumais-work force-pushed the lru-cache-example branch 3 times, most recently from c47ee9e to bbb321b Compare May 14, 2025 19:09

bhufmann requested changes May 15, 2025

View reviewed changes

marcdumais-work force-pushed the lru-cache-example branch 4 times, most recently from c96774e to 246a4e4 Compare May 15, 2025 15:00

marcdumais-work force-pushed the lru-cache-example branch from 246a4e4 to 242c4cc Compare May 15, 2025 15:26

bhufmann approved these changes May 15, 2025

View reviewed changes

PatrickTasse approved these changes May 15, 2025

View reviewed changes

MatthewKhouzam merged commit c10569f into eclipse-tracecompass:main May 16, 2025
5 of 6 checks passed


		#### Leads

		- How to explain the disproportional decrase in performance between the fully sized caches vs 95% sized results? It's a ~12x factor!

		@@ -0,0 +1,96 @@
		package org.eclipse.tracecompass.traceeventlogger.lrucachedemo;

		@@ -0,0 +1,249 @@
		package org.eclipse.tracecompass.traceeventlogger.lrucachedemo;

		@@ -0,0 +1,240 @@
		package org.eclipse.tracecompass.traceeventlogger.lrucachedemo;

[example] Add Instrumented Loading LRU cache example #66

[example] Add Instrumented Loading LRU cache example #66

Uh oh!

Conversation

marcdumais-work commented May 6, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What it does

How to test

Follow-ups

Review checklist

Uh oh!

bhufmann left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

bhufmann left a comment

Choose a reason for hiding this comment

Uh oh!

marcdumais-work commented May 15, 2025

Uh oh!

bhufmann left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

marcdumais-work commented May 6, 2025 •

edited

Loading