added keyword matching functionality for real-browser-monitor #6097

bhelm · 2025-09-03T09:03:05Z

📋 Overview

What problem does this pull request address?
- Real-browser monitors could only check HTTP status codes but couldn't verify that specific content was present on the page after JavaScript rendering. This made it impossible to detect when a web application loads successfully but displays error messages or missing content that would only be visible in the rendered page. This becomes a real deal breaker with modern web applications that only reference a single js file on the first request. Sometimes there are 100+ requests for additional js and content that are required to work in order for the page to display correctly, creating a lot potential for things to go wrong and the monitor to miss real problems.
What features or functionality does this pull request introduce or enhance?
- Adds optional keyword checking functionality to real-browser monitors, allowing users to specify a keyword that must be present (or absent with invert mode) in the rendered page content. The implementation extracts all visible text from the fully-loaded page using Playwright's textContent() method and performs keyword matching. This enables monitoring of dynamic web applications that rely heavily on JavaScript for content rendering.
Resolves: Real Browser Allow keyword lookup #4068

🛠️ Type of change

🐛 Bugfix (a non-breaking change that resolves an issue)
✨ New feature (a non-breaking change that adds new functionality)
⚠️ Breaking change (a fix or feature that alters existing functionality in a way that could cause issues)
🎨 User Interface (UI) updates
📄 New Documentation (addition of new documentation)
📄 Documentation Update (modification of existing documentation)
📄 Documentation Update Required (the change requires updates to related documentation)
🔧 Other (please specify):

📄 Checklist

🦿 LLM Assistance Declaration

This implementation was developed with assistance from an AI assistant (Claude) for:

Code architecture and implementation patterns
Test case development and E2E test scenarios
Code quality improvements and linting fixes
Pull request documentation

I (the submitter) reviewed the code closely and tested it manually. Not so close attention was given to the provided tests. I also verified that the keyword input field appears on the other monitors ui.

🧪 Testing

test coverage added:

Backend unit tests: test/backend-test/test-real-browser-keyword.js - Tests keyword logic, invert functionality, edge cases
E2E integration tests: test/e2e/specs/monitor-form.spec.js - Tests UI form behavior and monitor creation
All existing tests pass: Verified backward compatibility with existing functionality

Test results:

✅ 19/19 E2E tests passed
✅ All backend unit tests passed
✅ ESLint and StyleLint validation passed

🔧 Implementation Details

Files modified:

server/monitor-types/real-browser-monitor-type.js - Core keyword checking logic
src/pages/EditMonitor.vue - UI form updates
test/backend-test/test-real-browser-keyword.js - Unit tests (new)
test/e2e/specs/monitor-form.spec.js - E2E tests (enhanced)

Key features:

Text extraction using Playwright's page.textContent('body')
Text preprocessing (whitespace normalization)
Support for invert keyword functionality
Comprehensive error handling with informative messages
Full backward compatibility

📷 Screenshots or Visual Changes

UI Modifications: Added keyword and invert keyword fields to real-browser monitor form, matching the existing pattern used in HTTP keyword monitors.

Changes made:

Keyword input field (optional for real-browser monitors)
Invert keyword checkbox
Appropriate help text indicating the feature is optional
Consistent styling with existing monitor types

End result

https://vuetifyjs.com/en/getting-started/browser-support/ was choosen as the first request to this URL only references javascript and contains NO content.

Feature	Description
Keyword Field	Optional text input for specifying keywords to search for in rendered page content
Invert Keyword	Checkbox to reverse logic (monitor fails when keyword IS found)
Form Validation	Optional for real-browser (unlike required for HTTP keyword monitors)
Backward Compatibility	Existing real-browser monitors work unchanged

This PR resolves a real world problem and extends the coverage of uptime kuma to web 2.0 applications with minimal changes.

CommanderStorm

I have not run the code but it seems solid.
A few corners need a bit more scrubbing.

Have you looked Into if we can reuse the text monitor condition logic (see the dns monitor) to reduce duplicate code?
I would prefer to have less code for maintenance reasons ^^

CommanderStorm · 2025-09-04T09:01:18Z

server/monitor-types/real-browser-monitor-type.js

+                let textContent = await page.textContent("body");
+
+                if (textContent) {
+                    textContent = textContent.replace(/\s+/g, " ").trim();


Why this?
Could you add a comment?

comment is added. it removes duplicate whitespaces to make the keyword matching more predictable when matching multiple words.

how does it make more predicable?
What does this make more predictable?

Please actually document your reasoning why you think we should replace any whitespace at least one char wide with a space.

html collapes duplicate spaces <div>Hello my <p> world</p></div> would just read "hello my world" when rendered in html. maybe textContent would already handle that, but making sure its "hello my world" and not "Hello my world" makes it more predictable if the user is copying some phrase to match out of the browser instead of using the inspector to get the actual raw spacing.

maybe comment should be "remove duplicate whitespace as html also collapses them". Would that make more sense?

Please test that this is necessary and if it is, document why this is necessary and why it is the correct way of doing things.

I don't think your logic is quite as solid as you think it is:

hello<pre> </pre>world renders as:

hello

world

Also, how does this affect 	 and the other whitespace characters?
Please at least document why this is more expected than not doing this and how this is correct.

src/pages/EditMonitor.vue

test/backend-test/test-real-browser-keyword.js

test/e2e/specs/monitor-form.spec.js

server/monitor-types/real-browser-monitor-type.js

Co-authored-by: Frank Elsinga <[email protected]>

bhelm · 2025-09-09T14:42:03Z

The PR now includes a backend test for the "real browser" monitor that runs an actual chromium browser.
I have extracted the test setup into an seperate file so it can be used for currently untested and future functionality.

To be able to test the backend code without requiring an actual http:// or https:// website, i have allowed data:// urls. Data URLs are self-contained and are not subject to the local file:// inclusion vulnerability.

Also while reviwing the code and debugging problems with the test never finishing, i noticed that the browser's page or context may never be explicitely removed if errors occur during its execution. I made it more defensive by using a finally clause and explicitely closing page and context in any case, which may or may not resolve #3788

Thank you for providing a review and feedback. I hope it is all resolved and good now.

CommanderStorm · 2025-10-02T22:12:41Z

test/backend-test/test-real-browser-keyword.js

+const availableKeywords = {
+    found: "Hello", // Present in withHello pages
+    notFound: "Missing", // Not present in any pages
+    partial: "World", // Present in withHello pages


What does a partial keyword match even mean?
How is this different from found?

CommanderStorm · 2025-10-02T22:13:38Z

test/backend-test/test-real-browser-keyword.js

+    found: "Hello", // Present in withHello pages
+    notFound: "Missing", // Not present in any pages
+    partial: "World", // Present in withHello pages
+    numeric: "123", // Present in withNumeric page


how is this different from found?

CommanderStorm · 2025-10-02T22:15:02Z

test/backend-test/test-real-browser-keyword.js

+    notFound: "Missing", // Not present in any pages
+    partial: "World", // Present in withHello pages
+    numeric: "123", // Present in withNumeric page
+    special: "àáâãäå" // Present in withSpecialChars page


How is this an interesting case to test, what are we testing here?

CommanderStorm · 2025-10-02T22:22:31Z

server/monitor-types/real-browser-monitor-type.js

+            // Handle data: URLs which don't return a proper Response object
+            const isDataUrl = url.protocol === "data:";


I don't think this is usefull for monitoring purposes and for testing this is not very realistic.

Just spin up a local test server for the testcase, that should be fairly simple and not create such a weird branch

The data: url is a feature supported by all browsers and commonly used in testing. The problem was just that the monitor was over-filtering data urls and not getting a http status code from them, what the monitor expects.

Its much simpler, faster and more reliable than a local test server.

it saves the hassle of finding a free port for local binding and the error handling involved

race conditions when the server is not ready to respond when the test needs it or these wired problems where the port is not freed immediately and cannot be bound again for some seconds.

no problem with two tests running in parallel on the same network interface

uses less resources

causes less latency, runs faster

is less code

while its true that data: urls are not the production use case, the difference in the code path regarding the monitor code is very small, so the coverage is basicly the same.

CommanderStorm · 2025-10-02T22:30:46Z

test/backend-test/test-real-browser-keyword.js

+};
+
+test("Real Browser Monitor Integration Tests", {
+    skip: process.env.CI && !process.env.UPTIME_KUMA_ENABLE_BROWSER_TESTS


CommanderStorm · 2025-10-02T22:31:31Z

test/backend-test/test-real-browser-keyword.js

+    special: "àáâãäå" // Present in withSpecialChars page
+};
+
+test("Real Browser Monitor Integration Tests", {


these testcases are currently really really hard to read.

Please DAMP them up a bit.

https://testing.googleblog.com/2019/12/testing-on-toilet-tests-too-dry-make.html

bhelm added 2 commits September 3, 2025 10:32

added keyword matching functionality for real-browser-monitor

e0d7baa

removed obvious comments and empty lines

859debc

CommanderStorm requested changes Sep 4, 2025

View reviewed changes

bhelm and others added 4 commits September 5, 2025 10:50

remove optional handling for keyword description

e4e8600

Co-authored-by: Frank Elsinga <[email protected]>

setup real browser backend testing framework to use a real browser

127c63b

fixed linting errors

0997700

Merge branch 'master' into master

58dc1d3

bhelm requested a review from CommanderStorm September 9, 2025 15:11

Merge branch 'master' into master

94c1b11

CommanderStorm reviewed Oct 2, 2025

View reviewed changes

CommanderStorm marked this pull request as draft October 2, 2025 22:47

CommanderStorm added the pr:please address review comments this PR needs a bit more work to be mergable label Oct 2, 2025

		// Handle data: URLs which don't return a proper Response object
		const isDataUrl = url.protocol === "data:";

Uh oh!

added keyword matching functionality for real-browser-monitor #6097

Are you sure you want to change the base?

added keyword matching functionality for real-browser-monitor #6097

Conversation

bhelm commented Sep 3, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

📋 Overview

🛠️ Type of change

📄 Checklist

🦿 LLM Assistance Declaration

🧪 Testing

🔧 Implementation Details

📷 Screenshots or Visual Changes

Uh oh!

CommanderStorm left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

bhelm Oct 2, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

bhelm commented Sep 9, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

CommanderStorm Oct 2, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

bhelm commented Sep 3, 2025 •

edited

Loading

bhelm Oct 2, 2025 •

edited

Loading

CommanderStorm Oct 2, 2025 •

edited

Loading