Add remote write 2 sender compliance tests #194

pipiland2612 · 2025-10-28T11:37:09Z

What issue is this PR fixing?

Fixes remote_write_sender: Add support for Remote Write 2.0 #101

Changes

Completely rewrite current sender test replace with new test that is inspired by Add remote write 2 receivers compliance tests #182. TO be specifically I watch the work from receiver and flipping the functional to match a test for a sender
Latest run https://pipiland2612.github.io/

Since this is a very large work, I'd be happy to hear everyone opinions on this. Thanks for your time!

Signed-off-by: pipiland2612 <[email protected]>

remotewrite/sender/index.html

bwplotka

Thanks! Looks great from the high level, so far.

Small question, I will try to run it and check in details in the following days. Perhaps @saswatamcode or Juraj have time too

README.md

Signed-off-by: pipiland2612 <[email protected]>

bwplotka · 2025-11-07T15:17:35Z

README.md

 compliance tests for PromQL correctness. It's designed for smaller in-place quick unit tests, e.g. on per-PR basis, using docker based test framework. Useful as an acceptance tests
 for vendors or those who wish to maintain high Prometheus compatibility over time.

 ## Remote Write: Sender


TODO(to myself or others): Link receiver while we are at it.

bwplotka

Nice work!

Some comments, but generally great! I didn't review all tests in details, but some early thoughts:

This is good. Let's directly base it to main and do changes now, it's good!
Let's fix index.html (: (receiver mention when I ran sender)

Let's either update or remove circle CI that expects sender. Ideally we rely only on GH actions.
Let's move sender/receiver README to a single readme under remotewrite. Those are really similar (share same logic). Also when you run index.html It tries local ./result.json now and it makes no sense UNLESS we write shared docs for both sender/receiver.
Some of the tests are neither MUST, SHOULD or MAY - they are not in the spec e.g. optimizations like symbols must be deduplicated. Maybe it's obvious but maybe we need "OPTIMIZATION" rfc level?

{"Time":"2025-11-07T15:36:31.437028Z","Action":"pass","Package":"github.com/prometheus/compliance/remotewrite/sender","Elapsed":467.59}

remotewrite/receiver/index.html

remotewrite/sender/README.md

remotewrite/sender/batching_test.go

Signed-off-by: pipiland2612 <[email protected]>

pipiland2612 · 2025-11-08T11:01:16Z

Hi @bwplotka, thanks for reviewing.

Nice work!

Some comments, but generally great! I didn't review all tests in details, but some early thoughts:

This is good. Let's directly base it to main and do changes now, it's good!

Let's fix index.html (: (receiver mention when I ran sender)

Let's either update or remove circle CI that expects sender. Ideally we rely only on GH actions.

Let's move sender/receiver README to a single readme under remotewrite. Those are really similar (share same logic). Also when you run index.html It tries local ./result.json now and it makes no sense UNLESS we write shared docs for both sender/receiver.

Some of the tests are neither MUST, SHOULD or MAY - they are not in the spec e.g. optimizations like symbols must be deduplicated. Maybe it's obvious but maybe we need "OPTIMIZATION" rfc level?

Hmm strange, my index.html show the right thing:

I have removed the sender from the circle CI test: 248bad3
Move receiver/sender readme.md under cd3f5d2
Changing the test rfcLevel: 15874dd

PTAL thanks!

bwplotka

Looks amazing! I still need to look deeper thought.

Can we change the base to main?

Can we fix CI? (remove circleCI?)

remotewrite/README.md

bwplotka · 2025-11-13T10:21:41Z

remotewrite/README.md

+The test suite automatically downloads and runs Prometheus as the reference sender implementation. For testing custom senders, place the binary in the `bin/` directory.
+
+### For Receiver Tests
+A Prometheus server with Remote-Write Receiver enabled, as baseline:


This also requires Go 1.23+ no?

Hmm I dont know, probably not as the receiver readme.md doesn't mention this.

remotewrite/README.md

bwplotka

.

bwplotka · 2025-11-14T09:37:56Z

.circleci/config.yml

              dir:
                - alert_generator
                - promql
                - remotewrite/sender


We could create (copy) linting GH action from other Prometheus repos quickly and fix it.

bwplotka · 2025-11-14T15:06:02Z

Change base of this PR to main

bwplotka

Left some comments, so far!

Slowly going through (:

remotewrite/sender/backoff_test.go

remotewrite/sender/batching_test.go

Signed-off-by: pipiland2612 <[email protected]>

bwplotka · 2025-11-18T10:13:22Z

remotewrite/sender/rw1_compat_test.go

+				// RW 1.0 format doesn't have created_timestamp field.
+				// If sender is truly in RW 1.0 mode, this field should be 0/unset.
+				for _, ts := range req.Request.Timeseries {
+					should(t, int64(0) == ts.CreatedTimestamp, "RW 1.0 should not use created_timestamp field")


To be updated with Prometheus main

ping (: @pipiland2612

I would love to merge this compliance framework 👍🏽

Hi @bwplotka, thanks for pinging me. Solved at 386931a

Signed-off-by: pipiland2612 <[email protected]>

pipiland2612 added 12 commits October 27, 2025 14:50

init

562e28c

Signed-off-by: pipiland2612 <[email protected]>

fix should & may and test setup

3fbbaa0

Signed-off-by: pipiland2612 <[email protected]>

add go mod and go sum

f42eff6

Signed-off-by: pipiland2612 <[email protected]>

add parralell

a60451c

Signed-off-by: pipiland2612 <[email protected]>

add parralell and fix some tests

f6cd2a0

Signed-off-by: pipiland2612 <[email protected]>

add rfc and description

c89b3ae

Signed-off-by: pipiland2612 <[email protected]>

fix should and may so it fails test

46f4b13

Signed-off-by: pipiland2612 <[email protected]>

refactor tests

e931267

Signed-off-by: pipiland2612 <[email protected]>

fix comments

a28a7ef

Signed-off-by: pipiland2612 <[email protected]>

fix readme

1f93e87

Signed-off-by: pipiland2612 <[email protected]>

rm

fd48ceb

Signed-off-by: pipiland2612 <[email protected]>

add backoff test

3e64af3

Signed-off-by: pipiland2612 <[email protected]>

bwplotka reviewed Oct 28, 2025

View reviewed changes

remotewrite/sender/index.html Outdated Show resolved Hide resolved

bwplotka reviewed Oct 28, 2025

View reviewed changes

README.md Outdated Show resolved Hide resolved

pipiland2612 added 2 commits October 28, 2025 14:47

fix readme

b12aa94

Signed-off-by: pipiland2612 <[email protected]>

add common index.html

5d1e617

Signed-off-by: pipiland2612 <[email protected]>

bwplotka reviewed Nov 7, 2025

View reviewed changes

bwplotka reviewed Nov 8, 2025

View reviewed changes

remotewrite/receiver/index.html Outdated Show resolved Hide resolved

remotewrite/sender/README.md Outdated Show resolved Hide resolved

remotewrite/sender/batching_test.go Outdated Show resolved Hide resolved

pipiland2612 added 3 commits November 8, 2025 11:01

remove sender from circle ci test

248bad3

Signed-off-by: pipiland2612 <[email protected]>

add common readme.md

cd3f5d2

Signed-off-by: pipiland2612 <[email protected]>

change some test to optimization

15874dd

Signed-off-by: pipiland2612 <[email protected]>

pipiland2612 requested a review from bwplotka November 8, 2025 13:28

bwplotka reviewed Nov 14, 2025

View reviewed changes

bwplotka changed the base branch from exp-sender to main November 14, 2025 15:05

bwplotka reviewed Nov 14, 2025

View reviewed changes

pipiland2612 added 2 commits November 15, 2025 18:51

change some test to RECOMMENDED

bbe522e

Signed-off-by: pipiland2612 <[email protected]>

remove test

da28159

Signed-off-by: pipiland2612 <[email protected]>

pipiland2612 added 3 commits November 15, 2025 19:10

update readmd.md and remove sender from lint

597001c

Signed-off-by: pipiland2612 <[email protected]>

fix batch_size_reasonable

695ea8d

Signed-off-by: pipiland2612 <[email protected]>

fix comment

d8b3bcf

Signed-off-by: pipiland2612 <[email protected]>

bwplotka reviewed Nov 18, 2025

View reviewed changes

fix created timestamp to start timestamp

386931a

Signed-off-by: pipiland2612 <[email protected]>

Add remote write 2 sender compliance tests #194

Are you sure you want to change the base?

Add remote write 2 sender compliance tests #194

Uh oh!

Conversation

pipiland2612 commented Oct 28, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What issue is this PR fixing?

Changes

Uh oh!

Uh oh!

bwplotka left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

bwplotka Nov 7, 2025

Choose a reason for hiding this comment

Uh oh!

bwplotka left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

pipiland2612 commented Nov 8, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

bwplotka left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

bwplotka Nov 13, 2025

Choose a reason for hiding this comment

Uh oh!

pipiland2612 Nov 15, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

bwplotka left a comment

Choose a reason for hiding this comment

Uh oh!

bwplotka Nov 14, 2025

Choose a reason for hiding this comment

Uh oh!

bwplotka commented Nov 14, 2025

Uh oh!

bwplotka left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

bwplotka Nov 18, 2025

Choose a reason for hiding this comment

Uh oh!

bwplotka Nov 19, 2025

Choose a reason for hiding this comment

Uh oh!

pipiland2612 Nov 19, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

pipiland2612 commented Oct 28, 2025 •

edited

Loading

bwplotka left a comment •

edited

Loading

pipiland2612 commented Nov 8, 2025 •

edited

Loading

pipiland2612 Nov 19, 2025 •

edited

Loading