Skip to content

Added Label for Watch commands #5

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 8 commits into from
Nov 13, 2024

Conversation

psrvere
Copy link

@psrvere psrvere commented Nov 6, 2024

What is the Problem?

SDK provides Watch api to subscribe Watch commands. This returns push response ["GET", fingerprint, value] so that users can get fingerprint immediately.

Since multiplexing is allowed i.e. users can subscribe for multiple keys, example: GET.WATCH k1 and GET.WATCH k2, there is no way for SDK to know that if a received update is the first message for say k2 or an update message for k1.

What is the Solution?

PR adds watch label (uuid) in diceDB worker.go. It is generated with every watch subscription. Label is received only in the first response by SDK.

This PR adds two separate channels firstUpdateCh and updateCh for first response and subsequent updates of Watch subscriptions.

Watch api call is followed by Channel api call to get channel for updates. Earlier, Channel initialised initMsgChan routine which would check for updates from server.

Now Watch api call initialises initMsgChan and initMsgChan filters subscription first response and sends it to firstUpdateCh buffered channel which is then consumed by Watch. In First response, msg.Command is expected to be a valid uuid string. Label is not sent to the end user.

All subsequent responses are sent to updateCh which is returned as response to Channel api and users to consume updates on subscribed keys.

@psrvere psrvere marked this pull request as ready for review November 11, 2024 09:58
@psrvere psrvere changed the title Added watch label Added Watch Label Nov 11, 2024
@psrvere psrvere changed the title Added Watch Label Added Label for Watch commands Nov 11, 2024
Copy link
Collaborator

@JyotinderSingh JyotinderSingh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the working on this feature @psrvere! I have left some comments.

Comment on lines 555 to 563
label := msg.Command
_, err = uuid.Parse(label)

if err == nil {
select {
case c.firstUpdateCh <- msg:
if !timer.Stop() {
<-timer.C
}
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Trying to understand the flow here. Shouldn't we have multiple firstUpdateCh's - one for each Watch invocation associated with a unique uuid? The logic here could look up the uuid in a map and send it to the correct firstUpdateCh (for the respective watch command).

Copy link
Author

@psrvere psrvere Nov 13, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed. I have fixed this by creating a new channel whenever Watch is invoked and the same is stored in watchLabelFirstMsgChMap

Comment on lines 178 to 186
w.chOnce.Do(func() {
w.msgCh = newWatchCommandChannel(w)
// Create firstUpdateCh with buffer of 1 to avoid blocking
w.msgCh.firstUpdateCh = make(chan *WatchResult, 1)
w.msgCh.initMsgChan()
})

// Subscribe to the command
err := w.watchCommand(ctx, cmdName, args...)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The SDK here should generate a uuid and associate it with the given watch command (maybe create a map where they key is the uuid and the value is this invocation's respective firstMsgCh.

The uuid is sent to the server by the SDK and the server simply returns it back with the first response (note that the server should also be able to handle the case where no uuid is sent and in that case it simply returns an empty value)

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changed this, Now uuid is generated in SDK and sent to server.

@psrvere
Copy link
Author

psrvere commented Nov 13, 2024

@JyotinderSingh - worked on the comments. Please review.

Here is the flow for your understanding:

  • Watch() creates watchLabel (uuid) and dedicated firstMsgCh for each request. Their mapping is stored in watchLabelFirstMsgChMap
  • Both Watch() and Channel() initialise initMsgChan routine, with concurrency safety. This makes the order of calling these functions irrelevant
  • initMsgChan is responsible for routing first message to dedicated channel and updates to a common channel

Copy link
Collaborator

@JyotinderSingh JyotinderSingh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for addressing the reviews @psrvere! The changes look good, have just added a clarification comment.

Copy link
Collaborator

@JyotinderSingh JyotinderSingh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Awesome, thanks for adding this!

@JyotinderSingh JyotinderSingh merged commit 4ecce8f into DiceDB:master Nov 13, 2024
JyotinderSingh added a commit that referenced this pull request Nov 17, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants