Skip to content

change topicArns to map safe for concurrent access #3459

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 7 commits into from
Jun 24, 2024

Conversation

nenikola
Copy link
Contributor

Description

Changed native map to xsync.MapOf which is safe for concurrent access

Issue reference

We strive to have all PR being opened based on an issue, where the problem or feature have been discussed prior to implementation.

Please reference the issue this PR will close: #3430

Checklist

Please make sure you've completed the relevant tasks for this PR, out of the following list:

  • Code compiles correctly
  • Created/updated tests
  • Extended the documentation / Created issue in the https://github.com/dapr/docs/ repo: dapr/docs#[issue number]

@nenikola nenikola requested review from a team as code owners June 22, 2024 16:28
@yaron2
Copy link
Member

yaron2 commented Jun 22, 2024

Thanks for this PR. Can you please replace the usage of the xsync package with Go's sync.Map? It has the same methods. https://pkg.go.dev/golang.org/x/sync/syncmap

@nenikola
Copy link
Contributor Author

Thanks for this PR. Can you please replace the usage of the xsync package with Go's sync.Map? It has the same methods. https://pkg.go.dev/golang.org/x/sync/syncmap

@yaron2 sure, I opted for xsync just because I saw it being used in topicsLocker.

Also, I saw subscriptions and queues are also of type regular map and seems like both could potentially face similar concurrent write issue. Would it make sense to change those to sync.Map too?

Copy link
Contributor

@ItalyPaleAle ItalyPaleAle left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In this file we were doing synchronization using a mutex. For example, calls to getOrCreateTopic are wrapped around a lock:

s.topicsLocker.Lock(req.Topic)
defer s.topicsLocker.Unlock(req.Topic)

Rather than changing topicArns, it may make more sense to just ensure that wherever the access to the map was happening that wasn't wrapped in a log (and was causing the panic), uses the lock instead

@nenikola
Copy link
Contributor Author

@ItalyPaleAle so I did something similar to what is done in the Subscribe method by locking the topic when invoking getOrCreateTopic but would it make more sense for the method getOrCreateTopic to do the locking itself instead of its caller doing so?

The way it is now, caller of getOrCreateTopic needs to know that getOrCreateTopic method does unsafe map write, which shouldn't be the caller's concern per my understanding.

@ItalyPaleAle
Copy link
Contributor

@nenikola thanks for making the change.

I think adding a requirement to have a lock before calling a method is not an unusual thing. It is especially helpful in some cases such as when the caller may invoke the method twice. However perhaps we could add a comment to getOrCreateTopic to indicate that locks should be acquired by callers?

ItalyPaleAle
ItalyPaleAle previously approved these changes Jun 24, 2024
@nenikola
Copy link
Contributor Author

@ItalyPaleAle sounds good, let me add comment and then merge the changes. Thanks!

Copy link
Contributor

@ItalyPaleAle ItalyPaleAle left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for this contribution and for fixing the issue!

@ItalyPaleAle ItalyPaleAle merged commit d09ffe1 into dapr:main Jun 24, 2024
88 of 89 checks passed
@yaron2 yaron2 modified the milestones: v1.15, v1.14 Jul 31, 2024
@marcduiker
Copy link
Contributor

@holopin-bot @nenikola Thank you Nikola!

Copy link

holopin-bot bot commented Aug 15, 2024

Congratulations @nenikola, the maintainer of this repository has issued you a badge! Here it is: https://holopin.io/claim/clzv9vq4q125210cjyqv675ud2

This badge can only be claimed by you, so make sure that your GitHub account is linked to your Holopin account. You can manage those preferences here: https://holopin.io/account.
Or if you're new to Holopin, you can simply sign up with GitHub, which will do the trick!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

fatal error: concurrent map writes
4 participants