Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Investigate registrator state corruption bug when consul agent is down #4

Open
michal-franc opened this issue Jun 24, 2019 · 0 comments

Comments

@michal-franc
Copy link

Related to:
When a consul-agent collocated with registrator has died and a container was created on this same instance. We ended up with a corrupted registrator state that was registering phantom services to consul-cluster. Manual de-register was not helping as registrator was bringing phantom service back. Only recycling registrator on that instance helped.

Check internal - postmortem from 18.06.19.

Scenario 1:

  • registrator, consul-agent on one node
  • kill consul-agent
  • create container on that node
  • check if registrator internal state has been changed
  • start consul-agent
  • correct service should be registered with consul

Scenario 2:

  • registrator, consul-agent on one node
  • kill consul-agent
  • create container on that node
  • destroy container on that node
  • check if registrator internal state has been changed
  • start consul-agent
  • no service should be added to consul
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant