Add PubSub and BigTable support for Central hosted network controllers#2526
Open
Add PubSub and BigTable support for Central hosted network controllers#2526
Conversation
… ztcontroller feature flag
…stance for the whole library, add init/shutdown functions for it exposed to C
match other things already written
Tests currently need to be run with --test-threads=1. Seems like the instances of the pubsub emulator stomp on each other without that
…s in new CentralDB class This allows us to interchangeably use different listeners (pgsql, redis, pubsub) depending on configuration values passed into the constructor.
PubSub allows us to do schema validation, however it only allows one top level message at a time. Move other sub-message declarations under the main message declaration so that we can enable schema validation in the pubsub stream directly
Muuuuch easier to use external dependencies now Also tried out conan and vcpkg. Ran into dependency issues when solving for packages to install with conan. vcpkg is just obtuse as all hell to install and not easy to integrate
* Postgres Direct * Redis * BigTable
Actually these *should* be structs, not filling out json. Adding this to the tech debt backlog
Contributor
|
@glimberg I'm only just noticing this. Do you still want it merged in? I see it is still a draft. |
Contributor
Author
|
Eventually yes. Just making sure everything is stable over in the new environment before doing so |
Contributor
Author
|
@joseph-henry ready for your review. I know this is a huge one. Don't hesitate to reach out if you have any questions |
Contributor
Author
|
OK not sure why the github action is failing now but I'll look at it tomorrow |
Required to get the list of configs for a particular org that the controller has. Named it `linked_id` rather than `org_id` since we don't know what it will be linked to in CV2
Publish CTL_NONCE_UPDATE to PubSub when nonces are created or reused in getSSOAuthInfo(), with the network's frontend as a message attribute so only the correct CV frontend receives it. Listen for ZT1_AUTH_UPDATE messages and update sso_expiry.authentication_expiry_time accordingly, with a network existence check before applying. - Add sso_send_topic/sso_recv_topic to PubSubConfig - Add PubSubWriter::publishSSONonceUpdate() with frontend param - Add PubSubSSOListener class for inbound auth updates - Rename CV1_AUTH_UPDATE to ZT1_AUTH_UPDATE in sso.proto - Fix pre-existing connection pool leak in getSSOAuthInfo() catch block
Allow controllers to advertise which central version (cv1, cv2, or all) they are assigned to handle via a new configurable field. The value is persisted to the database on each heartbeat and validated at startup against the DB CHECK constraint.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This is a big one that goes along with other internal upcoming changes.
NotificationListenerinterface, and adds a 3rd method for message passing via GCP PubSub. Which one to use is configurable by local.conf controller settingsext/. This is likely more useful for Central controller builds than end user builds for the time being.