Commits on Aug 14, 2024

1. Add gNMI Extension field parsing support …

   gNMI allows for the use of an `extension` field in each top-level message of the gNMI RPCs: https://github.com/openconfig/reference/blob/master/rpc/gnmi/gnmi-extensions.md
   Given this is an arbitrary Protobyte payload, the default gNMI protobufs can't decode the payload contained within the field.
   This PR adds the necessary configuration options to load in an arbitrary Protobuf file per `extension` identifier, with a message-name to lookup the message type.
   
   In this change:
   1. A new function `DecodeExtension` is added. This function uses `protoreflect` to dynamically marshal arbitrary protoBytes into JSON. The loaded JSON is then put back in the Extension message as bytes (mainting type)
   2. The `Target` type has an `ExtensionProtoMap` added, allowing for the lookup of Extension IDs to loaded-in protobufs
   3. The required changes to the `TargetConfig` type to support loading in the new configuration
   4. Modified `collector.go` to output the gNMI message _after_ inlining the decoded protoBytes
   5. Loading in the protobufs was added to `app/target.go`: `parseExtensionProtos`. This uses the `Parser` provided by `protoreflect/desc/protoparse`
   6. Added functionality to `event.go` to insert the K/Vs provided by the Extension as Tags. Given we come from JSON, all numbers are float64, so the only 2 types supported currently are `string` and `float64`
   7. Minor helper function to turn the arbitrary JSON into an arbitrary map.
   
   This has been tested with a device emiting an `extension` field:
   ```
   [gnmic] target "edge01_test01": gNMI Subscribe Response: &{
   SubscriptionName:port_stats
   SubscriptionConfig:{"name":"port_stats","paths":["/interfaces/interface/"],"mode":"STREAM","stream-mode":"SAMPLE","encoding":"JSON","sample-interval":15000000000,"heartbeat-interval":15000000000,"outputs":["prom-scrape-output"]}
   Response:
   update:{timestamp:1723653363502452302  prefix:{elem:{name:"interfaces"}  elem:{name:"interface"  key:{key:"name"  value:"et-1/0/3"}}}
   update:{path:{elem:{name:"state"}  elem:{name:"hardware-port"}}  val:{json_val:"\"FPC1:PIC0:PORT3\""}}
   update:{path:{elem:{name:"state"}  elem:{name:"transceiver"}}  val:{json_val:"\"FPC1:PIC0:PORT3:Xcvr0\""}}}
   extension:{registered_ext:{id:1
       msg:"{\"systemId\":\"edge01_test01\",\"componentId\":65535,\"sensorName\":\"sensor_1005_2_1\",\"subscribedPath\":\"/interfaces/interface/\",\"streamedPath\":\"/interfaces/interface/\",\"component\":\"chassisd\",\"sequenceNumber\":\"770002\",\"payloadGetTimestamp\":\"1723653363502\",\"streamCreationTimestamp\":\"1723653361858\",\"exportTimestamp\":\"1723653363504\",\"streamId\":\"PERIODIC\"}"}}}
   ```
   Which is then properly rendered to a Prometheus metric:
   ```
   gnmi_interfaces_interface_state_hardware_port{component="chassisd",componentId="65535",hardware_port="FPC1:PIC0:PORT3",interface_name="et-1/0/3",metric_source="edge01_test01",subscription_name="port_stats",systemId="edge01_test01"} 1
   ```
   Note that some label-drop rules have been added to remove the spurious labels to avoid a cardinality explosion.

   

   Ichabond committed Aug 14, 2024
   Configuration menu

   • View commit details
   • Copy full SHA for a5e0eaa
   • Browse repository at this point

   Copy the full SHA

   a5e0eaa View commit details

   Browse the repository at this point in the history