Open to enhancements to the FoundationDB integration?

[jon-signal]

Hello!

We're starting our journey with FoundationDB and were pleased to discover that there's a first-class FoundationDB integration! As we've worked with it more, though, we've found that the existing integration doesn't quite meet our needs in a few key areas. Please let me be clear that I'm new to the world of custom checks and integrations, and it's entirely possible that I'm just misunderstanding things. That said, here's what we're finding ourselves wanting so far:

1. As near as I can tell, the existing integration can only really work with a single FoundationDB cluster. There are two components to that: first, I don't immediately see a mechanism in the check for per-instance tags (so there's no way to tag metrics from two different clusters with two different cluster names, for example). Second, the check has only a single _db instance and appears to ignore instance configuration entirely, and so there's no way to provide multiple FoundationDB cluster files to facilitate connections to different clusters:

   integrations-core/foundationdb/datadog_checks/foundationdb/check.py

   Lines 40 to 41 in ffeb442

   	def check(self, _):
   	status_data = self.fdb_status_data()

   (EDIT: This is incorrect! I just misunderstood the lifecycle of check instances!)
2. It's challenging to associate process-level metrics with roles in the current integration. The existing CPU utilization metric, for example, breaks things down by process but that's all. If we see that one process is reporting unusually high CPU utilization, we need to pull the process name, then use external tools to figure out what role(s) that process is performing, and then use that information to decide how to address the situation. It'd be great if the process-level metrics were tagged by role, too.
3. Some metrics that would be really helpful to us to understand cluster performance—like disk busy-ness—just aren't included in the metrics at all.

Would you folks be open to third-party contributions? I think the main things that would change are:

1. Provide some mechanism for specifying multiple clusterfiles
2. Add new tags/dimensions to existing metrics
3. Fundamentally change the topology of process/role metrics to more easily associate processes and roles
4. Modify parts of the dashboard to make use of the new metrics

This would be, I think the semantic versioning equivalent of a major version change, and I'm not sure how you think about those in terms of integration updates.

We intend to make these changes for our own installation regardless; would you be interested in a pull request to make comparable changes upstream?

Thanks!

[jon-signal]

Second, the check has only a single _db instance and appears to ignore instance configuration entirely

Actually, this may be an embarrassing (if foreseen!) consequence of me being new to custom checks/integrations. Do we wind up with one FoundationdbCheck per configured instance? If so, perhaps the concerns about multiple clusters are misplaced!

Regardless, we're still planning on making changes to the metrics themselves and would be happy to contribute those upstream if there's an appetite for that kind of change!

[jon-signal]

Do we wind up with one FoundationdbCheck per configured instance?

Indeed we do. The developer docs (which I regret not finding earlier!) say:

The Agent creates [an instance of a custom check] for each element contained in the instances sequence within the corresponding config file…