How do you collect usage analytics?

[ShubhamPalriwala]

hey hey!

really cool to try out the keboola mcp, worked well w my claude! i'm trying to understand how to do mcp servers analyse user events & track it. would like to know how y'all do it?

wasn't sure if this was the right place but let me know if you want to chat somewhere else.

[radcliffkey]

Hi Shubham!

Currently, we use two ways:

1. adding headers to Keboola API calls
2. adding metadata to Keboola objects (component configurations, transformations) created or updated by MCP tools.

Cheers, Rado

[mariankrotil]

Hi there,

Just to add on. We're also planning to introduce tracking via structured events PR#130. These events will follow a predefined schema and include details like tool inputs, outputs, relevant context, and error messages if any exception occurs. The goal is to make it easier to analyze how tools are used and debug issues when they happen.

Let us know if you’re exploring something similar or have thoughts on how you would like to consume such data.

[ShubhamPalriwala]

ahh okay, that's a great starting point! seems like a lot of companies are doing the same afaik!
what i’ve seen teams benefit from is being able to trace full user journeys and get answers to questions like:

• Which setups or configurations are performing better?
• Where are users dropping off or getting stuck?
• What’s the impact of changing prompts, tools, or flows?
• Are some models or toolchains consistently failing in certain patterns?
• How does performance vary across different models/users?

i believe these can be achieved by the structured logs y'all are planning but will require a lot of manual logging. i'm currently exploring on if building an sdk for something like this would be of help or not

[mariankrotil]

Yep, those are exactly the kinds of things we hope to get from the tracking and structured events. Definitely agree, at first, it’d take some manual effort to get the logging right, however, we have most of the functionality already implemented.

Curious to hear more about your approach with sdk, are you going towards something lightweight or more structured with built-in schemas? Currently, we’d like to first see how far we can get with our current approach and what kinds of insights it enables. but would be great to stay aligned and potentially collaborate as things evolve.

[ShubhamPalriwala]

yeah, i'd also love to see how y'all get everything going & get an understanding of the efforts, time, etc!
i'm thinking something lightweight that attaches the entire object we can extract & then you can configure what all you'd want & in what form like an OTEL exporter to Grafana, or anything else per se.

Would you be interested in getting on a call anytime? Can understand the problems you're facing & how you plan to tackle them