feat(llmobs): user processor can add metadata to span#15376
feat(llmobs): user processor can add metadata to span#15376matthgt wants to merge 1 commit intoDataDog:mainfrom
Conversation
matthgt
force-pushed
the
mr/llmobs-allow-user-set-metadata
branch
2 times, most recently
from
2d68aa9 to
b863e9c
Compare
matthgt
force-pushed
the
mr/llmobs-allow-user-set-metadata
branch
from
b863e9c to
05b2ea8
Compare
|
hey @matthgt! thank you for contributing this, I think the change is reasonable. I'll try to get you a review by EOW 😄 |
Kyle-Verhoog
left a comment
Kyle-Verhoog
left a comment
There was a problem hiding this comment.
nice and simple! thanks for the patience 😄
| --- | ||
| features: | ||
| - | | ||
| LLMObs: This update introduces the ability for users to add custom metadata to LLMObs spans |
There was a problem hiding this comment.
| LLMObs: This update introduces the ability for users to add custom metadata to LLMObs spans | |
| LLMObs: This update introduces the ability for users to add or override custom metadata to LLMObs spans |
|
This pull request has been automatically closed after a period of inactivity. |
brettlangdon
left a comment
brettlangdon
left a comment
There was a problem hiding this comment.
unncessary change to Cargo.lock?
There was a problem hiding this comment.
why are there changes to this file? I don't see any Cargo.toml updates, so this lock file should not have needed updating
|
This pull request has been automatically closed after a period of inactivity. |
3 participants
Description
LLMObs allows users to register a processor for LLMObsSpan in order to redact sensitive information being sent to datadog. However, if you wish to store this sensitive information and allow it to be linked from datadog there is no readily available way to do that. URLs in the input/output get redacted by sensitive data scanner (as we want them to) and they probably don't belong there anyway.
This change allows the registered processor to optionally add metadata to the LLMObsSpan. When they do, it is merged into the span's metadata. This eventually is shown in the datadog UI for LLMObs:
Testing
I've added tests to test_llmobs.py and tested the changes locally with our codebase.
Risks
clashing metadata keys with existing/expected ddtrace metadata?
Additional Notes
None