[logs-sdk] Remove LogData and extend SDK LogRecord to have instrumentation scope

[hectorhdzg]

Description

Removing LogData and extending SDK LogRecord to have instrumentation scope

Fixes #4313

Type of change

Please delete options that are not relevant.

• Breaking change (fix or feature that would cause existing functionality to not work as expected)
• This change requires a documentation update

How Has This Been Tested?

Please describe the tests that you ran to verify your changes. Provide instructions so we can reproduce. Please also list any relevant details for your test configuration

• Test A

[hectorhdzg]

Contrib tests expect an object like log.log_record returned in log_exporter.get_finished_logs()
I can update contrib tests for this breaking issue, let me know your thoughts

[DylanRussell]

What if instead of removing LogData, we remove LogRecord from the SDK, and update LogData to have 3 fields: the API LogRecord, the resource, and the instrumentation_scope..

I don't think we need a LogRecord in both the API and the SDK, when they are exactly the same except for 2 additional fields on the SDK version (resource and instrumentation_scope)..

This way we don't need to update 2 classes every time a change is made to the log data structure.

Also I think our implementation is confusing the 2 classes in a bunch of places maybe because they are named the same. For example I think Logger.emit should be taking an API LogRecord instead of an SDK one. We tell instrumentations to use the API LogRecord, and instrumentations therfore call Logger.emit with the API LogRecord, the SDK implementation should be adding instrumentation_scope and resource (to LogData) and then forward it along..

[hectorhdzg]

@DylanRussell we can also call it SDKLogRecord like Javascript https://github.com/open-telemetry/opentelemetry-js/blob/main/experimental/packages/sdk-logs/src/index.ts#L24

[DylanRussell]

Yeah I think i'm OK with that name too. But what do you think of it being just the 3 fields I mentioned above ? And switching Logger.emit in the SDK to take the API LogRecord and then create the SDK LogRecord and attach instrumentation_scope/resource to it..

That looks to be what javascript is doing too:

https://github.com/open-telemetry/opentelemetry-js/blob/99dde7786f52d8d7e3d080a0a69b9685104c29e2/experimental/packages/sdk-logs/src/Logger.ts#L42

[hectorhdzg]

@DylanRussell updated, still having SDKLogRecord inheriting API LogRecord, having the API LogRecord as an attribute feels really weird to me, I think it could cause more confusion an issues than solving them, let me know what you think.

[hectorhdzg]

@open-telemetry/python-maintainers this is definitely a big breaking change, do we want to use another branch to group the Logs updates together?, we discussed this last week SIG meeting, but trying to understand what is the best way to move forward here.

[DylanRussell]

In my opinion just have SdkLogRecord with a ApiLogRecord attribute, it should not inherit from it at all now.. We don't need another class with all the same attributes.

We just need a small wrapper class..

Logger.emit should accept an ApiLogRecord (not an SDK log record), it should then create the SdkLogRecord with the logger's instrumentation_scope and resource.

And LoggingHandler._translate should produce an ApiLogRecord..

[DylanRussell]

Looks like some instrumentations fail w/ this change (https://github.com/open-telemetry/opentelemetry-python-contrib/blob/f20fa77ad59d90aae4978ae28cb1a98b12fbb959/instrumentation-genai/opentelemetry-instrumentation-vertexai/tests/test_function_calling.py#L4 -- this instrumentation is accessing log_data.log_record.. If you keep log_record as an attribute that might pass)

[DylanRussell]

Mostly LGTM

[DylanRussell]

I know at one of the spec meetings we agreed it was a good idea to make a bunch of the breaking changes in one release.. Do we want to add the @Deprecation wrappers to some of these things first (ex: on LogData and the SDK LogRecord), wait a release, then do it, or just go ahead and make the change ?

[hectorhdzg]

I'm a little bit confused why genai tests are failing in contrib, looks like these are also testing sdk logs but not using current changes here, need to dig deeper

UPDATE: Events was sending incorrect types, fixed now

[DylanRussell]

K looks good to me again

[hectorhdzg]

I'm happy to resolve conflicts and update when release plan is defined

[hectorhdzg]

@xrmx I removed a lot of the code from #4771 my understanding those were temporary changes to let people know about this change, LogDeprecatedInitWarning is completely removed here

[hectorhdzg]

This PR should be ready for another look now, failing tests are about commit mismatch and actual check of breaking changes that is accurate in this case