Error writing spans to disk - 0.16.0 - When built from source

[gopal-vunet]

I got the below error while building the SDK from source, SDK version is 0.16.0. When I used the save SDK from public repository it was not having this issue.

Device Used - Android emulator API 35

An unexpected error happened while attempting to write the data in disk. 
java.io.FileNotFoundException: /data/user/0/io.vunet.vubanknative/cache/opentelemetry/signals/logs/1762856920473: open failed: ENOENT (No such file or directory)
at libcore.io.IoBridge.open(IoBridge.java:574)
at java.io.FileOutputStream.<init>(FileOutputStream.java:259)
at java.io.FileOutputStream.<init>(FileOutputStream.java:208)
at io.opentelemetry.contrib.disk.buffering.internal.storage.files.WritableFile.<init>(WritableFile.java:39)
at io.opentelemetry.contrib.disk.buffering.internal.storage.FolderManager.createWritableFile(FolderManager.java:77)
at io.opentelemetry.contrib.disk.buffering.internal.storage.Storage.write(Storage.java:61)
at io.opentelemetry.contrib.disk.buffering.internal.storage.Storage.write(Storage.java:69)
at io.opentelemetry.contrib.disk.buffering.internal.storage.Storage.write(Storage.java:47)
at io.opentelemetry.contrib.disk.buffering.internal.storage.FileSignalStorage.write(FileSignalStorage.java:48)
at io.opentelemetry.contrib.disk.buffering.storage.impl.FileLogRecordStorage.write(FileLogRecordStorage.java:45)
at io.opentelemetry.contrib.disk.buffering.internal.exporters.SignalStorageExporter.exportToStorage(SignalStorageExporter.java:34)
at io.opentelemetry.contrib.disk.buffering.exporters.LogRecordToDiskExporter.export(LogRecordToDiskExporter.java:40)
at io.opentelemetry.android.export.BufferDelegatingLogExporter$delegatingExporter$1.invoke(BufferDelegatingLogExporter.kt:23)
at io.opentelemetry.android.export.BufferDelegatingLogExporter$delegatingExporter$1.invoke(BufferDelegatingLogExporter.kt:23)
at io.opentelemetry.android.export.DelegatingExporter.export(DelegatingExporter.kt:91)
at io.opentelemetry.android.export.BufferDelegatingLogExporter.export(BufferDelegatingLogExporter.kt:34)
at io.opentelemetry.sdk.logs.export.BatchLogRecordProcessor$Worker.exportCurrentBatch(BatchLogRecordProcessor.java:321)
at io.opentelemetry.sdk.logs.export.BatchLogRecordProcessor$Worker.run(BatchLogRecordProcessor.java:238)
at java.lang.Thread.run(Thread.java:1012)
Caused by: android.system.ErrnoException: open failed: ENOENT (No such file or directory)
at libcore.io.Linux.open(Native Method)
at libcore.io.ForwardingOs.open(ForwardingOs.java:563)
at libcore.io.BlockGuardOs.open(BlockGuardOs.java:274)
at libcore.io.ForwardingOs.open(ForwardingOs.java:563)
at android.app.ActivityThread$AndroidOs.open(ActivityThread.java:8591)
at libcore.io.IoBridge.open(IoBridge.java:560)
at java.io.FileOutputStream.<init>(FileOutputStream.java:259) 
at java.io.FileOutputStream.<init>(FileOutputStream.java:208) 
at io.opentelemetry.contrib.disk.buffering.internal.storage.files.WritableFile.<init>(WritableFile.java:39) 
at io.opentelemetry.contrib.disk.buffering.internal.storage.FolderManager.createWritableFile(FolderManager.java:77) 
at io.opentelemetry.contrib.disk.buffering.internal.storage.Storage.write(Storage.java:61) 
at io.opentelemetry.contrib.disk.buffering.internal.storage.Storage.write(Storage.java:69) 
at io.opentelemetry.contrib.disk.buffering.internal.storage.Storage.write(Storage.java:47) 
at io.opentelemetry.contrib.disk.buffering.internal.storage.FileSignalStorage.write(FileSignalStorage.java:48) 
at io.opentelemetry.contrib.disk.buffering.storage.impl.FileLogRecordStorage.write(FileLogRecordStorage.java:45) 
at io.opentelemetry.contrib.disk.buffering.internal.exporters.SignalStorageExporter.exportToStorage(SignalStorageExporter.java:34) 
at io.opentelemetry.contrib.disk.buffering.exporters.LogRecordToDiskExporter.export(LogRecordToDiskExporter.java:40) 
at io.opentelemetry.android.export.BufferDelegatingLogExporter$delegatingExporter$1.invoke(BufferDelegatingLogExporter.kt:23) 
at io.opentelemetry.android.export.BufferDelegatingLogExporter$delegatingExporter$1.invoke(BufferDelegatingLogExporter.kt:23) 
at io.opentelemetry.android.export.DelegatingExporter.export(DelegatingExporter.kt:91) 
at io.opentelemetry.android.export.BufferDelegatingLogExporter.export(BufferDelegatingLogExporter.kt:34) 
at io.opentelemetry.sdk.logs.export.BatchLogRecordProcessor$Worker.exportCurrentBatch(BatchLogRecordProcessor.java:321) 
at io.opentelemetry.sdk.logs.export.BatchLogRecordProcessor$Worker.run(BatchLogRecordProcessor.java:238) 
at java.lang.Thread.run(Thread.java:1012) 

[LikeTheSalad]

When I used the save SDK from public repository it was not having this issue.

What do you mean by the public repository? Is it the OTel Android SDK from maven central?

In case it's maven central, can you confirm which version of the SDK you used from it that was working correctly?

[gopal-vunet]

When I used the save SDK from public repository it was not having this issue.

What do you mean by the public repository? Is it the OTel Android SDK from maven central?

In case it's maven central, can you confirm which version of the SDK you used from it that was working correctly?

Yes I meant "OTel Android SDK from maven central". And the version i used is 0.16.0-alpha

[gopal-vunet]

It seems the issues is happening with the SDK used from maven as well, but only on the device with API 35 not with API 36.

Apologies for setting the wrong perspective.

I've tested this on two devices API 35 and API 36. Only API 35 seems to have this issue, irrespective of whether the SDK is used from maven or built locally.

[LikeTheSalad]

Thank you for all the details, @gopal-vunet

Even though this seems like an issue related to differences across Android API levels, I still think it's important to address it, so thanks for raising it.

Now that I know how to reproduce it, I'll take a deeper look this week. At first glance it seems like there might be some issue around the destination folder not existing, which could be either due to some permission issues or because we needed to manually create them. In any case I can't tell for sure right now, so I'll keep this issue updated on what I can find.

[LikeTheSalad]

Hi @gopal-vunet

I've just created an Android API 35 emulator and ran the demo-app there, which uses the initializer function.

I found an issue with the initializer defaults, because one behavior for the initializer is that it should enable diskbuffering by default, and it seems like that's not happening right now (I'll create a PR to address this issue separately). So I had add this to the demo-app initializer to enable disk buffering for my test:

diskBuffering {
    enabled(true)
}

And then ran the app in the API 35 emulator and I couldn't reproduce the issue. I'm going to need some help to try and reproduce it locally, so please let me know if:

• Do you use the initializer? If not, how do you configure disk buffering in your app?
• Can you reproduce it using the demo-app?

[gopal-vunet]

Hi @LikeTheSalad

Thanks for looking into it.

Do you use the initializer? If not, how do you configure disk buffering in your app?

I haven't used the initializer, here is my initialization code:

    fun init(){
        val diskBufferingConfig = DiskBufferingConfig(maxCacheSize = 10_000_000, enabled = true)
        val config =
            OtelRumConfig()
                .setGlobalAttributes(
                    Attributes.builder()
                        .put(stringKey("app.version.name"), appVersionName)
                        .put(stringKey("app.version.code"), versionCode,)
                        .build()
                )
                .setDiskBufferingConfig(diskBufferingConfig)

        val baseUrl = BuildConfig.BASE_URL

        val spansIngestUrl = "${baseUrl}/v1/traces"
        val logsIngestUrl = "${baseUrl}/v1/logs"
        
        val sessionAwareLogProcessor = SessionAwareLogProcessor {
            rum?.getRumSessionId()
        }

        val otelRumBuilder: OpenTelemetryRumBuilder =
            OpenTelemetryRum.builder(this, config)
                .addSpanExporterCustomizer {
                    OtlpHttpSpanExporter.builder()
                        .setEndpoint(spansIngestUrl)
                        .build()
                }.addTracerProviderCustomizer { sdkTracerProviderBuilder, _ ->
                    sdkTracerProviderBuilder.addSpanProcessor(GlobalAttributesSpanProcessor())
                }
                .addLogRecordExporterCustomizer {
                    OtlpHttpLogRecordExporter.builder()
                        .setEndpoint(logsIngestUrl)
                        .build()
                }
                .addLoggerProviderCustomizer { sdkLoggerProviderBuilder, _ ->
                    sdkLoggerProviderBuilder.addLogRecordProcessor(sessionAwareLogProcessor)
                }

            rum = otelRumBuilder.build()
    }

Can you reproduce it using the demo-app?

I will try to reproduce this and update you.