Skip to content

Allow Name Aliasing on SpecifcAvroReader #3670

Open
@snoz2191

Description

@snoz2191

Problematic

Currently we have an error where the SpecificAvroReader is not able to deserialize correctly an Avro file while the flag for Specific Avro Reader is set to true.

Context

We have the following Avro schema:

{
  "namespace": "com.mycompany.data.schemas.avro",
  "type": "record",
  "name": "OnlineStatusChangedEvent",
  "aliases": ["OnlineStatusChange"],
  "fields": [
    {
      "name": "Id",
      "type": "long"
    },
    {
      "name": "currentStatus",
      "type": [
        "null",
        "string"
      ],
      "default": null
    },
    {
      "name": "previousStatus",
      "type": [
        "null",
        "string"
      ],
      "default": null
    }
  ]
}

And this generates, through the avro-maven-plugin a corresponding class which would fall under com.mycompany.data.schemas.avro.OnlineStatusChangedEvent. Now this change is both forwards and backwards compatible based on the latest Avro specification.

Nevertheless, when a consumer tries to deseriliaze an event written in this new schema, while having the previous schema and the associated old class, we get the following error:

Caused by: org.apache.kafka.common.errors.SerializationException: Could not find class com.mycompany.data.schemas.avro.OnlineStatusChangedEvent specified in writer's schema whilst finding reader's schema for a SpecificRecord.
	at io.confluent.kafka.serializers.AbstractKafkaAvroDeserializer.getSpecificReaderSchema(AbstractKafkaAvroDeserializer.java:336)
	at io.confluent.kafka.serializers.AbstractKafkaAvroDeserializer.getReaderSchema(AbstractKafkaAvroDeserializer.java:307)
	at io.confluent.kafka.serializers.AbstractKafkaAvroDeserializer.access$000(AbstractKafkaAvroDeserializer.java:57)
	at io.confluent.kafka.serializers.AbstractKafkaAvroDeserializer$1.load(AbstractKafkaAvroDeserializer.java:73)
	at io.confluent.kafka.serializers.AbstractKafkaAvroDeserializer$1.load(AbstractKafkaAvroDeserializer.java:68)
	at com.google.common.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3574)
	at com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2316)
	at com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2190)
	at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2080)
	... 24 common frames omitted

Goal/Expectation

If the Avro specification allows for name aliasing, the deserializer should be able to, in case the original class is not available, to search if one of the aliases mentioned has a corresponding class. Hence, we would ensure that our Deserializer allows for the same behaviour as the Avro schemas.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions