APICurio Caching

[Gourav2912]

Hello, I want to understand the caching mechanism used in apicurio for validation of schema, Is schema once fetched will remain in memory or everytime apicurio APIs are call for fetching schema at runtime using APICurio SDK?

Thanks in advance.

[EricWittmann]

Hi @Gourav2912 thanks for the question and apologies for the delay in responding. The Apicurio Registry server does not currently cache any data. We are exploring server-side caching options, but it's not a high priority at the moment.

Instead, it is expected that clients (SerDes, SDKs, CLIs, etc) will perform caching logic. For example, the Apicurio SerDes classes that we provide for Kafka use-cases will cache the schemas on the client-side.

Note that aggressive client-side caching should mostly be safe due to the immutability of artifact versions.

[davidradl]

HI @EricWittmann , How should I cache using the V2 Registry Client? Is this something that I need to implement myself? Or are are there classes / sample implementations I could copy / be inspired by. I am thinking that for deserialize - I will keep 2 maps with keys of globalId and contentId with the value of the schema as the value. My only concern is the maps blowing up by using all the heap, so maybe we would need some cache management. WDYT?

[EricWittmann]

The provided Kafka serializers/deserializers already do this caching. I believe the Java class we use to perform caching in that layer is this:

https://github.com/Apicurio/apicurio-registry/blob/main/schema-resolver/src/main/java/io/apicurio/registry/resolver/ERCache.java

If you look in the schema resolver code, you will find some usages:

https://github.com/search?q=repo%3AApicurio%2Fapicurio-registry%20path%3A%2F%5Eschema-resolver%5C%2Fsrc%5C%2F%2F%20ERCache&type=code

Your concern is valid - certainly some sort of LRU cache should be used, with a cap of some kind. For most use cases it's probably not needed, since any given Kafka app is likely to use a small number of schemas. But for edge cases an LRU cache could be important.

[davidradl]

Thanks Eric that is helpful. I am likely going to need to use this ERCache with the Registry Client and expose the cache configuration.