Cache Implementation in Cac Client

[pratikmishra356]

Idea is to fast lookup of the config for a given context, for this we will use cache where evaled config will be stored, and fallback will be the current eval logic i.e to iterate over all the condition , apply the given context and keep patching overrides
on top of default_configs

New Structure of Client

pub struct Client {
    tenant: String,
    reqw: Data<reqwest::RequestBuilder>,
    polling_interval: Duration,
    last_modified: Data<RwLock<DateTime<Utc>>>,
    config: Cache,
}

Function addition

fn get_hash_val(parameters same as eval)
{
 // generate the key string by json sorting the given context
 // lookup for evaled_value in cache , 
// if it is not present we will do eval and then set the value in cache and return
 }

Crate used for caching ->
For now we are going with mini_moka
we can set ttl, tti, number of keys in configuration

Structure of Cache
<String, Map<String,Value>>
key: this will sorted json string of (given context based on which eval happens)
value: this will be the full evaled config

[Datron]

@pratikmishra356 do we need a new function get_hash_val? We can do the following:

• for all functions, first check the cache
• if cache hit, return the data
• if cache miss, eval the context
• write to cache, return the data

[pratikmishra356]

@Datron Currently, we have only eval function where we dont have any kind of filter
get_full_config_state_with_filter, get_resolved_config includes filter parameters so we cannot use cache there for now.

Instead of new function we can include this in eval itself only if we are enabling this cache feature by default

[pratikmishra356]

Included cache in eval function itself

[pratikmishra356]

Perf cac eval cache hit : microseconds
50%ile -> 80
75%ile -> 105
90%ile -> 145
95%ile -> 181
99%ile -> 603

Perf cac eval cache miss : microseconds (this includes eval and updating cache) -> around 500 overrides
50%ile -> 2200
75%ile -> 2700
90%ile -> 3100
95%ile -> 3600
99%ile -> 5000