Question: Way to specify alternative lifetime for new entry in cached method?

[hcldan]

I have a cached async method that fetches some data. It's backed by a TimedSizedCache.
When running the method to get a new value to be cached, I have found that I may want to alter the lifetime of the entry I'm creating to something other than what the default lifetime of the cached entries are in the cache.

Is there a way to do this, and if not... any thoughts on how you think it might be possible with some work?

[jaemk]

Hey @hcldan , do you have an example you could share? My immediate thought is that the alternative lifetime could only be solved by making the cached value be an Arc (return an Arc from your function) so you have the flexibility to do what you need. When using the macros to memoize functions, the backing cache is a global so you can't really have a lifetime on the cached values since they will be owned by the global cache.

If it fits your use case, you could alternatively use and manage the cache instance yourself (without the macros), and that would allow you more freedom of types, but you lose the convenience of functions being memoized for you.

[hcldan]

Hey there, sorry for the late response.

So my example would be... I'm implementing a TlsResolver to fetch certs dynamically in the middle of the TLS handshake. Going and searching for a cert involves a fair bit of IO, and I'd like to cache the result for a time so that I don't have to go read the certs.

However, I don't want the lifetime of the item to extend beyond the expiry of the cert.
So when returning my result from the cached method, I'd love to be able to specify the TTL for the entry.

[hcldan]

Maybe the caching code could do something special if the function returned a Lifetime<T> or Result<Lifetime<T>, Error> that would implement providing the cache code with the Duration it should use.

Lifetime::new(T, Duration) something like that.

[jaemk]

I see, so what you're really after is a way to have a cached function specify a per-item ttl as part of the function return (I misunderstood, thought you meant "rust lifetime"). That's certainly doable, the macros support something implemented similarly in order to specify whether a value was cached or newly generated (see with_cached_flag).

Not sure when I'll get the chance to explore this (and figuring out how it will play/combine with with_cached_flag), so I'd suggest dropping the macro and using ExpiringSizedCache directly which lets you insert with a ttl and also supports concurrent reads. Depending on what your app looks like, you'd either need to store an Arc<RwLock<ExpiringSizedCached<K, V>>> in your webserver's state or make it a global static (what the macros do) by putting it in a lazy_static or once_cell::sync::Lazy, and doing the cache get/set yourself at the call site. More work than the macros, but more control - also note the macros only work with cache types that implement the Cached traits which ExpiringSizedCache does not, since my primary desire for it was to provide a cache supporting concurrent reads.

[hcldan]

So... I can see this both ways.
There's really no perfect fit for this and both TimedSizedCache and ExpiringSizedCache seem very similar and I'm not sure which one would be best.

I don't imagine my cache will be huge, so I don't think I'll be too worried about eviction based on size constraints. Either of these seem suitable, really.

It might be interesting to define a trait, Expires, where you could have a SomeCache<K, V> where v:Expires. That way there could be way more flexibility in what a consumer of this crate could cache. Simply creating a storage struct and implementing different Traits to interact with the cache.

trait Expires {
  /// Return `None` to use the default TTL as configured by the cache.
  /// Return `Some<Duration>>` to override cache default.
  fn ttl() -> Option<Duration>;
}

I see there is already a CanExpire trait, but the naming is a bit strange to me, only having an is_expired() function... I would have expected that trait to tell you if a value indeed could expire or not.

You could include some default impls for primitives or core types that just return None.
And you could even expand on this idea to implement other configurable traits that override cache defaults.

trait Refreshes {
  /// If true, fetches of this value will refresh the ttl for this entry in a TimedCache.
  fn time_refresh() -> bool
}

trait FallsBack<T> {
  /// A per-entry optional fallback value for cached methods returning an [Error]
  fn fallback() -> Option<T>;
}

If you're up for this idea I could maybe try my hand at it.

[hcldan]

I realize I can use the caches directly, but I really do like the interaction pattern of having the attribute macro on my function.

[hcldan]

More work than the macros, but more control - also note the macros only work with cache types that implement the Cached traits which ExpiringSizedCache does not, since my primary desire for it was to provide a cache supporting concurrent reads.

Is that not possible with macros? Maybe with a new macro? ![concurrent_read_cached()]

[jaemk]

Maybe with a new macro?

Yes, I've been meaning to add some new "CachedConcurrent" (or similarly named) trait, and then some macro variant, either a new macro entirely or new macro args to support annotating functions with caches that do not require synchronization to read.

Side note, the synchronized reads are primarily to increment cache metrics and to support traditional "LRU" functionality which requires mutating in some way on each read in order to maintain the "least recently used." The ExpiringSizedCache gets around this by expiring based on oldest insertion time instead of usage time, and also only evicts on insertions or explicit calls to "evict" which means it only needs to synchronize on writes (also doesn't record metrics).

The existing expiry trait was possibly poorly named since it was really adding support for defining a cache containing values that can tell you if they are expired/need to be evicted.

For those Refreshes and FallsBack:

• Refreshes is technically already supported on the Timed and TimedSized caches, see set_refresh or with_size_and_lifespan_and_refresh. To use with a macro, you would need to specify the cache type and constructor in the macro call, see https://github.com/jaemk/cached/blob/master/examples/kitchen_sink_proc_macro.rs#L54-L58
• FallsBack feels more like a macro option than a trait or something that cache types should implement/provide because it's tied to the function being wrapped and not the cache implementation. For this, the macros can take an additional optional arg default_on_error (or similarly named) which is a name of a function that returns the same value as the cache type and will be called if the wrapped function returns an error

[hcldan]

Just checking in again on my PR.  I know you are busy but this should be a rather quick ***@***.*** I am not understanding the current check failure...I think it's complaining that the readme was changed? But I thought I read in the contributing guidelines to update the readme. [check/fmt]: Checking code format... 7cargo fmt --check 8[check/readme]: Checking README.md... 9cargo readme --no-indent-headings | cmp README.md 10README.md - differ: byte 4364, line 89 11make: *** [Makefile:131: check/readme] Error 1 This is the diff for the readme:https://github.com/jaemk/cached/pull/240/files#diff-b335630551682c19a781afebcf4d07bf978fb1f8ac04c6bf87428ed5106870f5 CONTRIBUTING.md states this:The README.md is generated using cargo-readme from the crate level documentation in src/lib.rs. This means the README.md should never be modified by hand. To sync changes made to src/lib.rs, run make docsWhich I did, and checked in the result. Dan DumontE-mail: ***@***.***  ----- Original message ----- From: "James Kominick" ***@***.***> To: "jaemk/cached" ***@***.***> Cc: "Dan Dumont" ***@***.***>, "Mention" ***@***.***> Subject: Re: [jaemk/cached] Question: Way to specify alternative lifetime for new entry in cached method? (Issue #238)Date: Wed, Apr 30, 2025 9:54 PM [CAUTION: This email is from outside the organization. Unless you trust the sender, don't click links or open attachments as it may be a phishing email, which can steal your information and compromise your computer.] jaemk left a comment (jaemk/cached#238)Maybe with a new macro?Yes, I've been meaning to add some new "CachedConcurrent" (or similarly named) trait, and then some macro variant, either a new macro entirely or new macro args to support annotating functions with caches that do not require synchronization to read.Side note, the synchronized reads are primarily to increment cache metrics and to support traditional "LRU" functionality which requires mutating in some way on each read in order to maintain the "least recently used." The ExpiringSizedCache gets around this by expiring based on oldest insertion time instead of usage time, and also only evicts on insertions or explicit calls to "evict" which means it only needs to synchronize on writes (also doesn't record metrics).The existing expiry trait was possibly poorly named since it was really adding support for defining a cache containing values that can tell you if they are expired/need to be evicted.For those Refreshes and FallsBack:Refreshes is technically already supported on the Timed and TimedSized caches, see set_refresh or with_size_and_lifespan_and_refresh. To use with a macro, you would need to specify the cache type and constructor in the macro call, see https://github.com/jaemk/cached/blob/master/examples/kitchen_sink_proc_macro.rs#L54-L58FallsBack feels more like a macro option than a trait or something that cache types should implement/provide because it's tied to the function being wrapped and not the cache implementation. For this, the macros can take an additional optional arg default_on_error (or similarly named) which is a name of a function that returns the same value as the cache type and will be called if the wrapped function returns an error—Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you were mentioned.Message ID: ***@***.***>