Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

AI缓存插件的应用场景问题 #1459

Open
ilovedumplings opened this issue Oct 30, 2024 · 4 comments
Open

AI缓存插件的应用场景问题 #1459

ilovedumplings opened this issue Oct 30, 2024 · 4 comments

Comments

@ilovedumplings
Copy link

Why do you need it?

AI缓存插件,可以配置缓存过期时间,但是在对提问进行缓存的时候,对于一些具备时效性的提问,比如现在几点了,或者请告诉我昨天12点和今天12点美国都有什么新闻,这种具备明显时间关联性质的问题,如何避免缓存?

How could it be?

A clear and concise description of what you want to happen. You can explain more about input of the feature, and output of it.

Other related information

Add any other context or screenshots about the feature request here.
@johnlanni
Copy link
Collaborator

这个我考虑扩展一下 ai intent 插件，识别出意图后设置一个 header，然后基于不同 header 匹配不同路由，对特定 header 的路由可以关闭缓存插件

@ilovedumplings
Copy link
Author

intent

我的意思是针对同一个模型~也就是同一个路由这样去配置

@ilovedumplings
Copy link
Author

ai

ai intent这个插件有开放吗请问

@johnlanni
Copy link
Collaborator

johnlanni commented Oct 30, 2024 •
edited
Loading

intent

我的意思是针对同一个模型~也就是同一个路由这样去配置

那就要在 ai cache 插件里支持意图识别，这个应该不会考虑去做

ai intent这个插件有开放吗请问

https://github.com/alibaba/higress/tree/main/plugins/wasm-go/extensions/ai-intent

@CH3CHO CH3CHO mentioned this issue Nov 4, 2024
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@johnlanni @ilovedumplings