Skip to content

[features] Input QA prompt to VLM #123

@LukeLIN-web

Description

@LukeLIN-web

https://research.nvidia.com/labs/dir/pbench/

Support such features:
generate video, then

ask VLM

Question: Is the large truck positioned directly in front of the vehicle throughout the video?
Category: Space: Relationship
Answer: yes
Correct Answer: yes
Score: 1

Question: Does the truck veer off the road before coming to a stop on the shoulder?
Category: Time: Order
Answer: no
Correct Answer: yes
Score: 0

It is useful to evaluate some tasks such as #44

Metadata

Metadata

Assignees

Labels

No labels
No labels

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions