Update Python VLM example in README #1178
Conversation
Existing example uses an undefined "read_image" function, and using max_new_tokens in pipe.generate() resulted in an error with latest nightly. I updated the example to work out of the box. Makes it a bit longer, but this section is hidden by default in the README, so it doesn't add to visual clutter for people just visiting the repo.
README.md
Outdated
|
|
||
| # Choose GPU instead of CPU in the line below to run the model on Intel integrated or discrete GPU | ||
| pipe = ov_genai.VLMPipeline("/home/ubuntu/models/MiniCPM-V-2_6", "CPU") | ||
| config = ov_genai.GenerationConfig() |
There was a problem hiding this comment.
Config creation was intentionally avoided to preserve default model generation config.
There was a problem hiding this comment.
The current example sets max_new_tokens in the generate() call but that returns with an error:
ValueError: 'max_new_tokens' is unexpected parameter name. Use help(openvino_genai.VLMPipeline.generate) to get list of acceptable parameters.
So what is the best practice for adding max_new_tokens without adding a config?
There was a problem hiding this comment.
@popovaan, please take a look why max_new_tokens isn't accepted by VLMPipeline
There was a problem hiding this comment.
looks like after we have migrated to new Sampler, we have not updated a list of generation parameters accepted by Python. So, all manual parameters are ignored and we have to fix it in 2024.5
There was a problem hiding this comment.
Checked with 2024.5 RC4 that setting max_new_tokens directly works now so I removed the generation config from the example.
Co-authored-by: Vladimir Zlobin <[email protected]>
github-actions
bot
added
no-match-files
and removed
category: sampling
Existing example uses an undefined "read_image" function, and using max_new_tokens in pipe.generate() resulted in an error with latest nightly.
I updated the example to work out of the box. Makes it a bit longer, but this section is hidden by default in the README, so it doesn't add to visual clutter for people just visiting the repo. Also added links to the relevant samples.