Is text-gen-webui able to load the new meta-llama_Llama-3.2-11B-Vision? & Cannot load multimodal ext

[CallMeRive]

I have the model, downloaded it manually, but I cannot load it, cause Transformers doesn't recognize its architecture called "mllama". I understand that 'm' in "mllama" means multimodal, so probably I'd need the multimodal extension, but the multimodal extension doesn't want to load either, with these errors:

08:53:32-869239` INFO     Loading the extension "multimodal"
		Traceback (most recent call last)
 N:\text-generation-webui-main\server.py:279 in <module>                                                   
                                                                                                           
   278                 time.sleep(0.5)                                                                     
 > 279                 create_interface()                                                                  
   280                                                                                                     
                                                                                                           
 N:\text-generation-webui-main\server.py:166 in create_interface                                           
                                                                                                           
   165         extensions_module.create_extensions_tabs()  # Extensions tabs                               
 > 166         extensions_module.create_extensions_block()  # Extensions block                             
   167                                                                                                     
                                                                                                           
 N:\text-generation-webui-main\modules\extensions.py:199 in create_extensions_block                        
                                                                                                           
   198                 extension, _ = row                                                                  
 > 199                 extension.ui()                                                                      
   200                                                                                                     
                                                                                                           
 N:\text-generation-webui-main\extensions\multimodal\script.py:99 in ui                                    
                                                                                                           
    98     global multimodal_embedder                                                                      
 >  99     multimodal_embedder = MultimodalEmbedder(params)                                                
   100     with gr.Column():                                                                               
                                                                                                           
 N:\text-generation-webui-main\extensions\multimodal\multimodal_embedder.py:27 in __init__                 
                                                                                                           
    26     def __init__(self, params: dict):                                                               
 >  27         pipeline, source = load_pipeline(params)                                                    
    28         self.pipeline = pipeline                                                                    
                                                                                                           
 N:\text-generation-webui-main\extensions\multimodal\pipeline_loader.py:34 in load_pipeline                
                                                                                                           
   33     else:                                                                                            
 > 34         model_name = shared.args.model.lower()                                                       
   35         for k in pipeline_modules:                                                                   

AttributeError: 'NoneType' object has no attribute 'lower'

[CallMeRive]

Hey? Anybody reads this?

[ChrisWhiteSr]

I got it to work but man is it slow with my machine! 3060 12GB

good luck!

1. Run cmd_windows.bat in the text-generation-webui folder.
2. Set the HF_TOKEN like so: set HF_TOKEN=<YOUR_HF_TOKEN_HERE>
3. Lastly, run download-model.py with the model argument.
   Example: python download-model.py meta-llama/Llama-3.2-11B-Vision

[CallMeRive]

Your answer is about downloading the model, which is not a problem - I have the model.
My question was about loading it. Without the multimodal extension I get the error "The checkpoint you are trying to load has model type mllama but Transformers does not recognize this architecture." etc, but when I try to activate the multimodal extension, the error from the first post occurs.

I've updated the post to clarify this confusion.

[CalculonPrime]

Even if you loaded it, wouldn't oobabooga need to also add support for importing images for it to do anything? As I understand it Llama 3.2 "vision" models are about "image to text". Basically the opposite of stable diffusion. So you'd drag a photo into the (hypothetical) Web UI in the future, and then you could ask the text engine questions about it.

[CallMeRive]

That's why, I believe, I need the multimodal extension.
https://github.com/oobabooga/text-generation-webui/blob/main/extensions/multimodal/README.md