Pipelines loaded with dtype=torch.float16 cannot run with cpu device.

[r3KoR]

Hi there!

I am using a image to video for lipsync through Sonic and suddenly get this error. When I do shorter videos it does not seem to cause any issues I assume this is due to length?

The thing is I want to create an animated avatar for youtube videos and they are generally 10 minutes long but when I work with 1 minute or less it does not cause any error.

Here is the log just in case. Is anyone able to spot an error or can confirm it's the video length?

[START] Security scan
[DONE] Security scan
## ComfyUI-Manager: installing dependencies done.
** ComfyUI startup time: 2025-03-20 08:47:02.754
** Platform: Linux
** Python version: 3.11.10 (main, Sep  7 2024, 18:35:41) [GCC 11.4.0]
** Python executable: /workspace/ComfyUI/venv/bin/python
** ComfyUI Path: /workspace/ComfyUI
** ComfyUI Base Folder Path: /workspace/ComfyUI
** User directory: /workspace/ComfyUI/user
** ComfyUI-Manager config path: /workspace/ComfyUI/user/default/ComfyUI-Manager/config.ini
** Log path: /workspace/ComfyUI/user/comfyui.log

[notice] A new release of pip is available: 24.0 -> 25.0.1
[notice] To update, run: pip install --upgrade pip

[notice] A new release of pip is available: 24.0 -> 25.0.1
[notice] To update, run: pip install --upgrade pip

Prestartup times for custom nodes:
   0.0 seconds: /workspace/ComfyUI/custom_nodes/rgthree-comfy
   0.0 seconds: /workspace/ComfyUI/custom_nodes/comfyui-easy-use
  29.5 seconds: /workspace/ComfyUI/custom_nodes/comfyui-manager

Checkpoint files will always be loaded safely.
Total VRAM 24210 MB, total RAM 128012 MB
pytorch version: 2.6.0+cu124
Set vram state to: NORMAL_VRAM
Device: cuda:0 NVIDIA GeForce RTX 4090 : cudaMallocAsync
Using pytorch attention
ComfyUI version: 0.3.26
ComfyUI frontend version: 1.11.8
[Prompt Server] web root: /workspace/ComfyUI/venv/lib/python3.11/site-packages/comfyui_frontend_package/static
Note: NumExpr detected 32 cores but "NUMEXPR_MAX_THREADS" not set, so enforcing safe limit of 16.
NumExpr defaulting to 16 threads.
[ComfyUI-Easy-Use] server: v1.2.8 Loaded
[ComfyUI-Easy-Use] web root: /workspace/ComfyUI/custom_nodes/comfyui-easy-use/web_version/v2 Loaded
WAS Node Suite: OpenCV Python FFMPEG support is enabled
WAS Node Suite Warning: `ffmpeg_bin_path` is not set in `/workspace/ComfyUI/custom_nodes/was-node-suite-comfyui/was_suite_config.json` config file. Will attempt to use system ffmpeg binaries if available.
WAS Node Suite: Finished. Loaded 220 nodes successfully.

        "Art is the soul made visible." - George Crook

----------Jake Upgrade Nodes Loaded----------
generated new fontManager
[AnimateDiffEvo] - ERROR - No motion models found. Please download one and place in: ['/workspace/ComfyUI/custom_nodes/ComfyUI-AnimateDiff-Evolved/models', '/workspace/ComfyUI/models/animatediff_models']
[AnimateDiffEvo] - ERROR - No motion models found. Please download one and place in: ['/workspace/ComfyUI/custom_nodes/ComfyUI-AnimateDiff-Evolved/models', '/workspace/ComfyUI/models/animatediff_models', '/workspace/ComfyUI/custom_nodes/comfyui-animatediff-evolved/models']
Total VRAM 24210 MB, total RAM 128012 MB
pytorch version: 2.6.0+cu124
Set vram state to: NORMAL_VRAM
Device: cuda:0 NVIDIA GeForce RTX 4090 : cudaMallocAsync
Successfully imported omost_function from omost.py in the current directory
### Loading: ComfyUI-Inspire-Pack (V1.14.1)
### Loading: ComfyUI-Impact-Subpack (V1.2.9)
Creating new Ultralytics Settings v0.0.6 file ✅ 
View Ultralytics Settings with 'yolo settings' or at '/root/.config/Ultralytics/settings.json'
Update Settings with 'yolo settings key=value', i.e. 'yolo settings runs_dir=path/to/dir'. For help see https://docs.ultralytics.com/quickstart/#ultralytics-settings.
[Impact Subpack] ultralytics_bbox: /workspace/ComfyUI/models/ultralytics/bbox
[Impact Subpack] ultralytics_segm: /workspace/ComfyUI/models/ultralytics/segm

[rgthree-comfy] Loaded 42 exciting nodes. 🎉

### Loading: ComfyUI-Impact-Pack (V8.8.1)
[Impact Pack] Wildcards loading done.
### Loading: ComfyUI-Manager (V3.31.7)
[ComfyUI-Manager] network_mode: public
### ComfyUI Revision: 3238 [9aac21f8] *DETACHED | Released on '2025-03-09'

Import times for custom nodes:
   0.0 seconds: /workspace/ComfyUI/custom_nodes/sdxl-recommended-res-calc
   0.0 seconds: /workspace/ComfyUI/custom_nodes/websocket_image_save.py
   0.0 seconds: /workspace/ComfyUI/custom_nodes/ComfyUi_NNLatentUpscale
   0.0 seconds: /workspace/ComfyUI/custom_nodes/cg-use-everywhere
   0.0 seconds: /workspace/ComfyUI/custom_nodes/Ttl_ComfyUi_NNLatentUpscale
   0.0 seconds: /workspace/ComfyUI/custom_nodes/cg-image-picker
   0.0 seconds: /workspace/ComfyUI/custom_nodes/ComfyUI_HuggingFace_Downloader
   0.0 seconds: /workspace/ComfyUI/custom_nodes/civitai_comfy_nodes
   0.0 seconds: /workspace/ComfyUI/custom_nodes/comfyui_ipadapter_plus
   0.0 seconds: /workspace/ComfyUI/custom_nodes/ComfyUI-GGUF
   0.0 seconds: /workspace/ComfyUI/custom_nodes/comfyui-if_ai_hfdownloadernode
   0.1 seconds: /workspace/ComfyUI/custom_nodes/comfyui-kjnodes
   0.1 seconds: /workspace/ComfyUI/custom_nodes/ComfyMath
   0.1 seconds: /workspace/ComfyUI/custom_nodes/comfyui-custom-scripts
   0.1 seconds: /workspace/ComfyUI/custom_nodes/comfyui_ultimatesdupscale
   0.1 seconds: /workspace/ComfyUI/custom_nodes/derfuu_comfyui_moddednodes
   0.2 seconds: /workspace/ComfyUI/custom_nodes/comfyui-jakeupgrade
   0.2 seconds: /workspace/ComfyUI/custom_nodes/comfyui-inspire-pack
   0.2 seconds: /workspace/ComfyUI/custom_nodes/rgthree-comfy
   0.3 seconds: /workspace/ComfyUI/custom_nodes/comfyui-animatediff-evolved
   0.3 seconds: /workspace/ComfyUI/custom_nodes/ComfyUI-AnimateDiff-Evolved
   0.3 seconds: /workspace/ComfyUI/custom_nodes/comfyui-videohelpersuite
   0.4 seconds: /workspace/ComfyUI/custom_nodes/comfyui-impact-pack
   0.6 seconds: /workspace/ComfyUI/custom_nodes/comfyui-manager
   0.7 seconds: /workspace/ComfyUI/custom_nodes/comfyui-easy-use
   0.8 seconds: /workspace/ComfyUI/custom_nodes/comfyui-impact-subpack
   1.7 seconds: /workspace/ComfyUI/custom_nodes/comfyui-detail-daemon
   8.6 seconds: /workspace/ComfyUI/custom_nodes/ComfyUI_Sonic
   9.7 seconds: /workspace/ComfyUI/custom_nodes/was-node-suite-comfyui
  13.0 seconds: /workspace/ComfyUI/custom_nodes/ComfyUI-IF_AI_tools

Starting server

To see the GUI go to: http://0.0.0.0:8188
To see the GUI go to: http://[::]:8188
[ComfyUI-Manager] default cache updated: https://raw.githubusercontent.com/ltdrdata/ComfyUI-Manager/main/alter-list.json
[ComfyUI-Manager] default cache updated: https://raw.githubusercontent.com/ltdrdata/ComfyUI-Manager/main/model-list.json
[ComfyUI-Manager] default cache updated: https://raw.githubusercontent.com/ltdrdata/ComfyUI-Manager/main/github-stats.json
[ComfyUI-Manager] default cache updated: https://raw.githubusercontent.com/ltdrdata/ComfyUI-Manager/main/custom-node-list.json
[ComfyUI-Manager] default cache updated: https://raw.githubusercontent.com/ltdrdata/ComfyUI-Manager/main/extension-node-map.json
FETCH ComfyRegistry Data: 5/79
FETCH ComfyRegistry Data: 10/79
FETCH ComfyRegistry Data: 15/79
FETCH ComfyRegistry Data: 20/79
FETCH ComfyRegistry Data: 25/79
FETCH ComfyRegistry Data: 30/79
FETCH ComfyRegistry Data: 35/79
FETCH ComfyRegistry Data: 40/79
FETCH ComfyRegistry Data: 45/79
FETCH ComfyRegistry Data: 50/79
FETCH ComfyRegistry Data: 55/79
FETCH ComfyRegistry Data: 60/79
FETCH ComfyRegistry Data: 65/79
FETCH ComfyRegistry Data: 70/79
FETCH ComfyRegistry Data: 75/79
FETCH ComfyRegistry Data [DONE]
[ComfyUI-Manager] default cache updated: https://api.comfy.org/nodes
FETCH DATA from: https://raw.githubusercontent.com/ltdrdata/ComfyUI-Manager/main/custom-node-list.json [DONE]
[ComfyUI-Manager] All startup tasks have been completed.
got prompt
Input audio duration is 551.2620408163265 seconds, infer audio duration is: 551.0 seconds.
Requested to load CLIPVisionModelProjection
loaded completely 21689.567377853393 1208.09814453125 True
100%|████████████████████████████████████████████████████████████████████████████████████████████████████| 6887/6887 [00:04<00:00, 1606.47it/s]
None
***********Start infer  ***********
100%|███████████████████████████████████████████████████████████████████████████████████████████████████████| 25/25 [2:48:18<00:00, 403.95s/it]
Pipelines loaded with `dtype=torch.float16` cannot run with `cpu` device. It is not recommended to move them to `cpu` as running them will fail. Please make sure to use an accelerator to run the pipeline in inference, due to the lack of support for`float16` operations on this device in PyTorch. Please, remove the `torch_dtype=torch.float16` argument, or use another device for inference.
Requested to load AutoencodingEngine
loaded completely 19564.821048736572 186.42292404174805 True

I would really appreciate your help!

Thank you!

[hlky]

Hi @r3KoR. On our end the message you're seeing is a warning, it will not stop the model from loading, and it looks like the generation succeeded, if the node is moving the model to CUDA afterwards it may not be a problem. Can you link to the custom node you're using, the fix (if any) will likely need to come from there but we are happy to take a look and advise on a solution.

diffusers/src/diffusers/pipelines/pipeline_utils.py

Lines 490 to 496 in 2c1ed50

	logger.warning(
	"Pipelines loaded with `dtype=torch.float16` cannot run with `cpu` device. It"
	" is not recommended to move them to `cpu` as running them will fail. Please make"
	" sure to use an accelerator to run the pipeline in inference, due to the lack of"
	" support for`float16` operations on this device in PyTorch. Please, remove the"
	" `torch_dtype=torch.float16` argument, or use another device for inference."
	)

[r3KoR]

Hi @hlky thank you for the swift reply. I am currently out of town hence my delay in response. The custom node for this would be:

https://github.com/smthemex/ComfyUI_Sonic.git

More accurately the SonicSampler. Would it help if I attach the workflow here as well?

[hlky]

Would it help if I attach the workflow here as well?

Yes that would help to confirm why the warning only appears for longer videos, however after reviewing the code the warning appears to be harmless, the pipe is moved to weight_dtype during __init__, this is when the warning will appear, then pipe is moved to device during process, so it's ok at that point.

https://github.com/smthemex/ComfyUI_Sonic/blob/6595df02e72406fe23baebc12ac2088832d7ff9b/sonic.py#L219

self.pipe = pipe.to(dtype=weight_dtype)

https://github.com/smthemex/ComfyUI_Sonic/blob/6595df02e72406fe23baebc12ac2088832d7ff9b/sonic.py#L249

self.pipe.to(self.device)