cannot move remote server's Tensor to gpu.

[TriLoo]

My hostfile is writen as below:

localhost slots=8
192.168.10.101 slots=8

Then I launch a deepspeed script to run, but the program cannot move tensors to GPU, for example code:

batch = torch.rand((5, 30))
batch = batch.to(model.device)

where model is a DeepSpeedEngine. If I use only localhost in the hostfile, it can work, but if I include the remote server (192.168.10.101) into hostfile, looks like the tensor cannot be moved to GPU !

any advices ? thanks