how can i do reference if i have one picture and it's corresponding pose?

[fankeyu123]

No description provided.

[donydchen]

Hi, @fankeyu123, thanks for your interest in our work.

Our work mainly focuses on multi-view input, and two critical components of the encoder, i.e., multi-view transformer and cost volume, are both designed to extract cross-view information effectively. Therefore, it is not easy to use one single image as input directly.

Since you do have the corresponding pose, one potential way is to synthesize an additional view by warping the image to a nearby viewpoint (you may refer to the related code snippet here) and then using both the original image and the warped one as input. However, I have not tried any single-view input before, and I am unsure whether it will work.

Alternatively, you might also want to try on those single-view-based NVS models, such as Free3D.

[fankeyu123]

Thanks for answering

…

---Original--- From: "Donald ***@***.***> Date: Fri, Jun 21, 2024 16:57 PM To: ***@***.***>; Cc: "Keyu ***@***.******@***.***>; Subject: Re: [donydchen/mvsplat] how can i do reference if i have one pictureand it's corresponding pose? (Issue #37) Hi, @fankeyu123, thanks for your interest in our work. Our work mainly focuses on multi-view input, and two critical components of the encoder, i.e., multi-view transformer and cost volume, are both designed to extract cross-view information effectively. Therefore, it is not easy to use one single image as input directly. Since you do have the corresponding pose, one potential way is to synthesize an additional view by warping the image to a nearby viewpoint (you may refer to the related code snippet here) and then using both the original image and the warped one as input. However, I have not tried any single-view input before, and I am unsure whether it will work. Alternatively, you might also want to try on those single-view-based NVS models, such as Free3D. — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you were mentioned.Message ID: ***@***.***>