Take two inputs for one output

[lucydjo]

Hello,
I would like to have your opinion on the following feasibility study:

Is it possible to modify pix2pix to include :

Input 1: Image to be modified
input 2 : Vector image "prompt
Output : The image modified with the instructions of the pormpt image

The idea is to be able to modify the shape of the lips while retaining the original appearance. ( skin color, beard ).
I've created a dataset with 5000 images of lips/mouths and their position/lip shape, teeth...

Like this :

What do you think?
Can't wait to hear what you think!
Thx

[Vinitrajputt]

have you figured out, how to do that?