AU/Vista HDMI input through mipi port

[Duckodope]

Hi there, i wonder if it will be possible in the future to add a attach a female hdmi port to the vistas mipi input to transmit whatever signal you like. That would be pristine for gimbal operating to get framing perfect. Yes latency would go up but thats not a big problem if it stays below 200ms or smth like that.

Greets

[CorruptName]

This is already theoretically possible. Runcam makes "unlicensed" cameras using mipi. While I agree that this would be very cool, it is outside the scope of margerine (IMO).
All that is needed is the MIPI pinout and a HDMI->MIPI Adapter (usually a small amount of lag in these adapters). This process would also allow someone to use an Action Camera such as GoPro or any camera with HDMI out to to broadcast over DJI.
Perhaps Margerine can allow us to obtain the MIPI Pinout? Don't know where to look personally..
I asked runcam for the pinout long time ago, they declined calling it their "intellectual property" to which I laughed. If anything, it's DJI's IP.

Cheers, I too look forward to the day when we can broadcast HDMI over DJI. This would especially be nice for Simulators.

PS: I'm In no way saying that we couldn't use custom software to inject display data over USB into an air unit for broadcast, but your question was about MIPI - which I agree is the preferred method.

[CorruptName]

Hi there, i wonder if it will be possible in the future to add a attach a female hdmi port to the vistas mipi input to transmit whatever signal you like. That would be pristine for gimbal operating to get framing perfect. Yes latency would go up but thats not a big problem if it stays below 200ms or smth like that.

Greets

I looked a bit more and it seems that the pinout has already been discovered..

.

Not sure but you may need a 4 channel HDMI to MIPI board (most seem to be only 2 channels). If so, they ARE available. Seems the C780B board has 4 channels. GL

[Duckodope]

I think the hardest part now will be sourcing the socket for those dji mipi connectors

[CorruptName]

I agree

…

On Wed, May 4, 2022, 8:42 AM Duckodope ***@***.***> wrote: I think the hardest part now will be sourcing the socket for those dji mipi connectors — Reply to this email directly, view it on GitHub <#17 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AGFWWLZAVJM6UETOLQU7OT3VIJWCHANCNFSM5UF47RJQ> . You are receiving this because you commented.Message ID: ***@***.***>

[j005u]

It should be possible if you manage to source the connectors (I know someone who tried and wait and waited and never got them) and match the MIPI stream config exactly as the RTOS on the Vista/AU is expecting a very specifically configured video stream. I don't know the details but they can probably be reverse engineered from the RTOS binaries/logs.

[flyingbandit]

I think the hardest part now will be sourcing the socket for those dji mipi connectors

Perhaps just use an existing cable? Solder it to the hdmi adapter

[CorruptName]

That'll surely work for prototypes, but I think he meant for production. Although I think the $10 the cable cost is going to be trivial for most people who want this feature.

…

On Sat, May 7, 2022, 12:45 AM a1axx ***@***.***> wrote: I think the hardest part now will be sourcing the socket for those dji mipi connectors Perhaps just use an existing cable? — Reply to this email directly, view it on GitHub <#17 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AGFWWL6NWOVBJ7C4OWKTH3DVIXYODANCNFSM5UF47RJQ> . You are receiving this because you commented.Message ID: ***@***.***>

[Duckodope]

I think the hardest part now will be sourcing the socket for those dji mipi connectors

Perhaps just use an existing cable? Solder it to the hdmi adapter

In my opinion someone would need a socket soldeted to a custom hdmi to mipi converter. I dont think cutting an existing cable and then splicing and soldering the tiny coax cabels to a pcb is a viable or even possible option. Does anyone know what this socket is called? Let alone even knowing the manufacturer would be a huge step forward. I'm sadly not able to figure out what kind of chips it would need to convert the hdmi signal quickly enough into mipi. And how the signal even has to look in the first place.

If anyone needs help in CAD design for a project i would be happy to help in exchange for help with this problem.
:D

[CorruptName]

My understanding is that MIPI is a standard. Meaning that supplying the data on the correct pins should do it..? Not 100%. I fully disagree about the connection tho.. one can easily cut one end off of a DJI coaxial cable and solder the wires to the end of the ribbon cable that comes with these HDMI > MIPI Boards. Not the most elegant solution but would be fine for testing.

…

On Sat, May 7, 2022, 7:32 AM Duckodope ***@***.***> wrote: I think the hardest part now will be sourcing the socket for those dji mipi connectors Perhaps just use an existing cable? Solder it to the hdmi adapter In my opinion someone would need a socket soldeted to a custom hdmi to mipi converter. I dont think cutting an existing cable and then splicing and soldering the tiny coax cabels to a pcb is a viable or even possible option. Does anyone know what this socket is called? Let alone even knowing the manufacturer would be a huge step forward. I'm sadly not able to figure out what kind of chips it would need to convert the hdmi signal quickly enough into mipi. And how the signal even has to look in the first place. If anyone needs help in CAD design for a project i would be happy to help in exchange for help with this problem. :D — Reply to this email directly, view it on GitHub <#17 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AGFWWL4WAT744TP54ANT2SDVIZIDBANCNFSM5UF47RJQ> . You are receiving this because you commented.Message ID: ***@***.***>

[CorruptName]

Perhaps the easiest solution would be to pipe the display data into a rooted Air Unit over USB C and make the air unit transmit. I'm not sure if the Air Unit's type C is 3.0 or not.. would matter for bandwidth.. I'm 99% sure this is the easier solution and doesn't require modifications to hardware. Also I would bet this would be the lower latency solution.

…

On Sat, May 7, 2022, 7:37 AM Ryan ***@***.***> wrote: My understanding is that MIPI is a standard. Meaning that supplying the data on the correct pins should do it..? Not 100%. I fully disagree about the connection tho.. one can easily cut one end off of a DJI coaxial cable and solder the wires to the end of the ribbon cable that comes with these HDMI > MIPI Boards. Not the most elegant solution but would be fine for testing. On Sat, May 7, 2022, 7:32 AM Duckodope ***@***.***> wrote: > I think the hardest part now will be sourcing the socket for those dji > mipi connectors > > Perhaps just use an existing cable? Solder it to the hdmi adapter > > In my opinion someone would need a socket soldeted to a custom hdmi to > mipi converter. I dont think cutting an existing cable and then splicing > and soldering the tiny coax cabels to a pcb is a viable or even possible > option. Does anyone know what this socket is called? Let alone even knowing > the manufacturer would be a huge step forward. I'm sadly not able to figure > out what kind of chips it would need to convert the hdmi signal quickly > enough into mipi. And how the signal even has to look in the first place. > > If anyone needs help in CAD design for a project i would be happy to help > in exchange for help with this problem. > :D > > — > Reply to this email directly, view it on GitHub > <#17 (comment)>, > or unsubscribe > <https://github.com/notifications/unsubscribe-auth/AGFWWL4WAT744TP54ANT2SDVIZIDBANCNFSM5UF47RJQ> > . > You are receiving this because you commented.Message ID: > ***@***.***> >

[Duckodope]

Is this method you are talking about already working? Do you think you could use a hdmi to usbc converter to make this possible?

[CorruptName]

I am unaware of any current projects. Just one of the many, many things that are possible with root access. I would say (without looking) that a type C video adapter would not work. Certain thing would be required which I doubt they put into the air unit. The type C would have to support "HDMI Alt Mode" and I'm pretty sure that requires a GPU. I highly doubt this is supported by the hardware. Also it would still require supporting software. This doesn't mean that it's not possible, just can't use a Type C HDMI adapter. We could test if the USB Type C is 3.0 or greater by testing transfer rate from a fast micro SD to PC using Mass Storage Mode on PC since USB 2.0 only supports 480MBPS while USB3.0 Supports 5GBPS and 3.1 supports 10GBPS. This won't tell us if it is 3.0 or 3.1 but will tell us if it is at least 3.0. (unlikely to find a microSD faster than 5GBPS)

…

On Sat, May 7, 2022, 7:48 AM Duckodope ***@***.***> wrote: Is this method you are talking about already working? Do you think you could use a hdmi to usbc converter to make this possible? — Reply to this email directly, view it on GitHub <#17 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AGFWWL3B33FQWKKBCN6JOSLVIZKAXANCNFSM5UF47RJQ> . You are receiving this because you commented.Message ID: ***@***.***>

[wyldrlorenz]

Is there any news about this topic? I am looking for a HDMI Input feature on the air-unit side for a while now. The "splitting cable method" was also my last option.
Here are the mpi-hdmi adapters I would have tried:

https://www.digikey.de/short/vffprdv7
https://www.distec.de/produkte/tft-komponenten/tft-controller/detail/distec/prismamipi-hdmi/

[j005u]

No news that I'm aware of, don't think anyone is actively working on it. A couple of notes:

• MIPI is not a standard that auto negotiates feed configuration, you are expected to do that on your own. It's more or less just a dumb pipe. For MIPI in to work you have to match the exact video format as expected by RTOS from 60fps cams. These have not been documented.
• The pinout you posted has not been confirmed to be correct
• Air side only has USB 2.0 and while theoretically OTG capable, we haven't figured out how to connect any anything to it yet. For one, it's not wired up to power any connected devices.
• The only way to do it via USB would be to figure out how to enable OTG and then use a USB 2.0 capture card of some kind
• The above would also entail figuring out how to compile new kernel modules for the appropriate drivers and would need a bunch of custom software to get the data over to ground side. Displaying in real time on the Goggles is also not going to be simple, but it might be possible to use the simulator pipeline for h264 decoding

Both approaches are very hard at best. The MIPI one stands a better chance in my mind for sure, as it doesn't require a ton of very hard custom software. You'd still need to figure out the exact video stream format by reverse engineering the RTOS (or getting Caddx/Runcam to spill the beans) and finding a converter that is able to output that exact format.

I would say the more realistic option is to look at proffesional solutions that can do this, such as the DJI Transmission.

[vr2045]

I started down this path as well a little while back :) I quickly came to the conclusion that messing with the hot metal box transmitter thingie is a non-starter as it requires hacking deep into the guts of the thing. I settled on a man-in-the-middle on the MIPI stream itself as that would allow a more modular and clean solution.

The connector is this one. I confirmed this by building a small board here. The micro-coax cables fit properly.

The pinout referred to above is almost correct: its missing the I2C as well as clock signals that are mis-labelled as ground.
Unfortunately, I am off by one on the power on my board so I had to hack my hack to get the passthrough to work but I seem to have killed my camera in the process :(
Am planning to build another board that will have this issue fixed and allow me to probe the MIPI signals to figure out the data rate and other settings of the MIPI stream.

I looked at the I2C traffic to the camera briefly and its interesting: after initial camera configuration, the metal box sends what is probably auto exposure/gain to the camera using the 1MHz I2C protocol. Simply adding a MIPI source is unlikely to keep the metal box happy: a spoof of the camera is probably required at a minimum so it gets the right responses back from the camera: not too tough to do but work required here. With my little adapter, I should be able to capture the I2C data quite easily for analysis.

Havent put in all this data into the repo yet.

The above work is open source. Happy to collaborate if someone is interested.

On a commercial front, I'm currently building a Lattice FPGA (Crosslink NX 33U) based 1"x1" module (to be released in 3Q'23) that will handle MIPI <-> USB3 conversion. Should be able to hijack the MIPI stream and sniff/inject MIPI.