Cannot reconnect to wayvnc

[drigoskalwalker]

I have wayvnc server in my Linux machine.

The first time I connect it works fine.
The second the view close.

I'm using UltraVNC Viewer and TigerVNC Viewer on my Windows11
The RealVNC Viewer works, and reconnects normally

But I think it's a problem with wayvnc on reconnections.

[any1]

It helps if you follow the bug report template. This also needs a more thorough description of how to reproduce the problem.

[drigoskalwalker]

I'm sorry.
As the bug is simple to replicate, I thought you wanna try yourself.

Here is the trace log.
wayvnc-crash.log

WayVNC doens't crash, the viewer just cannot connect to it anymore.
Tried UltraVNC Viewer and TigerVNC Viewer

UltraVNC closes after tried to reconnect showing a black screen and then closes.
TigerVNC shows a error after tried to reconnect telling "Invalid pixel format"
wayvnc-crash_tigervnc.log

Tried the master version of WayVNC and other versions of the viewers with the same behavior.

RealVNC Viewer connect and reconnect normally even after the connection of these two clients cited above.
The UltraVNC Viewer and TigerVNC Viewer still cannot reconnect after RealVNC

In resume cannot connect anymore after the first connection.

I'm using Windows 11 in the client.

[any1]

This is probably a duplicate of #305.

If this was globally and trivially reproducible, it would be fixed already. This is why it helps to have as much information as possible; things such as what compositor you're running and the exact output from wayvnc -V.

[drigoskalwalker]

root@darkstar:~# wayvnc -V
wayvnc: 0.8.0
neatvnc: 0.8.1
aml: 0.3.0

I'm using LabWC 0.8.0

[any1]

Does it make a difference if you change the "preferred encoding" in the "compression" options menu in TigerVNC?

I see you're using RSA-AES. Does disabling it and using VeNCrypt (TLS) instead change anything? You can change that in the security menu if you're configured both on the server side.

[any1]

It occurs to me that when this happens, wayvnc may be stalled on a blocking system call. If you attach to it using gdb when this happens and get a backtrace, that should tell us where it's stuck. Do you know how to do that?

[Neustradamus]

@drigoskalwalker: Have you seen the latest @any1 comment?

[drigoskalwalker]

Hi sorry for the absence..
I tried to use GDB but I didn't get anything
Do I have to compile with debug enabled?

[any1]

There is a chance that we don't need debug symbols to figure this out, but they will definitely help. To enable debug symbols, you can pass the --buildtype=debug argument to meson.

After you've reproduced the issue, please issue the following command:

sudo gdb -p $(pidof wayvnc) --batch -ex "thread apply all where"

[drigoskalwalker]

Hi sorry for the wait again.
Here is the log file.
logwayvnc.txt

Thank you

[any1]

That backtrace looks normal. I did notice a race condition with resizing that trips up tigervnc a couple a weeks ago. Could that be what you're running into?

What happens if you remove this line? https://github.com/any1/wayvnc/blob/master/src/main.c#L937

[drigoskalwalker]

Same behavior
Both viewers are stuck in this block
https://github.com/any1/wayvnc/blob/f5c5fc856355b6e2f62047c7ec375fd8453f08d9/src/main.c#L2102C1-L2108C3

But I think when I try to reconnect the buffer or something that wayvnc is sending is not ordered or not right in the form that ultraVNC could read, so it rejects the connection.
TigerVNC shows that the stream is in the wrong format when I try to reconnect.
Somehow RealVNC deal with it.

I think neatvnc need to clean the buffer or reset the display on each reconnection.

[any1]

@drigoskalwalker, Can you try selecting a different "preferred encoding" in TigerVNC?

• In the connection menu, select "Options..."
• Select the "Compression" tab
• Unselect "Auto select"
• Select "Tight"

You can also play with other parameters and select other encodings if you want.

If that doesn't do anything, maybe try disabling the clipboard:

• Go to "Input" tab
• Unselect "Accept clipboard from server"
• Unselect "Send clipboard to server"

It might also help to know the following:

• Are you running a clipboard manager?
• Which wayland compositor are you running? What version?
• What sort of a GPU do you have?
• What is the exact command line that you use to run wayvnc?
• What does your config file look like (with password redacted)?

[drigoskalwalker]

Hi thanks for your response.

I tried Tight and several others encoders, same behavior.

I tried to disable clipboard but seems ultravnc doesn't have this option.

Are you running a clipboard manager? - Don't know what you mean.
Which wayland compositor are you running? What version? - LabWC 0.8.0
What sort of a GPU do you have? - NVidia RTX 3060
What is the exact command line that you use to run wayvnc? - wayvnc
What does your config file look like (with password redacted)?

use_relative_paths=true address=0.0.0.0 port=7900 enable_auth=true username=vnctest password=vnctest

You can have a try.
Just install UltraVNC Viewer or other VNC Viewer on Windows
To make a connect you have to use Real VNC Viewer as I said before.

the address:
abreucorp.ddns.net:65535

[any1]

Yeah, I can even reproduce the problem by connecting with tigervnc from Linux to your system.

The problem is that NVidia wants to use a 24 bit pixel format. but the clients aren't having it. They want each pixel to be 1, 2, or 4 bytes. We could convert to 32, but technically 24 is not a protocol violation, so I'd say that the clients are wrong.

[any1]

By the way, thanks for answering my questions and setting up the server. I really appreciate it!

[drigoskalwalker]

I'm glad for your support.
There's anything I could do to fix that, or help you with it?

[any1]

I think I can take it from here. I'll report the issue with TigerVNC and UltraVNC. However, supporting 24 bit buffers might be more work than they're willing to undertake.

[any1]

@maztheman, did you never run into this issue?

[drigoskalwalker]

I think I can take it from here. I'll report the issue with TigerVNC and UltraVNC. However, supporting 24 bit buffers might be more work than they're willing to undertake.

I really appreciate your effort.
What I can't understand is that, the first connection, all clients works.
After the first one, onlye RealVNC can connect.
But I'm using RealVNC to restart the wayvnc server, as RealVNC is really slow
UltraVNC Viewer is the way to go.

Thank you.

[maztheman]

@maztheman, did you never run into this issue?

I don't think so. However, it's been awhile..

[any1]

@drigoskalwalker, I pushed a fix to neatvnc just now. It should take care of the problem with both the tiger and the ultra.

[Neustradamus]

@any1: Thanks for your neatvnc fix here:

• any1/neatvnc@7220c2f

[drigoskalwalker]

@any1: Thanks for the path.
Now the reconnection works, the viewer don't close anymore.
But...I getting a grey screen, RealVNC Viewer is working normally.

[any1]

If you connect with RealVNC and then TigerVNC, do you also get a grey screen then?

[drigoskalwalker]

Hi, sorry the wait.
Everything is working now.

Tested Tiger VNC Viewer 1.14.0 and 1.14.1 - OK.
Tested Real VNC Viewer - OK
Tested UltraVNC Viewer 1.5-dev - Not working
Tested UltraVNC Viewer 1.650 - OK

• Only the test build VncViewer closes when I try to connect again ultravnc/UltraVNC#218 (comment)

Thank you again!
You did a great job!