Slicer crashes when sending red slice via OpenIGTLink

[mcfly3001]

Slicer crashes when trying to send an image of the red slice via OpenIGTLink. The crash only occurs when the red slice has reached a certain resolution. Small resolutions work fine.

Here the steps and code to reproduce the crash:
Start Slicer and open the python console window. Enter the following code:

def update_image(wti):
     wti.Update()
     wti.Modified()


# Create IGTLink server node
node = slicer.mrmlScene.GetFirstNodeByName("SliceStreaming")
connector_node = slicer.vtkMRMLIGTLConnectorNode()
connector_node.SetTypeServer(18944)
slicer.mrmlScene.AddNode(connector_node)
connector_node.Start()

# create window to image filter
layout_manager = slicer.app.layoutManager()
slice_widget = layout_manager.sliceWidget("Red")
slice_view = slice_widget.sliceView()
render_window = slice_view.renderWindow()
slice_logic = slice_widget.sliceLogic()
render_window = slice_view.renderWindow()
wti = vtk.vtkWindowToImageFilter()
wti.SetInput(render_window)
wti.Update()

# create volume node
volume_node = slicer.mrmlScene.AddNewNodeByClass("vtkMRMLScalarVolumeNode")
connector_node.RegisterOutgoingMRMLNode(volume_node)
volume_node.SetAndObserveImageData(wti.GetOutput())

# update volume when red slice changed
slice_logic.AddObserver(vtk.vtkCommand.ModifiedEvent, lambda logic, evt: update_image(wti))

Next, start the ImagerClient3.exe example script:

ImagerClient3.exe localhost 18944 30 2

Now, when dragging the slider of the red slice, the image is successsfully send to the client. To trigger the crash, change the layout such that the red slice is taking more space (e.g. red slice only). Slicer crashes.

[mcfly3001]

Hi, I am using Windows 10 as OS and tested this issue with Slicer 4.10 and 4.11. For both versions the behavior is the same. Thanks for taking care! Von: leochan2009 <notifications@github.com> Gesendet: Montag, 4. Mai 2020 15:21 An: openigtlink/SlicerOpenIGTLink <SlicerOpenIGTLink@noreply.github.com> Cc: mcfly3001 <andreas.kleinoeder@kleinoeder-development.de>; Author <author@noreply.github.com> Betreff: Re: [openigtlink/SlicerOpenIGTLink] Slicer crashes when sending red slice via OpenIGTLink (#103) Thanks for reporting! Could you tell me which slicer version are you using? And which OS platform you are using? — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#103 (comment)> , or unsubscribe <https://github.com/notifications/unsubscribe-auth/AI3W3RCQBJKI6P2NVB2YDPTRP26MVANCNFSM4MYQXG2Q> . <https://github.com/notifications/beacon/AI3W3RESC4ONOVLAS6DF443RP26MVA5CNFSM4MYQXG22YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEUUT4OI.gif>

[leochan2009]

Hi,

I have been trying to reproduce your issue using my Mac book, though was able to reproduce the crash... I am using slicer 4.10.2 and Openigtlink from the master branch.

[leochan2009]

The red slice size was increased from 600x400 to around 1300x850...
It seems like the image receiving end has some issue. It looks like the unpacking has some issue, the message from the console output seems wrong.

[leochan2009]

Hi,

In the ImageClient3.cpp file, you will need to change the line:
https://github.com/openigtlink/OpenIGTLink/blob/53d81eb9341ca24b958f30657243c89f8e864f60/Examples/Imager/ImagerClient3.cxx#L104
to
if (strcmp(headerMsg->GetDeviceName(), "Volume") == 0)
as the device name from the Slicer will be fixed to "Volume".
In this way, you will see the received images saved to the folder of the ImageClient.exe.

The crash didn't happen to my machine. but i see that many saved image files are of zero size . Also Another possible reason for the crash is the changing of image size dynamically. The design of the volume streaming is for fixed image/volume size during the transmission. The changing image/volume size might be error prone. If you have volumes/images with different sizes, it would be better create two volumes and send them independently.

[mcfly3001]

Hi,

thanks for testing. I also did some more testing. If I initially enlarge the red slice (i.e. Slicer in full screen and then only showing red slice) before starting the ImagerClient.exe everything works fine.
In this case I can dynamically resize the Slicer window and switch the layouts without a crash. On the other hand if the red slice has initially a smaller resolution, Slicer will crash if the red slice is increased to a higher resolution while the ImagerClient is connected.
What I forgot to mention: The ImagerClient also exits with the error message "Message size information and actual data size don't match".

As you said I also think that the change of the data size while connected leads to the crash. Maybe the OS also makes a difference.

Would the video streaming functionality also be based on a fixed size? I am trying to stream the content of a slice to another device. I havent found out though how to enable the video streaming functionality in Slicer yet.

[leochan2009]

Hi,
When accessing a memory that wasn't allocated, the program crashes. OS definitely makes differences.

video streaming functionality is also based on fixed sizes. VP9 codec is currently used in Slicer. Youtube actually use the same codec. To use the video streaming, you will need to use vtkMRMLStreamingVolumeNode, here are some document for your reference:
https://apidocs.slicer.org/master/classvtkMRMLStreamingVolumeNode.html
https://apidocs.slicer.org/master/classvtkStreamingVolumeCodecFactory.html
I don't have any sample python code with me, however, you will need to create a codec first using the vtkStreamingVolumeCodecFactory, assign the codec to the vtkMRMLStreamingVolumeNode and encode an image data.
you can look into the OpenIGTLinkIF module and its code for more information.

[leochan2009]

I was looking into the code and actually find that when you streaming image using vtkMRMLScalarVolumeNode, the openigtlink message actually get created whenever the volume is updated. so there should be fine for sending dynamic volume. vtkMRMLStreamingVolumeNode though requires fixed volume size. I am not sure how the crash happen when using vtkMRMLScalarVolumeNode. By the way, when running the imageclient3.exe, do you see the saved image files? some files are of byte zero, did this happen to you?

[mcfly3001]

Thanks for the detailed explanation concerning the video streaming. I will give this a try asap.
When using the imageclient3.exe I also see the saved image files and some of them have 0 bytes. My guess is that due the resize QT destroys/reinitializes the the red slice widget sometimes having a size of 0x0 pixels. Probably this leads to the 0 byte images?