GUI freezes when PredictedPointsArray contains nan

[ElTinmar]

The GUI freezes when jumping to labeled frame containing an instance predicted by the model that contains nan positions.

import sleap.io as sio
labels = sio.load_file('sleap/labels.v010.slp')

In [33]: labels.labeled_frames[252].instances[0].points
Out[33]: 
PredictedPointsArray([([575.65203857, 356.7387085 ], 0.91531068,  True, False, 'Head'),
                      ([572.88000488, 364.34875488], 0.80707204,  True, False, 'Eye_Left_Front'),
                      ([583.85784912, 361.18151855], 0.67773259,  True, False, 'Eye_Left_Back'),
                      ([568.13079834, 356.41497803], 0.84104878,  True, False, 'Eye_Right_Front'),
                      ([571.98321533, 348.07727051], 0.78071177,  True, False, 'Eye_Right_Back'),
                      ([599.86999512, 336.30239868], 0.78452855,  True, False, 'Swim_Bladder'),
                      ([596.07550049, 340.26177979], 0.5335865 ,  True, False, 'Tail_0'),
                      ([         nan,          nan], 0.        , False, False, 'Tail_1'),
                      ([         nan,          nan], 0.        , False, False, 'Tail_2'),
                      ([         nan,          nan], 0.        , False, False, 'Tail_3'),
                      ([         nan,          nan], 0.        , False, False, 'Tail_4'),
                      ([612.53485107, 264.25234985], 0.24509695,  True, False, 'Tail_5'),
                      ([612.91308594, 259.89797974], 0.20331819,  True, False, 'Tail_6'),
                      ([619.28857422, 248.93322754], 0.20466103,  True, False, 'Tail_7'),
                      ([         nan,          nan], 0.        , False, False, 'Tail_8'),
                      ([         nan,          nan], 0.        , False, False, 'Tail_9')],
                     dtype=[('xy', '<f8', (2,)), ('score', '<f8'), ('visible', '?'), ('complete', '?'), ('name', 'O')])

Loading that slp file into sleap-label and jumping to that frame causes the GUI to freeze.
If I replace the nan values with e.g. 100, the frame loads fine

In [56]:  labels.labeled_frames[252].instances[0].points
Out[56]: 
PredictedPointsArray([([575.65203857, 356.7387085 ], 0.91531068,  True, False, 'Head'),
                      ([572.88000488, 364.34875488], 0.80707204,  True, False, 'Eye_Left_Front'),
                      ([583.85784912, 361.18151855], 0.67773259,  True, False, 'Eye_Left_Back'),
                      ([568.13079834, 356.41497803], 0.84104878,  True, False, 'Eye_Right_Front'),
                      ([571.98321533, 348.07727051], 0.78071177,  True, False, 'Eye_Right_Back'),
                      ([599.86999512, 336.30239868], 0.78452855,  True, False, 'Swim_Bladder'),
                      ([596.07550049, 340.26177979], 0.5335865 ,  True, False, 'Tail_0'),
                      ([100.        , 100.        ], 0.        , False, False, 'Tail_1'),
                      ([100.        , 100.        ], 0.        , False, False, 'Tail_2'),
                      ([100.        , 100.        ], 0.        , False, False, 'Tail_3'),
                      ([100.        , 100.        ], 0.        , False, False, 'Tail_4'),
                      ([612.53485107, 264.25234985], 0.24509695,  True, False, 'Tail_5'),
                      ([612.91308594, 259.89797974], 0.20331819,  True, False, 'Tail_6'),
                      ([619.28857422, 248.93322754], 0.20466103,  True, False, 'Tail_7'),
                      ([100.        , 100.        ], 0.        , False, False, 'Tail_8'),
                      ([100.        , 100.        ], 0.        , False, False, 'Tail_9')],
                     dtype=[('xy', '<f8', (2,)), ('score', '<f8'), ('visible', '?'), ('complete', '?'), ('name', 'O')])

Your personal set up

SLEAP: 1.5.1
Numpy: 2.2.6
Python: 3.11.0
OS: Linux-6.14.0-33-generic-x86_64-with-glibc2.41 (Ubuntu 25.04)

[eberrigan]

Thanks so much @ElTinmar!

So jumping means clicking on it in the frame seeker at the bottom of the window? Does it also happen with arrows?

Are there any error messages in the logs?

[ElTinmar]

My bad I mean jumping to a frame in the Labeling Suggestions dock on the right, if that frame contains a predicted instance with NaNs . Either double-clicking with the mouse or pressing space to go to the next offending frame causes the issue

Nothing shows up on the terminal when the GUI freezes, are there other logs I can have a look at?

EDIT: it also happens if I navigate to the frame by clicking on the frame seeker at the bottom, or move with the arrow

[gitttt-1234]

Hi @ElTinmar !

I'm trying to reproduce your error, it seems to be working fine on my end. (tested on a frame with nan values, and the GUI doesn't seem to crash moving across suggestion frames with nan keypoints). Could you let me know how you installed SLEAP?

Thanks,

Divya

[ElTinmar]

Hi Divya!
Thanks a lot !!!

I created a conda environment with

conda create -n sleap python==3.11

then pip installed sleap within the conda environment:

pip install sleap[nn]==1.5.1

I tested the same file on Windows 11 and I have no issues, so it seems to be linux specific.

[gitttt-1234]

Hi @ElTinmar !

Could you try creating a new env with python 3.12 and then install sleap there or try the uv installation method, I think this could be because of a dependency conflict.

# (linux cpu install)
uv tool install "sleap[nn]" --index https://download.pytorch.org/whl/cpu --index https://pypi.org/simple --python 3.12

Let us know if you have any questions!

Thanks,

Divya

[ElTinmar]

The problem is still here when creating a python 3.12 environment or using uv.

I've narrowed it down to sleap/gui/widgets/video.py:1744

if I comment out:

#self.setPolygon(polygon)

The freezing / massive slowdown disappears (as well as the edges in the skeleton in the GUI).
It seems that passing QPolygonF with NaN values is messing up the rendering on my machine for some reason.

I also realized that the issue only shows with predicted instances because user instances get sanitized here:

sleap/gui/widgets/video.py:1866

        if not self.predicted:
           # Initialize missing nodes with random points marked as non-visible.
            fill_missing(
                self.instance,
                max_x=self.player.video.shape[2],
                max_y=self.player.video.shape[1],
            )

I'll investigate a bit more and come back to you if I find something interesting.

[gitttt-1234]

Hi @ElTinmar !

Could you share your slp file here (with associated videos or you can export it as a pkg.slp from the gui and share it instead of uploading all videos) ? It would be helpful to debug better from our side, I couldn't replicate this with my sample .slp file with nans in the predicted instances.

Thanks,

Divya

[ElTinmar]

I just uploaded the data, there is only one offending frame

import sleap_io as sio
labels = sio.load_file('labels.v022.slp')
labels.labeled_frames[501]

Out[3]: LabeledFrame(video=Video(filename="02_07dpf_wt_Thu_09_Oct_2025_17h26min06sec_fish_3.mp4", shape=(408616, 900, 900, 1), backend=MediaVideo), frame_idx=105353, instances=[PredictedInstance(points=[[261.0999450683594, 144.4190216064453], [260.22796630859375, 152.65480041503906], [nan, nan], [252.69107055664062, 141.2283477783203], [260.4349365234375, 132.5644073486328], [293.029296875, 125.16816711425781], [300.69171142578125, 121.11438751220703], [312.2689514160156, 116.5400390625], [320.66845703125, 109.41189575195312], [332.2195129394531, 105.09684753417969], [340.9866638183594, 100.90618896484375], [352.72998046875, 96.61775207519531], [364.6463623046875, 88.9901123046875], [373.1644592285156, 81.10855865478516], [381.04071044921875, 73.1665267944336], [388.85931396484375, 64.67637634277344]], track=None, score=0.99, tracking_score=nan)])

I the meantime, I have patched QtEdge.setLine in sleap/gui/widgets/video.py locally
to not draw an edge if either the src or dst node have nan coordinates

    def setLine(self, line):
        self._line = line
        polygon = QPolygonF()

+        def has_nan(point: QPointF) -> bool:
+            return np.isnan(point.x()) or np.isnan(point.y())
+
+       # dont plot edge if either src or dst is nan
+       if has_nan(line.p1()) or has_nan(line.p2()):
+           return

        if self.player.state.get("edge style", default="").lower() == "wedge":
            r = self.src.visible_radius / 2.0

            norm_a = line.normalVector()
            norm_a.setLength(r)

            norm_b = line.normalVector()
            norm_b.setLength(-r)

            polygon.append(norm_a.p2())
            polygon.append(line.p2())
            polygon.append(norm_b.p2())
            polygon.append(norm_a.p2())

        else:
            polygon.append(line.p1())
            polygon.append(line.p2())
        self.setPolygon(polygon)

which seems to resolve the issue on my side, although I don't fully understand why the problem is there in the first place.

Thank you so much for your help, let me know if I can do anything else? I'd be happy to share a video capture of my desktop if that helps

[gitttt-1234]

Hi @ElTinmar!

I tested the file on my end, and it seems to be working fine. This could be because of qt version mismatch. Are you using a conda env? Could you share the output of pip freeze from your env? (you might want to do uv tool uninstall sleap if you're running sleap-label from a conda env. If sleap is available as a uv tool, then even if you're in a conda env, it'll use the uv tool version)

Thanks,

Divya

[ElTinmar]

Hi Divya !

I removed everything and recreated a vanilla conda env with python 3.12 and only sleap installed via pip

conda create -n sleap python==3.12
conda activate sleap
pip install sleap[nn]==1.5.1

I checked sleap-label points to the right location:

$ which sleap-label
/home/martin/miniconda3/envs/sleap/bin/sleap-label

version of Qt:

from qtpy import QtCore
QtCore.qVersion()
Out[9]: '6.10.0'

Here is the output of pip freeze:

aiohappyeyeballs==2.6.1
aiohttp==3.13.1
aiosignal==1.4.0
albucore==0.0.24
albumentations==2.0.8
annotated-types==0.7.0
antlr4-python3-runtime==4.9.3
anyio==4.11.0
argon2-cffi==25.1.0
argon2-cffi-bindings==25.1.0
arrow==1.4.0
asttokens==3.0.0
async-lru==2.0.5
attrs==25.4.0
av==15.1.0
babel==2.17.0
beautifulsoup4==4.14.2
bleach==6.2.0
cattrs==25.3.0
certifi==2025.10.5
cffi==2.0.0
charset-normalizer==3.4.4
click==8.3.0
comm==0.2.3
contourpy==1.3.3
cycler==0.12.1
debugpy==1.8.17
decorator==5.2.1
defusedxml==0.7.1
executing==2.2.1
fastjsonschema==2.21.2
filelock==3.20.0
fonttools==4.60.1
fqdn==1.5.1
frozenlist==1.8.0
fsspec==2025.9.0
gitdb==4.0.12
GitPython==3.1.45
h11==0.16.0
h5py==3.15.1
hdmf==4.1.1
httpcore==1.0.9
httpx==0.28.1
hydra-core==1.3.2
idna==3.11
imageio==2.37.0
imageio-ffmpeg==0.6.0
imgstore==0.3.7
ipykernel==7.0.1
ipython==9.6.0
ipython_pygments_lexers==1.1.1
ipywidgets==8.1.7
isoduration==20.11.0
jedi==0.19.2
Jinja2==3.1.6
joblib==1.5.2
jsmin==3.0.1
json5==0.12.1
jsonpickle==4.1.1
jsonpointer==3.0.0
jsonschema==4.25.1
jsonschema-specifications==2025.9.1
jupyter==1.1.1
jupyter-console==6.6.3
jupyter-events==0.12.0
jupyter-lsp==2.3.0
jupyter_client==8.6.3
jupyter_core==5.9.1
jupyter_server==2.17.0
jupyter_server_terminals==0.5.3
jupyterlab==4.4.10
jupyterlab_pygments==0.3.0
jupyterlab_server==2.28.0
jupyterlab_widgets==3.0.15
kiwisolver==1.4.9
kornia==0.8.1
kornia_rs==0.1.9
lark==1.3.0
lazy_loader==0.4
lightning==2.5.5
lightning-utilities==0.15.2
loguru==0.7.3
markdown-it-py==4.0.0
MarkupSafe==3.0.3
matplotlib==3.10.7
matplotlib-inline==0.1.7
mdurl==0.1.2
mistune==3.1.4
mpmath==1.3.0
multidict==6.7.0
nbclient==0.10.2
nbconvert==7.16.6
nbformat==5.10.4
ndx-pose==0.2.2
nest-asyncio==1.6.0
networkx==3.5
nixio==1.5.4
notebook==7.4.7
notebook_shim==0.2.4
numpy==2.2.6
nvidia-cublas-cu12==12.8.4.1
nvidia-cuda-cupti-cu12==12.8.90
nvidia-cuda-nvrtc-cu12==12.8.93
nvidia-cuda-runtime-cu12==12.8.90
nvidia-cudnn-cu12==9.10.2.21
nvidia-cufft-cu12==11.3.3.83
nvidia-cufile-cu12==1.13.1.3
nvidia-curand-cu12==10.3.9.90
nvidia-cusolver-cu12==11.7.3.90
nvidia-cusparse-cu12==12.5.8.93
nvidia-cusparselt-cu12==0.7.1
nvidia-nccl-cu12==2.27.5
nvidia-nvjitlink-cu12==12.8.93
nvidia-nvshmem-cu12==3.3.20
nvidia-nvtx-cu12==12.8.90
omegaconf==2.3.0
opencv-python==4.12.0.88
opencv-python-headless==4.12.0.88
packaging==25.0
pandas==2.3.3
pandocfilters==1.5.1
parso==0.8.5
pexpect==4.9.0
pillow==12.0.0
platformdirs==4.5.0
prometheus_client==0.23.1
prompt_toolkit==3.0.52
propcache==0.4.1
protobuf==6.33.0
psutil==7.1.1
ptyprocess==0.7.0
pure_eval==0.2.3
pycparser==2.23
pydantic==2.12.3
pydantic_core==2.41.4
Pygments==2.19.2
pykalman==0.10.2
pymatreader==1.1.0
pynwb==3.1.2
pyparsing==3.2.5
PySide6==6.10.0
PySide6_Addons==6.10.0
PySide6_Essentials==6.10.0
python-dateutil==2.9.0.post0
python-json-logger==4.0.0
python-rapidjson==1.22
pytorch-lightning==2.5.5
pytz==2025.2
PyYAML==6.0.3
pyzmq==27.1.0
QtPy==2.4.3
referencing==0.37.0
requests==2.32.5
rfc3339-validator==0.1.4
rfc3986-validator==0.1.1
rfc3987-syntax==1.1.0
rich==14.2.0
rich-click==1.9.3
rpds-py==0.28.0
ruamel.yaml==0.18.16
ruamel.yaml.clib==0.2.14
scikit-base==0.12.6
scikit-image==0.25.2
scikit-learn==1.7.2
scipy==1.16.2
seaborn==0.13.2
Send2Trash==1.8.3
sentry-sdk==2.42.1
setuptools==80.9.0
shiboken6==6.10.0
simplejson==3.20.2
simsimd==6.5.3
six==1.17.0
sleap==1.5.1
sleap-io==0.5.6
sleap-nn==0.0.2
smmap==5.0.2
sniffio==1.3.1
soupsieve==2.8
stack-data==0.6.3
stringzilla==4.2.1
sympy==1.14.0
terminado==0.18.1
threadpoolctl==3.6.0
tifffile==2025.10.16
tinycss2==1.4.0
torch==2.9.0
torchmetrics==1.8.2
torchvision==0.24.0
tornado==6.5.2
tqdm==4.67.1
traitlets==5.14.3
triton==3.5.0
typing-inspection==0.4.2
typing_extensions==4.15.0
tzdata==2025.2
tzlocal==5.3.1
uri-template==1.3.0
urllib3==2.5.0
wandb==0.22.2
wcwidth==0.2.14
webcolors==24.11.1
webencodings==0.5.1
websocket-client==1.9.0
wheel==0.45.1
widgetsnbextension==4.0.14
xmltodict==1.0.2
yarl==1.22.0

What the bug looks like on my ubuntu machine

I open a slp project with an offending frame at index 240 in the suggestion table.
I move to a position slightly above frame 240 and press space to jump to next suggested frame. User labeled frames are fine, when I jump to frame 240, the GUI slows to a crawl. After a few seconds, the instance shows up over the previous frame, then the background video is redrawn

Screencast.From.2025-10-23.09-05-40.mp4

Thanks a lot for your help !!

EDIT: trying to resize the GUI while on the problematic frame seems to make the issue even more noticeable (maybe through multiple problematic repaintings?)

[ElTinmar]

If that helps, the issue seems to be in QtEdge.paint

I added to the QtEdge class:

import time
def paint(self, *args, **kwargs):
    t0 = time.perf_counter()
    super().paint(*args, **kwargs)
    print(f"painted edge: {time.perf_counter() - t0}")

For a normal frame, I get:

painted edge: 2.0763996872119606e-05
painted edge: 1.1489013559184968e-05
painted edge: 1.1060998076573014e-05
painted edge: 1.0463001672178507e-05
painted edge: 8.282004273496568e-06
painted edge: 1.1597003322094679e-05
painted edge: 1.2617005268111825e-05
painted edge: 1.4142991858534515e-05
painted edge: 1.253499067388475e-05
painted edge: 1.409099786542356e-05
painted edge: 1.4123987057246268e-05
painted edge: 8.35300306789577e-06
painted edge: 1.7471000319346786e-05
painted edge: 7.929003913886845e-06
painted edge: 8.614995749667287e-06
painted edge: 1.3873999705538154e-05
painted edge: 1.0463001672178507e-05
painted edge: 1.2245000107213855e-05
painted edge: 1.3336000847630203e-05

While for the frame with NaNs:

painted edge: 3.3831995096988976e-05
painted edge: 2.5058005121536553e-05
painted edge: 2.400900120846927e-05
painted edge: 1.9639002857729793e-05
painted edge: 2.9332004487514496e-05
painted edge: 2.057799429167062e-05
painted edge: 2.1667001419700682e-05
painted edge: 2.51320016104728e-05
painted edge: 1.877998874988407e-05
painted edge: 2.830299490597099e-05
painted edge: 2.019699604716152e-05
painted edge: 1.7147001926787198e-05
painted edge: 2.647200017236173e-05
painted edge: 2.6505993446335196e-05
painted edge: 2.057700476143509e-05
painted edge: 2.2741005523130298e-05
painted edge: 1.5893005183897913e-05
painted edge: 1.7655998817645013e-05
painted edge: 4.333001862003584 <--- The offending edge takes forever to paint

[ElTinmar]

Hi !

It seems to be a problem with the interaction between NaNs and antialiasing in the QtEdge on linux.
Here is a minimal working example that reproduces the bug on my machine:

import sys, time
from PySide6.QtWidgets import (
    QApplication, QGraphicsView, QGraphicsScene,
    QGraphicsPolygonItem, QGraphicsObject, QVBoxLayout, QWidget
)
from PySide6.QtGui import QPolygonF, QPainter, QPen, QColor, QBrush
from PySide6.QtCore import QPointF
import math

TRIGGER_BUG = True

class QtEdge(QGraphicsPolygonItem):
    def __init__(self, polygon, parent=None):
        super().__init__(polygon, parent)
        self.setBrush(QBrush(QColor("green")))
        self.setPen(QPen(QColor("green"), 2))

    def paint(self, painter, option, widget=None):
        if TRIGGER_BUG:
            painter.setRenderHint(QPainter.Antialiasing, True)
        else:
            painter.setRenderHint(QPainter.Antialiasing, False)
        t0 = time.perf_counter()
        super().paint(painter, option, widget)
        t1 = time.perf_counter()
        print(f"QtEdge.paint {t1-t0:.6f}s, {self.shape()}")

class QtInstance(QGraphicsObject):
    def __init__(self):
        super().__init__()
        
        points = [
            QPointF(float('nan'), float('nan')),
            QPointF(612.0, 264.0)
        ]
        poly = QPolygonF(points)
        self.edge = QtEdge(poly, parent=self)

    def boundingRect(self):
        return self.edge.boundingRect()

    def paint(self, painter, option, widget=None):
        pass

class VideoPlayer(QWidget):
    def __init__(self):
        super().__init__()
        self.view = QGraphicsView()
        self.view.setRenderHint(QPainter.Antialiasing, True)
        self.scene = QGraphicsScene(self)
        self.view.setScene(self.scene)
        self.scene.setSceneRect(0, 0, 620, 300)

        instance = QtInstance()
        self.scene.addItem(instance)

        layout = QVBoxLayout()
        layout.addWidget(self.view)
        self.setLayout(layout)

if __name__ == "__main__":
    app = QApplication(sys.argv)
    w = VideoPlayer()
    w.show()
    sys.exit(app.exec())

The bug is present in my sleap conda environment, or in a vanilla environment containing
only pyside6==6.10.

Overriding paint in the QtEdge class fixes the bug in SLEAP for me:

    def paint(self, painter, option, widget):
        painter.setRenderHint(QPainter.Antialiasing, False)
        super().paint(painter, option, widget)

[wjjjjyourFA]

me too. ... and I don't know how to solve it. 0.0.... I can understand the reason, but can't find this .py file which I need to edit.

[ElTinmar]

Heya,

You should be able to locate the folder where the python code was installed using the following command (if you installed in a conda or virtual environment, the environment needs to be activated):

python -c "import sleap, os; print(os.path.dirname(sleap.__file__))"

In my case, I installed it via conda/pip, it's in

/home/username/miniconda3/envs/sleap/lib/python3.11/site-packages/sleap

From that folder, navigate to gui/widgets/video.py
It might be a good idea to make a backup of the video.py file in case
anything goes wrong with the edit.
You need to add that function

def paint(self, painter, option, widget):
    painter.setRenderHint(QPainter.Antialiasing, False)
    super().paint(painter, option, widget)

inside the QtEdge class