Skip to content

fork() being used for multiprocessing crashes solving process with cplex #1906

@thomgeo

Description

@thomgeo

Version Checks (indicate both or one)

  • I have confirmed this bug exists on the lastest release of PyPSA-Eur.

  • I have confirmed this bug exists on the current master branch of PyPSA-Eur.

Issue Description

Dear all,

I have run into issues with PyPSA, getting the following error messages when solving a large pypsa network with cplex:

This process (pid=2637151) is multi-threaded, use of fork() may lead to deadlocks in the child.

Afterwards, the script crashes, which I assume is due to fork being used instead of spawn.

When interrupting the process early, and continuing to work on the networks, I further get this error message:

ERROR! Session/line number was not unique in database. History logging moved to new session 2301

which appears to be a result of working on a process after forks have been created.

Do you have any idea, what could cause this, and how this could be addressed? With Highs, I did not encounter these issues.

Many thanks in advance!

Reproducible Example

import pypsa

n = pypsa.Network("resources/networks/base_s_adm_2190seg__2040.nc")

n.optimize(solver_name="cplex")

Expected Behavior

network is solved and delivers results

Installed Versions

Details - tqdm=4.67.1=pyhd8ed1ab_1 - traitlets=5.14.3=pyhd8ed1ab_1 - typing-extensions=4.15.0=h396c80c_0 - typing_extensions=4.15.0=pyhcf101f3_0 - typing_utils=0.1.0=pyhd8ed1ab_1 - tzdata=2025b=h78e105d_0 - ukkonen=1.0.1=py312hd9148b4_6 - unicodedata2=17.0.0=py312h4c3975b_0 - unidecode=1.4.0=pyhcf101f3_1 - uri-template=1.3.0=pyhd8ed1ab_1 - uriparser=0.9.8=hac33072_0 - urllib3=2.5.0=pyhd8ed1ab_0 - validators=0.35.0=pyhd8ed1ab_0 - virtualenv=20.35.4=pyhd8ed1ab_0 - wayland=1.24.0=hd6090a7_1 - wcwidth=0.2.14=pyhd8ed1ab_0 - webcolors=25.10.0=pyhd8ed1ab_0 - webencodings=0.5.1=pyhd8ed1ab_3 - websocket-client=1.9.0=pyhd8ed1ab_0 - wheel=0.45.1=pyhd8ed1ab_1 - widgetsnbextension=3.6.10=pyhd8ed1ab_0 - wrapt=1.17.3=py312h4c3975b_1 - xarray=2025.6.1=pyhd8ed1ab_1 - xcb-util=0.4.1=h4f16b4b_2 - xcb-util-cursor=0.1.5=hb9d3cd8_0 - xcb-util-image=0.4.0=hb711507_2 - xcb-util-keysyms=0.4.1=hb711507_0 - xcb-util-renderutil=0.3.10=hb711507_0 - xcb-util-wm=0.4.2=hb711507_0 - xerces-c=3.2.5=h988505b_2 - xkeyboard-config=2.46=hb03c661_0 - xlrd=2.0.2=pyhd8ed1ab_0 - xorg-libice=1.1.2=hb9d3cd8_0 - xorg-libsm=1.2.6=he73a12e_0 - xorg-libx11=1.8.12=h4f16b4b_0 - xorg-libxau=1.0.12=hb9d3cd8_0 - xorg-libxcomposite=0.4.6=hb9d3cd8_2 - xorg-libxcursor=1.2.3=hb9d3cd8_0 - xorg-libxdamage=1.1.6=hb9d3cd8_0 - xorg-libxdmcp=1.1.5=hb9d3cd8_0 - xorg-libxext=1.3.6=hb9d3cd8_0 - xorg-libxfixes=6.0.2=hb03c661_0 - xorg-libxi=1.8.2=hb9d3cd8_0 - xorg-libxinerama=1.1.5=h5888daf_1 - xorg-libxrandr=1.5.4=hb9d3cd8_0 - xorg-libxrender=0.9.12=hb9d3cd8_0 - xorg-libxtst=1.2.5=hb9d3cd8_3 - xorg-libxxf86vm=1.1.6=hb9d3cd8_0 - xorg-xorgproto=2024.1=hb9d3cd8_1 - xyzservices=2025.10.0=pyhd8ed1ab_0 - xz=5.8.1=hbcc6ac9_2 - xz-gpl-tools=5.8.1=hbcc6ac9_2 - xz-tools=5.8.1=hb9d3cd8_2 - yaml=0.2.5=h280c20c_3 - yarl=1.22.0=py312h8a5da7c_0 - yte=1.8.1=pyha770c72_0 - zeromq=4.3.5=h387f397_9 - zict=3.0.0=pyhd8ed1ab_1 - zipp=3.23.0=pyhd8ed1ab_0 - zlib=1.3.1=hb9d3cd8_2 - zlib-ng=2.2.5=hde8ca8f_0 - zstandard=0.25.0=py312h5253ce2_0 - zstd=1.5.7=hb8e6e7a_2

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions