RunEngineResult is not JSON serializable when RunEngine flag call_returns_result=True.

[burkeds]

I am running the queue server in a docker container and following along with the tutorial for the queue server. My startup code is simply the startup code in the ophyd-async tutorial (https://blueskyproject.io/ophyd-async/main/user/tutorials/using-existing-devices.html).

I am controlling the queue server via qserver bash commands. I open the environment, clear the queue and add an item.

qserver queue add plan '{"name": "count", "args": [["det1", "det2"]], "kwargs": {"num": 10, "delay": 1}}'
qserver queue start

After the 10 counts a TypeError is raised.

[I 2024-03-27 10:34:37,551 bluesky_queueserver.manager.manager] Starting the plan: 2024-03-27T10:34:37.551885768Z {'name': 'count', 2024-03-27T10:34:37.551904800Z 'args': [['det1', 'det2']], 2024-03-27T10:34:37.551907759Z 'kwargs': {'num': 10, 'delay': 1}, 2024-03-27T10:34:37.551910601Z 'user': 'qserver-cli', 2024-03-27T10:34:37.551913431Z 'user_group': 'primary', 2024-03-27T10:34:37.551916201Z 'meta': {}, 2024-03-27T10:34:37.551919128Z 'item_uid': '577a0133-772d-49fc-8dfb-fd48020aec7b'}. 2024-03-27T10:34:37.552487927Z [I 2024-03-27 10:34:37,551 bluesky_queueserver.manager.worker] Starting execution of a plan ... 2024-03-27T10:34:37.552600568Z [I 2024-03-27 10:34:37,552 bluesky_queueserver.manager.worker] Starting a plan 'count'. 2024-03-27T10:34:37.740276573Z [I 2024-03-27 10:34:37,739 bluesky_queueserver.manager.plan_monitoring] New run was open: 'dd4812f0-a081-4e5a-b6fc-cc61e03d8f85' 2024-03-27T10:34:47.761402028Z [I 2024-03-27 10:34:47,757 bluesky_queueserver.manager.plan_monitoring] Run was closed: 'dd4812f0-a081-4e5a-b6fc-cc61e03d8f85' 2024-03-27T10:34:47.762916747Z [I 2024-03-27 10:34:47,762 bluesky_queueserver.manager.worker] The plan was exited. Plan state: completed 2024-03-27T10:34:48.266380104Z [E 2024-03-27 10:34:48,256 bluesky_queueserver.manager.comms] Exception occurred while processing the message {"method": "request_plan_report", "jsonrpc": "2.0", "id": "d444c704-6a51-4fd8-ae64-5d0e4b9d1d72"}: Object of type RunEngineResult is not JSON serializable 2024-03-27T10:34:48.266440726Z Traceback (most recent call last): 2024-03-27T10:34:48.266454138Z File "/usr/local/lib/python3.11/site-packages/bluesky_queueserver/manager/comms.py", line 231, in process_msg_thread 2024-03-27T10:34:48.266465405Z self.handle_msg(msg) 2024-03-27T10:34:48.266475309Z File "/usr/local/lib/python3.11/site-packages/bluesky_queueserver/manager/comms.py", line 248, in _handle_msg 2024-03-27T10:34:48.266485653Z response = response.json 2024-03-27T10:34:48.266495512Z ^^^^^^^^^^^^^ 2024-03-27T10:34:48.266505548Z File "/usr/local/lib/python3.11/site-packages/jsonrpc/base.py", line 87, in json 2024-03-27T10:34:48.266515789Z return self.serialize(self.data) 2024-03-27T10:34:48.266525646Z ^^^^^^^^^^^^^^^^^^^^^^^^^ 2024-03-27T10:34:48.266535633Z File "/usr/local/lib/python3.11/json/init.py", line 231, in dumps 2024-03-27T10:34:48.266545742Z return default_encoder.encode(obj) 2024-03-27T10:34:48.266555501Z ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2024-03-27T10:34:48.266565225Z File "/usr/local/lib/python3.11/json/encoder.py", line 200, in encode 2024-03-27T10:34:48.266575334Z chunks = self.iterencode(o, one_shot=True) 2024-03-27T10:34:48.266585084Z ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2024-03-27T10:34:48.266595065Z File "/usr/local/lib/python3.11/json/encoder.py", line 258, in iterencode 2024-03-27T10:34:48.266606025Z return _iterencode(o, 0) 2024-03-27T10:34:48.266616830Z ^^^^^^^^^^^^^^^^^ 2024-03-27T10:34:48.266626604Z File "/usr/local/lib/python3.11/json/encoder.py", line 180, in default 2024-03-27T10:34:48.266636720Z raise TypeError(f'Object of type {o.class.name} ' 2024-03-27T10:34:48.266646828Z TypeError: Object of type RunEngineResult is not JSON serializable 2024-03-27T10:34:48.759153544Z [E 2024-03-27 10:34:48,757 bluesky_queueserver.manager.manager] Failed to download plan report: Timeout occurred while processing the request. Stopping queue processing.

After some troubleshooting I discovered that the problem was with the instantiation of the RunEngine. If RunEngine is instantiated with the call_returns_result=True flag, the queue server seems unable to parse the RunEngineResult object.

RE = RunEngine({}, call_returns_result=True)

Setting this flag to false caused the error to no longer be raised.

RE = RunEngine()

[dmgav]

@burkeds Thank you. This is not a bug. The Queue Server is indeed not supporting plans that return result. This is on our TODO list. I'll try to look into it shortly. This should be a small change.