Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Service Task response containing "items" key cause TypeError exception on next task #478

Open
vtexier opened this issue Sep 8, 2023 · 4 comments
Open

Service Task response containing "items" key cause TypeError exception on next task #478

vtexier opened this issue Sep 8, 2023 · 4 comments
Labels
bug Something isn't working

Comments

@vtexier
Copy link

vtexier commented Sep 8, 2023

I created a Spiff Connector to a service returning a dict with the key "items" in it. The corresponding value of the key is a list.

This key cause a TypeError on the next talk in the process in the diagram. Here in the code:

https://github.com/sartography/spiff-arena/blob/8c7061b0402be1bc4b23e9d820c58b85356413c2/SpiffWorkflow/SpiffWorkflow/bpmn/serializer/helpers/registry.py#L49C1-L49C55

task.data is treated as an object (called obj in the code) and when obj.items() is called Python try to call the list value as a function.

Here is the full exception of the event on the next task:

Stacktrace:
TypeError: 'list' object is not callable
                                 ^^^^^^^^^^^
    items = [ (k, v) for k, v in obj.items() ]
  File "/app/venv/lib/python3.11/site-packages/SpiffWorkflow/bpmn/serializer/helpers/registry.py", line 49, in clean
    self.clean(obj)
  File "/app/venv/lib/python3.11/site-packages/SpiffWorkflow/bpmn/serializer/helpers/registry.py", line 42, in convert
                    ^^^^^^^^^^^^^^^
    return dict((k, self.convert(v)) for k, v in obj.items())
  File "/app/venv/lib/python3.11/site-packages/SpiffWorkflow/bpmn/serializer/helpers/dictionary.py", line 97, in <genexpr>
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    return dict((k, self.convert(v)) for k, v in obj.items())
  File "/app/venv/lib/python3.11/site-packages/SpiffWorkflow/bpmn/serializer/helpers/dictionary.py", line 97, in convert
           ^^^^^^^^^^^^^^^^^^^^
    return super().convert(obj)
  File "/app/venv/lib/python3.11/site-packages/SpiffWorkflow/bpmn/serializer/helpers/registry.py", line 43, in convert
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    'data': self.data_converter.convert(task.data),
  File "/app/venv/lib/python3.11/site-packages/SpiffWorkflow/bpmn/serializer/workflow.py", line 233, in task_to_dict
                          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    new_properties_json = self.serializer.task_to_dict(spiff_task)
  File "/app/src/spiffworkflow_backend/services/task_service.py", line 265, in update_task_model
    self.update_task_model(task_model, spiff_task)
  File "/app/src/spiffworkflow_backend/services/task_service.py", line 199, in update_task_model_with_spiff_task
    self.task_service.update_task_model_with_spiff_task(waiting_spiff_task)
  File "/app/src/spiffworkflow_backend/services/workflow_execution_service.py", line 198, in after_engine_steps
    self.after_engine_steps(bpmn_process_instance)
  File "/app/src/spiffworkflow_backend/services/workflow_execution_service.py", line 234, in on_exception
    self.delegate.on_exception(bpmn_process_instance)
  File "/app/src/spiffworkflow_backend/services/workflow_execution_service.py", line 91, in on_exception
    self.execution_strategy.on_exception(self.bpmn_process_instance)
  File "/app/src/spiffworkflow_backend/services/workflow_execution_service.py", line 422, in run_and_save
    execution_service.run_and_save(exit_at, save)
  File "/app/src/spiffworkflow_backend/services/process_instance_processor.py", line 1418, in _do_engine_steps
    self._do_engine_steps(exit_at, save, execution_strategy_name, execution_strategy)
  File "/app/src/spiffworkflow_backend/services/process_instance_processor.py", line 1383, in do_engine_steps
    yield
  File "/app/src/spiffworkflow_backend/services/process_instance_queue_service.py", line 98, in dequeued
Traceback (most recent call last):

During handling of the above exception, another exception occurred:

SpiffWorkflow.bpmn.exceptions.WorkflowTaskException: TypeError:'list' object is not callable. 
    raise wte
  File "/app/venv/lib/python3.11/site-packages/SpiffWorkflow/bpmn/PythonScriptEngine.py", line 78, in execute
    super().execute(task, script, methods)
  File "/app/src/spiffworkflow_backend/services/process_instance_processor.py", line 362, in execute
    raise e
  File "/app/src/spiffworkflow_backend/services/process_instance_processor.py", line 365, in execute
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    return task.workflow.script_engine.execute(task, self.script)
  File "/app/venv/lib/python3.11/site-packages/SpiffWorkflow/bpmn/specs/mixins/script_task.py", line 46, in _execute
           ^^^^^^^^^^^^^^^^^^^
    return self._execute(task)
  File "/app/venv/lib/python3.11/site-packages/SpiffWorkflow/bpmn/specs/mixins/script_task.py", line 31, in _run_hook
             ^^^^^^^^^^^^^^^^^^^^^^^
    result = self._run_hook(my_task)
  File "/app/venv/lib/python3.11/site-packages/SpiffWorkflow/specs/base.py", line 305, in _run
    raise exc
  File "/app/venv/lib/python3.11/site-packages/SpiffWorkflow/specs/base.py", line 316, in _run
             ^^^^^^^^^^^^^^^^^^^^^^^^^
    retval = self.task_spec._run(self)
  File "/app/venv/lib/python3.11/site-packages/SpiffWorkflow/task.py", line 589, in run
    task.run()
  File "/app/src/spiffworkflow_backend/services/workflow_execution_service.py", line 318, in spiff_run
    self.execution_strategy.spiff_run(self.bpmn_process_instance, exit_at)
  File "/app/src/spiffworkflow_backend/services/workflow_execution_service.py", line 408, in run_and_save
Traceback (most recent call last):

During handling of the above exception, another exception occurred:

TypeError: 'list' object is not callable
                ^^^^^^^^^^^
    for k, v in arg.items():
  File "/app/venv/lib/python3.11/site-packages/SpiffWorkflow/bpmn/PythonScriptEngineEnvironment.py", line 94, in __init__
              ^^^^^^
    self[k] = Box(v)
  File "/app/venv/lib/python3.11/site-packages/SpiffWorkflow/bpmn/PythonScriptEngineEnvironment.py", line 96, in __init__
           ^^^^^^^^^
    return Box(data)
  File "/app/venv/lib/python3.11/site-packages/SpiffWorkflow/bpmn/PythonScriptEngineEnvironment.py", line 149, in convert_to_box
    Box.convert_to_box(context)
  File "/app/venv/lib/python3.11/site-packages/SpiffWorkflow/bpmn/PythonScriptEngineEnvironment.py", line 158, in _prepare_context
    self._prepare_context(context)
  File "/app/venv/lib/python3.11/site-packages/SpiffWorkflow/bpmn/PythonScriptEngineEnvironment.py", line 47, in execute
    super().execute(script, context, external_methods)
  File "/app/src/spiffworkflow_backend/services/process_instance_processor.py", line 135, in execute
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    return self.environment.execute(script, context, external_methods)
  File "/app/venv/lib/python3.11/site-packages/SpiffWorkflow/bpmn/PythonScriptEngine.py", line 118, in _execute
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    return self._execute(script, task.data, external_methods or {})
  File "/app/venv/lib/python3.11/site-packages/SpiffWorkflow/bpmn/PythonScriptEngine.py", line 75, in execute
Traceback (most recent call last):
@vtexier
Copy link
Author

vtexier commented Sep 12, 2023

I am able to reproduce the problem with this simple code.
The problem is in the Box class. It overrides standard dict methods with dict keys.

from SpiffWorkflow.bpmn.PythonScriptEngineEnvironment import Box

# create a standard dict
d = {"items": [1,2], "toto": 'titi'}
# create a Box dict
b = Box(d)

# iterate items() on standard dict is OK
d.items()
# iterate items()  on Box dict is NOT OK
b.items()

Box is deprecated, but the deprecation message does not mention the new class to use...
Can you tell me what class to use in replacement of Box ?
I want to test this code with the new class replacing Box to see if the problem persist.

@jbirddog
Copy link
Contributor

Thanks for the update, we have been internally debating on Box for a bit. Hopefully the diff that replaces Box is all red lines. I do appreciate following up on this, will add more points to our discussion.

@jbirddog
Copy link
Contributor

jbirddog commented Sep 13, 2023 •
edited
Loading

Also forgot to address the replacement for Box - at least within arena is based on this line - https://github.com/sartography/spiff-arena/blob/main/spiffworkflow-backend/src/spiffworkflow_backend/services/process_instance_processor.py#L264

You'd want to switch it to NonTaskDataBasedScriptEngineEnvironment as the base class, but this hasn't been tested in a while and looks like it may lead to issues based on the comment above.

@github-project-automation github-project-automation bot moved this to New Issue in SpiffWorkflow Oct 6, 2023
@calexh-sar calexh-sar added the bug Something isn't working label Oct 6, 2023
@calexh-sar calexh-sar moved this from New Issue to Backlog in SpiffWorkflow Oct 6, 2023
@essweine
Copy link
Collaborator

I have opened sartography/SpiffWorkflow#359 in the SpiffWorkflow library to remove Box.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug Something isn't working
Projects
SpiffWorkflow
Status: Backlog
Milestone
No milestone
Development

No branches or pull requests
4 participants
@vtexier @essweine @calexh-sar @jbirddog