Add function to parse an escaped JSON string

[skayliu]

Is your feature request related to a problem? Please describe.
When use the REST connector to call a service, i got the response like below.
Now i need to extract the taskTodoList into an array to the result expression.

{
    "status": 200,
    "headers": {
        "date": "Fri, 12 Apr 2024 01:29:49 GMT",
        "server": "nginx",
        "transfer-encoding": "chunked",
        "x-content-type-options": "nosniff",
        "x-xss-protection": "1; mode=block",
        "x-frame-options": "SAMEORIGIN",
        "content-type": "application/json;charset=utf-8",
        "connection": "keep-alive",
        "strict-transport-security": "max-age=172800; includeSubDomains"
    },
    "body": {
        "RESPONSE": {
            "RETURN_CODE": "S",
            "RETURN_DATA": "{\"taskTodoList\":[{\"activityDefId\":\"obj_ca954d098ba000018cd81cfdc2602910\",\"activityDefName\":\"startTask\",\"activityModelExt\":{\"customUniqueId\":\"\",\"extendProperty\":\"\",\"id\":\"obj_ca954d098ba000018cd81cfdc2602910\",\"name\":\"startTask\",\"no\":1,\"sequential\":-1},\"activityType\":\"userTask\",\"async\":false,\"beginEngineNode\":\"2406:440:600::16e0:10018\",\"beginTime\":1712885389000,\"claimResourceId\":\"\",\"claimType\":0,\"controlState\":\"active\",\"customUniqueId\":\"\",\"delayTimes\":0,\"delegateUser\":\"\",\"dispatchId\":\"4a9cf601-b664-4cda-8581-ead5814ac6a0\",\"eAITask\":false,\"ext1\":\"\",\"ext2\":\"\",\"ext3\":\"\",\"ext4\":\"\",\"ext5\":\"\",\"ext6\":\"\",\"ext7\":0,\"ext8\":0.0,\"historyTask\":false,\"iOBD\":\"\",\"iOC\":\"\",\"iOR\":\"\",\"iOS\":\"\",\"id\":\"2c8eebfa-7f16-4c14-8f70-bd660d65e510\",\"monitor\":false,\"owner\":\"admin\",\"ownerDepartmentId\":\"5bc3a2dc-3bd2-4376-bcc3-5612e28e55fe\",\"ownerName\":\"admin\",\"parentTaskInstId\":\"00000000-0000-0000-0000-000000000000\",\"priority\":1,\"processDefId\":\"obj_1b01dfade2fd4c408f6426ee290743d0\",\"processDefVerId\":\"obj_1b01dfade2fd4c408f6426ee290743d0\",\"processGroupId\":\"obj_f953af04fb5944ca8128e262d363f62c\",\"processInstId\":\"e4710d5c-9078-4a47-8868-4d43805d5d23\",\"readState\":0,\"remindTimes\":0,\"root\":true,\"scopeId\":\"00000000-0000-0000-0000-000000000000\",\"securityLevel\":0,\"state\":1,\"target\":\"admin\",\"targetCompanyId\":\"8911e732-b42a-4556-853f-ad32761bcbee\",\"targetDepartmentId\":\"5bc3a2dc-3bd2-4376-bcc3-5612e28e55fe\",\"targetName\":\"admin\",\"targetRoleId\":\"d102c89d-55f3-4865-9d5c-c00b7f47b803\",\"targetRoleNo\":\"\",\"taskInfo\":\"\",\"title\":\"Test Zeebe Rest connector\",\"trash\":true}],\"processInstId\":\"e4710d5c-9078-4a47-8868-4d43805d5d23\"}",
            "RETURN_DESC": "sucess",
            "RETURN_STAMP": "2024-04-12 09:29:49:022"
        }
    }
}

As discuss in the forum:
How to get the taskTodoList from the response body
Parse a json string to FEEL context

Describe the solution you'd like
A new built-in function to handle the JSON string like JSON.parse().

// Function signature
from json(json: string): Any

// Examples
// 1) JSON object to FEEL context
from json("{\"a\": 1, \"b\": 2}")
// {a: 1, b: 2}

// 2) JSON literal to FEEL value
from json("1")
// 1

from json("\"a\"")
// "a"

from json("true")
// true

from json("null")
// null

// 3) JSON array to FEEL list
from json("[1, 2, 3]")
// [1, 2, 3]

// 4) JSON string of a date/time/date-time to FEEL string 
from json("\"2023-06-14\"")
// "2023-06-14" 

from json("\"14:55:00\"")
// "14:55:00" 

from json("\"2023-06-14T14:55:00\"")
// "2023-06-14T14:55:00" 

// 5) JSON string of a duration to FEEL string 
from json("\"P1Y\"")
// "P1Y" 

from json("\"PT2H\"")
// "PT2H" 

// 6) invalid JSON to FEEL null 
from json("invalid")
// null 

Related issues

• Stringify JSON objects #602
• Camunda Autormation Platform 7:
• Zeebe broker:
• OMG issues:
  • https://issues.omg.org/browse/DMN16-68
  • https://issues.omg.org/browse/DMN15-102

[saig0]

@skayliu thank you for raising it. I agree that parsing a JSON string into a FEEL value would be useful. I saw similar requests before. 👍

I updated the issue description and proposed a new built-in function. Please have a look if the examples match your requirements.

The issue of parsing a JSON string is closely related to the revert function of creating a JSON string from a FEEL value: #602.

[skayliu]

@saig0, which libray would you like to support this feature?

[saig0]

@skayliu we could implement the JSON parser ourselves, similar to the FEEL parser. There is an example here.

Or, we use Jackson library. We use the library already in Camunda 8.

It could be fun writing the parser ourselves and we don't need a new library for it. If it doesn't work or is too complicated then we should switch to Jackson.

[sbuettner]

For the sake of NIH I would recommend we go with Jackson as its already a proven solution in this space.

[berndruecker]

Hi @saig0 - is this scheduled in any way? Also stumbled upon this - especially in the context of AI results that often encode Json as a String. JSON.parse would be super helpful.

I think we should definitely rely on Jackson for doing it!

[saig0]

@berndruecker thank you for your input. Currently, the function is not scheduled for any release.

[falko]

Now that DMN 1.6 finally standardized the FEEL-JSON mapping it would be a logical next step to add it as a FEEL conversion function in DMN 1.7.

See DMN 1.6 Table 48: Mapping between FEEL and JSON domains

[falko]

As a workaround for flat JSON objects with only string values one can build something with string functions and the context conversion function:

{
  json: choices[1].message.content,
  pairStrings: split(substring(json, 3, string length(json) - 4), "\",\""),
  keyValuePairs: for pair in pairStrings return split(pair, "\":\""),
  entries: for kv in keyValuePairs return { "key": kv[1], "value": kv[2] },
  contextFromJson: context(entries)
}

Full Example in FEEL Playground

Using our extract function this could be probably made even stronger with regular expression.