Throw users a bone? Show how to invoke step function locally USING the ASL file? Show how to see the local logs?

[jpwynn]

Trying the ruby 2.7 Stock Trader "example". (All the AWS guides seems to be pointedly silent on the subject of locally executing a step function as defined in a SAM project). There's a fair number of stackoverflow questions on this topic, none of which explain how to do it.

So a single, complete "how to test stock Trader step function locally" would be a most welcome addition to the docs since it includes multiple lambdas, a DB table and a step function.

I think I might be about half-way there... maybe someone else can help document the incantations needed to make it work locally, and to be able to view log results afterwards.

I did eventually find (by trial and error) that when using aws stepfunctions create-state-machine is is possible to specify a file (rather than enter the json on the command line per the online examples, which, let's face it, is a useless example).

And I did find out (again by trial and error) that one (apparently) needs to manually create a NEW json file with all the resource variables in the ASL file de-referenced by 'local' ARNs?

And (once again by trial an error) I found that the edits to that COPY of the ASL file go something like this:

${StockCheckerFunctionArn} -> arn:aws:lambda:us-east-1:123456789012:function:StockCheckerFunctionArn
${StockSellerFunctionArn} -> arn:aws:lambda:us-east-1:123456789012:function:StockSellerFunctionArn
${StockBuyerFunctionArn} -> arn:aws:lambda:us-east-1:123456789012:function:StockBuyerFunctionArn
${DDBPutItem} -> arn:aws:lambda:us-east-1:123456789012:transaction_table:DDBPutItem
${DDBTable} -> arn:aws:lambda:us-east-1:123456789012:transaction_table:TransactionTable

So if the edited ASL file is saved in a new folder in the SAM project, for example as edata/step1.asl.json, the step function can almost be tested locally using:

Mac terminal window # 1: sam local start-lambda
Mac terminal window # 2: docker run -p 8083:8083 amazon/aws-stepfunctions-local
Mac terminal window # 3 (note the use of file://): aws stepfunctions --endpoint http://localhost:8083 create-state-machine --definition file://edata/step1.asl.json --name StockTradingStateMachine --role-arn "arn:aws:iam::012345678901:role/StockTradingStateMachineRole"
then, to RUN it:
Mac terminal window # 4: aws stepfunctions --endpoint http://localhost:8083 describe-execution --execution-arn arn:aws:states:us-east-1:123456789012:execution:StockTradingStateMachine:test

The output of terminal # 4 finally shows something:

{
    "executionArn": "arn:aws:states:us-east-1:123456789012:execution:StockTradingStateMachine:test",
    "stateMachineArn": "arn:aws:states:us-east-1:123456789012:stateMachine:StockTradingStateMachine",
    "name": "test",
    "status": "RUNNING",
    "startDate": "2021-04-21T14:55:12.961000-10:00",
    "input": "{}",
    "inputDetails": {
        "included": true
    }
}

But the output stream in terminal # 2 shows the error:

2021-04-22 00:58:34.179: arn:aws:states:us-east-1:123456789012:execution:StockTradingStateMachine:test : {"Type":"ExecutionFailed","PreviousEventId":20,"ExecutionFailedEventDetails":{"Error":"Lambda.AWSLambdaException","Cause":"The security token included in the request is invalid. (Service: AWSLambda; Status Code: 403; Error Code: UnrecognizedClientException; Request ID: 2cd8222b-5f6e-4b1e-8263-d34438d713c9; Proxy: null)"}}

The examples I've found also fail to show how to provide a test input event to a step function. (The -e flag used when locally invoking Lambdas does not seem to work.)

[davidjb]

The page at https://docs.aws.amazon.com/step-functions/latest/dg/sfn-local-lambda.html covers the general steps of running lambdas and Step Functions Local, but it needs to be adapted for the example you mentioned. That page definitely ends in a cliff when it comes to wanting to create a state machine from a SAM CLI ASL definition.

The SAM CLI Node templates are using sed in their respective Makefiles to replace the SAM CLI-templated ASL's variables. A somewhat cleaner workaround is to dynamically replace those variables using yq like so:

aws stepfunctions create-state-machine \
  --endpoint http://localhost:8083 \
  --role-arn "arn:aws:iam::012345678901:role/DummyRole" \
  --name "StockTradingStateMachine" \
  --definition "$(yq ' \
    (.. | select(has("Resource")) | .Resource) |= sub("\${(.*)Arn}", "arn:aws:lambda:us-east-1:123456789012:function:${1}") \
    (.. | select(has("MaxConcurrency")) | .MaxConcurrency) |= 1 \
    ' edata/step1.asl.json --output-format=json --indent=0)"

This is akin to what sed is doing but supports any input that yq does, particularly ASLs in YAML. It avoids use of a temporary file and is more extensible than simple sed string replacement but it does of course require yq. The MaxConcurrency line above useful for deterministic mocking (see https://docs.aws.amazon.com/step-functions/latest/dg/sfn-local-mock-cfg-file.html#mock-cfg-mckd-resp-sect) but can be removed if not testing using mocks.

I've not seen the security token error mentioned but it may be because you need to call start-execution first before trying to describe the execution. This is mentioned in Step 6 of the Testing Step Functions Local guide; this has worked for me with Step Functions Local.

As for sending test input to a Step Function execution, then that's done by aws stepfunctions start-execution --input '{}', where the {} is your JSON-encoded input. The docs on AWS CLI's start-execution command have more info.