AWS Stepfunctions – Introduction

AWS Step Functions is a service offered by AWS that can be used to coordinate communication between multiple processes. It not only helps with coordination, but the great advantage is that it can last up to a year before timing out!

In a recent project, we had a requirement to generate media files, a process that can take a considerable amount of time. Since the various steps occur over an extended period, this made it an ideal use case for building a state machine or saga.

Rather than manually tracking each step by crafting a Saga, I used a Step Function to orchestrate the flow.

You can create a Step Function using the intuitive user interface in the AWS Console. The workflow can be defined through drag-and-drop functionality, and the definition is stored using AWS State Language in the background.

I prefer coding, so in the next article, I will focus more on the generated code rather than the UI. However, everything mentioned here can also be accomplished using the user interface.

The graphical user interface for creating AWS Step Functions

When you are new to Step Functions, it can take a while to see the power of it. One important step I’ll encourage you to learn is the AWS State Language. It’s pretty straightforward to grasp and will save you time when implementing Infrastructure as Code (IaC) and storing your Step Function definitions in your Git repository. Let’s dive into the details.

AWS State Language

The AWS state language describes the entire flow, including all the steps and transitions in your Step Function. The most important section will be the States block, in which all the individual steps are described. The description of the States languages can be found here https://states-language.net/spec.html
Apart from the states, you can have a comment explaining the purpose of the step function., and an indication in which state the state function should start.

A definition of a state function can be found below:

{
  "Comment":"<a comment>",
  "StartsAt" : "<state to start the step function>"
  "States": {
    "<StateName>" : {
      "Type" : "<Type of the pass>",
      "Parameters" : {
        <Parameters that the state expects as key values>
      }
      "Next": "<The step to execute next>"
      "ResultPath": "<A JsonPath where to store the location of the step>"
    }
}

Here is an example of how a function might look. We will dig into more details later on.

{
  "Comment": "Video Generation",
  "StartAt": "Parse Json",
  "States": {
    "Parse Json": {
      "Type": "Pass",
      "Parameters": {
        "input.$": "States.StringToJson($.detail.data)"
      },
      "Next": "FetchRequestedEventContent"
    },
    "FetchRequestedEventContent": {
      "Type": "Pass",
      "Next": "SendMessageToSQS",
      "Parameters": {
        "InspectionIdentifier.$": "$.input.MyIdentifier",
        "Version.$": "$.input.Version"
      },
      "ResultPath": "$.MessageEventContent"
    },
    "SendMessageToSQS": {
      "Type": "Task",
      "Resource": "arn:aws:states:::sqs:sendMessage.waitForTaskToken",
      "HeartbeatSeconds": 300,
      "Parameters": {
        "QueueUrl": "myQueueUrl",
        "MessageBody.$": "$.ThisPartINeed",
        "MessageAttributes": {
          "TaskToken": {
            "DataType": "String",
            "StringValue.$": "$$.Task.Token"
          }
        }
      },
      "Next": "Build Event",
      "ResultPath": null
    },....

Understanding the States Block and the anatomy of the a state

Looking closer to the States block, you find multiples states, with each their own name. Each individual state has these major components

Type
Next
Parameters
ResultPath
OutputPath

Let us dive deeper into each one over here.

Type

This parameter specifies the action your step function will take when it arrives in the state. There are hundreds of different types, so I won’t list them all here to keep things concise. For more information on the available types, you can refer to the AWS documentation.

There are 2 major type categories

Actions: These types differ based on the action. Some examples:
- Invoke a Lambda function
- Add a message to an SQS queue
- Start an EC2 instance
- Delete an object on S3
- …
Flow: These types define in which flow certain actions will be executed. You can add if/else branches, loop over a JSON array
- Choice: Chose a next state to execute using a conditional If/Else logic.
- Parallel: Execute multiple tasks before going to the next state.
- Map: Execute a workflow for each item in an array.
- Pass: Transform data
- Wait: Wait for a certain timespan before moving on to the next state.
- Success: Marks the Step Function as a success
- Fail: Marks the Step Function as a failure.

An import state type is, the pass state. This pass state will give you the ability to transform the internal state of your step function, and pass specific data to other types. Understanding this state, will give you a lot of flexibility and power up your Step Function in the long run!

This parameter is rather obvious and describes the next state that will be executed. This refers to the name of the next state.

Parameters

The Parameters can be used to build a specific JSON object from the states input, that will be passed to the action.

InputPath

In the InputPath you can filter your Step Function JSON state and only pas a part or certain JSON portion to the action.

OutputPath

In the OutputPath you can define which part of the action you would like to keep.

ResultPath

The ResultPath might look similar to the OutputPath, but it has a different purpose. When you set this path, it will add the result of the action, to the current state of the function! This is extremely handy if you want to keep track of certain action that are already defined and what the outcome was. If you do not require the result of the Action, you can set this to null

Now you have a brief understanding how a Step Function looks, in the upcoming post we will explore nd uncover more details of the AWS State Language!