A Serverless Bash REPL

Evaluating Bash code on a custom AWS Lambda runtime.

A Read-Eval-Print-Loop (REPL) is an interactive program that takes a user input, evaluates it, and returns the output. Such environments are helpful for experimentation and prototyping, and REPLs for many languages are available as online applications, even if the language is not web-native. This article describes creating a Bash REPL as a web application using an AWS Lambda Function as the basis for an HTTP API to evaluate Bash code and return the output as a response to the client.

various bash commands being executed by a lambda function with custom bash runtime.

Create a Custom Bash 'Server'

My plan for building an API to evaluate Bash code is to create a Bash web server using a custom Lambda runtime. The AWS docs describe this process in Tutorial: Building a custom runtime — AWS Lambda (amazon.com).

Because Lambda events are JSON objects, parsing them in bash will be a chore unless using some tool like jq. Thus, I’ve included the jq AMD64 binary with my Lambda function handler code.

./lambda_handler
├── bin
│   └── jq
├── bootstrap
└── function.sh

The bootstrap script is executed first and sets up the runtime environment. I’ve modified the runtime PATH variable to include /var/task/bin which is the directory housing the jq binary.

#!/bin/sh

# bootstrap.sh 

# Initialization - load function handler
# equal to 'source /var/task/function.sh'
source $LAMBDA_TASK_ROOT/"$(echo $_HANDLER | cut -d. -f1).sh"

# Add `jq` to PATH
export PATH="/var/task/bin:$PATH"

# Processing
while true
do
    HEADERS="$(mktemp)"
    # Get an event. The HTTP request will block until one is received
    EVENT_DATA=$(curl -sS -LD "$HEADERS" "http://${AWS_LAMBDA_RUNTIME_API}/2018-06-01/runtime/invocation/next")

    # Extract request ID by scraping response headers received above
    REQUEST_ID=$(grep -Fi Lambda-Runtime-Aws-Request-Id "$HEADERS" | tr -d '[:space:]' | cut -d: -f2)

    # Run the handler function from the script
    RESPONSE=$($(echo "$_HANDLER" | cut -d. -f2) "$EVENT_DATA")

    # Send the response
    curl "http://${AWS_LAMBDA_RUNTIME_API}/2018-06-01/runtime/invocation/$REQUEST_ID/response"  -d "$RESPONSE"
done

My Bash server will have two routes:

A health check — GET /api/health
Code evaluation — GET /api/exec/{_src}

This is where using an API Gateway becomes helpful since it enables me to define these routes explicitly. When the gateway sends an API request to the Lambda function, its event will contain the request method and route information conveniently together as the property .requestContext.resourceId. Because we installed jq earlier, getting the request’s resource ID is trivial. And because our server runs on Bash, we can evaluate incoming source code easily using eval.

# Handles the incoming APIGatewayProxy event and echoes a response.
function handler () {
    EVENT_DATA=$1

    # i.e.,
    # {
    #  "version": "1.0",
    #  "resource": "/api/exec/{_src}",
    #  "path": "/api/exec/test",
    #  "httpMethod": "GET",
    #  "headers": { ...
    #  ...
    echo "EVENT_DATA: $EVENT_DATA" 

    resource_id=$(echo "$EVENT_DATA" | jq -r '.requestContext.resourceId')

    # Serves a health check response (Ok).
    # GET /api/health
    if [ "$resource_id" == "GET /api/health" ]; then
        echo '{"statusCode": 200, "body": "Ok."}'
        return
    fi

    # Executes {_src} as a bash script and returns the output.
    # GET /api/exec/{_src}
    if [ "$resource_id" == "GET /api/exec/{_src}" ]; then
        # decode incoming source code from base64.
        _src=$(echo "$EVENT_DATA" | jq -r '.pathParameters._src')
        _src=$(echo $_src | base64 -d)

        # evaluate source code and capture the result.
        if ! result=$(eval "$_src" 2>&1); then
            # sanitize the error message.
            result="Error: $(echo $result | grep -oP 'line\s+\d+:\s+\K.+')"
        fi

        # using jq to build a response object.
        RESPONSE=$(
            jq -n \
                --arg status_code 200 \
                --arg result "$result" \
            '{
                "statusCode": $status_code,
                "body": $result
            }'
        )


        echo "$RESPONSE"
        return
    fi

    # fallthrough
    echo '{"statusCode": 501, "body": "Not implemented."}'
}

Provision Cloud Resources

I’ve used Terraform to provision the project’s cloud infrastructure. It's a standard setup that zips the lambda_handler directory from the previous stage and implements the OS-only provided.al2023 runtime.

data "archive_file" "handler_function_payload" {
    type             = "zip"
    source_dir      = "${path.module}/lambda_handler"
    output_file_mode = "0666"
    output_path      = "${path.module}/dist/lambda_handler.zip"
}

module "lambda_function" {
    source  = "gitlab.com/ben_goodman/lambda-function/aws"
    version = "3.0.0"

    org              = "${var.resource_namespace}-${terraform.workspace}"
    project_name     = "bash-repl"
    lambda_payload   = data.archive_file.handler_function_payload
    function_name    = "bash-exec"
    function_handler = "function.handler"
    publish          = true
    memory_size      = 512
    runtime          = "provided.al2023"
}

resource "aws_lambda_permission" "allow_default_gateway" {
    statement_id  = "bash-repl-gateway-exec"
    action        = "lambda:InvokeFunction"
    function_name = module.lambda_function.name
    principal     = "apigateway.amazonaws.com"
    source_arn    = "${aws_apigatewayv2_api.default.execution_arn}/*/*"
}

Likewise, the API Gateway config is standard, too. The route definitions are included here.

resource "aws_apigatewayv2_api" "default" {
    name          = "bash-repl-api-gateway-${terraform.workspace}"
    protocol_type = "HTTP"
}


resource "aws_cloudwatch_log_group" "default" {
    name = "/aws/bash-repl_gateway_${terraform.workspace}/${aws_apigatewayv2_api.default.name}"
    retention_in_days = 30
}


resource "aws_apigatewayv2_stage" "default" {
    api_id = aws_apigatewayv2_api.default.id
    name        = "$default"
    auto_deploy = true

    access_log_settings {
        destination_arn = aws_cloudwatch_log_group.default.arn

        format = jsonencode({
            requestId               = "$context.requestId"
            sourceIp                = "$context.identity.sourceIp"
            requestTime             = "$context.requestTime"
            protocol                = "$context.protocol"
            httpMethod              = "$context.httpMethod"
            resourcePath            = "$context.resourcePath"
            routeKey                = "$context.routeKey"
            status                  = "$context.status"
            responseLength          = "$context.responseLength"
            integrationErrorMessage = "$context.integrationErrorMessage"
        })
    }
}


resource "aws_apigatewayv2_integration" "default" {
    api_id = aws_apigatewayv2_api.default.id
    integration_uri    = module.lambda_function.invoke_arn
    integration_type   = "AWS_PROXY"
    integration_method = "POST"
}


resource "aws_apigatewayv2_route" "health_check" {
    api_id = aws_apigatewayv2_api.default.id
    target    = "integrations/${aws_apigatewayv2_integration.default.id}"
    route_key = "GET /api/health"
}


resource "aws_apigatewayv2_route" "exec_bash" {
    api_id = aws_apigatewayv2_api.default.id
    target    = "integrations/${aws_apigatewayv2_integration.default.id}"
    route_key = "GET /api/exec/{_src}"
}

The complete code for this section is available at .aws/terraform · main · projects / Apps / Bash REPL · GitLab.

At this stage, the API is now ready to test.

src="$(echo 'echo "hello, world"' | base64)"

curl -X GET \
  https://ezb24gwwx1.execute-api.us-east-1.amazonaws.com/api/exec/$src

# hello, world

This is only half of the project, though. The next step is to create a simple GUI for entering input and displaying the results.

Create a Simple UI

Firstly, a custom React hook is created to interface the UI component with the API.

export const useEval = () => {
    return {
        evalCode: async (code: string): Promise<string> => {
            // encode the code to base64
            const encoded = btoa(code)
            const resp = await fetch(`./api/exec/${encoded}`)
            return await resp.text()
        }
    }
}

Then a React component which imitates a terminal emulator. I developed this one specifically for this projectL @bgoodman/react-console-component.

import React from 'react'
import { createRoot } from 'react-dom/client'
import { useEval } from './hooks/useEval'
import { Console } from '@bgoodman/react-console-component'
import { lightTheme } from './themes'
import { ThemePicker } from './components/ThemePicker'

export const App = () => {
    const [theme, setTheme] = React.useState(lightTheme)
    const { evalCode } = useEval()
    const handleExec = async (input: string) => await evalCode(input)
    return (
        <>
            <ThemePicker onChange={setTheme} />
            <Console
                onEnter={handleExec}
                theme={theme}
                height='90vh'
            />
        </>
    )
}


const container = document.getElementById('root')
const root = createRoot(container!)
root.render(
    <>
        <App/>
    </>
)

Conclusion

Is this safe? Is this something I want to host freely? No, not really, and by the time you read this, the URLs encountered for this project will no longer work. But hey, you can always clone the project and deploy your own Bash REPL using the AWS free tier.

The project’s source is available at projects / Apps / Bash REPL · GitLab.