Deploying AWS Lambda-Compatible Functions in Amazon EKS using TriggerMesh KLR

中文版 – Custom AWS Lambda Runtimes were introduced at re:Invent 2018. Knative is an open source project to build, deploy, and manage serverless workloads. This post by Sebastien Goasguen explains that TriggerMesh’s Knative Lambda Runtime is a custom runtime that can run a Lambda function on Knative running on an Amazon EKS cluster.

–Arun

AWS Lambda has become the leading serverless offering worldwide. The ability to easily connect together AWS services has opened the door for true cloud-native applications, and enabled users to innovate tremendously faster. At the same time, Kubernetes has emerged as the leader in container orchestration and the leading enabler of cloud-native applications on-premise. Amazon Elastic Container Service for Kubernetes (EKS) became generally available last year.

The two worlds of serverless and containers could seem at odds, with serverless removing any infrastructure burden and putting the focus on application architecture and services, while Kubernetes still exposes containers and infrastructure concerns to its users and operators.

However, these two worlds seem bound to converge in several ways. One example is Knative, an open source project designed to use Kubernetes to build source-centric and container-based applications that can run anywhere. Knative is made up of three components: building, serving, and eventing. Building takes source code and generates a container as transparently to the users as possible. Serving takes those containers and offers features like scaling to 0. Finally, eventing offers a way to take cloud events, wherever they are, and use them to trigger functions, very much like AWS event sources can be used to trigger AWS Lambda functions.

With Knative being developed, one has to wonder whether it would be possible to run an AWS Lambda function within Knative, in an EKS cluster or even in another Kubernetes cluster (including on-premise). You might want to do this to unify workload management under Kubernetes, or to keep all workloads, including functions, under one “roof,” or perhaps you simply believe that AWS Lambda functions should become the standard for functions across serverless providers.

In this post, we want to show you how this is already a reality: you can indeed run an AWS Lambda function in Knative, using the TriggerMesh Knative Lambda Runtimes (KLR).

KLR are Knative objects called build-templates which use a custom implementation of the AWS Lambda custom runtime API. Lambda custom runtimes opened up a great integration point to the open source community. At TriggerMesh, we decided to implement what we call a function invoker to expose the Lambda custom runtime API. We then used this to build Knative templates for Go, Ruby, Node, and Python functions.

This is a very exciting development that we are eager to share with you. The rest of this post is a step-by-step walkthrough that will guide you through creating an EKS cluster and running TriggerMesh KLR functions. You will:

• Create an EKS cluster with eksctl
• Install Knative
• Install the TriggerMesh Knative CLI
• Deploy a function in EKS using TriggerMesh KLR

The picture below depicts the overall setup that we are going to put in place.

In the remainder of this post, we assume that you have previously configured your AWS CLI (e.g aws) and your Kubernetes CLI (e.g kubectl). If this is not the case, head over to the following links:

• Install and configure the AWS CLI.
• Install and configure the Kubernetes CLI.

Create an EKS Cluster

You’ll first need to set up an Amazon EKS cluster, using eksctl with the Cluster config file mechanism. Start by downloading these prerequisites:

• eksctl
  kubectl

With all the necessary tools installed, you can launch your EKS cluster. In this example, we’re deploying the cluster in us-east-2, our Ohio region; you can replace the AWS_REGION with any region that supports Amazon EKS.

Deploy Cluster

export AWS_REGION=us-east-1
export ACCOUNT_ID=$(aws sts get-caller-identity –query “Account” –output text)
Once you’ve exported the region, create the ClusterConfig as follows:

cat >cluster.yaml <<EOF
apiVersion: eksctl.io/v1alpha4
kind: ClusterConfig
metadata:
  name: knative
  region: ${AWS_REGION}
  version: "1.11"

nodeGroups:
- name: knodes
  desiredCapacity: 3
  instanceType: m5.large
EOF

eksctl create cluster --kubeconfig eksknative.yaml -f cluster.yaml

alias knative='kubectl --kubeconfig=eksknative.yaml'

knative get nodes
NAME                             STATUS    ROLES     AGE       VERSION
ip-192-168-10-163.ec2.internal   Ready     <none>    1m        v1.11.5
ip-192-168-12-106.ec2.internal   Ready     <none>    1m        v1.11.5
ip-192-168-43-220.ec2.internal   Ready     <none>    1m        v1.11.5

knative apply --filename https://github.com/knative/serving/releases/download/v0.4.0/istio-crds.yaml && \
knative apply --filename https://github.com/knative/serving/releases/download/v0.4.0/istio.yaml

knative apply --filename https://github.com/knative/serving/releases/download/v0.4.0/serving.yaml \
--filename https://github.com/knative/build/releases/download/v0.4.0/build.yaml \
--filename https://github.com/knative/serving/releases/download/v0.4.0/monitoring.yaml \
--filename https://raw.githubusercontent.com/knative/serving/v0.4.0/third_party/config/build/clusterrole.yaml

knative get pods -n knative-serving
NAME                          READY     STATUS    RESTARTS   AGE
activator-6f7d494f55-57r9k    2/2       Running   1          2m
autoscaler-5cb4d56d69-mj4lm   2/2       Running   1          2m
controller-6d65444c78-tlpmm   1/1       Running   0          2m
webhook-55f88654fb-ppczx      1/1       Running   0          2m
knative get pods -n knative-build
NAME                                READY     STATUS    RESTARTS   AGE
build-controller-68dfb74954-5d68l   1/1       Running   0          2m
build-webhook-866fd64885-95sq4      1/1       Running   0          2m

$ knative get svc istio-ingressgateway -o json -n istio-system | jq -r .status.loadBalancer.ingress[0].hostname
af375ca60465511e9911e02b74097eec-2072757389.us-east-1.elb.amazonaws.com

go get github.com/triggermesh/tm

which tm

alias tmk='tm --config=eksknative.yaml'

tmk get services
NAMESPACE    SERVICE
tmk get builds
NAMESPACE    BUILD

tmk deploy buildtemplate -f https://raw.githubusercontent.com/triggermesh/knative-lambda-runtime/master/python-3.7/buildtemplate.yaml

tmk get buildtemplates
NAMESPACE    NAME                        READY
default      knative-python37-runtime    True

tmk set registry-auth dockerhub
Registry: index.docker.io
Username: runseb
Password: **********
Registry credentials set

tmk deploy service python-test -f https://github.com/serverless/examples \
                              --build-template knative-python37-runtime \
                              --build-argument DIRECTORY=aws-python-simple-http-endpoint \
                              --build-argument HANDLER=handler.endpoint \
                              --registry-host dockerhub \
                              --wait

curl -H 'Host: python-test.default.svc.cluster.local' http://af375ca60465511e9911e02b74097eec-2072757389.us-east-1.elb.amazonaws.com
{"statusCode": 200, "body": "{\"message\": \"Hello, the current time is 15:05:53.694014\"}"}

eksctl delete cluster knative