Deploying AWS Lambda functions often involves packaging various dependencies. Managing these dependencies directly within Lambda can be cumbersome and error-prone. Using Docker and Amazon Elastic Container Registry (ECR) can streamline this process, ensuring consistency and reliability in your deployments. In this blog, we will walk through how to install pip packages in Lambda packages using Docker and ECR.

AWS Lambda is a powerful service that allows you to run code without provisioning or managing servers. However, managing dependencies for your Lambda functions can be challenging, especially when dealing with complex Python packages. By leveraging Docker, you can package your Lambda function and its dependencies in a container, which ensures a consistent runtime environment. Additionally, using ECR to store your Docker images simplifies the deployment process.

Prerequisites

Before we get started, make sure you have the following installed and configured:

• AWS CLI
• Docker

Step 1: Install AWS CLI and Configure

First, install the AWS CLI and configure it with your credentials.

Installation

• On Windows: Download the AWS CLI MSI installer from the official website. Run the installer and follow the on-screen instructions.
• On macOS: Use Homebrew to install the AWS CLI:

brew install awscli

• On Linux: Use the following commands:

curl "https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip" -o "awscliv2.zip"
unzip awscliv2.zip
sudo ./aws/install

Configure the AWS CLI by running:

aws configure

Enter your AWS Access Key ID, Secret Access Key, Default region name, and Default output format when prompted.

Step 2: Create an ECR Repository

If you haven't already created an ECR repository, do so by running:

aws ecr create-repository --repository-name my-lambda-repo --region us-east-1

This command creates a new ECR repository where you'll store your Docker images.

Step 3: Authenticate Docker to ECR

Authenticate your Docker CLI to your ECR registry, make sure you are signed into Docker desktop and have it running on your computer:

aws ecr get-login-password --region us-east-1 | docker login --username AWS --password-stdin <aws_account_id>.dkr.ecr.us-east-1.amazonaws.com

Replace <aws_account_id> with your actual AWS account ID.

Step 4: Create a .dockerignore File

Create a .dockerignore file in your project directory to exclude unnecessary files from the Docker build context:

__pycache__
.git
tests
*.md
*.txt

Step 5: Create a Dockerfile

Create a Dockerfile in your project directory. This file defines the environment in which your Lambda function will run and specifies the necessary dependencies.

# Builder stage - you can use other AWS runtimes and python versions
FROM public.ecr.aws/lambda/python:3.12-x86_64 as builder

# Install Python packages (replace as needed)
RUN pip install --no-cache-dir requests numpy pandas --target "/var/task"

# Final stage
FROM public.ecr.aws/lambda/python:3.12-x86_64

# Copy necessary files from the builder stage - add whatever code you need
COPY --from=builder /var/task /var/task
COPY handler.py ./

# Set the CMD to your handler
CMD ["handler.lambda_handler"]

Step 6: Build the Docker Image

Build the Docker image using the following command:

docker build -t my-lambda-image .

This command creates a Docker image named my-lambda-image.

Step 7: Tag the Docker Image

Tag the Docker image for pushing to your ECR repository:

docker tag my-lambda-image <aws_account_id>.dkr.ecr.us-east-1.amazonaws.com/my-lambda-repo:latest

Step 8: Push the Docker Image to ECR

Push the Docker image to your ECR repository:

docker push <aws_account_id>.dkr.ecr.us-east-1.amazonaws.com/my-lambda-repo:latest

Step 9: Update Lambda Function

Finally, update your Lambda function to use the new Docker image. You can do this through the AWS Management Console or using the AWS CLI:

aws lambda update-function-code --function-name my-lambda-function --image-uri <aws_account_id>.dkr.ecr.us-east-1.amazonaws.com/my-lambda-repo:latest

Replace my-lambda-function with the name of your Lambda function.

Alternatively, you can update your Lambda function through the AWS Management Console:

1. Navigate to the AWS Lambda console.
2. Select your Lambda function from the list.
3. In the "Code" tab, choose "Image" as the code source.
4. Select the ECR image you pushed earlier.
5. Save the changes.

Creating a Lambda Function from Scratch with a Docker Image

You can also create a new AWS Lambda function from scratch using a Docker image. This is useful when you are starting a new project or want to create a fresh Lambda function with a specific Docker environment. Here's how to do it through the AWS Management Console:

1. Navigate to the AWS Lambda console.
2. Click on "Create function."
3. Select "Container image" as the code source.
4. Specify your container image details by selecting the ECR repository and image tag you pushed earlier.
5. Configure the remaining settings (function name, role, etc.).
6. Click on "Create function."

This process sets up a new Lambda function that uses your Docker image as its execution environment, ensuring all dependencies and configurations are included from the start.

Conclusion

Using Docker and ECR to manage dependencies for your AWS Lambda functions significantly streamlines the deployment process. This method ensures that your functions have a consistent runtime environment, reducing the potential for errors caused by missing or incompatible dependencies.

By following the steps outlined in this tutorial, you can efficiently package, test, and deploy your Lambda functions with all necessary dependencies included. This approach not only simplifies the deployment process but also enhances the reliability and performance of your serverless applications.

Remember to subscribe to the channel and support our content creation efforts:

• Subscribe on YouTube
• Support on Buy Me a Coffee
• Hire me for your IoT projects on UpWork