AWS Lambda pip packages: Build a Python layer fast

Video Tutorial (Optional)

Watch first if you want to see the full workflow for installing pip packages for AWS Lambda using a helper function and a Lambda Layer.

Project Overview

AWS Lambda + pip packages (pyjwt): In this tutorial, you install a pip dependency (example: pyjwt) for an AWS Lambda Python function by running pip install inside a helper Lambda, zipping the result, and deploying it as a Lambda Layer so your main function can import the package.

This approach is a fast, pragmatic method often used for simple projects and testing. It is not presented as best practice for all production workloads.

The only prerequisite to follow along is an AWS account.

Time: 20 to 40 minutes
Skill level: Beginner to Intermediate
What you will build: A Lambda Layer containing a pip-installed package (example: pyjwt) that you can attach to a Lambda function

If you have not yet, you can subscribe and support the channel here:

Subscribe: Youtube
Support: https://www.buymeacoffee.com/mmshilleh

Parts List

From ShillehTek

No ShillehTek parts are required for this AWS tutorial.

External

AWS account (Lambda + S3 access)
AWS Lambda function (target) running Python
AWS Lambda function (helper) running Python (matching runtime and architecture)
Amazon S3 bucket for storing the generated zip
Python package to install (example used: pyjwt)
Helper script reference: https://github.com/shillehbean/youtube-p2/blob/main/lambda_pip_helper.py

Note: Keep the Python version and CPU architecture consistent across your helper Lambda, your target Lambda, and the Layer (for example, Python 3.12 and arm64 vs x64). The helper Lambda also needs permission to write the zip file to your S3 bucket.

Step-by-Step Guide

Step 1 - Start with the Lambda code that needs a pip package

Goal: Define the target Lambda function code that imports a third-party package (example: pyjwt).

What to do: Use a Python Lambda handler similar to the following, which encodes and decodes a JWT using pyjwt.

Code:

import json
import jwt

def lambda_handler(event, context):
    secret_key = 'secret-key'

    # Data payload you want to encode
    payload = {
        'user_id': 123,
        'email': 'user@example.com'
    }

    try:
        # Encoding the payload with the secret key
        encoded_jwt = jwt.encode(payload, secret_key, algorithm='HS256')
        print(f"Encoded JWT: {encoded_jwt}")

        # Decoding the payload with the secret key
        decoded_jwt = jwt.decode(encoded_jwt, secret_key, algorithms=['HS256'])
        print(f"Decoded JWT: {decoded_jwt}")

        return {
            'statusCode': 200,
            'body': json.dumps({
                'message': 'JWT encoded and decoded successfully!',
                'encoded': encoded_jwt,
                'decoded': decoded_jwt
            })
        }

    except Exception as e:
        print(e)
        return {
            'statusCode': 500,
            'body': json.dumps({
                'message': 'An error occurred',
                'error': str(e)
            })
        }

Expected result: You have a Lambda function that requires pyjwt, but it will not run successfully until the dependency is available in the Lambda environment.

Step 2 - Create a helper Lambda that installs packages into /tmp

Goal: Use a separate Lambda function to run pip install in the Lambda environment, targeting the /tmp directory, then zip the installed files and upload them to S3.

What to do: Create a new Lambda function using the Python version that matches your target runtime (example: Python 3.12). Also match the CPU architecture (arm64 vs x64) between helper and target to avoid compatibility issues.

Use the helper script referenced here:

https://github.com/shillehbean/youtube-p2/blob/main/lambda_pip_helper.py

This helper script uses Python’s subprocess tooling to execute pip installs into /tmp. You can replace the script’s packages list with any packages you need.

Expected result: The helper Lambda is ready to install the dependency, package it, and upload a zip file to your S3 bucket.

Step 3 - Run the helper Lambda and verify the zip is in S3

Goal: Produce the dependency zip archive in S3 so it can be turned into a Lambda Layer.

What to do: Execute the helper Lambda function. Confirm it uploads the zipped dependency package to your designated S3 bucket.

Keep these considerations in mind:

Resource limits: Large packages may require increasing memory, timeout, and ephemeral storage settings in the helper Lambda configuration.
S3 and IAM permissions: Ensure the helper Lambda execution role can write to your S3 bucket, and confirm the script’s bucket configuration matches your bucket and region.

Expected result: You can see the uploaded zip file in your S3 bucket after the helper Lambda completes.

Step 4 - Create a Lambda Layer from the zip and attach it to your function

Goal: Make the pip package available to your original Lambda function via a Layer.

What to do: Download the zip from S3 and create a new Lambda Layer in the AWS Lambda console:

In the Lambda console, open Layers and select Create layer.
Upload the zip file you downloaded from S3.
Select the Compatible runtimes that match your function (example: Python 3.12).
Create the layer.

Then add the Layer to your target Lambda function:

Open the target Lambda function.
In the Layers section, click Add a layer.
Select Custom layers, choose your layer and version, and add it.

Expected result: Your Lambda function can import and use the third-party package (example: pyjwt) when you test or invoke the function.

Conclusion

You set up an AWS Lambda workflow to install pip packages (like pyjwt) by using a helper Lambda to build a zip in /tmp, uploading it to S3, and publishing it as a Lambda Layer. This keeps your function code separate from dependencies and avoids local build mismatches with the Lambda runtime.

Want to support ShillehTek and grab parts for your next build? Visit ShillehTek.com. If you want help customizing an IoT or cloud integration project, check out our consulting: https://shillehtek.com/pages/iot-consulting. You can also hire me here: https://www.upwork.com/freelancers/~017060e77e9d8a1157.