Your Web News in One Place

An Interest In:

Web News this Week

Search Archive

Some of Our Sources

View All Sources

Help Webnuz

Referal links:

Sign up for GreenGeeks web hosting
January 24, 2023 12:29 am GMT

Lambda with Oracle Layers

Overview

Recently, I came across a need to finally use Lambda in AWS. I was looking to query an Oracle database and export the Oracle table to a CSV file. While using Python I realized the Oracle client is not available in Lambda so I needed a way to add libraries and dependancies in order to execute my code. Lambda layers allows you to do this. You can add several layers to your Lambda function. This article documents how to create the Oracle Layer for Python in Lambda with the required libraries.

The main limitation that I discovered using Lambda layers is the size limit of 250 megs for all your code and libraries. Fortunately, I was just under this limitation but there are several ways to get around this if you run into this problem. One is to break your code into various micro services. The other is to to use EFS and place all of your libraries in the EFS volume and mount the volume on your Lambda function.

Building the Oracle Layer

Optional, but preferred, commission an EC2 instance so you can start from a clean environment.

Verify the Python version

python3 --version

Create your project directory

mkdir project

Create a directory called python

mkdir python

Make library directory

mkdir lib

Install cx_Oracle Python library

pip3 install cx_Oracle -t python/

Download the latest Oracle Client from https://www.oracle.com/database/technologies/instant-client/linux-x86-64-downloads.html

wget https://download.oracle.com/otn_software/linux/instantclient/218000/instantclient-basic-linux.x64-21.8.0.0.0dbru.zip -O oracli.zip

Unzip the Oracle Client into the library folder

unzip -j oracli.zip -d lib/

Ensure Libaio is insalled on the EC2 instance (it should be)

sudo yum install libaio -y

Copy Libaio to library

cp /lib64/libaio.so.1 lib/libaio.so.1

Zip up all your libraries to be used for the Lambda Layer

zip -y -r oracletable.zip python/ lib/

Copy your zipped library file to a S3 bucket so that the Lambda function can access it.

Adding Layer into Lambda

  • Copy the url to be use when creating the layer
  • Go to lambda and create a layer using AWS Lambda Layer documentation.
  • Create the Lambda function using AWS Lambda Documentation.
  • Add your code
  • Add the layer your created
  • Run the Lambda function
  • Using CloudFormations

Below is an example of the CloudFormation code to create a Lambda Layer and Lambda function.

Note: Amazon Python Power Tools Layer documentation is at https://awslabs.github.io/aws-lambda-powertools-python/2.6.0/

Role:  Type: 'AWS::IAM::Role'  Properties:    RoleName: lambda-role-name    AssumeRolePolicyDocument:      Version: "2012-10-17"      Statement:        - Effect: Allow          Principal:            Service:              - lambda.amazonaws.com              - s3.amazonaws.com          Action:            - 'sts:AssumeRole'    Path: /    Policies:      - PolicyName: lambda-policy        PolicyDocument:          Version: "2012-10-17"          Statement:# Add access to secrets manager to give access to Lambda function to use            - Effect: Allow              Action:                 - secretsmanager:GetResourcePolicy                - secretsmanager:GetSecretValue              Resource: 'arn:aws:secretsmanager:<region>:<account>:secret:<secretname>'            - Effect: "Allow"              Action:                - s3:ListBucket              Resource:                - !Sub 'arn:aws:s3:::<S3 Bucket>'            - Effect: "Allow"              Action:                - s3:GetObject                - s3:PutObject                - s3:PutObjectAcl              Resource:                - !Sub 'arn:aws:s3:::<S3 Bucket>/*'                - !Sub 'arn:aws:s3:::<S3 Bucket>/*'    ManagedPolicyArns:      - 'arn:aws:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole'      - 'arn:aws:iam::aws:policy/service-role/AWSLambdaVPCAccessExecutionRole'# Create the Oracle Layer for PythonOrcaleLayer:  Type: AWS::Lambda::LayerVersion  Properties:    CompatibleRuntimes:      - python3.7    Content:      S3Bucket: <S3 Lambda Bucket with code>      S3Key: <layer ip file>    Description: "Oracle Layer"    LayerName: oracle-layer# Create Lambda FunctionOracleFunction:  Type: AWS::Lambda::Function  Properties:    FunctionName: oracle-lambda    Handler: oraclelambda.lambda_handler    Layers:# Oracle Client Library Layer      - !Ref OracleLayer# Add Python Power Tools      - 'arn:aws:lambda:us-east-1:017000801446:layer:AWSLambdaPowertoolsPythonV2:13'    Role:      Fn::GetAtt:        - Role        - Arn    Environment:#Pass Secret Name, Target Bucket, and S3 Prefix to the Python code      Variables:        SecretName: <secret name>        TargetBucket: <target bucket>        S3Prefix: <target prefix># Location of the Python Code    Code:      S3Bucket: <S3 Lambda Bucket with code>      S3Key: <pthon code zip file>    Runtime: python3.7    Timeout: 90    TracingConfig:      Mode: Active    VpcConfig:      SecurityGroupIds: <security group id>        - !Ref SecurityGroupId      SubnetIds:        - <subnet 1>        - <subnet 2>

Whats next?

Currently, the process for updating the layers and code is very manual. Im looking to find ways to automate getting the CloudFormation templates and Lambda to get updated when new changes are made.


Original Link: https://dev.to/aws-builders/lambda-with-oracle-layers-30od

Share this article:    Share on Facebook
View Full Article

Dev To

An online community for sharing and discovering great ideas, having debates, and making friends

More About this Source Visit Dev To