Skip to content

Commit

Permalink
Initial commit
Browse files Browse the repository at this point in the history
  • Loading branch information
Davide Gallitelli committed Aug 22, 2021
1 parent 8c4db19 commit 77147c8
Show file tree
Hide file tree
Showing 24 changed files with 852 additions and 17 deletions.
1 change: 1 addition & 0 deletions CHANGELOG
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@

1 change: 1 addition & 0 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@

# Contributing Guidelines

Thank you for your interest in contributing to our project. Whether it's a bug report, new feature, correction, or additional
Expand Down
25 changes: 13 additions & 12 deletions LICENSE
Original file line number Diff line number Diff line change
@@ -1,15 +1,16 @@
Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.
MIT No Attribution

Permission is hereby granted, free of charge, to any person obtaining a copy of
this software and associated documentation files (the "Software"), to deal in
the Software without restriction, including without limitation the rights to
use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of
the Software, and to permit persons to whom the Software is furnished to do so.
Copyright (c) 2021 Amazon.com, Inc. or its affiliates. All Rights Reserved.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS
FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR
COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER
IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
Permission is hereby granted, free of charge, to any person obtaining a copy of this
software and associated documentation files (the "Software"), to deal in the Software
without restriction, including without limitation the rights to use, copy, modify,
merge, publish, distribute, sublicense, and/or sell copies of the Software, and to
permit persons to whom the Software is furnished to do so.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED,
INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A
PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT
HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE
SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
107 changes: 102 additions & 5 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,11 +1,108 @@
## My Project
# rekognition-video-face-blur-cdk-app

TODO: Fill this README out!
## About

Be sure to:
This code sample demonstrates how AWS Step Functions can be used to orchestrate AWS Lambda functions that call Amazon Rekognition for face detections, and use OpenCV to blur these detections per frame in the video. Customers can use Amazon Rekognition to enforce privacy/anonymity in videos.

* Change the title in this README
* Edit your repository description on GitHub
Original video frame | Blurred video image
:-------------------------:|:-------------------------:
![input](images/input.gif) | ![output](images/output.gif)

### Architecture

This code sample demonstrates how AWS Step Functions can be used to orchestrate AWS Lambda functions that call Amazon Rekognition for face detections, and use OpenCV to blur these detections per frame in the video.

![rekognition-video-face-blur-cdk-app Architecture](images/rekognition-video-face-blur-cdk-app.png)

### Container image support in AWS Lambda
Providing your serverless functions access to OpenCV is easier than ever with [Container Image Support](https://aws.amazon.com/blogs/aws/new-for-aws-lambda-container-image-support/). Instead of uploading a code package to AWS Lambda, your function's code instead resides in a Docker image that is hosted in [Amazon Elastic Container Registry](https://aws.amazon.com/ecr/).

```dockerfile
FROM public.ecr.aws/lambda/python:3.7
# Install the function's dependencies
# Copy file requirements.txt from your project folder and install
# the requirements in the app directory.
COPY requirements.txt .
RUN pip install -r requirements.txt
# Copy helper functions
COPY video_processor.py video_processor.py
# Copy handler function (from the local app directory)
COPY app.py .
# Overwrite the command by providing a different command directly in the template.
CMD ["app.lambda_function"]
```

### Project structure

This project contains source code and supporting files for a serverless application that you can deploy with the AWS CDK. It includes the following files and folders.

- rekognition_video_face_blurring_cdk/ - CDK python code for deploying the application
- rekopoc-apply-faces-to-video-docker/ - Code for Lambda function: uses OpenCV to blur faces per frame in video, uploads final result to output S3 bucket.
- rekopoc-check-status/ - Code for Lambda function: Gets face detection results for the Amazon Rekognition Video analysis.
- rekopoc-get-timestamps-faces/ - Code for Lambda function: Gets bounding boxes of detected faces and associated timestamps.
- rekopoc-start-face-detect/ - Code for Lambda function: is triggered by a S3 event when a new .mp4 or .mov video file is uploaded, starts asynchronous detection of faces in a stored video and starts the execution of AWS Step Functions' State Machine.
- requirements.txt - Required packages for deploying the AWS CDK application

The application uses several AWS resources, including AWS Step Functions, Lambda functions and S3 buckets. These resources are defined in the `rekognition_video_face_blurring_cdk_stack.py` that resides inside the rekognition_video_face_blurring_cdk folder in this project. You can update the python code to add AWS resources through the same deployment process that updates your application code.
### AWS Step Functions workflow
AWS Step Functions is a low-code visual workflow service used to orchestrate AWS services, automate business processes, and build serverless applications.
In this code sample, AWS Step Functions is used to orchestrate the calls and manage the flow of data between AWS Lambda functions. The AWS Step Functions workflow for this application consists of AWS Lambda functions SFN states as well as Choice, Wait, Fail and Succeed SFN states.

AWS Step Functions Workflow | AWS Step Functions (SFN) States
:-------------------------:|:-------------------------:
![rekognition-video-face-blur-cdk-app Step Functions Workflow](images/rekognition-video-face-blur-cdk-app-step-functions-graph.png) | ![rekognition-video-face-blur-cdk-app Step Functions Workflow](images/rekognition-video-face-blur-cdk-app-step-functions-graph-details.png)

## Deploy the sample application

### Deploy the AWS CDK application
The AWS Cloud Development Kit (AWS CDK) is an open source software development framework to define your cloud application resources using familiar programming languages. This project uses the AWS CDK in Python.

* AWS CDK - [Getting started with the AWS CDK](https://docs.aws.amazon.com/cdk/latest/guide/getting_started.html)
* AWS CDK on GitHub - [AWS CDK on Github - Contribute!](https://github.com/aws/aws-cdk)
* AWS CDK API Reference (for Python) - [AWS CDK Python API Reference](https://docs.aws.amazon.com/cdk/api/latest/python/modules.html)
* Docker - [Install Docker community edition](https://hub.docker.com/search/?type=edition&offering=community)

To build and deploy your application for the first time, ensure you have Docker running and have configured your AWS's credentials.
The easiest way to satisfy this requirement is to issue the following command
```bash
aws configure
```
After you have finished with these steps, run the following in your shell:

```bash
npm install -g aws-cdk
pip install -r requirements.txt
cdk bootstrap
cdk deploy
```

- The first command will install the AWS CDK Toolkit globally using Node Package Manager.
- The second command will install all the python packages needed by the AWS CDK using pip package manager.
- The third command will provision initial resources that the AWS CDK needs to perform the deployment.
These resources include an Amazon S3 bucket for storing files and IAM roles that grant permissions needed to perform deployments.
- Finally, `cdk deploy` will deploy the stack.


### Cleanup

To delete the sample application that you created, use the AWS CDK.
```bash
cdk destroy
```

## Resources

See the [Getting started with the AWS CDK](https://docs.aws.amazon.com/cdk/latest/guide/getting_started.html) for an introduction to important AWS CDK concepts and description of how to install and configure the AWS CDK.

Next, you can browse the AWS Samples/AWS CDK repository on GitHub. These examples each provide a demonstration of a common service implementation, or infrastructure pattern that could be useful in your use of the CDK for building your own infrastructure: [AWS CDK Examples](https://github.com/aws-samples/aws-cdk-examples)

## Credits

* Adrian Rosebrock, Blur and anonymize faces with OpenCV and Python, PyImageSearch,
https://www.pyimagesearch.com/2020/04/06/blur-and-anonymize-faces-with-opencv-and-python/,
accessed on 3 August 2021
* Jon Slominski, <a href=https://github.com/aws-samples/rekognition-face-blur-sam-app>aws-samples/rekognition-face-blur-sam-app</a>, accessed on 3 August 2021
* Video by <a href="https://www.pexels.com/@pixabay?utm_content=attributionCopyText&utm_medium=referral&utm_source=pexels">Pixabay</a> from <a href="https://www.pexels.com/video/video-of-people-walking-855564/?utm_content=attributionCopyText&utm_medium=referral&utm_source=pexels">Pexels</a>

## Security

Expand Down
34 changes: 34 additions & 0 deletions app.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
#!/usr/bin/env python3
import os

from aws_cdk import core as cdk

# For consistency with TypeScript code, `cdk` is the preferred import name for
# the CDK's core module. The following line also imports it as `core` for use
# with examples from the CDK Developer's Guide, which are in the process of
# being updated to use `cdk`. You may delete this import if you don't need it.
from aws_cdk import core

from stack.rekognition_video_face_blurring_cdk_stack import RekognitionVideoFaceBlurringCdkStack


app = core.App()
RekognitionVideoFaceBlurringCdkStack(app, "RekognitionVideoFaceBlurringCdkStack",
# If you don't specify 'env', this stack will be environment-agnostic.
# Account/Region-dependent features and context lookups will not work,
# but a single synthesized template can be deployed anywhere.

# Uncomment the next line to specialize this stack for the AWS Account
# and Region that are implied by the current CLI configuration.

#env=core.Environment(account=os.getenv('CDK_DEFAULT_ACCOUNT'), region=os.getenv('CDK_DEFAULT_REGION')),

# Uncomment the next line if you know exactly what Account and Region you
# want to deploy the stack to. */

#env=core.Environment(account='123456789012', region='us-east-1'),

# For more information, see https://docs.aws.amazon.com/cdk/latest/guide/environments.html
)

app.synth()
17 changes: 17 additions & 0 deletions cdk.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
{
"app": "python3 app.py",
"context": {
"@aws-cdk/aws-apigateway:usagePlanKeyOrderInsensitiveId": true,
"@aws-cdk/core:enableStackNameDuplicates": "true",
"aws-cdk:enableDiffNoFail": "true",
"@aws-cdk/core:stackRelativeExports": "true",
"@aws-cdk/aws-ecr-assets:dockerIgnoreSupport": true,
"@aws-cdk/aws-secretsmanager:parseOwnedSecretName": true,
"@aws-cdk/aws-kms:defaultKeyPolicies": true,
"@aws-cdk/aws-s3:grantWriteWithoutAcl": true,
"@aws-cdk/aws-ecs-patterns:removeDefaultDesiredCount": true,
"@aws-cdk/aws-rds:lowercaseDbIdentifier": true,
"@aws-cdk/aws-efs:defaultEncryptionAtRest": true,
"@aws-cdk/aws-lambda:recognizeVersionProps": true
}
}
Binary file added images/input.gif
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added images/output.gif
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added images/rekognition-video-face-blur-cdk-app.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
74 changes: 74 additions & 0 deletions requirements.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,74 @@
attrs==20.3.0
aws-cdk.assets==1.116.0
aws-cdk.aws-apigateway==1.116.0
aws-cdk.aws-applicationautoscaling==1.116.0
aws-cdk.aws-autoscaling==1.116.0
aws-cdk.aws-autoscaling-common==1.116.0
aws-cdk.aws-autoscaling-hooktargets==1.116.0
aws-cdk.aws-certificatemanager==1.116.0
aws-cdk.aws-cloudformation==1.116.0
aws-cdk.aws-cloudfront==1.116.0
aws-cdk.aws-cloudwatch==1.116.0
aws-cdk.aws-codebuild==1.116.0
aws-cdk.aws-codecommit==1.116.0
aws-cdk.aws-codeguruprofiler==1.116.0
aws-cdk.aws-codestarnotifications==1.116.0
aws-cdk.aws-cognito==1.116.0
aws-cdk.aws-dynamodb==1.116.0
aws-cdk.aws-ec2==1.116.0
aws-cdk.aws-ecr==1.116.0
aws-cdk.aws-ecr-assets==1.116.0
aws-cdk.aws-ecs==1.116.0
aws-cdk.aws-efs==1.116.0
aws-cdk.aws-eks==1.116.0
aws-cdk.aws-elasticloadbalancing==1.116.0
aws-cdk.aws-elasticloadbalancingv2==1.116.0
aws-cdk.aws-events==1.116.0
aws-cdk.aws-globalaccelerator==1.116.0
aws-cdk.aws-iam==1.116.0
aws-cdk.aws-kinesis==1.116.0
aws-cdk.aws-kms==1.116.0
aws-cdk.aws-lambda==1.116.0
aws-cdk.aws-lambda-event-sources==1.116.0
aws-cdk.aws-logs==1.116.0
aws-cdk.aws-route53==1.116.0
aws-cdk.aws-route53-targets==1.116.0
aws-cdk.aws-s3==1.116.0
aws-cdk.aws-s3-assets==1.116.0
aws-cdk.aws-s3-notifications==1.116.0
aws-cdk.aws-sam==1.116.0
aws-cdk.aws-secretsmanager==1.116.0
aws-cdk.aws-servicediscovery==1.116.0
aws-cdk.aws-signer==1.116.0
aws-cdk.aws-sns==1.116.0
aws-cdk.aws-sns-subscriptions==1.116.0
aws-cdk.aws-sqs==1.116.0
aws-cdk.aws-ssm==1.116.0
aws-cdk.aws-stepfunctions==1.116.0
aws-cdk.aws-stepfunctions-tasks==1.116.0
aws-cdk.cloud-assembly-schema==1.116.0
aws-cdk.core==1.116.0
aws-cdk.custom-resources==1.116.0
aws-cdk.cx-api==1.116.0
aws-cdk.lambda-layer-awscli==1.116.0
aws-cdk.lambda-layer-kubectl==1.116.0
aws-cdk.region-info==1.116.0
awscli==1.20.12
boto3==1.18.12
botocore==1.21.12
cattrs==1.6.0
colorama==0.4.3
constructs==3.3.112
docutils==0.15.2
jmespath==0.10.0
jsii==1.32.0
numpy==1.21.1
publication==0.0.3
pyasn1==0.4.8
python-dateutil==2.8.2
PyYAML==5.4.1
rsa==4.7.2
s3transfer==0.5.0
six==1.16.0
typing-extensions==3.10.0.0
urllib3==1.26.6
43 changes: 43 additions & 0 deletions setup.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,43 @@
import setuptools


with open("README.md") as fp:
long_description = fp.read()


setuptools.setup(
name="rekognition_video_face_blurring_cdk",
version="0.0.1",

description="An empty CDK Python app",
long_description=long_description,
long_description_content_type="text/markdown",

author="author",

package_dir={"": "rekognition_video_face_blurring_cdk"},
packages=setuptools.find_packages(where="rekognition_video_face_blurring_cdk"),

install_requires=[
"aws-cdk.core==1.110.1",
],

python_requires=">=3.6",

classifiers=[
"Development Status :: 4 - Beta",

"Intended Audience :: Developers",

"Programming Language :: JavaScript",
"Programming Language :: Python :: 3 :: Only",
"Programming Language :: Python :: 3.6",
"Programming Language :: Python :: 3.7",
"Programming Language :: Python :: 3.8",

"Topic :: Software Development :: Code Generators",
"Topic :: Utilities",

"Typing :: Typed",
],
)
Binary file added stack/.DS_Store
Binary file not shown.
Binary file added stack/lambdas/.DS_Store
Binary file not shown.
16 changes: 16 additions & 0 deletions stack/lambdas/rekopoc-apply-faces-to-video-docker/Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
FROM public.ecr.aws/lambda/python:3.7
# Install the function's dependencies
# Copy file requirements.txt from your project folder and install
# the requirements in the app directory.

COPY requirements.txt .
RUN pip install -r requirements.txt

# Copy helper functions
COPY video_processor.py video_processor.py

# Copy handler function (from the local app directory)
COPY app.py .

# Overwrite the command by providing a different command directly in the template.
CMD ["app.lambda_function"]
Loading

0 comments on commit 77147c8

Please sign in to comment.