Skip to content

Commit

Permalink
PHP: S3 Express Basics scenario
Browse files Browse the repository at this point in the history
  • Loading branch information
beqqrry-aws authored and ford-at-aws committed Oct 24, 2024
1 parent 5718285 commit 417118d
Show file tree
Hide file tree
Showing 22 changed files with 2,100 additions and 187 deletions.
62 changes: 62 additions & 0 deletions .doc_gen/metadata/ec2_metadata.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -1570,6 +1570,19 @@ ec2_DescribeInstances:
- aws-cli.bash-linux.ec2.aws_cli_error_log
services:
ec2: {DescribeInstances}
ec2_DescribeRouteTables:
languages:
PHP:
versions:
- sdk_version: 3
github: php/example_code/ec2
sdkguide:
excerpts:
- description:
snippet_tags:
- php.example_code.ec2.service.describeRouteTables
services:
ec2: {DescribeRouteTables}
ec2_DescribeRegions:
languages:
Rust:
Expand Down Expand Up @@ -1971,6 +1984,16 @@ ec2_CreateSubnet:
ec2: {CreateSubnet}
ec2_CreateVpc:
languages:
PHP:
versions:
- sdk_version: 3
github: php/example_code/ec2
sdkguide:
excerpts:
- description:
snippet_tags:
- php.example_code.ec2.service.createVpc

Ruby:
versions:
- sdk_version: 3
Expand All @@ -1982,6 +2005,45 @@ ec2_CreateVpc:
- ec2.Ruby.createVpc
services:
ec2: {CreateVpc}
ec2_DeleteVpc:
languages:
PHP:
versions:
- sdk_version: 3
github: php/example_code/ec2
sdkguide:
excerpts:
- description:
snippet_tags:
- php.example_code.ec2.service.deleteVpc
services:
ec2: {DeleteVpc}
ec2_CreateVpcEndpoint:
languages:
PHP:
versions:
- sdk_version: 3
github: php/example_code/ec2
sdkguide:
excerpts:
- description:
snippet_tags:
- php.example_code.ec2.service.createVpcEndpoint
services:
ec2: {CreateVpcEndpoint}
ec2_DeleteVpcEndpoint:
languages:
PHP:
versions:
- sdk_version: 3
github: php/example_code/ec2
sdkguide:
excerpts:
- description:
snippet_tags:
- php.example_code.ec2.service.deleteVpcEndpoint
services:
ec2: {DeleteVpcEndpoint}
ec2_DescribeIamInstanceProfileAssociations:
languages:
.NET:
Expand Down
35 changes: 35 additions & 0 deletions .doc_gen/metadata/s3_metadata.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -434,6 +434,15 @@ s3_DeleteObject:
snippet_tags:
- gov2.workflows.s3.ObjectLock.S3Actions.struct
- gov2.workflows.s3.ObjectLock.DeleteObject
PHP:
versions:
- sdk_version: 3
github: php/example_code/s3
sdkguide:
excerpts:
- description:
snippet_tags:
- php.example_code.s3.service.deleteObject
Python:
versions:
- sdk_version: 3
Expand Down Expand Up @@ -3458,3 +3467,29 @@ s3_Scenario_DeleteAllObjects:
- javascriptv3/example_code/s3/scenarios/delete-all-objects.js
services:
s3: {DeleteObjects, ListObjectsV2}
s3_Scenario_ExpressBasics:
title: Learn the basics of Amazon S3 Express One Zone with an &AWS; SDK
title_abbrev: Learn the basics of S3 Express One Zone
synopsis_list:
- Set up a VPC and VPC Endpoint
- Set up the S3 Express Policies, Roles, and User to work with S3 Express buckets
- Create two S3 Clients
- Create two buckets
- Create an object and copy it over
- Demonstrate performance difference
- Populate the buckets to show the lexicographical difference
- Prompt the user to see if they want to clean up the resources
category: Basics
languages:
PHP:
versions:
- sdk_version: 3
github: php/example_code/s3
sdkguide:
excerpts:
- description:
snippet_tags:
- php.example_code.s3.ExpressBasics
- php.example_code.s3.service.S3Service
services:
s3: {CreateVpc, DescribeRouteTables, CreateVpcEndpoint, CreateBucket, CopyObject, GetObject, PutObject, ListObjects, DeleteObject, DeleteBucket, DeleteVpcEndpoint, DeleteVpc}
5 changes: 3 additions & 2 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -7,8 +7,9 @@
.idea
*.iml
.metadata
.phpunit.result.cache
.recommenders
.phpunit.result.cache
.phpunit.cache
.swiftpm
.venv
.vs
Expand All @@ -34,4 +35,4 @@ rust_dev_preview
kotlin/services/**/build/
kotlin/services/**/gradle/
kotlin/services/**/gradlew
kotlin/services/**/gradlew.bat
kotlin/services/**/gradlew.bat
47 changes: 47 additions & 0 deletions basics_scenarios/s3express_scenario/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,47 @@
# Amazon S3 Batch Basics Scenario

## Introduction
This scenario demonstrates how to use the AWS SDK to interact with S3 Express One Zone operations.

## Setting up Resources
To successfully run this basic scenario, the program requires an IAM (Identity and Access Management) role. The program creates the IAM role by using an AWS CloudFormation template.

## Service Operations Invoked
The program performs the following tasks:

1. **Set up a VPC and VPC Endpoint**:
- EC2 Operation: `CreateVpc`
- EC2 Operation: `DescribeRouteTables`
- EC2 Operation: `CreateVpcEndpoint`

2. **Set up the S3 Express Policies, Roles, and User to work with S3 Express buckets**:
- CDK Operation to load a CloudFormation template

3. **Create *two* S3 Clients**:
- S3 instantiate client, once with elevated Express permissions

4. **Create two buckets**:
- S3 Operation`CreateBucket`

5. **Create an object and copy it over**:
- S3 Operation`CreateSession`
- S3 Operation`PutObject`
- S3 Operation`CopyObject`

6. **Demonstrate performance difference**:
- S3 Operation`GetObject`

7. **Populate the buckets to show the lexicographical difference**:
- S3 Operation `PutObject`
- S3 Operation `ListObjectsV2`

8. **Prompt the user to see if they want to clean up the resources **:
- S3 Operation `DeleteObject`
- S3 Operation `DeleteBucket`
- EC2 Operation `DeleteVpcEndpoint`
- EC2 Operation `DeleteVpc`

## Usage
1. Clone the repository or download the source code.
2. Open the code in your preferred IDE.
3. Invoke the main method in the `S3ExpressBasics` class.
144 changes: 144 additions & 0 deletions basics_scenarios/s3express_scenario/SPECIFICATION.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,144 @@
# S3 Express One Zone Basics Scenario Specification

## Overview
This SDK Basics scenario demonstrates how to interact with S3 Express One Zone (S3 Express) using the AWS SDK. It demonstrates various tasks such as creating a Directory bucket, using sessions for fast connect times, and downloading objects. Finally this scenario demonstrates how to clean up resources.

## Resources and User Input
This scenario requires a {bunch of stuff}.
This program is intended for users not familiar with S3 Express to easily get up and running.

## Scenario Program Flow

The S3 Express Basics scenario follows these steps:

1. **Set up a VPC and VPC Endpoint**:
- Explain to the user that running this code locally will show a lower performance increase than running it on an EC2 instance in the same AZ as the Directory bucket.
- Prompt the user for whether they are running the code locally or on an EC2 instance. If they are running it locally, skip the rest of step 1.
- Using the EC2 Client, create a VPC.
- Get the Route Table for the VPC using `DescribeRouteTables`.
- Using the EC2 client, create a VPC Endpoint with the created VPC and set the Route Table.
- **Exception Handling**: There are no unique errors for these operations.

2. **Set up the S3 Express Policies, Roles, and User to work with S3 Express buckets**:
- Create the policies, roles, and user.
- Use STS to assume the role and create S3 Express credentials.
- **Exception Handling**: Check for `InvalidArgumentException`.

3. **Create *two* S3 Clients**:
- Create a regular S3 Client using normal methods.
- Create an S3 Client with the created credentials.
- **Exception Handling**: Check for `InvalidArgumentException`.

4. **Create two buckets**:
- Create a regular S3 bucket with the normal S3 Client.
- Create a directory bucket *also using the normal S3 Client*.
- **Exception Handling**: Check for `BucketAlreadyExists`.

5. **Create an object and copy it over**:
- Create a basic object with text in the normal S3 bucket with the normal client.
- Use CopyObject to copy the object from the normal bucket to the directory bucket, using the S3 Express client.
- **Exception Handling**: No relevant unique errors for this operation.

6. **Demonstrate performance difference**:
- Use GetObject to download the object X (1000) times using the S3 Express client. Measure how long it takes. If possible in the language being used, run these in parallel.
- Use GetObject to download the object X (1000) times using the normal S3 client. Measure how long it takes. If possible in the language being used, run these in parallel.
- **Exception Handling**: Check for `NoSuchKey`.

7. **Populate the buckets to show the lexicographical difference**:
- In both buckets, add:
- `other/objectkey`, `alt/objectkey`, `other/alt/objectkey`
- Use `ListObjects` for each bucket and show the difference in ordering.
- **Exception Handling**: Check for `NoSuchBucket`.

8. **Prompt the user to see if they want to clean up the resources**:
- **Description**: If the user chose to do so, clean up relevant resources. Let the user know if any resources could not be deleted.
- **Exception Handling**: Any relevant errors related to resource deletion.

### Program execution
The following shows the output of the Amazon S3 Batch program in the console.

```
--------------------------------------
Welcome to the Amazon S3 Express Basics demo using PHP!
--------------------------------------
Let's get started! First, please not that S3 Express One Zone works best when working within the AWS infrastructure,
specifically when working in the same Availability Zone. To see the best results in this example, and when you implement
Directory buckets into your infrastructure, it is best to put your Compute resources in the same AZ as your Directory
bucket.
1. First, we'll set up a new VPC and VPC Endpoint if this program is running in an EC2 instance in the same AZ as your Directory buckets will be.
Skipping the VPC setup. Don't forget to use this in production!
2. Policies, users, and roles with CDK.
Now, we'll set up some policies, roles, and a user. This user will only have permissions to do S3 Express One Zone actions.
3. Create an additional client using the credentials with S3 Express permissions.
This client is created with the credentials associated with the user account with the S3 Express policy attached, so it can perform S3 Express operations.
All the roles and policies were created an attached to the user. Then, a new S3 Client and Service were created using that user's credentials.
We can now use this client to make calls to S3 Express operations. Keeping permissions in mind (and adhering to least-privilege) is crucial to S3 Express.
3. Create two buckets.
Now we will create a Directory bucket, which is the linchpin of the S3 Express One Zone service.
Directory buckets behave in different ways from regular S3 buckets, which we will explore here.
We'll also create a normal bucket, put an object into the normal bucket, and copy it over to the Directory bucket.
Now, let's create the actual Directory bucket, as well as a regular bucket.
Great! Both buckets were created.
5. Create an object and copy it over.
We'll create a basic object consisting of some text and upload it to the normal bucket.
Next, we'll copy the object into the Directory bucket using the regular client.
This works fine, because Copy operations are not restricted for Directory buckets.
It worked! It's important to remember the user permissions when interacting with Directory buckets.
Instead of validating permissions on every call as normal buckets do, Directory buckets utilize the user credentials and session token to validate.
This allows for much faster connection speeds on every call. For single calls, this is low, but for many concurrent calls, this adds up to a lot of time saved.
6. Demonstrate performance difference.
Now, let's do a performance test. We'll download the same object from each bucket 1000 times and compare the total time needed. Note: the performance difference will be much more pronounced if this example is run in an EC2 instance in the same AZ as the bucket.
The directory bucket took 2464126625 nanoseconds, while the normal bucket took 1989507125.
That's a difference of -474619500 nanoseconds, or -0.4746195 seconds.
7. Populate the buckets to show the lexicographical difference.
Now let's explore how Directory buckets store objects in a different manner to regular buckets.
The key is in the name "Directory!"
Where regular buckets store their key/value pairs in a flat manner, Directory buckets use actual directories/folders.
This allows for more rapid indexing, traversing, and therefore retrieval times!
The more segmented your bucket is, with lots of directories, sub-directories, and objects, the more efficient it becomes.
This structural difference also causes ListObjects to behave differently, which can cause unexpected results.
Let's add a few more objects with layered directories as see how the output of ListObjects changes.
Directory bucket content
other/basic-text-object
other/alt/basic-text-object
basic-text-object
alt/basic-text-object
Normal bucket content
alt/basic-text-object
basic-text-object
other/alt/basic-text-object
other/basic-text-object
Notice how the normal bucket lists objects in lexicographical order, while the directory bucket does not. This is because the normal bucket considers the whole "key" to be the object identifies, while the directory bucket actually creates directories and uses the object "key" as a path to the object.
That's it for our tour of the basic operations for S3 Express One Zone.
```

## SOS Tags

The following table describes the metadata used in this SDK Getting Started Scenario.


| action | metadata file | metadata key |
|--------------------------------|--------------------------------|------------------------------------------|
| `CreateVpc` | ec2_metadata.yml | ec2_CreateVpc |
| `DescribeRouteTables` | ec2_metadata.yml | ec2_DescribeRouteTables |
| `CreateVpcEndpoint` | ec2_metadata.yml | ec2_CreateVpcEndpoint |
| `CreateBucket` | s3_metadata.yml | s3_CreateBucket |
| `CopyObject` | s3_metadata.yml | s3_CopyObject |
| `GetObject` | s3_metadata.yml | s3_GetObject |
| `PutObject` | s3_metadata.yml | s3_PutObject |
| `ListObjects` | s3_metadata.yml | s3_ListObjectsV2 |
| `DeleteObject` | s3_metadata.yml | s3_DeleteObject |
| `DeleteBucket` | s3_metadata.yml | s3_DeleteBucket |
| `DeleteVpcEndpoint` | ec2_metadata.yml | ec2_DeleteVpcEndpoint |
| `DeleteVpc` | ec2_metadata.yml | ec2_DeleteVpc |
| ------------------------------ | ------------------------------ | ---------------------------------------- |
5 changes: 5 additions & 0 deletions php/example_code/aws_utilities/TestableReadline.php
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,11 @@ function testable_readline($prompt)
return readline($prompt);
}

// This is a helper function which makes it easy to ask the user to press enter to continue.
function pressEnter(){
testable_readline("Press enter to continue.\n");
}

trait TestableReadline {
function testable_readline($prompt)
{
Expand Down
1 change: 1 addition & 0 deletions php/example_code/composer.json
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,7 @@
"DynamoDb\\": "dynamodb/",
"DynamoDb\\Basics\\": "dynamodb/dynamodb_basics",
"DynamoDb\\PartiQL_Basics\\": "dynamodb/partiql_basics",
"Ec2\\": "ec2/",
"Glue\\": "glue/",
"Iam\\": "iam/",
"Kms\\": "kms/",
Expand Down
12 changes: 6 additions & 6 deletions php/example_code/composer.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Loading

0 comments on commit 417118d

Please sign in to comment.