Skip to content

Commit 9f83d89

Browse files
authored
Update aws.md
1 parent 1cee396 commit 9f83d89

File tree

1 file changed

+39
-12
lines changed

1 file changed

+39
-12
lines changed

cloud/aws.md

Lines changed: 39 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -246,21 +246,48 @@ the UW community. The following is working notes on recreating the S3 pseudo-dri
246246

247247
- ran `sudo apt update` and `sudo apt -y upgrade` in preparation
248248
- `sudo apt -y install s3cmd` followed by `s3cmd --version` > 2.4.0 so far so good
249-
- Intermezzo: Using `s3cmd`: How to copy an AWS S3 bucket to a Kopah bucket
250-
- `s3cmd --configure` produces ten fill-in-the-blank questions:
251-
- Access Key: *Note: Not in use in favor of bucket-specific keys*
252-
- Secret Key:
253-
- Default Region: US
254-
- S3 Endpoint: s3.amazonaws.com
255-
- DNS-style bucket+hostname:port template for accessing a bucket: %(bucket)s.s3.amazonaws.com
256-
- Encryption password:
257-
- Path to GPG program: /usr/bin/gpg *Note: This is the correct path to the Gnu Privacy Guard utility*
258-
- Use HTTPS protocol: True
259-
- HTTP Proxy server name:
260-
- HTTP Proxy server port: 0
249+
250+
251+
#### Intermezzo: Using `s3cmd`
252+
253+
254+
How to copy an AWS S3 bucket to a Kopah bucket
255+
256+
257+
- `s3cmd --configure` produces ten fill-in-the-blank questions:
258+
- Access Key, Secret Key, Default Region (US), S3 Endpoint (s3.amazonaws.com)
259+
- DNS-style bucket+hostname:port template for accessing a bucket: %(bucket)s.s3.amazonaws.com
260+
- Encryption password
261+
- Path to GPG program: /usr/bin/gpg *Note: This is the correct path to the Gnu Privacy Guard utility*
262+
- Use HTTPS protocol: True
263+
- HTTP Proxy server name:
264+
- HTTP Proxy server port: 0
261265
- `s3cmd sync s3://<aws-source>/ s3://<kopah-destination> --add-header "x-amz-copy-source: s3://<aws-source>"`
262266

263267

268+
The configuration file associated with `s3cmd` is called ~/.s3cfg with contents as follows:
269+
270+
271+
```
272+
host_base = s3.kopah.uw.edu
273+
host_bucket = s3.kopah.uw.edu/%(spatialtranscriptomic)
274+
use_https = True
275+
public_url_use_https = True
276+
# Login credentials
277+
access_key = Q301NARGYDO93WP90QFK
278+
secret_key = asdfowiQEFQEF243562SDFGDwertertQWQER
279+
```
280+
281+
The keys are examples.
282+
283+
- Empty a bucket and delete it: `s3cmd rb s3://spatialtranscriptomic --recursive --force`
284+
- Make a bucket: `s3cmd mb s3://spatialtranscriptomic`
285+
- List contents: `s3cmd ls; s3cmd ls s3://spatialtranscriptomic`
286+
- Synchronize a local directory `./` with a kopah bucket: `s3cmd sync ./ s3://spatialtranscriptomic`
287+
- This can take some time to apparently latch (5 mins?) when `./` is an AWS S3 bucket mounted via `mount-s3` (see above)
288+
289+
290+
264291
## aspirations
265292

266293

0 commit comments

Comments
 (0)