Skip to content

Commit 3a83e63

Browse files
cmckeencmckeen
authored
ATEAM-10574 update docs for local testing (meltwater#173)
Fixes meltwater#99
1 parent f3d3321 commit 3a83e63

File tree

4 files changed

+314
-5
lines changed

4 files changed

+314
-5
lines changed

‎.drone-local.yml

+245
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,245 @@
1+
---
2+
3+
# This is an example of a Drone pipeline that can be run locally
4+
kind: pipeline
5+
name: local-pipeline
6+
7+
platform:
8+
os: linux
9+
arch: amd64
10+
11+
clone:
12+
depth: 50
13+
14+
steps:
15+
# This is needed for the tags. And the tags are needed to determine version.
16+
- name: fetch
17+
image: docker:git
18+
commands:
19+
- git fetch --tags -f
20+
21+
- name: configure-buckets
22+
image: minio/mc:RELEASE.2020-10-03T02-54-56Z
23+
commands:
24+
- sleep 5
25+
- mc config host add minio http://minio:9000 AKIAIOSFODNN7EXAMPLE wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY
26+
- mc mb --region=eu-west-1 minio/drone-cache-bucket
27+
- mc admin user add minio foo barbarbar
28+
- "echo '{\"Version\": \"2012-10-17\", \"Statement\": [ { \"Action\": [ \"s3:GetObject\", \"s3:PutObject\", \"s3:DeleteObject\", \"s3:CreateBucket\", \"s3:DeleteBucket\" ], \"Effect\": \"Allow\", \"Resource\": [ \"arn:aws:s3:::s3-round-trip-with-role/*\", \"arn:aws:s3:::s3-round-trip-with-role\" ], \"Sid\": \"\" } ] }' >> /tmp/policy.json"
29+
- mc admin policy add minio userpolicy /tmp/policy.json
30+
- mc admin policy set minio userpolicy user=foo
31+
32+
- name: build
33+
image: golang:1.14.4-alpine3.12
34+
commands:
35+
- apk add --update make git
36+
- make drone-cache
37+
environment:
38+
CGO_ENABLED: 0
39+
40+
- name: lint
41+
image: golang:1.14.4-alpine3.12
42+
commands:
43+
- apk add --update make git curl
44+
- make lint
45+
environment:
46+
CGO_ENABLED: 0
47+
48+
- name: test
49+
image: golang:1.14.4-alpine3.12
50+
commands:
51+
- go test -mod=vendor -short -cover -tags=integration ./...
52+
environment:
53+
CGO_ENABLED: 0
54+
TEST_S3_ENDPOINT: minio:9000
55+
TEST_GCS_ENDPOINT: http://fakegcs:4443/storage/v1/
56+
TEST_STORAGE_EMULATOR_HOST: fakegcs:4443
57+
TEST_SFTP_HOST: sftp
58+
TEST_AZURITE_URL: azurite:10000
59+
volumes:
60+
- name: testdata
61+
path: /drone/src/tmp/testdata/cache
62+
63+
- name: rebuild-cache
64+
image: drone-cache:MyTestTag
65+
pull: if-not-exists
66+
settings:
67+
bucket: drone-cache-bucket
68+
mount:
69+
- vendor
70+
rebuild: true
71+
region: eu-west-1
72+
path_style: true
73+
endpoint: minio:9000
74+
exit_code: true
75+
environment:
76+
AWS_ACCESS_KEY_ID: AKIAIOSFODNN7EXAMPLE
77+
AWS_SECRET_ACCESS_KEY: wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY
78+
79+
- name: rebuild-cache-with-key
80+
image: drone-cache:MyTestTag
81+
pull: if-not-exists
82+
settings:
83+
bucket: drone-cache-bucket
84+
cache_key: "{{ .Repo.Name }}_{{ checksum \"go.mod\" }}_{{ checksum \"go.sum\" }}_{{ arch }}_{{ os }}"
85+
mount:
86+
- vendor
87+
rebuild: true
88+
region: eu-west-1
89+
path_style: true
90+
endpoint: minio:9000
91+
exit_code: true
92+
environment:
93+
AWS_ACCESS_KEY_ID: AKIAIOSFODNN7EXAMPLE
94+
AWS_SECRET_ACCESS_KEY: wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY
95+
96+
- name: rebuild-cache-with-gzip
97+
image: drone-cache:MyTestTag
98+
pull: if-not-exists
99+
settings:
100+
archive_format: gzip
101+
bucket: drone-cache-bucket
102+
cache_key: gzip
103+
mount:
104+
- vendor
105+
rebuild: true
106+
region: eu-west-1
107+
path_style: true
108+
endpoint: minio:9000
109+
exit_code: true
110+
environment:
111+
AWS_ACCESS_KEY_ID: AKIAIOSFODNN7EXAMPLE
112+
AWS_SECRET_ACCESS_KEY: wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY
113+
114+
- name: rebuild-cache-with-filesystem
115+
image: drone-cache:MyTestTag
116+
pull: if-not-exists
117+
settings:
118+
archive_format: gzip
119+
backend: filesystem
120+
cache_key: volume
121+
mount:
122+
- vendor
123+
rebuild: true
124+
exit_code: true
125+
volumes:
126+
- name: cache
127+
path: /tmp/cache
128+
129+
- name: restore-cache-with-key
130+
image: drone-cache:MyTestTag
131+
pull: if-not-exists
132+
settings:
133+
bucket: drone-cache-bucket
134+
cache_key: "{{ .Repo.Name }}_{{ checksum \"go.mod\" }}_{{ checksum \"go.sum\" }}_{{ arch }}_{{ os }}"
135+
mount:
136+
- vendor
137+
region: eu-west-1
138+
pull: always
139+
restore: true
140+
path_style: true
141+
endpoint: minio:9000
142+
exit_code: true
143+
environment:
144+
AWS_ACCESS_KEY_ID: AKIAIOSFODNN7EXAMPLE
145+
AWS_SECRET_ACCESS_KEY: wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY
146+
147+
- name: restore-cache-with-gzip
148+
image: drone-cache:MyTestTag
149+
pull: if-not-exists
150+
settings:
151+
archive_format: gzip
152+
bucket: drone-cache-bucket
153+
cache_key: gzip
154+
mount:
155+
- vendor
156+
region: eu-west-1
157+
restore: true
158+
path_style: true
159+
endpoint: minio:9000
160+
exit_code: true
161+
environment:
162+
AWS_ACCESS_KEY_ID: AKIAIOSFODNN7EXAMPLE
163+
AWS_SECRET_ACCESS_KEY: wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY
164+
165+
- name: restore-cache-with-filesystem
166+
image: drone-cache:MyTestTag
167+
pull: if-not-exists
168+
settings:
169+
archive_format: gzip
170+
backend: filesystem
171+
cache_key: volume
172+
mount:
173+
- vendor
174+
restore: true
175+
exit_code: true
176+
volumes:
177+
- name: cache
178+
path: /tmp/cache
179+
180+
- name: restore-cache-debug
181+
image: drone-cache:MyTestTag
182+
pull: if-not-exists
183+
settings:
184+
debug: true
185+
restore: true
186+
pull: always
187+
188+
- name: restore-cache
189+
image: drone-cache:MyTestTag
190+
pull: if-not-exists
191+
settings:
192+
bucket: drone-cache-bucket
193+
mount:
194+
- vendor
195+
region: eu-west-1
196+
restore: true
197+
path_style: true
198+
endpoint: minio:9000
199+
exit_code: true
200+
environment:
201+
AWS_ACCESS_KEY_ID: AKIAIOSFODNN7EXAMPLE
202+
AWS_SECRET_ACCESS_KEY: wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY
203+
204+
services:
205+
- name: minio
206+
image: minio/minio:RELEASE.2020-11-06T23-17-07Z
207+
commands:
208+
- minio server /data
209+
environment:
210+
MINIO_ACCESS_KEY: AKIAIOSFODNN7EXAMPLE
211+
MINIO_REGION: eu-west-1
212+
MINIO_SECRET_KEY: wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY
213+
ports:
214+
- 9000
215+
- name: fakegcs
216+
image: fsouza/fake-gcs-server:1.18.3
217+
ports:
218+
- 4443
219+
commands:
220+
- fake-gcs-server -public-host fakegcs -scheme http
221+
- name: sftp
222+
image: atmoz/sftp:alpine
223+
ports:
224+
- 22
225+
commands:
226+
- /entrypoint foo:pass:::sftp_test bar:pass:::plugin_test
227+
- name: azurite
228+
image: mcr.microsoft.com/azure-storage/azurite:3.10.0
229+
commands:
230+
- azurite-blob --blobHost 0.0.0.0
231+
ports:
232+
- 10000
233+
234+
volumes:
235+
- name: cache
236+
temp: {}
237+
- name: testdata
238+
temp: {}
239+
240+
trigger:
241+
branch:
242+
- master
243+
event:
244+
-
245+
...

‎.goreleaser-local.yml

+53
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,53 @@
1+
project_name: drone-cache
2+
before:
3+
hooks:
4+
- make clean
5+
- make vendor
6+
dist: target/dist
7+
builds:
8+
-
9+
goos:
10+
- linux
11+
goarch:
12+
- amd64
13+
14+
env:
15+
- CGO_ENABLED=0
16+
flags:
17+
-tags netgo
18+
ldflags: >
19+
-s
20+
-w
21+
-X main.Version={{.Version}}
22+
-X main.Commit={{.ShortCommit}}
23+
-X main.Date={{.Date}}
24+
hooks:
25+
# Ignores failures, this is just best effort. Alternative: find target/dist -name drone-cache -type f -exec upx {} +
26+
post: find target/dist -name drone-cache -type f -print0 | xargs -n 1 -0 upx
27+
checksum:
28+
name_template: "{{ .ProjectName }}_{{ .Version }}_checksums.txt" # default
29+
snapshot:
30+
name_template: "{{ .Tag }}-SNAPSHOT-{{.ShortCommit}}" # default
31+
dockers:
32+
-
33+
goos: linux
34+
goarch: amd64
35+
goarm: ''
36+
37+
# Modify the image template name here as desired
38+
image_templates:
39+
- "drone-cache:MyTestTag"
40+
41+
dockerfile: Dockerfile
42+
build_flag_templates:
43+
- "--label=org.label-schema.schema-version=1.0"
44+
- "--label=version={{.Version}}"
45+
- "--label=org.label-schema.name={{.ProjectName}}"
46+
- "--label=org.label-schema.version={{.Version}}"
47+
- "--label=org.label-schema.build-date={{.Date}}"
48+
- "--label=org.label-schema.vcs-ref={{.FullCommit}}"
49+
- "--label=org.label-schema.docker.dockerfile=Dockerfile"
50+
- "--label=org.opencontainers.image.name={{.ProjectName}}"
51+
- "--label=org.opencontainers.image.version={{.Version}}"
52+
- "--label=org.opencontainers.image.created={{.Date}}"
53+
- "--label=org.opencontainers.image.revision={{.FullCommit}}"

‎CHANGELOG.md

+6-5
Original file line numberDiff line numberDiff line change
@@ -7,6 +7,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
77

88
## Unreleased
99

10+
- [#99](https://github.com/meltwater/drone-cache/issues/99) Document building images and pushing locally for PR testing
1011
- [#133](https://github.com/meltwater/drone-cache/pull/133) backend/s3: Fixed Anonymous Credentials Error on public buckets.
1112
- Fixes [#132](https://github.com/meltwater/drone-cache/issues/132)
1213
- [#138](https://github.com/meltwater/drone-cache/pull/138) backend/gcs: Fix GCS to pass credentials correctly when `GCS_ENDPOINT` is not set.
@@ -20,11 +21,11 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
2021
- [#86](https://github.com/meltwater/drone-cache/pull/86) Add backend operation timeout option that cancels request if they take longer than given duration. `BACKEND_OPERATION_TIMEOUT`, `backend.operation-timeot`. Default value is `3 minutes`.
2122
- [#86](https://github.com/meltwater/drone-cache/pull/86) Customize the cache key in the path. Adds a new `remote_root` option to customize it. Defaults to `repo.name`.
2223
- Fixes [#97](https://github.com/meltwater/drone-cache/issues/97).
23-
[#89](https://github.com/meltwater/drone-cache/pull/89) Add Azure Storage Backend.
24-
[#84](https://github.com/meltwater/drone-cache/pull/84) Adds compression level option.
25-
[#77](https://github.com/meltwater/drone-cache/pull/77) Adds a new hidden CLI flag to be used for tests.
26-
[#73](https://github.com/meltwater/drone-cache/pull/73) Add Google Cloud storage support
27-
[#68](https://github.com/meltwater/drone-cache/pull/68) Introduces new storage backend, sFTP.
24+
[#89](https://github.com/meltwater/drone-cache/pull/89) Add Azure Storage Backend.
25+
[#84](https://github.com/meltwater/drone-cache/pull/84) Adds compression level option.
26+
[#77](https://github.com/meltwater/drone-cache/pull/77) Adds a new hidden CLI flag to be used for tests.
27+
[#73](https://github.com/meltwater/drone-cache/pull/73) Add Google Cloud storage support
28+
[#68](https://github.com/meltwater/drone-cache/pull/68) Introduces new storage backend, sFTP.
2829

2930
### Changed
3031

‎CONTRIBUTING.md

+10
Original file line numberDiff line numberDiff line change
@@ -64,3 +64,13 @@ the requirements below.
6464
7. Update [plugin index](https://github.com/drone/drone-plugin-index/blob/master/content/meltwater/drone-cache/index.md) using [DOCS](./DOCS.md).
6565

6666
> **Keep in mind that users usually use the `latest` tagged images in their pipeline, please make sure you do not interfere with their working workflow.**
67+
68+
## Testing Locally
69+
70+
Want to test locally without opening a PR? Follow the steps below to build a local image of drone-cache and run the Drone pipeline against it.
71+
72+
0. Make sure you have the [Drone CLI](https://docs.drone.io/cli/install/), [Docker](https://docs.docker.com/get-docker/), and [GoReleaser](https://goreleaser.com/install/) installed locally.
73+
1. Update the `image_templates` key in `drone-cache/.goreleaser-local.yml` to reflect the name you'd like your image to have, then run `goreleaser release --config=.goreleaser-local.yml --snapshot --skip-publish --rm-dist` to build the image.
74+
2. Update the `image: drone-cache:MyTestTag` entries in the `local-pipeline` pipeline in the `.drone.yml` with the name of the image that you created (there are several of these).
75+
3. Run the Drone pipeline locally with `drone exec ./.drone-local.yml --branch MyBranchName`
76+

0 commit comments

Comments
 (0)