Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

refactor build tooling #114

Merged
merged 7 commits into from
Jul 31, 2019
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions .dockerignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
.git
.github
.vscode
coverage.out
local-example
tmp
15 changes: 11 additions & 4 deletions .drone.yml
Original file line number Diff line number Diff line change
Expand Up @@ -7,14 +7,15 @@ build_config: &build_config
image: golang:1
environment:
- GO111MODULE=on
- GOPROXY=https://proxy.golang.org
- GOPROXY=https://proxy.golang.org
- CGO_ENABLED=0
commands:
- go build -a -ldflags "-X main.version=${DRONE_TAG:-other} -X main.rev=${DRONE_COMMIT}"

publish_config: &publish_config
image: plugins/docker
repo: nytimes/drone-gke
dockerfile: Dockerfile
secrets:
- docker_username
- docker_password
Expand All @@ -24,7 +25,7 @@ pipeline:
image: golang:1
environment:
- GO111MODULE=on
- GOPROXY=https://proxy.golang.org
- GOPROXY=https://proxy.golang.org
- CGO_ENABLED=0
commands:
- go mod download
Expand All @@ -44,6 +45,8 @@ pipeline:

publish_develop:
<<: *publish_config
build_args:
- GCLOUD_SDK_TAG=alpine
tag:
- "develop"
when:
Expand All @@ -52,19 +55,23 @@ pipeline:

publish_latest:
<<: *publish_config
build_args:
- GCLOUD_SDK_TAG=alpine
auto_tag: true

publish_1_10:
<<: *publish_config
build_args:
- GCLOUD_SDK_TAG=217.0.0-alpine
auto_tag: true
auto_tag_suffix: "k8s-1.10"
dockerfile: Dockerfile.1.10

publish_1_11:
<<: *publish_config
build_args:
- GCLOUD_SDK_TAG=241.0.0-alpine
auto_tag: true
auto_tag_suffix: "k8s-1.11"
dockerfile: Dockerfile.1.11

slack:
image: plugins/slack
Expand Down
21 changes: 21 additions & 0 deletions .editorconfig
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
root = true
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure you want to check this in?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@oliver-nyt happy to remove, but i think projects tend to set this to prevent configurations higher up in the filesystem tree (e.g, user settings) from resulting in auto generated changes that may not be desired within the project.. here are a few selected examples from the projects listed here:

htop
qemu
SpringBoot

wdyt? keep or remove?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Your call. I personally don't like checking in user tools specific stuff like editor configs (this is for VSCode only, right?) but I see your point, it makes life of some users easier. Interesting that so many projects do it.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

editorconfig isn't specific to vscode: https://editorconfig.org/#download 😁

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.


[*]
indent_style = space
indent_size = 2
end_of_line = lf
charset = utf-8
trim_trailing_whitespace = true
insert_final_newline = true

[*.md]
trim_trailing_whitespace = false

[*.go]
indent_style = tab

[{go.mod,go.sum}]
indent_style = tab

[Makefile]
indent_style = tab
3 changes: 3 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -1 +1,4 @@
.vscode
/drone-gke
coverage.out
tmp
244 changes: 241 additions & 3 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Contributing to Drone-GKE
# Contributing to drone-gke

Drone-GKE is an open source project started by handful of developers at The New York Times and open to the entire open source community.
`drone-gke` is an open source project started by a handful of developers at The New York Times and open to the entire open source community.

We really appreciate your help!

Expand Down Expand Up @@ -28,6 +28,244 @@ Before submitting changes, please follow these guidelines:
1. Make sure code follows the ['Go Code Review Comments'](https://github.com/golang/go/wiki/CodeReviewComments).
1. Open a Pull Request.

## Development

### Workflow

The git workflow follows [GitFlow](https://datasift.github.io/gitflow/IntroducingGitFlow.html).

New features should be based on the `master` branch.

### Dependencies

#### Core

```sh
# run tests for required core dependencies
make check
```

#### Go

[Go Modules](https://blog.golang.org/using-go-modules) is used to manage dependencies.

### Build, Test, Push

#### Building the `drone-gke` executable

```sh
make drone-gke
```

#### Testing the `drone-gke` executable

```sh
make test-coverage
```

#### Building the `drone-gke` docker image

Using a custom repo name:

```sh
docker_repo_name=alice make docker-image
```

Using a custom tag:

```sh
docker_tag=alpha make docker-image
```

#### Pushing `drone-gke` docker images

> :warning: By default, `make docker-push` will attempt to push to [`nytimes/drone-gke`](https://hub.docker.com/r/nytimes/drone-gke)

Push existing image to custom repo using custom tag:

```sh
docker_repo_name=alice docker_tag=alpha make docker-push
```

### Running `drone-gke` locally

> :warning: Valid GCP service account credentials are required. See ["Creating test serivce account credentials"](#creating-test-service-account-credentials) for more details.

#### Configuring [`TOKEN`](DOCS.md#secrets)

If you created test service account credentials using ["Creating test serivce account credentials"](#creating-test-service-account-credentials), those credentials will be used by default.

If using service account credentials that you created manually:

```sh
# replace with correct path
export TOKEN="$(cat /path/to/keyfile.json)"
```

#### Configuring cluster related values

If you have an existing `gcloud` configuration for `container/cluster`, `compute/zone`, or `compute/region`, those values will be used by default.

Otherwise manually configure the following using appropriate values:

```sh
export PLUGIN_CLUSTER=dev
export PLUGIN_ZONE=us-east1-b
# -- or --
export PLUGIN_REGION=us-east1
```

By default, `PLUGIN_NAMESPACE` will be set to `drone-gke-test`.

If you'd like to use the namespace currently configured with `kubectl`:

```sh
export PLUGIN_NAMESPACE=$(kubectl config view --minify --output 'jsonpath={..namespace}')
```

Otherwise manually configure a different namespace value:

```sh
export PLUGIN_NAMESPACE=my-test-ns
```

#### Running a `drone-gke` docker image

> :warning: Assumes `TOKEN` and any cluster related parameters have already been set / exported within the current terminal session

Using sample configurations within `local-example`:

```sh
# `dry-run` and `verbose` are both enabled by default
make run

# disable via the respective `PLUGIN_*` environment variables
PLUGIN_DRY_RUN=0 PLUGIN_VERBOSE=0 make run
```

Using custom configurations with default files:

```sh
# Deploy manifest templates in my-custom-configs

# my-custom-configs
# ├── .kube.sec.yml
# ├── .kube.yml
# └── vars.json

export CONFIG_HOME=$(pwd)/my-custom-configs
make run
```

Using custom configurations with _custom_ files:

```sh
# Deploy manifest templates in my-super-custom-configs

# my-super-custom-configs
# ├── app.yml
# ├── custom-vars.json
# └── secrets.yml

export CONFIG_HOME=$(pwd)/my-super-custom-configs
export PLUGIN_SECRET_TEMPLATE=${CONFIG_HOME}/secrets.yml
export PLUGIN_TEMPLATE=${CONFIG_HOME}/app.yml
export PLUGIN_VARS="$(cat ${CONFIG_HOME}/custom-vars.json)"
make run
```

If you've built the docker image using a custom repo or tag:

```sh
export docker_repo_name=alice
export docker_tag=alpha
make run
```

To print the commands that would be executed, without _actually_ executing them:

```sh
make run -n
```

### Tips and Tricks

#### `make e2e`

`make e2e` is an alias for:

```sh
make drone-gke
make test-coverage
make docker-image
make test-resources
make run
```

Build and test the `drone-gke` binary, build a `drone-gke` docker image tagged using a custom repo name, and run that docker image against a custom test cluster:

```sh
export docker_repo_name=alice
export PLUGIN_CLUSTER=my-test-cluster
make e2e
```

#### `make release`

> :warning: By default, `make release` will attempt to push to [`nytimes/drone-gke`](https://hub.docker.com/r/nytimes/drone-gke)

`make release` is an alias for:

```sh
make drone-gke
make test-coverage
make docker-image
make docker-push
```

Build, test, and push new image using custom repo, tag values:

```sh
export docker_repo_name=alice
export docker_tag=alpha
make release
```

#### Creating test serivce account credentials

> :warning: The following requires access to an active GCP project, a GCP user account with sufficient privileges, properly configured `gcloud`, and `terraform` executables. Please review ["Creating and managing service accounts"](https://cloud.google.com/iam/docs/creating-managing-service-accounts) and the Terraform ["Google Cloud Platform Provider"](https://www.terraform.io/docs/providers/google/index.html) documentation before continuing.

##### Setup

Verify `gcloud` configuration:

```sh
# the current / active GCP project will be the owner of your test service account resources
gcloud config list
```

##### Creation and configuration

Run following to verify the test resources that will be created / configured:

```sh
make terraform/test.tfplan
```

Run the following to create / configure the test resources:

```sh
make test-resources
```

##### Clean up

Once you've finished testing, don't forget to delete these test resources:

```sh
make destroy-test-resources
```

## License

Unless otherwise noted, the Drone-GKE source files are distributed under the Apache 2.0-style license found in the LICENSE file.
Unless otherwise noted, `drone-gke` is distributed under the Apache 2.0-style license found in [LICENSE](LICENSE).
Loading