The Jenkins project is a sub-project of our overarching DevOps Tool-Chain (DOTC) project. This project — and its peer projects — is designed to handle the automated deployment of common DevOps tool-chain services onto STIG-hardened, EL7-compatible Amazon EC2 instances and related AWS resources. Specifically, this project provides CloudFormation (CFn) templates for:
- Security Groups
- Simple Storage Service (S3)
- Identity and Access Management (IAM)
- "Classic" Elastic LoadBalancer
- Application LoadBalancer (a.k.a., "ELBv2")
- Standalone EC2 Jenkins Master instance
- Autoscale EC2 Jenkins Master instance
- Standalone EC2 "agent" instance
- Autoscale EC2 "agent" instance
- Parent Master Standalone Instance deployment
- Parent Master Autoscale deployment
Additionally, scripts are provided to automate the deployment of the Jenkins Master and Agent services (see the Jenkins Distributed Builds guide) onto the relevant EC2 instances - whether stand-alone or managed via AWS's AutoScaling service.
These templates are intended for use within AWS VPCs. It is further expected that the deployed-to VPCs will be configured with public and private subnets. All Jenkins elements other than the Elastic LoadBalancer(s) are expected to be deployed into private subnets. The Elastic LoadBalancers provide transit of Internet-originating web UI requests to the the Jenkins Master node's web-based management-interface.
These templates may work outside of such a networking-topology but have not been so tested.
It is generally expected that the use of the various, individual-service templates will be run either via the "parent" templates or some other control framework (e.g. Jenkins).
The included "parent" template allows for a kind of "one-button" deployment method, executed with wholly AWS native tools. The "parent" templates attempt to make it so that all the template-user needs to worry about is populating the template's fields and ensuring that CFn can find the child templates.
In order to use the "parent" template, it is recommended that the child templates be hosted in an S3 bucket separate from the one created for backups by this stack-set. The template-hosting bucket may be public or not. The files may be set to public or not. CFn typically has sufficient privileges to read the templates from a bucket without requiring the containing bucket or files being set public. Use of S3 for hosting eliminates the need to find other hosting-locations or sort out access-particulars of those hosting locations.
The EC2-related templates — either autoscale or instance — currently require that the scripts be anonymously curl-able. The scripts can still be hosted in a non-public S3 bucket, however, either:
- The scripts' file-ACLs will need to allow
public-read; or, - The scripts will need to be referenced by way of pre-signed URLs. This limitation may change in future releases — likely via an enhancement to the IAM template or fetch-method.
It will be necessary to deploy the Jenkins Master prior to deploying the Jenkins agent node(s). These templates assume the use of JNLP for configuring communications between master and agent nodes. The JNLP connection requires a unique access-token per agent that can only be generated once the master server is online.
These templates (currently) do not include Route53 functionality. It is assumed that the requisite Route53 or other DNS aliases will be configured separate from the instantiation of the public-facing ELB.
- While there are templates provided for standalone Master and Agents, they are provided mostly for "completeness". It's expected most users of these templates will desire the availability-enhancements accorded by the use of the auto-scaling templates.
- The ELBv2 template is similarly provided for "completeness".
See the docs subdirectory. It currently contains the following documents:
- ProjectLayout.md: Describes the overall layout for this git project.
- Automated Deployment with AWS Native Tooling: Collection of documents describing using (primarily) the AWS CLI to directly-deploy CloudFormation stacks from this project's templates.
- Master Server Elements
- Consolidated Deployment
- "Fresh" Deployment
- Upgrade "In Place" - OS/AMI/Instance-only
- Upgrade "In Place" - Jenkins Service
- Parallel Upgrade - OS and/or Jenkins Service
- "Template-at-a-time" Deployment
- Consolidated Deployment
- Agent Server Elements
- Master Server Elements
- Automated Deployment with Jenkins Overlays: Collection of documents describing using the included Jenkins pipeline-definitions to encapsulate the running of this project's templates.
- Miscellaneous Topics