Skip to content

Commit f53cc1c

Browse files
pjdufourpjdufour
committed
initial work
0 parents  commit f53cc1c

File tree

51 files changed

+1805
-0
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

51 files changed

+1805
-0
lines changed

‎.gitignore‎

Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,16 @@
1+
*.DS_Store
2+
*.swp
3+
secret
4+
secret.yml
5+
.vagrant
6+
hosts
7+
ansible.log
8+
*.pyc
9+
packer_cache
10+
output-virtualbox-ovf
11+
*.box
12+
box
13+
build
14+
builds
15+
geoserver.war
16+
geonodes.py

‎.travis.yml‎

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,13 @@
1+
---
2+
language: python
3+
python: "2.7"
4+
5+
before_install:
6+
- sudo apt-get update -qq
7+
8+
install:
9+
- pip install ansible
10+
- "printf '[defaults]\nroles_path = ansible/roles:ansible/extra_roles/common/ubuntu' > ansible.cfg"
11+
12+
script:
13+
- ansible-playbook -i ansible/inventory_localhost ansible/test.yml --syntax-check

‎README.md‎

Lines changed: 209 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,209 @@
1+
# GeoNode DevOps (geonode-devops)
2+
3+
DevOps tools for developing & deploying [GeoNode](http://geonode.org/), including [Ansible](https://www.ansible.com/), [Packer](https://www.packer.io/), [Vagrant](https://www.vagrantup.com/), and [Fabric](http://www.fabfile.org/) configuration files for building and managing Ubuntu and CentOS GeoNode boxes.
4+
5+
# Installation
6+
7+
On the control/host machine, you'll need to install [Ansible](https://www.ansible.com/), [Packer](https://www.packer.io/), [Vagrant](https://www.vagrantup.com/), and [Fabric](http://www.fabfile.org).
8+
9+
**Quick Install**
10+
11+
If you wish to use a python virtual environment on your host/control machine, be sure to install `virutalenv` and `virtualenvwrapper`. Your `~/.bash_aliases` file should look something like the following:
12+
13+
```
14+
export VIRTUALENVWRAPPER_PYTHON=/usr/bin/python
15+
export WORKON_HOME=~/.venvs
16+
source /usr/local/bin/virtualenvwrapper.sh
17+
export PIP_DOWNLOAD_CACHE=$HOME/.pip-downloads
18+
```
19+
20+
To quickly install [Ansible](https://www.ansible.com/) and [Fabric](http://www.fabfile.org), run the following:
21+
22+
```
23+
sudo apt-get install python-dev # if not already installed
24+
sudo easy_install pip # if pip is not already installed
25+
sudo pip install virtualenv virtualenvwrapper
26+
# cd into project directory
27+
sudo pip install -r requirements.txt
28+
```
29+
30+
## Ansible
31+
32+
Ansible is an agent-less provisioning tool for managing the state of machines. It is used by both [Packer](https://www.packer.io/) and [Vagrant](https://www.vagrantup.com/). By sharing a common Ansible `playbook` for configuring production machines, building test boxes via Packer, and development boxes for Vagrant, we're able to have dev-prod parity.
33+
34+
To get [Ansible](https://www.ansible.com/) follow the relevant section below. Also see http://docs.ansible.com/ansible/intro_installation.html#getting-ansible for more information.
35+
36+
#### Mac OS X & Ubuntu
37+
38+
```
39+
sudo easy_install pip # if pip is not already installed
40+
sudo pip install ansible
41+
```
42+
43+
## Packer
44+
45+
[Packer](https://www.packer.io/) can be used to build virtual machine images.
46+
47+
#### Mac OS X
48+
49+
```
50+
sudo mkdir -p /opt/packer/bin
51+
cd /opt/packer/
52+
sudo wget https://releases.hashicorp.com/packer/0.10.0/packer_0.10.0_darwin_amd64.zip
53+
sudo unzip packer_0.10.0_darwin_amd64.zip
54+
sudo mv packer /opt/packer/bin
55+
cd /usr/local/bin/
56+
sudo ln -s /opt/packer/bin/packer packer
57+
```
58+
59+
#### Ubuntu
60+
61+
```
62+
sudo mkdir -p /opt/packer/bin
63+
cd /opt/packer/
64+
sudo wget 'https://releases.hashicorp.com/packer/0.10.0/packer_0.10.0_linux_amd64.zip'
65+
sudo unzip packer_0.10.0_linux_amd64.zip
66+
sudo mv packer /opt/packer/bin
67+
cd /usr/bin/
68+
sudo ln -s /opt/packer/bin/packer packer
69+
```
70+
71+
## Fabric
72+
73+
[Fabric](http://www.fabfile.org/) provides an easy command line interface for executing remote shell commands and for transferring files between machines. Fabric is extremely useful for transferring files and managing remote servers.
74+
75+
Follow directions at http://www.fabfile.org/installing.html to install fabric or follow shortcuts below.
76+
77+
#### Mac OS X & Ubuntu
78+
79+
```
80+
sudo pip install fabric
81+
```
82+
83+
Once fabric is installed, create a `geonodes.py` file in the same directory as the `fabfile.py`. `geonodes.py` is in `.gitignore` so will not be committed. This file includes connection and other information, so that fab commands are streamlined.
84+
85+
```javascript
86+
GEONODE_INSTANCES = {
87+
"devgeonode": {
88+
"ident": "~/auth/keys/devgeonode.pem",
89+
"host": "dev.geonode.example.com",
90+
"user": "ubuntu",
91+
"type": "geoshape"
92+
},
93+
"prodgeonode": {
94+
"ident": "~/auth/keys/prodgeonode.pem",
95+
"host": "prod.geonode.example.com",
96+
"user": "ubuntu",
97+
"type": "geoshape"
98+
}
99+
}
100+
```
101+
102+
# Usage
103+
104+
Create a `secret.yml` file in the project root.
105+
106+
## Vagrant
107+
108+
To add the Centos 6.4 vagrant box to your control machine, run:
109+
110+
```
111+
vagrant box add --name "centos/6.4" http://developer.nrel.gov/downloads/vagrant-boxes/CentOS-6.4-x86_64-v20131103.box
112+
113+
```
114+
115+
To add an Ubuntu 14.04 ("Trusty") vagrant box to your control machine, run:
116+
117+
```
118+
vagrant box add ubuntu/trusty64
119+
120+
```
121+
122+
To launch the GeoNode virtual machine run:
123+
124+
```
125+
vagrant up
126+
```
127+
128+
To re-provision the machine run:
129+
130+
```
131+
vagrant provision
132+
```
133+
134+
## Packer
135+
136+
You can also use [Packer](https://www.packer.io/) to build a virtual machine.
137+
138+
To build a base box for CentOS 6.4, run:
139+
140+
```
141+
packer build -var 'ansible_playbook=ansible/centos_base.yml' -var 'ansible_secret=secret.yml' -var 'ansible_os=centos' packer/centos64.json
142+
```
143+
144+
To build a base box for Ubuntu 14.04, run:
145+
146+
```
147+
packer build -var 'ansible_playbook=ansible/ubuntu_base.yml' -var 'ansible_secret=secret.yml' -var 'ansible_os=ubuntu' packer/ubuntu1404.json
148+
```
149+
150+
#### Adding Your New Box
151+
152+
After you created your box, you can add with:
153+
154+
```
155+
vagrant box add --name "geonode_base" packer_virtualbox-ovf_virtualbox.box
156+
```
157+
158+
Add the box to the [Vagrantfile](https://github.com/pjdufour/geonode-devops/blob/master/Vagrantfile) to provision with it.
159+
160+
## Launch
161+
162+
Once the image is provisioned, ssh into the machine via:
163+
164+
```
165+
# cd into geonode-devops.git directory
166+
vagrant ssh
167+
```
168+
169+
Once in the virtual machine, run:
170+
171+
```
172+
workon geonode
173+
cd geonode
174+
paver stop
175+
paver reset_hard
176+
paver setup
177+
paver start -b 0.0.0.0:8000 # Launches Django and GeoServer. Listens to all addresses on port 8000.
178+
```
179+
180+
## Fabric
181+
182+
[Fabric](http://www.fabfile.org/) provides an easy command line interface for executing remote shell commands and for transferring files between machines. For GeoNode, Fabric can be used to import large files, `updatelayers`, manage GeoServer restart cron jobs, and backup a remote GeoNode locally.
183+
184+
To get started, change directory (`cd`) into the main fabric directory (`./fabric`) with the `fabfile.py`. When you call fab, start with `gn:geonodehost` so that the host and identity key are loaded automatically from `geonodes.py`.
185+
186+
To see a list of tasks run:
187+
188+
```
189+
fab -l
190+
```
191+
192+
To see the long description of a task run:
193+
194+
```
195+
fab -d taskname
196+
```
197+
198+
A few examples:
199+
200+
```
201+
fab gn:devgeonode,prodgeonode lsb_release
202+
fab gn:devgeonode inspect_geoshape
203+
fab gn:devgeonode restart_geoshape
204+
fab gn:prodgeonode updatelayers:t=geoshape
205+
fab gn:prodgeonode importlayers:t=geoshape,local=~/data/*.zip,drop=/opt/drop,user=admin,overwrite=1,private=1
206+
fab gn:prodgeonode addgmail_geoshape:email,password
207+
fab gn:prodgeonode cron_restart_geoserver:'00 04 * * *'
208+
fab gn:prodgeonode backup_geonode:t=geoshape,remote=/opt/backups/20150707,local=~/backups
209+
```

‎Vagrantfile‎

Lines changed: 26 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,26 @@
1+
# -*- mode: ruby -*-
2+
# vi: set ft=ruby :
3+
4+
Vagrant.configure(2) do |config|
5+
6+
config.vm.box = "ubuntu/trusty64"
7+
8+
config.vm.network "forwarded_port", guest: 8000, host: 8000
9+
config.vm.network "forwarded_port", guest: 8080, host: 8080
10+
11+
config.vm.synced_folder "~/workspaces/public/geonode.git", "/home/vagrant/geonode"
12+
13+
config.vm.provider "virtualbox" do |vb|\
14+
vb.gui = true
15+
vb.cpus = 2
16+
vb.memory = 4096
17+
end
18+
19+
config.vm.provision "ansible" do |ansible|
20+
ansible.playbook = "ansible/ubuntu_geonode.yml"
21+
ansible.host_key_checking = false
22+
ansible.verbose = "v"
23+
ansible.raw_arguments = []
24+
end
25+
26+
end

‎ansible.cfg‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,4 @@
1+
[defaults]
2+
display_skipped_hosts = False
3+
roles_path = ansible/roles:ansible/extra_roles/common/ubuntu
4+
log_path = ansible.log

‎ansible/centos_base.yml‎

Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,17 @@
1+
---
2+
3+
- name: Creates a GeoNode development virtual machine
4+
hosts: all
5+
remote_user: vagrant
6+
7+
roles:
8+
- common_networking_centos # Configure iptables for outside access
9+
- common_base_centos # Installs basic libraries, including GCC, PIP
10+
- common_java_centos # Installs Java
11+
- common_db_centos # Installs gdal & postgres, builds PostGIS from source, and creates template_postgis
12+
- common_memcached_centos
13+
- common_rabbitmq_centos # Required by celery, which is used by GeoNode
14+
- common_webserver_tomcat_centos # Installs Tomcat and Configures Service
15+
- common_webserver_django_centos # Installs dependencies, and configures uwsgi service
16+
- common_webserver_proxy_centos # Installs NGINX, and configures service
17+
# - geonode # Not ready yet.

‎ansible/extra_roles/common/centos/common_base_centos/tasks/install_easyinstall.yml‎

Lines changed: 18 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,18 @@
1+
---
2+
3+
- name: "Download ez_setup"
4+
become: yes
5+
become_user: root
6+
get_url: url="{{ URI_SOURCE.ez_setup }}" dest="/usr/src"
7+
any_errors_fatal: true
8+
9+
- name: "Install Easy Install"
10+
become: yes
11+
become_user: root
12+
shell: "{{ item }}"
13+
args:
14+
chdir: "/usr/src"
15+
with_items:
16+
- "python2.7 ez_setup.py"
17+
- "rm -r ez_setup.py setuptools*"
18+
any_errors_fatal: true

‎ansible/extra_roles/common/centos/common_base_centos/tasks/main.yml‎

Lines changed: 59 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,59 @@
1+
---
2+
3+
- name: Install Basic Libraries
4+
become: yes
5+
become_user: root
6+
yum: name={{ item }} state=latest
7+
with_items:
8+
- vim
9+
- wget
10+
- unzip
11+
- telnet
12+
- nmap
13+
- git
14+
15+
- name: Install Compiliers & Kernel Modules
16+
become: yes
17+
become_user: root
18+
yum: name={{ item }} state=latest
19+
with_items:
20+
- gcc
21+
- gcc-c++
22+
- kernel-headers
23+
- kernel-devel
24+
- dkms
25+
26+
##############
27+
# Python 2.7 should have been installed via pre_tasks
28+
29+
- name: "Check if Easy Install is Installed"
30+
stat: path=/usr/bin/easy_install-2.7
31+
register: e
32+
33+
- name: "Install Easy Install"
34+
include: install_easyinstall.yml
35+
when: not (e.stat.isreg is defined and e.stat.isreg)
36+
37+
- name: "Check if PIP is Installed"
38+
stat: path=/usr/bin/pip
39+
register: e
40+
41+
- name: "Install PIP"
42+
become: yes
43+
become_user: root
44+
shell: "easy_install-2.7 pip"
45+
args:
46+
chdir: "/usr/src"
47+
when: not (e.stat.isreg is defined and e.stat.isreg)
48+
49+
- name: "Check if virtualenv is Installed"
50+
stat: path=/usr/bin/virtualenv
51+
register: e
52+
53+
- name: "Install virtualenv"
54+
become: yes
55+
become_user: root
56+
shell: "easy_install-2.7 virtualenv"
57+
args:
58+
chdir: "/usr/src"
59+
when: not (e.stat.isreg is defined and e.stat.isreg)

‎ansible/extra_roles/common/centos/common_db_centos/handlers/main.yml‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
---
2+
3+
- name: restart postgresql-9.2
4+
become: yes
5+
become_user: root
6+
service: name=postgresql-9.2 state=restarted

0 commit comments

Comments
 (0)