Skip to content

Commit 827e7ed

Browse files
authored
cleaned up code blocks; scans conform to 1 template for sections
* initial work on sphinx docs; much left to do * first pass at docs complete; still has some warts * added requirements for readthedocs * added requirements for readthedocs * added requirements for readthedocs * cleaned up code blocks; scans conform to 1 template for sections * trying to fix code blocks not rendering on readthedocs * trying to fix code blocks not rendering on readthedocs * trying to fix code blocks not rendering on readthedocs * trying to fix code blocks not rendering on readthedocs * trying to fix code blocks not rendering on readthedocs * trying to fix code blocks not rendering on readthedocs
1 parent 8923d17 commit 827e7ed

File tree

14 files changed

+245
-166
lines changed

14 files changed

+245
-166
lines changed

docs/conf.py

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -12,6 +12,7 @@
1212
#
1313
import os
1414
import sys
15+
import sphinx_rtd_theme
1516

1617
sys.path.insert(0, os.path.abspath(".."))
1718

@@ -58,7 +59,10 @@
5859
# The theme to use for HTML and HTML Help pages. See the documentation for
5960
# a list of builtin themes.
6061
#
62+
pygments_style = "sphinx"
63+
6164
html_theme = "sphinx_rtd_theme"
65+
html_theme_path = [sphinx_rtd_theme.get_html_theme_path()]
6266

6367

6468
# Add any paths that contain custom static files (such as style sheets) here,

docs/modifications/new_wrapper.rst

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -4,17 +4,17 @@ Creating a New Wrapper Scan
44
If for whatever reason you want something other than FullScan, the process for defining a new scan is relatively simple.
55
The ``HTBScan`` is a good example.
66

7-
1. Define your new class, inheriting from :class:`luigi.WrapperTask` and use the ``inherits`` decorator to include any scan you want to utilize
7+
1. Define your new class, inheriting from **luigi.WrapperTask** and use the ``inherits`` decorator to include any scan you want to utilize
88

9-
.. code-block::
9+
.. code-block:: python
1010
1111
@inherits(SearchsploitScan, AquatoneScan, GobusterScan, WebanalyzeScan)
1212
class HTBScan(luigi.WrapperTask):
1313
...
1414
1515
2. Include all parameters needed by any of the scans passed to ``inherits``
1616

17-
.. code-block::
17+
.. code-block:: python
1818
1919
def requires(self):
2020
""" HTBScan is a wrapper, as such it requires any Tasks that it wraps. """
@@ -36,7 +36,7 @@ The ``HTBScan`` is a good example.
3636
3737
3. ``yield`` from each scan, keeping in mind that some of the parameters won't be universal (i.e. need to be removed/added)
3838

39-
.. code-block::
39+
.. code-block:: python
4040
4141
def requires(self):
4242
""" HTBScan is a wrapper, as such it requires any Tasks that it wraps. """

docs/overview/installation.rst

Lines changed: 5 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -13,7 +13,7 @@ Manual Steps
1313

1414
First, the manual steps to get cmd2 installed in a virtual environment are as follows (and shown below)
1515

16-
.. code-block::
16+
.. code-block:: console
1717
1818
apt install pipenv
1919
git clone https://github.com/epi052/recon-pipeline.git
@@ -52,12 +52,12 @@ for the auto installer to function:
5252
- derivative of debian (some tools are installed using apt)
5353

5454
The alternatives would be to manually install each tool or to modify the distro-specific portions of the commands
55-
laid out in ``recon.__init__``. For example, on Fedora, you could change the package manager from ``apt-get`` to
55+
laid out in ``recon.__init__.py``. For example, on Fedora, you could change the package manager from ``apt-get`` to
5656
``dnf`` and remove any ``apt-get`` specific options.
5757

58-
Example from ``recon-pipeline/recon/__init__.py``
58+
Example from ``recon.__init__.py``
5959

60-
.. code-block::
60+
.. code-block:: python
6161
6262
"pipenv": {
6363
"installed": False,
@@ -67,16 +67,10 @@ Example from ``recon-pipeline/recon/__init__.py``
6767
6868
would become
6969

70-
.. code-block::
70+
.. code-block:: python
7171
7272
"pipenv": {
7373
"installed": False,
7474
"dependencies": None,
7575
"commands": ["sudo dnf install -y pipenv"],
7676
},
77-
78-
79-
80-
81-
82-

docs/overview/scope.rst

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -7,7 +7,7 @@ The pipeline expects a file that describes the target's scope to be provided as
77
``--target-file`` option. The target file can consist of domains, ip addresses, and ip ranges, one per line. Ip
88
addresses and ip ranges can be mixed/matched, but domains cannot.
99

10-
.. code-block::
10+
.. code-block:: console
1111
1212
tesla.com
1313
tesla.cn
@@ -17,7 +17,7 @@ addresses and ip ranges can be mixed/matched, but domains cannot.
1717
Some bug bounty scopes have expressly verboten subdomains and/or top-level domains, for that there is the
1818
``--exempt-list`` option. The exempt list follows the same rules as the target file.
1919

20-
.. code-block::
20+
.. code-block:: console
2121
2222
shop.eu.teslamotors.com
2323
energysupport.tesla.com

recon/amass.py

Lines changed: 21 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -10,18 +10,30 @@
1010

1111
@inherits(TargetList)
1212
class AmassScan(ExternalProgramTask):
13-
""" Run amass scan to perform subdomain enumeration of given domain(s).
13+
""" Run ``amass`` scan to perform subdomain enumeration of given domain(s).
1414
15-
Expects TARGET_FILE.domains file to be a text file with one top-level domain per line.
15+
Note:
16+
Expects **TARGET_FILE.domains** file to be a text file with one top-level domain per line.
1617
17-
Commands are similar to the following
18+
Install:
19+
.. code-block:: console
1820
19-
amass enum -ip -brute -active -min-for-recursive 3 -df tesla -json amass.tesla.json
21+
sudo apt-get install -y -q amass
22+
23+
Basic Example:
24+
.. code-block:: console
25+
26+
amass enum -ip -brute -active -min-for-recursive 3 -df tesla -json amass.tesla.json
27+
28+
Luigi Example:
29+
.. code-block:: console
30+
31+
PYTHONPATH=$(pwd) luigi --local-scheduler --module recon.amass AmassScan --target-file tesla
2032
2133
Args:
2234
exempt_list: Path to a file providing blacklisted subdomains, one per line.
23-
target_file: specifies the file on disk containing a list of ips or domains *--* Required by upstream Task
24-
results_dir: specifes the directory on disk to which all Task results are written *--* Required by upstream Task
35+
target_file: specifies the file on disk containing a list of ips or domains *Required by upstream Task*
36+
results_dir: specifes the directory on disk to which all Task results are written *Required by upstream Task*
2537
"""
2638

2739
exempt_list = luigi.Parameter(default="")
@@ -83,9 +95,9 @@ class ParseAmassOutput(luigi.Task):
8395
""" Read amass JSON results and create categorized entries into ip|subdomain files.
8496
8597
Args:
86-
target_file: specifies the file on disk containing a list of ips or domains *--* Required by upstream Task
87-
exempt_list: Path to a file providing blacklisted subdomains, one per line. *--* Optional for upstream Task
88-
results_dir: specifes the directory on disk to which all Task results are written *--* Required by upstream Task
98+
target_file: specifies the file on disk containing a list of ips or domains *Required by upstream Task*
99+
exempt_list: Path to a file providing blacklisted subdomains, one per line. *Optional by upstream Task*
100+
results_dir: specifes the directory on disk to which all Task results are written *Required by upstream Task*
89101
"""
90102

91103
def requires(self):

recon/masscan.py

Lines changed: 27 additions & 15 deletions
Original file line numberDiff line numberDiff line change
@@ -14,25 +14,37 @@
1414

1515
@inherits(TargetList, ParseAmassOutput)
1616
class MasscanScan(luigi.Task):
17-
""" Run masscan against a target specified via the TargetList Task.
17+
""" Run ``masscan`` against a target specified via the TargetList Task.
1818
19-
Masscan commands are structured like the example below. When specified, --top_ports is processed and
20-
then ultimately passed to --ports.
19+
Note:
20+
When specified, ``--top_ports`` is processed and then ultimately passed to ``--ports``.
2121
22-
masscan -v --open-only --banners --rate 1000 -e tun0 -oJ masscan.tesla.json --ports 80,443,22,21 -iL tesla.ips
22+
Install:
23+
.. code-block:: console
2324
24-
The corresponding luigi command is shown below.
25+
git clone https://github.com/robertdavidgraham/masscan /tmp/masscan
26+
make -s -j -C /tmp/masscan
27+
sudo mv /tmp/masscan/bin/masscan /usr/local/bin/masscan
28+
rm -rf /tmp/masscan
2529
26-
PYTHONPATH=$(pwd) luigi --local-scheduler --module recon.masscan Masscan --target-file tesla --ports 80,443,22,21
30+
Basic Example:
31+
.. code-block:: console
32+
33+
masscan -v --open-only --banners --rate 1000 -e tun0 -oJ masscan.tesla.json --ports 80,443,22,21 -iL tesla.ips
34+
35+
Luigi Example:
36+
.. code-block:: console
37+
38+
PYTHONPATH=$(pwd) luigi --local-scheduler --module recon.masscan Masscan --target-file tesla --ports 80,443,22,21
2739
2840
Args:
2941
rate: desired rate for transmitting packets (packets per second)
3042
interface: use the named raw network interface, such as "eth0"
3143
top_ports: Scan top N most popular ports
3244
ports: specifies the port(s) to be scanned
33-
target_file: specifies the file on disk containing a list of ips or domains *--* Required by upstream Task
34-
exempt_list: Path to a file providing blacklisted subdomains, one per line. *--* Optional for upstream Task
35-
results_dir: specifies the directory on disk to which all Task results are written *--* Optional for upstream Task
45+
target_file: specifies the file on disk containing a list of ips or domains *Required by upstream Task*
46+
results_dir: specifes the directory on disk to which all Task results are written *Required by upstream Task*
47+
exempt_list: Path to a file providing blacklisted subdomains, one per line. *Optional by upstream Task*
3648
"""
3749

3850
rate = luigi.Parameter(default=defaults.get("masscan-rate", ""))
@@ -114,12 +126,12 @@ class ParseMasscanOutput(luigi.Task):
114126
""" Read masscan JSON results and create a pickled dictionary of pertinent information for processing.
115127
116128
Args:
117-
top_ports: Scan top N most popular ports *--* Required by upstream Task
118-
ports: specifies the port(s) to be scanned *--* Required by upstream Task
119-
interface: use the named raw network interface, such as "eth0" *--* Required by upstream Task
120-
rate: desired rate for transmitting packets (packets per second) *--* Required by upstream Task
121-
target_file: specifies the file on disk containing a list of ips or domains *--* Required by upstream Task
122-
results_dir: specifes the directory on disk to which all Task results are written *--* Required by upstream Task
129+
top_ports: Scan top N most popular ports *Required by upstream Task*
130+
ports: specifies the port(s) to be scanned *Required by upstream Task*
131+
interface: use the named raw network interface, such as "eth0" *Required by upstream Task*
132+
rate: desired rate for transmitting packets (packets per second) *Required by upstream Task*
133+
target_file: specifies the file on disk containing a list of ips or domains *Required by upstream Task*
134+
results_dir: specifes the directory on disk to which all Task results are written *Required by upstream Task*
123135
"""
124136

125137
def requires(self):

recon/nmap.py

Lines changed: 35 additions & 23 deletions
Original file line numberDiff line numberDiff line change
@@ -13,24 +13,30 @@
1313

1414
@inherits(ParseMasscanOutput)
1515
class ThreadedNmapScan(luigi.Task):
16-
""" Run nmap against specific targets and ports gained from the ParseMasscanOutput Task.
16+
""" Run ``nmap`` against specific targets and ports gained from the ParseMasscanOutput Task.
1717
18-
nmap commands are structured like the example below.
18+
Install:
19+
``nmap`` is already on your system if you're using kali. If you're not using kali, refer to your own
20+
distributions instructions for installing ``nmap``.
1921
20-
nmap --open -sT -sC -T 4 -sV -Pn -p 43,25,21,53,22 -oA htb-targets-nmap-results/nmap.10.10.10.155-tcp 10.10.10.155
22+
Basic Example:
23+
.. code-block:: console
2124
22-
The corresponding luigi command is shown below.
25+
nmap --open -sT -sC -T 4 -sV -Pn -p 43,25,21,53,22 -oA htb-targets-nmap-results/nmap.10.10.10.155-tcp 10.10.10.155
2326
24-
PYTHONPATH=$(pwd) luigi --local-scheduler --module recon.nmap ThreadedNmap --target-file htb-targets --top-ports 5000
27+
Luigi Example:
28+
.. code-block:: console
29+
30+
PYTHONPATH=$(pwd) luigi --local-scheduler --module recon.nmap ThreadedNmap --target-file htb-targets --top-ports 5000
2531
2632
Args:
2733
threads: number of threads for parallel nmap command execution
28-
rate: desired rate for transmitting packets (packets per second) *--* Required by upstream Task
29-
interface: use the named raw network interface, such as "eth0" *--* Required by upstream Task
30-
top_ports: Scan top N most popular ports *--* Required by upstream Task
31-
ports: specifies the port(s) to be scanned *--* Required by upstream Task
32-
target_file: specifies the file on disk containing a list of ips or domains *--* Required by upstream Task
33-
results_dir: specifes the directory on disk to which all Task results are written *--* Required by upstream Task
34+
rate: desired rate for transmitting packets (packets per second) *Required by upstream Task*
35+
interface: use the named raw network interface, such as "eth0" *Required by upstream Task*
36+
top_ports: Scan top N most popular ports *Required by upstream Task*
37+
ports: specifies the port(s) to be scanned *Required by upstream Task*
38+
target_file: specifies the file on disk containing a list of ips or domains *Required by upstream Task*
39+
results_dir: specifes the directory on disk to which all Task results are written *Required by upstream Task*
3440
"""
3541

3642
threads = luigi.Parameter(default=defaults.get("threads", ""))
@@ -127,24 +133,30 @@ def run(self):
127133

128134
@inherits(ThreadedNmapScan)
129135
class SearchsploitScan(luigi.Task):
130-
""" Run searchcploit against each nmap*.xml file in the TARGET-nmap-results directory and write results to disk.
136+
""" Run ``searchcploit`` against each ``nmap*.xml`` file in the **TARGET-nmap-results** directory and write results to disk.
137+
138+
Install:
139+
``searchcploit`` is already on your system if you're using kali. If you're not using kali, refer to your own
140+
distributions instructions for installing ``searchcploit``.
131141
132-
searchsploit commands are structured like the example below.
142+
Basic Example:
143+
.. code-block:: console
133144
134-
searchsploit --nmap htb-targets-nmap-results/nmap.10.10.10.155-tcp.xml
145+
searchsploit --nmap htb-targets-nmap-results/nmap.10.10.10.155-tcp.xml
135146
136-
The corresponding luigi command is shown below.
147+
Luigi Example:
148+
.. code-block:: console
137149
138-
PYTHONPATH=$(pwd) luigi --local-scheduler --module recon.nmap Searchsploit --target-file htb-targets --top-ports 5000
150+
PYTHONPATH=$(pwd) luigi --local-scheduler --module recon.nmap Searchsploit --target-file htb-targets --top-ports 5000
139151
140152
Args:
141-
threads: number of threads for parallel nmap command execution *--* Required by upstream Task
142-
rate: desired rate for transmitting packets (packets per second) *--* Required by upstream Task
143-
interface: use the named raw network interface, such as "eth0" *--* Required by upstream Task
144-
top_ports: Scan top N most popular ports *--* Required by upstream Task
145-
ports: specifies the port(s) to be scanned *--* Required by upstream Task
146-
target_file: specifies the file on disk containing a list of ips or domains *--* Required by upstream Task
147-
results_dir: specifies the directory on disk to which all Task results are written *--* Required by upstream Task
153+
threads: number of threads for parallel nmap command execution *Required by upstream Task*
154+
rate: desired rate for transmitting packets (packets per second) *Required by upstream Task*
155+
interface: use the named raw network interface, such as "eth0" *Required by upstream Task*
156+
top_ports: Scan top N most popular ports *Required by upstream Task*
157+
ports: specifies the port(s) to be scanned *Required by upstream Task*
158+
target_file: specifies the file on disk containing a list of ips or domains *Required by upstream Task*
159+
results_dir: specifies the directory on disk to which all Task results are written *Required by upstream Task*
148160
"""
149161

150162
def requires(self):

recon/web/aquatone.py

Lines changed: 22 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -12,24 +12,35 @@
1212
class AquatoneScan(luigi.Task):
1313
""" Screenshot all web targets and generate HTML report.
1414
15-
aquatone commands are structured like the example below.
15+
Install:
16+
.. code-block:: console
1617
17-
cat webtargets.tesla.txt | /opt/aquatone -scan-timeout 900 -threads 20
18+
mkdir /tmp/aquatone
19+
wget -q https://github.com/michenriksen/aquatone/releases/download/v1.7.0/aquatone_linux_amd64_1.7.0.zip -O /tmp/aquatone/aquatone.zip
20+
unzip /tmp/aquatone/aquatone.zip -d /tmp/aquatone
21+
sudo mv /tmp/aquatone/aquatone /usr/local/bin/aquatone
22+
rm -rf /tmp/aquatone
1823
19-
An example of the corresponding luigi command is shown below.
24+
Basic Example:
25+
``aquatone`` commands are structured like the example below.
2026
21-
PYTHONPATH=$(pwd) luigi --local-scheduler --module recon.web.aquatone AquatoneScan --target-file tesla --top-ports 1000
27+
``cat webtargets.tesla.txt | /opt/aquatone -scan-timeout 900 -threads 20``
28+
29+
Luigi Example:
30+
.. code-block:: python
31+
32+
PYTHONPATH=$(pwd) luigi --local-scheduler --module recon.web.aquatone AquatoneScan --target-file tesla --top-ports 1000
2233
2334
Args:
2435
threads: number of threads for parallel aquatone command execution
2536
scan_timeout: timeout in miliseconds for aquatone port scans
26-
exempt_list: Path to a file providing blacklisted subdomains, one per line. *--* Optional for upstream Task
27-
top_ports: Scan top N most popular ports *--* Required by upstream Task
28-
ports: specifies the port(s) to be scanned *--* Required by upstream Task
29-
interface: use the named raw network interface, such as "eth0" *--* Required by upstream Task
30-
rate: desired rate for transmitting packets (packets per second) *--* Required by upstream Task
31-
target_file: specifies the file on disk containing a list of ips or domains *--* Required by upstream Task
32-
results_dir: specifes the directory on disk to which all Task results are written *--* Required by upstream Task
37+
exempt_list: Path to a file providing blacklisted subdomains, one per line. *Optional by upstream Task*
38+
top_ports: Scan top N most popular ports *Required by upstream Task*
39+
ports: specifies the port(s) to be scanned *Required by upstream Task*
40+
interface: use the named raw network interface, such as "eth0" *Required by upstream Task*
41+
rate: desired rate for transmitting packets (packets per second) *Required by upstream Task*
42+
target_file: specifies the file on disk containing a list of ips or domains *Required by upstream Task*
43+
results_dir: specifes the directory on disk to which all Task results are written *Required by upstream Task*
3344
"""
3445

3546
threads = luigi.Parameter(default=defaults.get("threads", ""))

0 commit comments

Comments
 (0)