Skip to content

Commit b2e2b52

Browse files
jcapolinojcapolino
committed
Merge branch 'feature-python3-support' into 'master'
Adding python 3 support into cb-feeds. See merge request carbonblack/integrations/libs/feed!1
2 parents d5d9257 + 2acb788 commit b2e2b52

30 files changed

+3052
-528
lines changed

‎.travis.yml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
language: python
22
python:
3-
- "2.7"
3+
- "3.6"
44
install:
55
- "pip install -r requirements.txt"
66
- "python setup.py install"

‎CHANGELOG.md

Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,15 @@
1+
CHANGELOG.md
2+
# Carbon Black EDR Alliance Feed Library Changelog
3+
4+
## v1.0.0
5+
#### Features
6+
* Converted to python3
7+
* Added handling of sha256, ja3, ja3s and query reports
8+
* Added unit tests
9+
10+
> _NOTE: Not all examples not converted at this time!_
11+
12+
## v0.8.0
13+
#### Features
14+
* Initial Release
15+

‎README.md

Lines changed: 51 additions & 25 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,6 @@
11
# Carbon Black Feeds [![Build Status](https://travis-ci.org/carbonblack/cbfeeds.svg?branch=master)](https://travis-ci.org/carbonblack/cbfeeds)
22

3+
34
## License
45

56
Use of the Carbon Black Feeds API is governed by the license found in LICENSE.md.
@@ -28,6 +29,15 @@ The CB Response 5.0+ server adds support for two new types of indicators:
2829
The CB Response 6.1+ server adds support for one new type of indicator:
2930

3031
* IPv6 addresses
32+
33+
The CB Response 7.0+ server adds support for one new type of indicator:
34+
35+
* Binary SHA-256
36+
37+
The CB Response 7.3+ server adds support for two new types of indicator:
38+
39+
* Ja3 hash
40+
* Ja3s hash
3141

3242
Please note that query IOC types have significant differences as compared to MD5s, IPv4 and IPv6 addresses, and DNS names. Please see notes below regarding their usage.
3343

@@ -36,6 +46,8 @@ easy to add support for feed data from any input source.
3646

3747
Example feed creation scripts are included. See the 'Examples' section in this document for a listing of the examples.
3848

49+
> _**NOTE:** As of this version, python 3 is a requirement._
50+
3951
## Using the Carbon Black Feeds API
4052

4153
The Carbon Black Feeds API (CBFAPI) is found on github at:
@@ -109,20 +121,22 @@ Each `report` has report metadata and a list of IOCs.
109121

110122
`feedinfo` is a JSON structure with the following entries:
111123

112-
| name | status | description |
113-
| -------------- | -------- |-------------|
114-
| `name` | REQUIRED | Internal name; must not include spaces or special characters. See Notes. |
115-
| `display_name` | REQUIRED | Display name for the user interface. |
116-
| `provider_url` | REQUIRED | Human-consumpable link to view more information about this feed. |
117-
| `summary` | REQUIRED | A short description of this feed. |
118-
| `tech_data` | REQUIRED | More detailed technical description, to include data sharing requirements (if any) |
119-
| `icon` | OPTIONAL | A base64 encoded version of the image to use in the user interface |
120-
| `icon_small` | OPTIONAL | A base64 encoded version of a smaller icon |
121-
| `category` | OPTIONAL | Category of the feed i.e. Open Source, Partner, Connector, First Party etc. |
124+
| name | status | description |
125+
| ---------------- | -------- |-------------|
126+
| `display_name` | REQUIRED | Display name for the user interface. |
127+
| `name` | REQUIRED | Internal name; must not include spaces or special characters. See Notes. |
128+
| `provider_url` | REQUIRED | Human-consumpable link to view more information about this feed. |
129+
| `summary` | REQUIRED | A short description of this feed. |
130+
| `tech_data` | REQUIRED | More detailed technical description, to include data sharing requirements (if any) |
131+
| `category` | _OPTIONAL_ | Category of the feed i.e. Open Source, Partner, Connector, First Party etc. |
132+
| `icon` | _OPTIONAL_ | A base64 encoded version of the image to use in the user interface |
133+
| `icon_small` | _OPTIONAL_ | A base64 encoded version of a smaller icon |
134+
| `provider_rating`| _OPTIONAL_ | Provider rating for the feed. |
135+
| `version` | _OPTIONAL_ | Version of the feed source. |
122136

123137
Notes:
124138

125-
The 'name' field must not include spaces or special characters. Typically, it should be unique per-feed on a single server.
139+
The 'name' field cannot not include spaces or special characters. Typically, it should be unique per-feed on a single server.
126140

127141
#### Icon
128142

@@ -136,15 +150,15 @@ Explanation of `category` parameters:
136150

137151
| Category Name | Description |
138152
| ------------- | ----------- |
139-
| `Partner` | Proprietary threat intelligence provided to the Threat Intelligence Cloud via a partner agreement. |
140-
| `Open Source` | Open Source intelligence that is generally available to the public |
153+
| `Carbon Black` | Intelligence based on output from host-based integrations |
141154
| `Carbon Black First Party` | Intelligence generated inside the Threat Intelligence Cloud by the Carbon Black Research team |
142155
| `Connectors` | Intelligence connectors from third party technologies Carbon Black have integrated with |
143-
| `Carbon Black` | Intelligence based on output from host-based integrations |
144156
| `Meta-feed` | Includes a theme-based aggregate of selected intelligence indicators from other feeds |
157+
| `Partner` | Proprietary threat intelligence provided to the Threat Intelligence Cloud via a partner agreement. |
158+
| `Open Source` | Open Source intelligence that is generally available to the public |
145159

146160

147-
An example `feedinfo` structure, from the generate_tor_feed.py script:
161+
An example `feedinfo` structure, from the `generate_tor_feed.py` script:
148162

149163
```
150164
"feedinfo": {
@@ -165,14 +179,14 @@ A `report` is a JSON structure with the following entries:
165179

166180
| name | status | description |
167181
| -------------- | -------- |-------------|
168-
| `timestamp` | REQUIRED | Time this report was last updated, in seconds since epoch (GMT). This should always be updated whenever the content of the report changes.|
169182
| `id` | REQUIRED | A report id, must be unique per feed `name` for the lifetime of the feed. Must be alphanumeric (including no spaces).|
183+
| `iocs` | REQUIRED | The IOCs for this report. A match on __any__ IOC will cause the activity to be tagged with this report id. The IOC format is described below.|
170184
| `link` | REQUIRED | Human-consumbable link to information about this report.|
171-
| `title` | REQUIRED | A one-line title describing this report.|
172185
| `score` | REQUIRED | The severity of this report from -100 to 100, with 100 most critical.|
173-
| `iocs` | REQUIRED | The IOCs for this report. A match on __any__ IOC will cause the activity to be tagged with this report id. The IOC format is described below.|
174-
| `tags` | OPTIONAL | A comma separated list of identifiers to tag the report. |
175-
| `description` | OPTIONAL | A description of the report. |
186+
| `timestamp` | REQUIRED | Time this report was last updated, in seconds since epoch (GMT). This should always be updated whenever the content of the report changes.|
187+
| `title` | REQUIRED | A one-line title describing this report.|
188+
| `description` | _OPTIONAL_ | A description of the report. |
189+
| `tags` | _OPTIONAL_ | A comma separated list of identifiers to tag the report. |
176190

177191
### iocs
178192

@@ -190,15 +204,27 @@ CB Response 6.1+ supports all 5.0 IOCs and adds one additional type:
190204

191205
* ipv6 addresses
192206

207+
The CB Response 7.0+ server adds support for one new type of indicator:
208+
209+
* Binary SHA-256
210+
211+
The CB Response 7.3+ server adds support for two new types of indicator:
212+
213+
* Ja3 hash
214+
* Ja3s hash
215+
193216
`iocs` is a structure with one or more of these entries:
194217

195218
| name | status | description |
196219
| -------------- | -------- |-------------|
197-
| `ipv4` | OPTIONAL | A list of IPv4 addresses in dotted decimal form|
198-
| `ipv6` | OPTIONAL | A list of IPv6 addresses|
199-
| `dns` | OPTIONAL | A list of domain names|
200-
| `md5` | OPTIONAL | A list of md5s|
201-
| `query` | OPTIONAL | A query of type "events" or "modules"|
220+
| `dns` | _OPTIONAL_ | A list of domain names|
221+
| `ipv4` | _OPTIONAL_ | A list of IPv4 addresses in dotted decimal form|
222+
| `ipv6` | _OPTIONAL_ | A list of IPv6 addresses|
223+
| `ja3` | _OPTIONAL_ | A list of ja3 hashes (md5)|
224+
| `ja3s` | _OPTIONAL_ | A list of ja3s hashes (md5)|
225+
| `md5` | _OPTIONAL_ | A list of md5s|
226+
| `query` | _OPTIONAL_ | A query of type "events" or "modules"|
227+
| `sha256` | _OPTIONAL_ | A list of sha-256s|
202228

203229
An example `reports` list with two `report` structures, each with one IPv4 IOC, from the example_tor.py script:
204230

‎cbfeeds/__init__.py

Lines changed: 7 additions & 15 deletions
Original file line numberDiff line numberDiff line change
@@ -1,17 +1,9 @@
1-
class CbException(Exception):
2-
pass
3-
4-
class CbIconError(CbException):
5-
pass
6-
7-
class CbInvalidFeed(CbException):
8-
pass
9-
10-
class CbInvalidReport(CbException):
11-
pass
12-
13-
from feed import CbFeed
14-
from feed import CbFeedInfo
15-
from feed import CbReport
1+
# coding: utf-8
2+
# Carbon Black EDR Copyright © 2013-2020 VMware, Inc. All Rights Reserved.
3+
################################################################################
164

5+
__all__ = ["CbFeed", "CbFeedInfo", "CbReport", "CbIconError", "CbInvalidFeed", "CbInvalidFeedInfo", "CbInvalidReport",
6+
"CbException"]
177

8+
from .exceptions import CbException, CbIconError, CbInvalidFeed, CbInvalidFeedInfo, CbInvalidReport
9+
from .feed import CbFeed, CbFeedInfo, CbReport

‎cbfeeds/exceptions.py

Lines changed: 31 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,31 @@
1+
# coding: utf-8
2+
# Carbon Black EDR Copyright © 2013-2020 VMware, Inc. All Rights Reserved.
3+
################################################################################
4+
5+
__all__ = ["CbException", "CbIconError", "CbInvalidFeed", "CbInvalidFeedInfo", "CbInvalidReport"]
6+
7+
8+
# CBFeeds Exception set
9+
class CbException(Exception):
10+
"""CBFeeds base exception class"""
11+
pass
12+
13+
14+
class CbIconError(CbException):
15+
"""Exception for icon related issues"""
16+
pass
17+
18+
19+
class CbInvalidFeed(CbException):
20+
"""Exception for problems with overall feed structure"""
21+
pass
22+
23+
24+
class CbInvalidFeedInfo(CbException):
25+
"""Exception for problems with feedinfo information"""
26+
pass
27+
28+
29+
class CbInvalidReport(CbException):
30+
"""Exception for problems with report information"""
31+
pass

0 commit comments

Comments
 (0)