Feature/319 refactor search suggestions

.github/workflows/common-tests.yml

@@ -0,0 +1,32 @@
+name: Common
+
+on:
+  pull_request:
+    paths:
+      - .github/workflows/common-tests.yml
+      - "packages/common/**"
+
+jobs:
+  tests:
+    name: Tests
+    runs-on: ubuntu-latest
+    timeout-minutes: 15
+    steps:
+      - uses: actions/checkout@v4
+
+      - name: Setup node
+        uses: actions/setup-node@v3
+        with:
+          node-version: 18
+
+      - run: yarn install --immutable
+        working-directory: .
+
+      - run: yarn workspace @red-kite/common test:cicd
+        working-directory: .
+
+      - uses: tanmen/jest-reporter@v1
+        if: always()
+        with:
+          github-token: ${{ secrets.GITHUB_TOKEN }}
+          result-file: ./packages/common/test/jest-output.json

.github/workflows/stalker-app-tests.yml

@@ -5,6 +5,7 @@ on:
     paths:
       - .github/workflows/frontend.yml
       - "packages/frontend/stalker-app/**"
+      - "packages/common/**"
 
 jobs:
   tests:

.vscode/settings.json

@@ -17,5 +17,5 @@
   "[scss]": {
     "editor.defaultFormatter": "esbenp.prettier-vscode"
   },
-  "typescript.preferences.importModuleSpecifier": "relative"
+  "typescript.preferences.importModuleSpecifier": "shortest"
 }

devspace.yaml

@@ -504,8 +504,8 @@ deployments:
         repo: https://charts.bitnami.com/bitnami
         version: "28.0.0"
       upgradeArgs:
-      - --timeout
-      - 1200s
+        - --timeout
+        - 1200s
       values:
         controller:
           nodeSelector: $(echo ${KAFKA_K8S_NODE_SELECTOR})
@@ -556,7 +556,7 @@ deployments:
             value: "no"
         provisioning:
           enabled: true
-          parallel: 10
+          parallel: 1
           nodeSelector: $(echo ${KAFKA_K8S_NODE_SELECTOR})
           auth:
             tls:

docs/docs/concepts/findings.md

@@ -63,6 +63,23 @@ log_finding(
 )
 ```
 
+### Attaching information to a domain
+
+Adding fields and using a custom finding type will add a finding to the domain resource.
+
+```python
+from stalker_job_sdk import DomainFinding, log_finding, TextField
+
+hostname = "example.com"
+log_finding(
+    DomainFinding(
+        "MyCustomHostnameFinding", hostname, None, "Domain info", [
+          TextField("myfield", "Field Title", "Finding data")
+        ]
+    )
+)
+```
+
 ## IpFinding
 
 An ip finding creates a new host. IP addresses are in the IPv4 format.
@@ -93,9 +110,26 @@ log_finding(
 )
 ```
 
+### Attaching information to a host
+
+Adding fields and using a custom finding type will add a finding to the host resource.
+
+```python
+from stalker_job_sdk import IpFinding, log_finding, TextField
+ip = "0.0.0.0"
+mask = 16
+log_finding(
+    IpFinding(
+      "MyCustomIpFinding", ip, "New Info", [
+          TextField("myfield", "Field Title", "Finding data")
+        ]
+    )
+)
+```
+
 ## IpRangeFinding
 
-An ip range finding creates a new ip range for a project. IP addresses are in the IPv4 format.
+An ip range finding creates a new ip range for a project. IP addresses are in the IPv4 format. `IpRangeFindings` can also be used to attach information to an IP Range resource.
 
 | Field  | Type   | Description                                                                                              |
 | ------ | ------ | -------------------------------------------------------------------------------------------------------- |
@@ -116,19 +150,32 @@ Example:
 Using the python SDK, you can emit this finding with the following code:
 
 ```python
-from stalker_job_sdk import IpFinding, log_finding
+from stalker_job_sdk import IpRangeFinding, log_finding
 ip = "0.0.0.0"
 mask = 16
 log_finding(
     IpRangeFinding(
-        ip, mask
+        'IpRangeFinding', ip, mask, None, [], "IpRangeFinding"
     )
 )
 ```
 
-> You can't attach fields to an IP range as they are different than other ressources.
+### Attaching information to an IP range
 
-Which is equivalent to the following python code, but with more metadata:
+Adding fields and using a custom finding type will add a finding to the IP range resource.
+
+```python
+from stalker_job_sdk import IpRangeFinding, log_finding, TextField
+ip = "0.0.0.0"
+mask = 16
+log_finding(
+    IpRangeFinding(
+        'IpRangeFinding', ip, mask, "Finding title", [
+            TextField("myfield", "Field Title", "Finding data")
+        ]
+    )
+)
+```
 
 ## HostnameIpFinding
 
@@ -192,7 +239,7 @@ Example:
 Using the python SDK, you can emit this finding with the following code:
 
 ```python
-from stalker_job_sdk import PortFinding, log_finding
+from stalker_job_sdk import PortFinding, log_finding, TextField
 port = 80
 ip = "1.2.3.4"
 log_finding(
@@ -208,6 +255,29 @@ log_finding(
 )
 ```
 
+### Attaching information to a port
+
+Adding fields and using a custom finding type will add a finding to the port resource.
+
+```python
+from stalker_job_sdk import PortFinding, log_finding, TextField
+port = 80
+ip = "1.2.3.4"
+log_finding(
+    PortFinding(
+        "MyCustomPortFinding",
+        ip,
+        port,
+        "tcp",
+        "New port data",
+        [
+          TextField("protocol", "This is a TCP port", "tcp"), 
+          TextField("myfield", "Field Title", "Finding data")
+        ],
+    )
+)
+```
+
 ## WebsiteFinding
 
 The `WebsiteFinding` will create a website resource. Websites are made from 4 characteristics: an IP address, a domain name, a port number
@@ -269,57 +339,74 @@ log_finding(
 )
 ```
 
+### Attaching information to a website
+
+Adding fields and using a custom finding type will add a finding to the website resource.
+
+```python
+from stalker_job_sdk import WebsiteFinding, log_finding, TextField
+port = 80
+ip = "1.2.3.4"
+domain = "example.com"
+path = "/"
+ssl = False
+
+log_finding(
+    WebsiteFinding(
+        "MyCustomWebsiteFinding",
+        ip,
+        port,
+        domain,
+        path,
+        ssl,
+        "New website data",
+        [
+          TextField("myfield", "Field Title", "Finding data")
+        ],
+    )
+)
+```
+
 ## CustomFinding
 
-Dynamic findings allow jobs to attach custom data to resources.
+Custom findings attach finding field information to a resource. There are custom findings for every type of resources. When you do not specify the _type of finding_ that you are logging, you are creating a custom finding for the associated resource type.
 
-| Field        | Description                                                     |
-| ------------ | --------------------------------------------------------------- |
-| `domainName` | The domain to which to attach the custom finding                |
-| `host`       | The host to which to attach the custom finding                  |
-| `port`       | The port to which to attach the custom finding                  |
-| `fields`     | A list of [fields](#dynamic-fields) containing the finding data |
+| SDK finding class | Resources |
+| ----------------- | --------- |
+| HostnameFinding   | Domains   |
+| IpFinding         | Hosts     |
+| IpRangeFinding    | IP ranges |
+| PortFinding       | Ports     |
+| WebsiteFinding    | Websites  |
 
-Examples:
 
-```json
-{
-  "type": "CustomFinding",
-  "host": "1.2.3.4",
-  "port": 80,
-  "fields": [
-    {
-      "type": "image",
-      "data": "data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAgAAAAIAQMAAAD+wSzIAAAABlBMVEX///+/v7+jQ3Y5AAAADklEQVQI12P4AIX8EAgALgAD/aNpbtEAAAAASUVORK5CYII"
-    }
-  ]
-}
-```
+Here is an example of a **custom finding** for a port with the python SDK. In this example, the port will show the custom information _This port
+runs an HTTP server_, with a text field attached to it:
 
-```json
-{
-  "type": "CustomFinding",
-  "domainName": "red-kite.io",
-  "fields": [
-    {
-      "type": "text",
-      "label": "Domain greatness level",
-      "data": "This domain is great, would recommend"
-    }
-  ]
-}
+```python
+from stalker_job_sdk import PortFinding, log_finding, TextField
+port = 80
+ip = "0.0.0.0"
+log_finding(
+    PortFinding(
+        "PortFunFact", ip, port, "tcp", "This is a fun fact about a port", [
+          TextField('myfieldkey', 'My field title', 'My field data')
+        ]
+    )
+)
 ```
 
-Here is an example of a custom finding for a port with the python SDK. In this example, the port will show the custom information _This port
-runs an HTTP server_:
+Notice how the key `PortFunFact` can be anything, how information is provided through `TextField`s and how the finding type is not provided to use the default value.
+
+To compare, here is an example of how to create a port with the `PortFinding` class, which here is **not** used as a custom finding. You will see that the key is `PortFinding`, no fields are provided, and the type is `PortFinding` as well:
 
 ```python
 from stalker_job_sdk import PortFinding, log_finding
 port = 80
 ip = "0.0.0.0"
 log_finding(
     PortFinding(
-        "PortFunFact", ip, port, "tcp", "This is a fun fact about a port"
+        "PortFinding", ip, port, "tcp", None, None, "PortFinding"
     )
 )
 ```

docs/docs/concepts/project.md

@@ -6,13 +6,9 @@ description: Organizing your data through projects
 
 # Projects
 
-Projects are the way to organize and centralize resources. Resources are unique per project, and deleting the project will delete all its
+Projects are the way to organize and centralize [resources](./resources.md). Resources are unique per project, and deleting the project will delete all its
 resources and related information.
 
 The project's name must be unique, but it can always be changed. You can also add a logo for display purposes, but it is not mandatory.
 
-## Subnets
-
-In the case where a target owns a public subnet, you can add the different subnets in the projects page.
-
-For a subnet of `127.0.0.1/24`, you would simply add `127.0.0.1` in the _IP Address_ field, and the `/24` in the _Short Mask_ field.
+> Red Kite allows to work on multiple projects at once, but using the global project filter in the navigation bar, you can also focus your work on a single project at a time. It will pre-filter data in displays such as tables and metrics.

docs/docs/concepts/resources.md → docs/docs/concepts/resources.mdx

@@ -4,6 +4,8 @@ title: Resources
 description: What are resources
 ---
 
+import EnterpriseNotice from "../../src/components/EnterpriseNotice";
+
 # Resources
 
 Resources represent the core entities of an exposed network. They are used to store and show the data found by
@@ -22,29 +24,40 @@ up-to-date. Want to dive deeper? Check out the section on [learn more about find
 ## Types of Resources
 
 Resources come in various types, each created by specific findings. Some findings are generated through the user interface, while others
-originate from the API. Regardless of their origin, every resource is tied to a specific project.
+originate from the API. Regardless of their origin, every resource is [tied to a specific project](./project.md).
 
 ### Domains
 
 The domains represent domain names or a subdomains, such as `example.com` or `subdomain.example.com`. They store and display DNS-related
 information and can be managed via the `Domains` page in the user interface.
 
 Domains can be created using the `HostnameFinding`, via the API. They can also be created through the user interface's `Add domains`
-functionality.
+functionality. Adding a new domain will seed the automation process and start a scan.
 
 Typically, a domain resolves to one or more IP addresses, which are represented as host resources. A domain can be linked to one or more
 hosts through the `HostnameIpFinding`. If a `HostnameIpFinding` identifies a new domain or host, it will create these resources
 automatically.
 
-Importantly, each domain's name, combined with its project identifier, must is unique within the database.
+The combination of a domain's name and its project identifier is unique in the database.
+
+
+### IP Ranges
+
+IP ranges consist in a network IP address and a network mask, and allow to designate full subnetworks as part of a project. These ranges would be owned, for instance, by your target, and they are a likely place to find relevant [hosts](#hosts). They can be found in the user interface under the `IP ranges` page.
+
+You can create an IP range by either adding it in the interface through the `Add IP ranges` capabilities, or by emitting an `IpRangeFinding` in a job. When an IP range is added, a **scan** for the range is **immediatly started**. A scan is also launched every two weeks to find new hosts and refresh data.
+
+The combination of an IP range's IP, mask and project identifier is unique in the database.
+
+> At the moment, only IPv4 addresses are supported.
 
 ### Hosts
 
 The hosts represent an exposed IP address: or a computer's network interface listening on the network. Hosts are leveraged to represent the
 links between _domains_, hosts and _ports_. They can be seen in the user interface under the `Hosts` page.
 
 A host can be created through the `IpFinding` for a standalone host, or through a `HostnameIpFinding` for a host that is linked to a
-_domain_. `IpFinding`s can be emitted by the API through the user interface's `Add hosts` capabilities.
+_domain_. `IpFinding`s can be emitted by the API through the user interface's `Add hosts` capabilities. Adding a new host will seed the automation process and start a scan.
 
 An existing host can be linked to a _domain_ through the `HostnameIpFinding`. A host can be linked to one or many domains.
 
@@ -136,8 +149,9 @@ by remembering its existence.
 
 ### Exporting Resources
 
-In Red Kite Enterprise, resources can be exported from the list views in the `JSON` or `CSV` format. The `JSON` format is recommended as it
-is more flexible than CSV, and therefore better suited to the task.
+<EnterpriseNotice />
+
+Resources can be exported from the list views in the `JSON` or `CSV` format. The `JSON` format is intended to be used by programs, while the `CSV` format is designed for humans.
 
 ### Merging Websites