Design and development of the communicator component for the new agent

[TomasTurina]

Description

As part of the development of the new agent MVP, it is necessary to develop a new communicator module.

Take into account the following:

• Use the POC developed as a model to develop this MVP.
• Integrate with the wazuh-http-request library.
• The module should be able to:
  • Register with the manager. The uuid generation will be done during the registration process.
  • Request a token (authenticate) from the manager.
  • Send stateless and stateful messages to the manager.
  • Receive commands from the manager.
• Use the concurrency control module for task scheduling.

For more considerations (including API endpoints), see the spike issue.

[sdvendramini]

The branch test/25-server-mock was added to work with communicator module.

The server mock acepts:

/authentication

• Request

curl -X POST \
  http://localhost:8080/authentication \
  -H "Content-Type: application/json" \
  -H "Accept: application/json" \
  -H "Accept-Charset: utf-8" \
  -d '{"uuid":"agent_uuid","password":"123456"}'

• Response (json not yet defined)

eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXUyJ9.eyJpc3MiOiJzb21lLWF1dGgtc2VydmVyIiwic2FtcGxlIjoidGVzdCJ9.E6b_onD9_9-1rFwBVZN1I62yoy14iHXGmYKJ_RXJ-bM

/agents

• Request

curl -X POST \
  http://localhost:8080/agents \
  -H "Content-Type: application/json" \
  -H "Accept: application/json" \
  -H "Accept-Charset: utf-8" \
  -H "Authorization: Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXUyJ9.eyJpc3MiOiJzb21lLWF1dGgtc2VydmVyIiwic2FtcGxlIjoidGVzdCJ9.E6b_onD9_9-1rFwBVZN1I62yoy14iHXGmYKJ_RXJ-bM" \
  -d  '{"uuid":"agent_uuid","name":"agent_name"}'

• Response (json not yet defined)

agent_key

/stateless

• Request

curl -X POST \
  http://localhost:8080/stateless \
  -H "Content-Type: application/json" \
  -H "Accept: application/json" \
  -H "Accept-Charset: utf-8" \
  -H "Authorization: Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXUyJ9.eyJpc3MiOiJzb21lLWF1dGgtc2VydmVyIiwic2FtcGxlIjoidGVzdCJ9.E6b_onD9_9-1rFwBVZN1I62yoy14iHXGmYKJ_RXJ-bM" \
  -d  '{"uuid":"agent_uuid","event":{"events":[{"event_id":1,"event_data":"data1"},{"event_id":2,"event_data":"data2"}]}}'

• Response (json not yet defined)

Stateless post request received

/commands

• Request

curl -X GET \
  http://localhost:8080/commands?uuid=agent_uuid \
  -H "Content-Type: application/json" \
  -H "Accept: application/json" \
  -H "Accept-Charset: utf-8" '
  -H "Authorization: Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXUyJ9.eyJpc3MiOiJzb21lLWF1dGgtc2VydmVyIiwic2FtcGxlIjoidGVzdCJ9.E6b_onD9_9-1rFwBVZN1I62yoy14iHXGmYKJ_RXJ-bM" -i

• Response (json not yet defined)

HTTP/1.1 200 OK
Server: Boost.Beast
Content-Type: json
Content-Length: 327

{
    "commands":
    [
        {
            "origin":
            {
                "module": "upgrade_module"
            },
            "command": "upgrade_update_status",
            "parameters":
            {
                "agents":
                [
                    20
                ],
                "error": 0,
                "data": "Upgrade Successful",
                "status": "Done"
            }
        },
        {
            "origin":
            {
                "module": "upgrade_module"
            },
            "command": "upgrade_update_status",
            "parameters":
            {
                "agents":
                [
                    20
                ],
                "error": 0,
                "data": "Upgrade Successful",
                "status": "Done"
            }
        }
    ]
}

OR

HTTP/1.1 408 Request Timeout
Server: Boost.Beast
Content-Type: json
Content-Length: 0

[aritosteles]

Update

07/02/24

First look at the issue. Evaluating possible repo layouts.

07/03/24

Created working branch:
https://github.com/wazuh/wazuh-agent/tree/enhancement/25-communicator-module-mvp
Started laying out the repo structure and creating CMakeLists.txt for the main project and child projects.

07/04/24

Modified makefiles and repo structure for Agent and Communicator module.
Investigating dependency management with vcpkg.
Investigating wazuh-http-request.

07/05/24

Project's repository structure and CMakeLists.txt have been modified to allow building the main project and each child project in standalone mode.
After a meeting with @juliancnn where he showed the engine repo structure and the use of CPM to manage dependencies, I have included CPM in Agent project and I am working on modifying wazuh-http-request CMakeLists.txt to remove vcpkg so that all dependencies can be managed by CPM in the main Agent project.

07/08/24

Researched CPM capabilities and dependency packages availability.
Discarded CPM, cleaned up CMakeLists.txt of all references to CPM.
Copied wazuh-http-request into repository.

07/10/24

Removed wazuh-http-request since we will use Boost instead.
Done minor fixes.

07/11/24

Collaborated in preparing first PR.

07/12/24

Worked on solving review issues.
Squashed related commits to try and have a cleaner history.

07/16/24

Still working on the commits to clean up history. Ran into some problems running standalone build of the modules due to vcpkg setup.

07/17/24

Solved issue with vcpkg setup and CMakeLists. Working on rebasing the branch to accommodate recent merges.

07/18/24

Completed the rebase to master branch.

07/19/24

Fixed review comments to PR.

07/22/24

Rebased branch again due to new merges. Worked on Communicator module. Created thread to keep the token valid.

07/23/24

Worked on the task manager. Added an exit flag for the tasks to return.
Continued working on the token validity task.
Started working on integrating the config parser.

07/24/24

Configuration parser: Added default config file creation.
Communicator: Added manager_ip and port settings from config file.
TaskManager: implemented use of condition_variable.

07/25/24

PR: Addressed all review comments.
Communicator-Configuration Parser: Implemented a dependency injection solution to decouple the agent, the configuration parser and the modules. Some work left to do if the solution is approved.
Communicator-TaskManager: Better encapsulated the synchronization internals by moving code into a member function of the communicator.

07/26/24

Communicator-Configuration Parser: Communicator now receives only a pointer to a function to get configuration settings.
Communicator-TaskManager: Further enhanced encapsulation of the synchronization internals and implemented token renewal function as a coroutine.

07/29/24

Addressed various issues in PR.

07/30/24

PR: Various fixes
CommandDispatcher: added module

07/31/24

Worked on the coroutine to get commands from manager. It was decided that this should be part of the communicator itself so command dispatcher module was removed.

08/01/24

Finished work on retrieving commands from the manager. PR is ready for review.

08/02/24

Fixex issues on PR. Added uuid to communicator.

[sdvendramini]

Update

07/03/24

I caught up with the progress of the issue and we started to analyze how to reorganize cmakefiles.

07/04/24

We continue with the reorganization of the repository structure. I start to modify the PoC server mock to use with the new agent.

07/08/24

Researched CPM capabilities and dependency packages availability.

07/10/24

Dependencies with vcpkg added and first request and respective test with boost and gtest added. Also two more endpoint was modified in server mock.

07/11/24

Registration endpoint added along with minor fixes. We create first PR.

07/12/24

Worked on solving review issues.
Squashed related commits to try and have a cleaner history.

07/15/24

I still work on cleaning up history.

07/16/24

Still working on the commits to clean up history. Also working on setup debugger on new environment

07/26/24

Work on the design of the agent's registration through cli.

07/29/24

Still working on the design of the agent's registration through cli.

07/30/24

Working on the solution of the agent's resitration through cli: https://github.com/wazuh/wazuh-agent/tree/enhancement/25-communicator-module-mvp-register-cli

07/31/24

Still working on the solution of the agent's resitration through cli: https://github.com/wazuh/wazuh-agent/tree/enhancement/25-communicator-module-mvp-register-cli

08/01/24

I have created a Draf PR with the changes and we are working to bring all the work together.

08/02/24

The PR was opened, reviewed, corrected and merged

08/05/24

The PR was corrected and merged.