Skip to content

Conversation

aritosteles
Copy link
Contributor

@aritosteles aritosteles commented Jul 7, 2025

Description

To port SCA from 6.x to 5.x we need to replace the YAML parsing library used so we avoid adding new external dependencies to 5.x. This replacement needs only to be done for the 5.x migration.

Results and Evidence

YAML Document is loaded and policy parsed successfully:

image

Tests pass:

YamlDocument and YamlNode tests:

 ctest -R YamlWrapperTest --verbose
UpdateCTestConfiguration  from :/home/ariel/Documents/wazuh-agent/src/build/DartConfiguration.tcl
UpdateCTestConfiguration  from :/home/ariel/Documents/wazuh-agent/src/build/DartConfiguration.tcl
Test project /home/ariel/Documents/wazuh-agent/src/build
Constructing a list of tests
Done constructing a list of tests
Updating test list for fixtures
Added 0 tests to meet fixture requirements
Checking test dependency graph...
Checking test dependency graph end
test 39
    Start 39: YamlWrapperTest

39: Test command: /home/ariel/Documents/wazuh-agent/src/build/modules/sca/tests/yaml_wrapper_test
39: Test timeout computed to be: 10000000
39: Running main() from /home/ariel/Documents/vcpkg/buildtrees/gtest/src/v1.15.2-41f5afb119.clean/googletest/src/gtest_main.cc
39: [==========] Running 15 tests from 1 test suite.
39: [----------] Global test environment set-up.
39: [----------] 15 tests from YamlWrapperTest
39: [ RUN      ] YamlWrapperTest.YamlDocDefaultConstructor
39: [       OK ] YamlWrapperTest.YamlDocDefaultConstructor (0 ms)
39: [ RUN      ] YamlWrapperTest.YamlDocLoadInvalidString
39: [       OK ] YamlWrapperTest.YamlDocLoadInvalidString (0 ms)
39: [ RUN      ] YamlWrapperTest.YamlDocLoadValidString
39: [       OK ] YamlWrapperTest.YamlDocLoadValidString (0 ms)
39: [ RUN      ] YamlWrapperTest.YamlDocGetRoot
39: [       OK ] YamlWrapperTest.YamlDocGetRoot (0 ms)
39: [ RUN      ] YamlWrapperTest.YamlNodeSubscriptOperatorAndGetNodeType
39: [       OK ] YamlWrapperTest.YamlNodeSubscriptOperatorAndGetNodeType (0 ms)
39: [ RUN      ] YamlWrapperTest.YamlNodeIs_Scalar_Sequence_Map
39: [       OK ] YamlWrapperTest.YamlNodeIs_Scalar_Sequence_Map (0 ms)
39: [ RUN      ] YamlWrapperTest.YamlNodeHasKey
39: [       OK ] YamlWrapperTest.YamlNodeHasKey (0 ms)
39: [ RUN      ] YamlWrapperTest.YamlNodeAsString
39: [       OK ] YamlWrapperTest.YamlNodeAsString (0 ms)
39: [ RUN      ] YamlWrapperTest.YamlNodeAsMap
39: [       OK ] YamlWrapperTest.YamlNodeAsMap (0 ms)
39: [ RUN      ] YamlWrapperTest.YamlNodeRemoveKey
39: [       OK ] YamlWrapperTest.YamlNodeRemoveKey (0 ms)
39: [ RUN      ] YamlWrapperTest.YamlNodeAsSequence
39: [       OK ] YamlWrapperTest.YamlNodeAsSequence (0 ms)
39: [ RUN      ] YamlWrapperTest.YamlNodeAppendToSequence
39: [       OK ] YamlWrapperTest.YamlNodeAppendToSequence (0 ms)
39: [ RUN      ] YamlWrapperTest.YamlNodeSetScalarValue
39: [       OK ] YamlWrapperTest.YamlNodeSetScalarValue (0 ms)
39: [ RUN      ] YamlWrapperTest.YamlNodeSequenceCreation
39: [       OK ] YamlWrapperTest.YamlNodeSequenceCreation (0 ms)
39: [ RUN      ] YamlWrapperTest.YamlNodeClone
39: [       OK ] YamlWrapperTest.YamlNodeClone (0 ms)
39: [----------] 15 tests from YamlWrapperTest (0 ms total)
39: 
39: [----------] Global test environment tear-down
39: [==========] 15 tests from 1 test suite ran. (0 ms total)
39: [  PASSED  ] 15 tests.
1/1 Test #39: YamlWrapperTest ..................   Passed    0.00 sec

The following tests passed:
        YamlWrapperTest

100% tests passed, 0 tests failed out of 1

PolicyParser tests are mostly the same as existed in 6.x except for a couple that became irrelevant.

PolicyParserTest:

ctest -R PolicyParserTest --verbose
UpdateCTestConfiguration  from :/home/ariel/Documents/wazuh-agent/src/build/DartConfiguration.tcl
UpdateCTestConfiguration  from :/home/ariel/Documents/wazuh-agent/src/build/DartConfiguration.tcl
Test project /home/ariel/Documents/wazuh-agent/src/build
Constructing a list of tests
Done constructing a list of tests
Updating test list for fixtures
Added 0 tests to meet fixture requirements
Checking test dependency graph...
Checking test dependency graph end
test 38
    Start 38: SCAPolicyParserTest

38: Test command: /home/ariel/Documents/wazuh-agent/src/build/modules/sca/tests/sca_policy_parser_test
38: Test timeout computed to be: 10000000
38: Running main() from /home/ariel/Documents/vcpkg/buildtrees/gtest/src/v1.15.2-41f5afb119.clean/googletest/src/gtest_main.cc
38: [==========] Running 10 tests from 1 test suite.
38: [----------] Global test environment set-up.
38: [----------] 10 tests from PolicyParserTest
38: [ RUN      ] PolicyParserTest.InvalidYamlFileNotThrows
38: [       OK ] PolicyParserTest.InvalidYamlFileNotThrows (0 ms)
38: [ RUN      ] PolicyParserTest.YamlSequenceIsValid
38: [       OK ] PolicyParserTest.YamlSequenceIsValid (0 ms)
38: [ RUN      ] PolicyParserTest.YamlMapIsValid
38: [       OK ] PolicyParserTest.YamlMapIsValid (0 ms)
38: [ RUN      ] PolicyParserTest.ConstructorExtractsVariables
38: [       OK ] PolicyParserTest.ConstructorExtractsVariables (0 ms)
38: [ RUN      ] PolicyParserTest.MissingPolicyReturnsNullopt
38: [       OK ] PolicyParserTest.MissingPolicyReturnsNullopt (0 ms)
38: [ RUN      ] PolicyParserTest.EmptyRequirementsReturnsNullopt
38: [       OK ] PolicyParserTest.EmptyRequirementsReturnsNullopt (0 ms)
38: [ RUN      ] PolicyParserTest.MissingChecksReturnsNullopt
38: [       OK ] PolicyParserTest.MissingChecksReturnsNullopt (0 ms)
38: [ RUN      ] PolicyParserTest.InvalidConditionReturnsNullopt
38: [       OK ] PolicyParserTest.InvalidConditionReturnsNullopt (0 ms)
38: [ RUN      ] PolicyParserTest.InvalidRuleIsHandledGracefully
38: [       OK ] PolicyParserTest.InvalidRuleIsHandledGracefully (0 ms)
38: [ RUN      ] PolicyParserTest.YamlNodeToJsonParsesMapWithSequenceValues
38: [       OK ] PolicyParserTest.YamlNodeToJsonParsesMapWithSequenceValues (0 ms)
38: [----------] 10 tests from PolicyParserTest (0 ms total)
38: 
38: [----------] Global test environment tear-down
38: [==========] 10 tests from 1 test suite ran. (0 ms total)
38: [  PASSED  ] 10 tests.
1/1 Test #38: SCAPolicyParserTest ..............   Passed    0.00 sec

The following tests passed:
        SCAPolicyParserTest

100% tests passed, 0 tests failed out of 1

Full set of agent tests:

Test project /home/ariel/Documents/wazuh-agent/src/build
      Start 38: SCAPolicyParserTest
      Start  1: byteArrayHelperTests
      Start  2: cmdHelperTests
      Start  3: sysInfoPackagesLinuxHelper_unit_test
 1/61 Test  #1: byteArrayHelperTests .....................   Passed    0.00 sec
      Start  4: sysInfoPackagesBerkeleyDB_unit_test
 2/61 Test #38: SCAPolicyParserTest ......................   Passed    0.00 sec
      Start  5: sysInfoNetworkLinux_unit_test
 3/61 Test  #3: sysInfoPackagesLinuxHelper_unit_test .....   Passed    0.00 sec
      Start  6: sysInfoRpm_unit_test
 4/61 Test  #4: sysInfoPackagesBerkeleyDB_unit_test ......   Passed    0.00 sec
      Start  7: sysInfoPackageLinuxParserRPM_unit_test
 5/61 Test  #5: sysInfoNetworkLinux_unit_test ............   Passed    0.00 sec
      Start  8: sysinfo_unit_test
 6/61 Test  #2: cmdHelperTests ...........................   Passed    0.00 sec
      Start  9: sysInfoPackages_unit_test
 7/61 Test  #6: sysInfoRpm_unit_test .....................   Passed    0.00 sec
      Start 10: sysInfoPort_unit_test
 8/61 Test  #7: sysInfoPackageLinuxParserRPM_unit_test ...   Passed    0.00 sec
      Start 11: dbsync_unit_test
 9/61 Test  #9: sysInfoPackages_unit_test ................   Passed    0.00 sec
      Start 12: dbsyncPipelineFactory_unit_test
10/61 Test #10: sysInfoPort_unit_test ....................   Passed    0.00 sec
      Start 13: dbengine_unit_test
11/61 Test  #8: sysinfo_unit_test ........................   Passed    0.01 sec
      Start 14: fim_integration_test
12/61 Test #13: dbengine_unit_test .......................   Passed    0.00 sec
      Start 15: Filesystem
13/61 Test #15: Filesystem ...............................   Passed    0.00 sec
      Start 16: FileIO
14/61 Test #12: dbsyncPipelineFactory_unit_test ..........   Passed    0.01 sec
      Start 17: globHelperTests
15/61 Test #16: FileIO ...................................   Passed    0.00 sec
      Start 18: hashHelperTests
16/61 Test #17: globHelperTests ..........................   Passed    0.00 sec
      Start 19: jsonHelperTests
17/61 Test #19: jsonHelperTests ..........................   Passed    0.00 sec
      Start 20: linuxHelperTests
18/61 Test #18: hashHelperTests ..........................   Passed    0.01 sec
      Start 21: LoggerTest
19/61 Test #21: LoggerTest ...............................   Passed    0.00 sec
      Start 22: mapWrapperTests
20/61 Test #22: mapWrapperTests ..........................   Passed    0.00 sec
      Start 23: pipelineHelperTests
21/61 Test #20: linuxHelperTests .........................   Passed    0.02 sec
      Start 24: sqliteWrapperTests
22/61 Test #23: pipelineHelperTests ......................   Passed    0.01 sec
      Start 25: stringHelperTests
23/61 Test #25: stringHelperTests ........................   Passed    0.00 sec
      Start 26: threadDispatcherTests
24/61 Test #26: threadDispatcherTests ....................   Passed    0.00 sec
      Start 27: timeHelperTests
25/61 Test #27: timeHelperTests ..........................   Passed    0.01 sec
      Start 28: SCATest
26/61 Test #28: SCATest ..................................   Passed    0.00 sec
      Start 29: SCAPolicyLoaderTest
27/61 Test #29: SCAPolicyLoaderTest ......................   Passed    0.00 sec
      Start 30: SCAEventHandlerTest
28/61 Test #30: SCAEventHandlerTest ......................   Passed    0.01 sec
      Start 31: SCAUtilsTest
29/61 Test #31: SCAUtilsTest .............................   Passed    0.00 sec
      Start 32: CheckConditionEvaluatorTest
30/61 Test #32: CheckConditionEvaluatorTest ..............   Passed    0.00 sec
      Start 33: RuleCreationTest
31/61 Test #33: RuleCreationTest .........................   Passed    0.00 sec
      Start 34: FileRuleEvaluatorTest
32/61 Test #34: FileRuleEvaluatorTest ....................   Passed    0.00 sec
      Start 35: CommandRuleEvaluatorTest
33/61 Test #35: CommandRuleEvaluatorTest .................   Passed    0.00 sec
      Start 36: DirectoryRuleEvaluatorTest
34/61 Test #36: DirectoryRuleEvaluatorTest ...............   Passed    0.00 sec
      Start 37: ProcessRuleEvaluatorTest
35/61 Test #37: ProcessRuleEvaluatorTest .................   Passed    0.00 sec
      Start 39: ModuleManagerTest
36/61 Test #24: sqliteWrapperTests .......................   Passed    0.04 sec
      Start 40: AgentInfoTest
37/61 Test #40: AgentInfoTest ............................   Passed    0.00 sec
      Start 41: AgentInfoPersistenceTest
38/61 Test #11: dbsync_unit_test .........................   Passed    0.08 sec
      Start 42: CentralizedConfiguration
39/61 Test #41: AgentInfoPersistenceTest .................   Passed    0.00 sec
      Start 43: CommandHandlerTest
40/61 Test #42: CentralizedConfiguration .................   Passed    0.00 sec
      Start 44: CommandStoreTest
41/61 Test #44: CommandStoreTest .........................   Passed    0.00 sec
      Start 45: CommunicatorTest
42/61 Test #39: ModuleManagerTest ........................   Passed    0.03 sec
      Start 46: ConfigParserTest
43/61 Test #46: ConfigParserTest .........................   Passed    0.00 sec
      Start 47: ConfigParserUtilsTest
44/61 Test #47: ConfigParserUtilsTest ....................   Passed    0.00 sec
      Start 48: HttpClientTest
45/61 Test #48: HttpClientTest ...........................   Passed    0.00 sec
      Start 49: HttpSocketTest
46/61 Test #49: HttpSocketTest ...........................   Passed    0.00 sec
      Start 50: HttpsSocketTest
47/61 Test #50: HttpsSocketTest ..........................   Passed    0.01 sec
      Start 51: InstanceCommunicatorTest
48/61 Test #14: fim_integration_test .....................   Passed    0.21 sec
      Start 52: MultiTypeQueueTest
49/61 Test #52: MultiTypeQueueTest .......................   Passed    0.00 sec
      Start 53: StorageTest
50/61 Test #53: StorageTest ..............................   Passed    0.00 sec
      Start 54: SQLiteManager_test
51/61 Test #54: SQLiteManager_test .......................   Passed    0.06 sec
      Start 55: TaskManagerTest
52/61 Test #55: TaskManagerTest ..........................   Passed    0.05 sec
      Start 56: RestartHandler
53/61 Test #43: CommandHandlerTest .......................   Passed    2.01 sec
      Start 57: AgentTest
54/61 Test #57: AgentTest ................................   Passed    0.00 sec
      Start 58: AgentEnrollmentTest
55/61 Test #58: AgentEnrollmentTest ......................   Passed    0.00 sec
      Start 59: SignalHandlerTest
56/61 Test #59: SignalHandlerTest ........................   Passed    0.00 sec
      Start 60: MessageQueueUtilsTest
57/61 Test #60: MessageQueueUtilsTest ....................   Passed    0.00 sec
      Start 61: InstanceHandlerTest
58/61 Test #61: InstanceHandlerTest ......................   Passed    0.00 sec
59/61 Test #51: InstanceCommunicatorTest .................   Passed    7.00 sec
60/61 Test #45: CommunicatorTest .........................   Passed    8.01 sec
61/61 Test #56: RestartHandler ...........................   Passed   11.01 sec

100% tests passed, 0 tests failed out of 61

Total Test time (real) =  11.35 sec

@aritosteles aritosteles self-assigned this Jul 7, 2025
@aritosteles aritosteles force-pushed the enhancement/30599-replace-yaml-cpp-with-libyaml branch from 7d01940 to e828afd Compare July 7, 2025 13:20
Copy link
Member

@jr0me jr0me left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A few notes on const correctness and removal of writes to standard output. We should add tests since it's a lot of code to cover.

try
{
if (!IsValidYamlFile(filename.string()))
std::cout << "*** PolicyParser(" << filename.c_str() << ")\n";
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should remove all couts.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

{
std::vector<std::string> values;
for (const auto& item : yamlNode)
auto items = yamlNode.AsSequence();
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can be const

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

if (root.HasKey("variables"))
{
for (const auto& var : variables)
auto variablesNode = root["variables"];
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can be const

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

{
std::vector<std::string> values;
for (const auto& item : yamlNode)
auto items = yamlNode.AsSequence();
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can be const?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

try
{
if (!IsValidYamlFile(filename.string()))
std::cout << "*** PolicyParser(" << filename.c_str() << ")\n";
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There shouldn't be any couts

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Those were in a separate commit ready to be removed, done

return nullptr;
}

std::cout << "*** Policy parsed successfully\n";
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There a few more couts, can we remove them?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

#include <filesystem>
#include <string>

class YamlNode;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this forward declaration needed?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, it isn´t. Removed.

Comment on lines 21 to 23
YamlDocument()
: m_loaded(false) {};
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we move this to the source file?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

Comment on lines 33 to 37
bool IsValidDocument() const
{
return m_loaded;
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we move this to the source file?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

Comment on lines 31 to 52
YamlNode() = default;

/// @brief Gets the type of the YAML node
/// @return Type of the node
Type GetNodeType() const
{
return m_type;
}

/// @brief Gets the type of the YAML node as a string
/// @return String representation of the node type
std::string GetNodeTypeAsString() const
{
switch (m_type)
{
case Type::Scalar: return "Scalar";
case Type::Sequence: return "Sequence";
case Type::Mapping: return "Mapping";
case Type::Undefined: // fallthrough
default: return "Undefined";
}
}

/// @brief Checks if the YAML node is a scalar
/// @return True if the node is a scalar, false otherwise
bool IsScalar() const
{
return m_type == Type::Scalar;
}

/// @brief Checks if the YAML node is a sequence
/// @return True if the node is a sequence, false otherwise
bool IsSequence() const
{
return m_type == Type::Sequence;
}

/// @brief Checks if the YAML node is a map
/// @return True if the node is a map, false otherwise
bool IsMap() const
{
return m_type == Type::Mapping;
}

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Move to src file please.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

@aritosteles aritosteles force-pushed the enhancement/30599-replace-yaml-cpp-with-libyaml branch 4 times, most recently from 03e6414 to a703eff Compare July 8, 2025 00:22
@aritosteles aritosteles force-pushed the enhancement/30599-replace-yaml-cpp-with-libyaml branch 2 times, most recently from 25c0516 to 86a66d4 Compare July 8, 2025 00:36
@aritosteles aritosteles force-pushed the enhancement/30599-replace-yaml-cpp-with-libyaml branch from 86a66d4 to bf23261 Compare July 8, 2025 01:10
@aritosteles aritosteles requested a review from jr0me July 8, 2025 01:13
@aritosteles aritosteles force-pushed the enhancement/30599-replace-yaml-cpp-with-libyaml branch from b59153b to 6838ef0 Compare July 8, 2025 19:32
@jr0me jr0me merged commit 7e8df06 into enhancement/30270-migrate-sca-6x-code-to-5x-and-compile Jul 8, 2025
0 of 7 checks passed
@jr0me jr0me deleted the enhancement/30599-replace-yaml-cpp-with-libyaml branch July 8, 2025 19:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants