Allow agent to override fleet settings

[michalpristas]

This pull request introduces support for a new "Fleet override" capability, allowing persisted configuration from a file to override configuration received from Fleet, but only if explicitly enabled by capabilities. The changes include the implementation of the new capability type, its integration into the configuration processing flow, and comprehensive unit tests to ensure correct behavior.

Opted for capabilities rather than FF or agent config option as it is clear this is set on purpose. It is not coming from fleet and cannot override itself.

Fleet override capability implementation and integration:

• Added a new capability type, fleet_override, to the capabilities system, including parsing from YAML, internal representation, and logic to determine if Fleet override is allowed (AllowFleetOverride).
• Updated the capabilitiesManager to support the new fleetOverrideCaps and wire it into the capability loading process.

Configuration override logic:

• Implemented the applyPersistedConfig function, which applies persisted configuration from a file on disk if the Fleet override capability is enabled. This function is now called during configuration processing in the coordinator.
• Added the necessary import for os to support file operations in the coordinator.

Testing:

• Added unit tests for applyPersistedConfig to verify correct merging of persisted configuration based on the Fleet override capability, including both enabled and disabled scenarios.

Manual testing:

• For manual testing build and enroll to fleet to a policy with monitoring disabled.

• Verify http server is not running with curl http://localhost:6774/liveness -I

• Update elastic-agent.yml with following content

agent:
  monitoring:
    enabled: true
    use_output: default
    logs: false
    metrics: true
    http:
      port: 6774

• Create capabilities file like so:

capabilities:
- fleet_override:
  rule: allow

• Restart agent and perform curl again curl http://localhost:6774/liveness -I

[elasticmachine]

Pinging @elastic/elastic-agent-control-plane (Team:Elastic-Agent-Control-Plane)

[swiatekm]

The logic looks fine to me but I don't like where it's located in the codebase. Ideally, this change should try to make reasoning about config loading easier, not harder.

[swiatekm]

LGTM

[elasticmachine]

💚 Build Succeeded

• Buildkite Build
• Commit: 1651558

History

• 💚 Build #31307 succeeded 4ae51b3
• 💔 Build #31306 failed df77bf7

cc @michalpristas

[cmacknz]

Are you sure you just want to log on failure? In the context of agentless, this would leave the pod running without valid configuration wouldn'it it?

How would you alert on that or gate promotion on this?

[cmacknz]

Can you use a more complete example with inputs and output overrides to ensure they work?

[cmacknz]

Capabilities is a nice way to control this, good idea 👍

The capabilities.yml file has to be read from the config path, do we do permissions checks on ownership in this directory? This new capability is quite powerful so I think we want to do some extra work to make sure that the file was not created by an arbitrary user. For example if agent is root, the config path and capabilities.yml file need to be owned by root too.

I don't see any on the capabilities.yml file itself:

elastic-agent/internal/pkg/capabilities/capabilities.go

Lines 43 to 58 in 7d6cf59

	func LoadFile(capsFile string, log *logger.Logger) (Capabilities, error) {
	// load capabilities from file
	fd, err := os.Open(capsFile)
	if errors.Is(err, fs.ErrNotExist) {
	// No file, return an empty capabilities manager
	log.Infof("Capabilities file not found in %s", capsFile)
	return &capabilitiesManager{}, nil
	}
	if err != nil {
	return nil, err
	}
	// We successfully opened the file, pass it through to Load
	defer fd.Close()
	return Load(fd, log)
	}