Handlebars template validation and documentation #1030
Conversation
|
test integrations |
| pkgEntries, err := fs.Glob(fsys, "agent/**/*.hbs") | ||
| if err != nil && !errors.Is(err, fs.ErrNotExist) { | ||
| return nil, err | ||
| } |
There was a problem hiding this comment.
According to the spec, this files could also be links.
Would this catch even the links thanks to the filesystem ? Or is it needed to add a special case for link files?
There was a problem hiding this comment.
i've updated the PR to include the linked files. For this i've changed the aproach of glob and used nested readDirs to get into the files.
i've added tests and test packages to check this works as expected with the links 👍🏻
|
|
||
| // validateFile validates a single Handlebars file located at filePath. | ||
| // it parses the file using the raymond library to check for syntax errors. | ||
| func validateFile(filePath string) error { |
There was a problem hiding this comment.
I'd rename this to be more specific, what about validateHandlebarTemplateFile or validateHandlebarTemplate?
There was a problem hiding this comment.
yup, i've gone around this one :D as the "parent" function is very similar... i will think around this
…bsolute path for linked files
|
test integrations |
|
Created or updated PR in integrations repository to test this version. Check elastic/integrations#16224 |
| // If fs.ReadFile fails (likely due to linked file path outside filesystem boundary), | ||
| // fall back to absolute path approach like linkedfiles.FS does | ||
| absolutePath := fsys.Path(filePath) | ||
| if content, err = os.ReadFile(absolutePath); err != nil { | ||
| return err | ||
| } |
There was a problem hiding this comment.
According to the comment, is this already done by the linkedfiles.FS ? If so, maybe it could be removed from here.
There was a problem hiding this comment.
this refers to the logic at getLinkedFileChecksum where the file content is read to get the checksum. If the file is not within fsys fs.ReadFile fails, as is not in its scope. If this happens, we use os.ReadFile (like in getLinkedFileChecksum) to read the file with the absolute path of it
| Available Handlebars helpers include: | ||
| - `contains` (checks if item is in array/string), | ||
| - `escape_string` (wraps string in single quotes and escapes them), | ||
| - `escape_multiline_string` (escapes multiline strings without wrapping), | ||
| - `to_json` (converts object to JSON string), and | ||
| - `url_encode` (URI encodes string). |
There was a problem hiding this comment.
These helpers are defined in Fleet (Kibana), it could be added a reference here in the description that the full list of helpers is in Fleet Kibana.
At least, I've found that they are defined here:
https://github.com/elastic/kibana/blob/70749d9216d7a5de6ce1e7a028d153d5390ad3ac/x-pack/platform/plugins/shared/fleet/server/services/epm/agent/agent.ts#L226
But not sure about adding this link since it could be outdated in the future.
There was a problem hiding this comment.
i took a list from that file and created the text with it, i will add the permlink to the file as a warning that these list can be updated so they can check the fleet code also
| } | ||
| err = validateHandlebarsEntry(fsys, dir, linkFile.IncludedFilePath) | ||
| if err != nil { | ||
| return err |
There was a problem hiding this comment.
Could it be defined a specerrors.ValidationErrors here in this function and add all the errors found in a given folder?
I was thinking if it could be possible to show all the errors at once to the user.
There was a problem hiding this comment.
fixed this and adding more context to the error 👍🏻
an example of the output when testing one of the failed integration:
Error: building package failed: invalid content found in built zip package: found 1 validation error:
1. invalid handlebars template: error validating data_stream/alerts_events_v2/agent/stream/gcs.yml.hbs: Parse error on line 49:
Expecting OpenEndBlock, got: 'EOF'
… tests for clarity
💚 Build Succeeded
History
|
|
after running the test against the current integrations, two of them had errors regarding some template parsing. |
| Config template file for inputs defined in the policy_templates section of the top level manifest. | ||
| The template should use standard Handlebars syntax (e.g., `{{vars.key}}`, `{{#if vars.condition}}`, `{{#each vars.items}}`) | ||
| and must compile to valid YAML. | ||
| Available Handlebars helpers include: |
There was a problem hiding this comment.
Can you please document the metadata variables that are available in templates too. See elastic/kibana#241140
4 participants
What does this PR do?
Includes a validation rule for verifying the
.hbsfiles used at input tempate and stream template are correct.Also adds more information on the spec regarding the format of these templates and the helpers that are available at fleet when rendering them.
Linked file are also verified
.hbs.linkWhy is it important?
If a template arrives fleet without being checked, the can be errors when fleet renders the template. This is a way to prevent this before the package is installed.
Checklist
Added unit tests and validation test with test packages
test/packagesthat prove my change is effective.spec/changelog.yml.Related issues
Relates #21