PoC of running granular tests inside #1447
Conversation
| @click.option('--delimiter', default="-", help='create tmt test strucure based on delimiter') | ||
| @click.option('--kstestdir', default=".", help='position where are located additional functions for rhinstaller tests') | ||
| def main(path, extension, metadata, delimiter, kstestdir): | ||
| file_list = list_files(path, "*." + extension) |
There was a problem hiding this comment.
Only the executable .sh files are tests (the directory structure is in shameful state, we have a jira ticket to move tests to a separate directory).
Example of identifying test files when generating permian test cases:
There was a problem hiding this comment.
Ah, yes, would be better to organize it, but is is not big deal for me right now. yep I'm not checking executable bit. but probably executable bit should not be major factor, as it is passed through container, so we should avoid to use executable bits for tests as possible I think.
There was a problem hiding this comment.
Yes, we should fix that but until that I think you should check it when identifying the tests, so that for example post-lib-keyboard.sh (which is a library included in tests) does not get into the main.fmf.
There was a problem hiding this comment.
Only the executable .sh files are tests (the directory structure is in shameful state, we have a jira ticket to move tests to a separate directory).
For the reference: https://issues.redhat.com/browse/INSTALLER-3259
There was a problem hiding this comment.
okay, I'll check for executable bit, to avoid using libraries.
| /tests: | ||
| /anabot: | ||
| /1: | ||
| /x: |
There was a problem hiding this comment.
Just out of curiosity - why all those slashes (on all levels)?
There was a problem hiding this comment.
these slashes creates FMF file structures https://fmf.readthedocs.io/en/stable/features.html#hierarchy , as this is equvalent to creating dir structure to do same thing as well.
plans/granular.fmf
Outdated
There was a problem hiding this comment.
I wonder about the hardcoded values (guest, boot.iso URL) - those are just some temporary ones used during the development, right?
There was a problem hiding this comment.
Right this is more less implementation detail for development right now.
There was a problem hiding this comment.
As discussed internally with rvykydal, we'd like to retain the definitions (TESTTYPE, test enabled/disabled) within the test scripts as the source of truth. Is my understanding correct that it will remain this way and this extractor script is merely a helper tool to translate the metadata into a form that used by TMT?
There was a problem hiding this comment.
Exactly, this tool just extract the data from tests and construct FMF/TMT metadata. with extracted data. (in production I can imagine it will be also git commit/push hook to have it up to date)
jscotka
force-pushed
the
tmt_poc
branch
2 times, most recently
from
9c1f4f8 to
944dad9
Compare
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
|
/test-os-variants |
|
@jscotka can you please rebase the PR on current main so that we can run ci (required because of recent changes) |
jscotka
force-pushed
the
tmt_poc
branch
2 times, most recently
from
ad46b73 to
1997a28
Compare
|
/test-tmt |
| @@ -7,7 +7,8 @@ | |||
|
|
|||
| set -eu | |||
|
|
|||
| BASEDIR=$(dirname $(dirname $(dirname $(realpath $0)))) | |||
| SCRIPT_DIR=$( cd -- "$( dirname -- "${BASH_SOURCE[0]}" )" &> /dev/null && pwd ) | |||
| BASEDIR="$(dirname $(dirname $SCRIPT_DIR ))" | |||
Check warning
Code scanning / ShellCheck
SC2046
|
/test-os-variants --testtype smoke |
Test metadata convertor from tests to TMT format
Run each test separately via TMT
python3 scripts/tmt_test_metadata_extractor.pytmt -vvd run plan --name granular