Ideas for CLI reorganization

[mvidner]

• Kanban: https://trello.com/c/3aUupc2F/4623-3-1-left-research-reorganize-agamas-cli
• This summary is based on the complete CLI help which can be read at https://github.com/agama-project/agama-project.github.io/blob/main/docs/user/cli.md

CLI principle: agama [OBJECT] COMMAND ...

These commands have no object:

• agama probe
• agama install
• agama finish
• agama download

These are the objects that have commands:

• agama config ...
• agama profile ...
• agama logs ...
• agama questions ...
• agama auth ...

CLI principle: work with unix pipes via stdio

1. for input, accept stdin (denoted - in place of PATH or URL)
2. for output, prefer stdout, user can specify --output FILENAME (-o)

Commands:

• agama logs store [-d DEST_UGH]

  This is a justified exception to the stdio principle because the output is always binary data, so save it by default and print the filename.

  But the command behavior is inconsistent:

  • -d mylogs complains "Invalid destination path", -d ./mylogs works (writing ./mylogs.tar.gz)
  • -d . overwrites ./agama-logs.tar.gz, while no -d at all uses /tmp/agama-logs-$TIMESTAMP.tar.gz

  Suggestion: --output FILENAME exact not so friendly...

• agama download URL DESTINATION

  becomes agama download [--output PATH] URL
  download says it is an exception to the stdout rule because what? binary?
  IMHO no, just remove it (not make optional) and say in help

  Suggestion:

  "Earlier versions of agama had the destination as 2nd argument. Use --output now"

• agama config show

  Outputs to stdout, good.

  Suggestion: add --output FILENAME

• agama config load

  Inputs from stdin.

  Suggestion: agama config load URL_OR_PATH

  Transition help text: "Earlier versions of this command took stardard input by default. Use '-' as the argument now."

• agama profile validate, is a predicate on an external resource

  Input from URL_OR_PATH already, output is status only, good

• agama profile evaluate

  Suggestion: add --output FILENAME

  Should it include validate?

• agama profile autoyast URL

  combines conversion with downloading... explain that it is OK because that's
  what legacy AutoYaST does

• agama profile import URL

  Input as URL only which is inconsistent but I don't see a use case with path/stdin...

CLI principle: names make sense

TODO: agama config and agama profile are related, won't users confuse them? Suggestion: keep separate but explain, config is internal state, profile deals with AutoYaST?

[imobachgs]

In general it looks good, but I have a few comments.

• About agama download, the command writes the progress to the stdout already. It tells you where it is looking for the file (e.g., when using a YaST-like URL). Otherwise, it fails and you do not have an idea which devices it tried. Would be writing that info to the stderr a good practice?
• agama profile validate works with a configuration, so IMHO it should be moved to agama config.
• I would not say config is "internal state", it sounds like something the user should not touch. It is the result of processing the profile, but I do not have a good (and concise) way to explain it 😅.

[jreidinger]

I do not like suggestion for agama config load URL_OR_PATH as config is json only..so it means that url also have to point to json only and we already have for url other commands. And the command is exactly profile import which do necessary conversion and supports various urls. config load for me would be good if it can just accept path on fs.

[jreidinger]

regarding that config/profile I would explain it as config is agama native json format...profile is for stuff that require preprocessing like downloading, autoyast converion of jsonnet processing.

[joseivanlopez]

About profile and config. I am not able to see a big difference between them.

Current commands are:

> agama profile
Manage auto-installation profiles (retrieving, applying, etc.)

Usage: agama profile <COMMAND>

Commands:
  autoyast  Download the autoyast profile and print resulting json
  validate  Validate a profile using JSON Schema
  evaluate  Evaluate a profile, injecting the hardware information from D-Bus
  import    Process autoinstallation profile and loads it into agama
  help      Print this message or the help of the given subcommand(s)

Options:
  -h, --help  Print help

> agama config
Inspect or change the installation settings

Usage: agama config <COMMAND>

Commands:
  show  Generate an installation profile with the current settings
  load  Read and load a profile from the standard input
  edit  Edit and update installation option using an external editor
  help  Print this message or the help of the given subcommand(s)

Options:
  -h, --help  Print help (see more with '--help')

What about merging both into something like this?

> agama config convert-autoyast <url>
Download AutoYaST profile and generate JSON.

> agama config evaluate <jsonnet>
Evaluate JSONNET file and generate JSON.

> agama config validate <json>
Validate JSON.

> agama config load <json>
Set the given config (JSON).

> agama config edit
Edit the current config (JSON).

> agama config show
Print the current config (JSON).

For applying an AutoYaST profile:

> agama config convert-autoyast ftp://.. | agama config load

Or

> agama download https://... | agama config convert-autoyast | agama config load

For applying a JSONNET config:

> agama config evaluate config.jsonnet | agama config load

[imobachgs]

Another option:

> agama config generate <source>
Generate JSON from source (url | jsonnet | json | ...).

> agama config validate <source>
Validate config from source.

> agama config load <source>
Set config from source (url | jsonnet | json | ...).

> agama config edit
Edit the current config (JSON).

> agama config show
Print the current config (JSON).

I like this one. And if the profile is an "script", in my opinion, it could be a different option instead of inst.auto.

[mvidner]

Another option:

> agama config generate <source>
Generate JSON from source (url | jsonnet | json | ...).

> agama config load <source>
Set config from source (url | jsonnet | json | ...).

Generate and load are indistinguishable!

edit: ah, they differ in printing / applying the result

[imobachgs]

And agama config load should only receive JSON IIUC.

[joseivanlopez]

I think the three commands (agama config generate, agama config validate and agama config load) could receive an url, jsonnet or json.

For example:

> agama config load <url> # download, generate json, validate json and set config
> agama config validate <url> # download, generate json and validate json
> agama config generate <url> # download and generate json

What do you think?

[mvidner]

We consider

1. source of config (stdin, file, URL)
2. type of config (json, jsonnet, AY XML, ERB)

So Ivan is saying: allow (only?) URL as source, allow any type for agama config (input-consumer)

[joseivanlopez]

BTW, when we designed the current CLI we tried to follow this https://clig.dev.

[mvidner]

Before I summarize the discussion so far,
a call for usage: where is the CLI used now?
So far I see

• autoinstallation/bin/agama-auto uses agama profile import (and no other profile or config commands)

[imobachgs]

It is only used there. But it is just another front-end to Agama.

[jreidinger]

in fact I intentionally write that profile import for usage of agama-auto which was more complicated and contain more logic before.

[imobachgs]

While working on a different task, I recalled a use case we discussed in the past: being able to load the assets from the remote location. Let's say that you have a profile and a bunch of scripts in your hard disk:

$ ls
sles.jsonnet
pre-script.sh

It would be nice to not have to copy the files over the network or put them on a web server. Something like (using the old design):

agama profile import sles.jsonnet --upload

But thinking twice, perhaps just adding a command to upload them might be enough. But still you need to reference them from the CLI. Any thoughts on this? Does this feature make sense?

[joseivanlopez]

What do you mean? To upload files from a machine to a remote machine?

I would make agama work with dirs and tar files:

> ls sles-config/
sles.jsonnet
pre-script.sh

# Upload dir files to a remote
> agama --host test config load sles-config/

# Upload tar file to a remote 
> agama --host test config load sles-config.tar

# Download tar file from a remote
> agama config load ftp://test/sles-config.tar

[jreidinger]

I would just add command to simply upload it..otherwise we will have to support it on many places like validate or generate.

[imobachgs]

Actually, you do not need to upload the profile, it is sent through HTTP. The idea is that you can use relative URLs in your profile and it still works.

{
  scripts:
    post: [
     { name: "my-pre-script.sh", url: "./pre-script.sh" }
    ]
}

This example above will not work from a remote host, because the pre-script.sh is not accessible to. I find it reasonable that it does not work, but I wonder if we could do it better.

However, while thinking about this example, I think we do not need to introduce any new option (just make those files available automatically -not defined how-).

[mvidner]

Sorry I don't understand what @imobachgs means with the upload.

And since we are talking about downloads and uploads, we do need to consider the case when the client and the (target) host are different machines,

[imobachgs]

Please, forget about uploading/downloading, we can implement that case later.

[mvidner]

What single CLI command will be used in agama-auto.service? Josef designed agama profile import for this.

[imobachgs]

There is no need to have a dedicated command. We could just combine generate and load (if we go with the generate approach).

[mvidner]

OK, so config generate is liked.
I assume it replaces profile evaluate and profile autoyast.
The remaining profile commands get merged into agama config?
agama config import stays?
agama download stays?

[joseivanlopez]

I think the current agama config and agama profile commands can be replaced by the new 5 agama config commands proposed at #2289 (reply in thread). The rest of commands (e.g., agama download) stays.

[mvidner]

So here is the proposal with respect to command existence (disregarding their options)
and we need to decide if we care about discarding the script execution that profile import did,
and related to that, what is our position on agama --host TARGET ... vs. ssh target ...

--- cli-a       2025-04-29 10:53:53.561794683 +0200
+++ cli-b       2025-04-29 11:07:43.622355381 +0200
@@ -1,26 +1,24 @@
 ## `agama`
 ## `agama config`
+## `agama config generate` (combines `profile evaluate` and `profile autoyast`)
+## `agama config validate` (renamed from `profile validate`)
 ## `agama config show`
 ## `agama config load`
 ## `agama config edit`
 ## `agama probe`
 ## `agama install`
-## `agama profile`
-## `agama profile autoyast`
-## `agama profile validate`
-## `agama profile evaluate`
-## `agama profile import`
+## (`agama profile import` removed, including script execution? agama CLI vs ssh?)
 ## `agama questions`
 ## `agama questions mode`
 ## `agama questions answers`
 ## `agama questions list`
 ## `agama questions ask`
 ## `agama logs`
 ## `agama logs store`
 ## `agama logs list`
 ## `agama auth`
 ## `agama auth login`
 ## `agama auth logout`
 ## `agama auth show`
 ## `agama download`
 ## `agama finish`

[imobachgs]

We decided (discussed with @jreidinger) to drop support for those scripts. At the beginning it made sense because a) we did not have support for scripts and b) the CLI allowed to set individual values directly from the command-line. We can bring the feature back later if needed.

About the --host argument we decided to go with it because it makes the interaction with the Agama host easier. A few examples:

• You can import a profile on your disk without having to copy the file/s or to make it available through HTTP/FTP/whatever.
• It allows to download the logs using a single command.
• From a developer's perspective, it is much easier to test the CLI because you do not need to copy to the host after each compilation.

[mvidner]

candidate CLI principle: minimalism

Should the individual commands do just a bare minimum and be composed with other commands? Taken to the extreme, we would have only agama dowmload URL support the URLs and pipe to agama ... - where the other commands would support only local paths and stdin.
This is in opposition to Ivan's kinda-maximalistic proposal in #2289 (reply in thread)

[imobachgs]

Bear in mind that agama download URL is not enough for AutoYaST, as it could imply rules/classes and other stuff. The idea behind agama download is to serve as an ancillary command that accepts YaST-specific URLs.

Having said that, and summarizing what I understood from your comments, I would go with a minimalistic approach:

> agama config generate <url|file> # generate a JSON file (from an XML, Jsonnet, JSON file)
> agama config load [file] # applies the configuration (file or stdin)
> agama config show # no changes, prints the configuration
> agama config validate [file] # validates the profile (file or stdin)
> agama config edit # opens the profile in an editor and applies the changes on save

Note: I am not sure whether we should support reading from stdin and files (or just stdin is enough).

Does it make sense?

[mvidner]

Thanks, I keep forgetting that AutoYaST may involve more files from a single URL

[mvidner]

@imobachgs: but then, if agama generate does the heavy lifting and produces JSON stdout... how do we resolve relative references (#2305) in that JSON? I am afraid that we are contradicting ourselves today :(

and if we stayed with relurl:// we would still have the same problem

whoever resolves the relative references has to know the original base URL. Unix pipes break that, don't they?

[imobachgs]

I would say that agama config generate is the responsible for resolving the URLs. After all, it is the only command that knows the base URL of the profile.

[mvidner]

The format which I think is best suited for the result of this discussion is the following:

agama-project/agama-project.github.io#74 "CLI reorganization: desired state of CLI help"

It is a diff from the current generated summary of the CLI help to its desired state. Once we "Approve" it, it should not be merged but instead serve as the design document for a further Agama PR.