feat: TOOLS-3092 Introduce list-versions and diff versions commands for Aerospike configuration management #75

a-spiker · 2025-08-01T07:15:56Z

Summary:
This PR significantly enhances the asconfig CLI by introducing two new commands designed to improve Aerospike configuration analysis and management:

list-versions Command:
- A new top-level command asconfig list-versions has been added.
- This command allows users to quickly list all supported Aerospike server schema versions.
- It includes a --verbose (or -v) flag that, when enabled, displays the output in a detailed, numbered format, providing a clear overview of available versions.
diff versions Command:
- A new sub-command asconfig diff versions <version1> <version2> has been implemented.
- This command enables users to compare the configuration schema between any two specified Aerospike server versions.
- It provides insights into which configuration parameters have been added, removed, or modified between versions, making it easier to understand changes when planning upgrades or migrations.

These new commands provide essential tools for Aerospike administrators and developers to manage and understand configuration changes more effectively.

…ing Aerospike server - Introduced a new `--server` flag for the `diff` command to allow comparison of a local configuration file against the live configuration from a running Aerospike server. - Updated error handling for argument counts specific to server diff. - Refactored the diff command to separate file-to-file and server-to-file comparison logic.

…ent validation - Introduced new test cases for the `--server` flag in the diff command, covering various scenarios including missing arguments, invalid formats, and non-existent files. - Added validation tests for server diff arguments to ensure proper error handling. - Enhanced existing tests for file diff functionality to include edge cases and argument validation.

- Removed outdated versions of aerospike-management-lib from go.sum to streamline dependencies and ensure compatibility with the latest library versions.

Replaces reflect.DeepEqual with a custom valuesEqual function for more robust type-aware comparison, handling cases like int vs string, slices, and booleans. Ensures server and local configs are parsed through the same path to normalize data types before diffing, improving accuracy of configuration diffs.

Refactored the valuesEqual function to compare values by string conversion first, improving type flexibility and preventing panics on slice comparison. Added extensive unit tests for valuesEqual, isSlice, slicesEqual, diffFlatMaps, and argument validation in diff_test.go to ensure correctness and robustness of diff logic.

Refactored slicesEqual to compare slices in an order-agnostic way using frequency counting for O(n) complexity. Added convertToStringSlice to handle different slice types and ensure consistent comparison. Updated tests to validate new behavior, including mixed types and order variations.

Introduces a warning message indicating the server diff feature is in beta Since the generate command is in beta and diff with server uses generate functionality

Refactors the diff command to use a subcommand structure, moving server diff functionality to a new 'diff server' subcommand. Simplifies flag handling, updates help text and examples, and streamlines the diff logic by removing custom value comparison in favor of reflect.DeepEqual.

This commit removes extensive low-level unit tests for helper functions such as valuesEqual, isSlice, and slicesEqual, as well as redundant server diff argument validation and flag validation tests. The remaining tests focus on higher-level command behavior and integration, streamlining the test suite and reducing maintenance overhead.

Expanded the short and long descriptions of the diff command to clarify support for comparing a config file against a running server. Marked the server comparison feature as BETA in the server subcommand descriptions.

Improved the short and long descriptions for the diff and server subcommands to clarify that comparisons are made against a running server's configuration. Updated examples and added notes about supported file formats for better user guidance.

Moved the --format flag from persistent to local flags in the diff command and hid it in the server diff subcommand, as the format will be auto-detected. Added a debug log to show the detected local file format.

Introduces 'files' and 'server' subcommands for the diff command, improving clarity and extensibility. The legacy behavior is preserved for backward compatibility, with a warning message. Adds tests for both legacy and new subcommand usage, including argument validation.

Changed the diff test assertion to check if the output contains the expected result instead of requiring an exact match. This allows for additional warnings or messages in the output without failing the test.

Improved the help text and examples for the diff, diff files, and diff server commands to clarify usage, supported formats, and default behaviors. Added more detailed flag descriptions and updated example host/port formats for consistency.

update server diff usage example Co-authored-by: dwelch-spike <[email protected]>

Introduces a new CLI command `list-versions` to display available Aerospike server versions, with optional table formatting. Adds corresponding unit tests and updates schema loading logic to only include JSON files.

…ions command Updated the list-versions command to replace the "table" flag with "verbose" for output formatting. Adjusted corresponding test cases to reflect this change, ensuring consistency in command usage.

… configurations Introduces a new command `diff versions` to compare configuration changes between two specified Aerospike server versions. This feature includes validation for version order, schema loading, and detailed output options for added, removed, or modified configurations. Updates the command structure to enhance usability and maintainability.

dwelch-spike

I think this is a really useful addition to asconfig, nice work! I have some thoughts about the implementation that I left in comments, mainly...

I'm worried that some of the formatting, printing, and parsing is too tightly coupled to values in the schemas. Those values can and have changed, sometimes we add new ones etc. If possible, we should try to reduce the coupling to concrete fields in the schema and generate output dynamically. Let me know what you think, if this introduces too much complexity we can do something else.

It might be worth checking out this library https://github.com/josephburnett/jd it seems very popular and looks like it can generate diff output.

Nice work, let me know when you want me to take another look.

dwelch-spike · 2025-08-19T18:04:30Z

cmd/diff.go

+func newDiffVersionsCmd() *cobra.Command {
+	cmd := &cobra.Command{
+		Use:   "versions [flags] <version1> <version2>",
+		Short: "Diff Aerospike server versions.",


Suggested change

Short: "Diff Aerospike server versions.",

Short: "Show configuration file difference between versions of the Aerospike server.",

suggestion: thinking about how to clearly describe this

dwelch-spike · 2025-08-19T18:12:48Z

cmd/diff.go

+}
+
+// isVersionLessEqual compares two version strings (X.X.X) and returns true if v1 <= v2
+func isVersionLessEqual(v1, v2 string) bool {


Alot of our tools have their own implementation of this. Could we use the one in the management liv here? https://github.com/aerospike/aerospike-management-lib/blob/504da4668525110dd75aa2bd5aa33d52a12e8233/utils.go#L72

If not, we should add this to tools common go so we can unify how our tools compare versions.

dwelch-spike · 2025-08-19T18:19:42Z

cmd/list_versions.go

+
+func newListVersionsCmd() *cobra.Command {
+	res := &cobra.Command{
+		Use:   "list-versions",


what do you think of adding a list command and "versions" as a subcommand, so that the useage becomes list versions that way we can easily add more listings later.

dwelch-spike · 2025-08-19T18:22:02Z

cmd/diff_utils.go

+}
+
+// compareSchemas compares two schema objects and returns a summary of changes
+func compareSchemas(file1Obj, file2Obj map[string]interface{}, newVersion, oldVersion string) (ChangeSummary, error) {


Suggested change

func compareSchemas(file1Obj, file2Obj map[string]interface{}, newVersion, oldVersion string) (ChangeSummary, error) {

func compareSchemas(schema_left, schema_right map[string]interface{}, newVersion, oldVersion string) (ChangeSummary, error) {

nit: I think these names could be clearer. maybe "schema_old, schema_new" ?

dwelch-spike · 2025-08-19T18:39:34Z

cmd/diff_utils.go

+	for _, change := range changes {
+		section := extractSection(change.Path)
+
+		sectionChanges := summary.Sections[section]


won't this panic since summary.Sections[section] does not exist yet? I'd just initialize it to a new empty object

dwelch-spike · 2025-08-19T18:40:23Z

cmd/diff_utils.go

+	// Define valid top-level sections
+	validSections := map[string]bool{
+		"service":    true,
+		"logging":    true,
+		"network":    true,
+		"namespaces": true,
+		"mod-lua":    true,
+		"security":   true,
+		"xdr":        true,
+	}
+
+	// Look for valid top-level section in the path
+	for _, part := range parts {
+		if part == "" || part == "properties" || part == "items" || isNumeric(part) {
+			continue
+		}


It would be nice to avoid hardcoded tables like this so they don't need to be updated when the schemas/configs change. Maybe we could use the first element in the json path as the section and then exclude items we know we don't want? What do you think?

dwelch-spike · 2025-08-19T18:41:15Z

cmd/diff_utils.go

+	sectionOrder := []string{
+		"service",
+		"logging",
+		"network",
+		"namespaces",
+		"mod-lua",
+		"security",
+		"xdr",
+		"general",
+	}


It's nice to avoid hardcoding these where possible, since they may change. Can we sort alphabetically or something similar instead?

dwelch-spike · 2025-08-19T18:45:29Z

cmd/diff_utils.go

+}
+
+// printValueProperties prints properties of a value
+func printValueProperties(value interface{}, options DiffOptions) {


If we ever introduce new values to the schema, they will need to be added here. Is there some way we can generate this text dynamically? For example, by marshalling to json/yaml or a custom format?

dwelch-spike · 2025-08-19T18:49:56Z

cmd/diff_utils.go

+		// In verbose mode, print additional properties
+		// Print type information
+		if typeVal, exists := propMap["type"]; exists {
+			fmt.Printf("%sType: %v\n", prefix, typeVal)
+		}
+
+		// Print min/max values
+		if minVal, exists := propMap["minimum"]; exists {
+			fmt.Printf("%sMinimum: %v\n", prefix, minVal)
+		}
+		if maxVal, exists := propMap["maximum"]; exists {
+			fmt.Printf("%sMaximum: %v\n", prefix, maxVal)
+		}
+
+		// Print enum values if they exist
+		if enumVals, exists := propMap["enum"].([]interface{}); exists {
+			fmt.Printf("%sAllowed values: %v\n", prefix, enumVals)
+		}


Are these value properties or object properties?

dwelch-spike · 2025-08-19T19:01:14Z

cmd/diff_utils.go

+	"strconv"
+	"strings"
+
+	"github.com/wI2L/jsondiff"


Maybe worth taking a look at https://github.com/josephburnett/jd it has more stars and seems to generate diffs as text output. It might be worth experimenting with, maybe it could simplify this code.

tanmayja

can we have some tests around these commands?

tanmayja · 2025-08-20T04:39:14Z

cmd/diff.go

+	}
+
+	cmd.Flags().BoolP("verbose", "v", false, "Show detailed information about property changes (type, default values, etc.)")
+	cmd.Flags().StringP("filter-path", "f", "", "Filter results to only show properties under the specified path (e.g., 'service', 'namespaces')")


Does it take nested path also? or just the main context?

a-spiker · 2025-08-21T13:36:55Z

@dwelch-spike I had few things in for this feature, let me know if they sounds good.

Feature meant for very basic persona [newly onboarded] - make the diff output more human readable
Abstract the schemas from customer - Hide our schemas structure, they are internal to us and may change.
Diff output syntax is a value add for this feature - Thus not showing straight diff output between two json structures from a library [in diff format directly]
Make the Diff syntax as much agnostic output. But at high level our sections will not change viz security, namespaces, logging etc

a-spiker · 2025-11-11T10:10:58Z

Due to linting overhaul, this PR will be discarded, changes now part of #84

a-spiker and others added 21 commits July 4, 2025 08:53

chore: update go.sum to remove obsolete aerospike-management-lib entries

3531294

- Removed outdated versions of aerospike-management-lib from go.sum to streamline dependencies and ensure compatibility with the latest library versions.

Merge branch 'main' into TOOLS-3077

85d89ea

feat: Add beta warning and adjust log levels in diff command

48df97f

Introduces a warning message indicating the server diff feature is in beta Since the generate command is in beta and diff with server uses generate functionality

docs: Update diff command descriptions for server comparison

5fc646b

Expanded the short and long descriptions of the diff command to clarify support for comparing a config file against a running server. Marked the server comparison feature as BETA in the server subcommand descriptions.

fix: Refactor format flag handling in diff commands

ad345f6

Moved the --format flag from persistent to local flags in the diff command and hid it in the server diff subcommand, as the format will be auto-detected. Added a debug log to show the detected local file format.

fix: Update diff tests to use substring match

645adf9

Changed the diff test assertion to check if the output contains the expected result instead of requiring an exact match. This allows for additional warnings or messages in the output without failing the test.

Update cmd/diff.go

d6b3799

update server diff usage example Co-authored-by: dwelch-spike <[email protected]>

Merge branch 'main' into TOOLS-3092-1

e7bc39d

feat: Add list-versions command to list available server versions

ab1c405

Introduces a new CLI command `list-versions` to display available Aerospike server versions, with optional table formatting. Adds corresponding unit tests and updates schema loading logic to only include JSON files.

refactor: TOOLS-3092 Rename table format flag to verbose in list-vers…

623634a

…ions command Updated the list-versions command to replace the "table" flag with "verbose" for output formatting. Adjusted corresponding test cases to reflect this change, ensuring consistency in command usage.

a-spiker changed the base branch from main to TOOLS-3077 August 1, 2025 07:16

a-spiker requested a review from dwelch-spike August 1, 2025 19:45

a-spiker changed the base branch from TOOLS-3077 to main August 7, 2025 11:21

a-spiker changed the base branch from main to TOOLS-3077 August 7, 2025 11:22

Base automatically changed from TOOLS-3077 to main August 19, 2025 04:57

a-spiker added 2 commits August 19, 2025 21:58

Merge branch 'main' into TOOLS-3092-1

c0230b1

chore: Update schema submodule to latest commit f99d3c1

3d007d8

dwelch-spike reviewed Aug 19, 2025

View reviewed changes

tanmayja reviewed Aug 20, 2025

View reviewed changes

	Short: "Diff Aerospike server versions.",
	Short: "Show configuration file difference between versions of the Aerospike server.",

	func compareSchemas(file1Obj, file2Obj map[string]interface{}, newVersion, oldVersion string) (ChangeSummary, error) {
	func compareSchemas(schema_left, schema_right map[string]interface{}, newVersion, oldVersion string) (ChangeSummary, error) {

feat: TOOLS-3092 Introduce list-versions and diff versions commands for Aerospike configuration management #75

Are you sure you want to change the base?

feat: TOOLS-3092 Introduce list-versions and diff versions commands for Aerospike configuration management #75

Uh oh!

Conversation

a-spiker commented Aug 1, 2025

Uh oh!

dwelch-spike left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

tanmayja left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

a-spiker commented Aug 21, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

a-spiker commented Nov 11, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

a-spiker commented Aug 21, 2025 •

edited

Loading