Add `infrahubctl object` command #328

dgarros · 2025-03-30T13:07:19Z

Add infrahub object load and infrahub object validate commands to load objects defined in Yaml format into Infrahub.

These commands are meant to be used in an idempotent way and as such they would work better for models with a Human Friendly ID (hfid) defined.

File format

All object files must start with the following format, all other format will be automatically ignored
Each file is intended for one specific top level kind, but one file can include multiple nested objects of any kind.
The kind of the top level object must be defined in spec/kind

---
apiVersion: infrahub.app/v1
kind: Object
spec:
  kind: <NamespaceName>
  data:
    - [,,,]

Below is a more detailed example with nested objects and with related objects referenced by their HFIDs

---
apiVersion: infrahub.app/v1
kind: Object
spec:
  kind: TestingPerson
  data:
    - name: Alex Thompson
      height: 180
      animals:  # Relationship of cardinality many that will create/update nodes as needed
                      # The owner of the dog will be automatically inserted as long as their is a bidirectional relationship
                      # between TestingPerson and TestingDog
        kind: TestingDog         # The kind must be provided when the relationship is based on a generic
        data:
          - name: Max
            weight: 25
            breed: Golden Retriever
            color: "#FFD700"
      tags:           # The kind is optional here because there is only one option possible (not a generic)
        data:
          - name: Dog Lover    # New Tag will be created if it doesn't exist


    - name: Mike Johnson
      height: 175
      best_friends: # Relationship of cardinality many that referenced existing nodes based on their HFID
        - [Jane Smith, Max]
        - [Sarah Williams, Charlie]
      tags:
        - Veterinarian    # Existing Node referenced by its HFID in string format
        - [Breeder]       # Existing Node referenced by its HFID in list format

codecov · 2025-03-30T15:56:36Z

Codecov Report

Attention: Patch coverage is 78.74016% with 27 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
infrahub_sdk/spec/object.py	83.92%	9 Missing and 9 partials ⚠️
infrahub_sdk/ctl/object.py	25.00%	9 Missing ⚠️

@@             Coverage Diff             @@
##           develop     #328      +/-   ##
===========================================
+ Coverage    71.32%   72.27%   +0.94%     
===========================================
  Files           87       87              
  Lines         7871     7970      +99     
  Branches      1517     1544      +27     
===========================================
+ Hits          5614     5760     +146     
+ Misses        1865     1808      -57     
- Partials       392      402      +10

Flag	Coverage Δ
integration-tests	`25.10% <70.07%> (+2.30%)`	⬆️
python-3.10	`45.58% <38.58%> (+0.12%)`	⬆️
python-3.11	`45.60% <38.58%> (+0.12%)`	⬆️
python-3.12	`45.58% <38.58%> (+0.10%)`	⬆️
python-3.13	`45.58% <38.58%> (+0.10%)`	⬆️
python-3.9	`44.49% <38.58%> (+0.11%)`	⬆️
python-filler-3.12	`24.35% <0.00%> (-0.31%)`	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines	Coverage Δ
infrahub_sdk/ctl/cli_commands.py	`58.13% <100.00%> (ø)`
infrahub_sdk/testing/schemas/animal.py	`97.10% <100.00%> (+0.04%)`	⬆️
infrahub_sdk/yaml.py	`82.27% <100.00%> (+3.79%)`	⬆️
infrahub_sdk/ctl/object.py	`58.82% <25.00%> (-6.57%)`	⬇️
infrahub_sdk/spec/object.py	`84.90% <83.92%> (+61.71%)`	⬆️

... and 2 files with indirect coverage changes

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

ogenstad

One thing around this that isn't part of this PR the code already looked like this before but if we look at the example from the PR description:

---
apiVersion: infrahub.app/v1
kind: Object
spec:
  kind: TestingPerson
  data:
    - name: Mike Johnson
      height: 175
      best_friends: # Relationship of cardinality many that referenced existing nodes based on their HFID
        - [Jane Smith, Max]
        - [Sarah Williams, Charlie]

If TestPerson "Mike Johnson" existed going into this and had other best_friends relationships defined I think that those earlier relationships would be replaced with the ones defined here instead of just ensuring that the ones above also exist.

I'm not certain how I'd want it to work here, just highlighting to raise the issue. I think there are merits to both approaches, just not sure what users would expect to happen. Any thoughts?

poetry.lock

ogenstad · 2025-03-31T14:24:49Z

infrahub_sdk/spec/object.py

+
+    @property
+    def is_bidirectional(self) -> bool:
+        return bool(self.peer_rel)


What does is_bidirectional mean in this context?

Is it tied to this relationship direction in any way? If so it seems like something else would be required here. If not I don't quite know what the property should return.

class RelationshipDirection(InfrahubStringEnum): BIDIR = "bidirectional" OUTBOUND = "outbound" INBOUND = "inbound"

is_bidirectional here means that a relationship with the same identifier exist on the remote peer, it's different than RelationshipDirection but I understand it's confusing.

I'll try to find a better name

pyproject.toml

ajtmccarty · 2025-04-01T19:44:28Z

infrahub_sdk/ctl/object.py

@@ -40,8 +40,28 @@ async def load(
    client = initialize_client()

    for file in files:
-        file.validate_content()
-        schema = await client.schema.get(kind=file.spec.kind, branch=branch)
+        await file.validate_format(client=client, branch=branch)


not sure if this fits with how code is organized in the SDK, but in terms of SOLID, the validate_format method of InfrahubObjectFileData should be its own component: ObjectFileDataValidator or something
Something like this

class ObjectFileDataValidator: def __init__(self, client):... async def validate(self, branch, file): ...

ajtmccarty · 2025-04-01T19:45:37Z

infrahub_sdk/ctl/object.py

-        for item in file.spec.data:
-            await file.spec.create_node(client=client, schema=schema, data=item, branch=branch)
+    for file in files:
+        await file.process(client=client, branch=branch)


same SOLID comment here as for validate_format a couple lines above
also, process seems more like a create method to me

ajtmccarty · 2025-04-01T19:53:53Z

infrahub_sdk/spec/object.py

+
+
+async def get_relationship_info(
+    client: InfrahubClient, schema: MainSchemaTypesAPI, key: str, value: Any, branch: str | None = None


key and value could have more descriptive names. key seems like name to me, but not exactly sure about value. maybe a more accurate type-hint would help for value too

ajtmccarty · 2025-04-01T19:58:04Z

infrahub_sdk/spec/object.py

+                            identifier=key, message=rel_info.reason_relationship_not_valid or "Invalid relationship"
+                        )
+                    )
+                # TODO: Validate the sub-object too


does this need a follow-up issue?

github-actions bot added the type/documentation Improvements or additions to documentation label Mar 30, 2025

dgarros requested review from a team and removed request for a team March 31, 2025 06:07

dgarros marked this pull request as ready for review March 31, 2025 06:59

ogenstad approved these changes Mar 31, 2025

View reviewed changes

dgarros added 9 commits April 1, 2025 18:24

Add integration tests for infrahubctl object command

d6b3577

Add method to validate format of object file

ba046a3

Update doc for infrahubctl and fix for hidden commands

abc97b6

Pin version of infrahub-testcontainers for Python 3.9

e5d7063

Pin testcontaines for python 3.10 too

52f3c26

Set can_send_already_matched_responses for pytest-httpx

da5ed4c

Add httpx_mock setting for test_cli.py

4800c43

Revert infrahub-testcontainers to 1.1.9 for all versions of Python

b760dbf

Cleanup poetry file

b74995b

dgarros force-pushed the dga-20250330-object-cmd branch from d1ff928 to b74995b Compare April 1, 2025 16:29

ajtmccarty reviewed Apr 1, 2025

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add `infrahubctl object` command #328

Add `infrahubctl object` command #328

dgarros commented Mar 30, 2025 •

edited

Loading

codecov bot commented Mar 30, 2025 •

edited

Loading

ogenstad left a comment

ogenstad Mar 31, 2025

dgarros Apr 1, 2025

ajtmccarty Apr 1, 2025

ajtmccarty Apr 1, 2025

ajtmccarty Apr 1, 2025

ajtmccarty Apr 1, 2025



		async def get_relationship_info(
		client: InfrahubClient, schema: MainSchemaTypesAPI, key: str, value: Any, branch: str \| None = None

Add infrahubctl object command #328

Are you sure you want to change the base?

Add infrahubctl object command #328

Conversation

dgarros commented Mar 30, 2025 • edited Loading

File format

codecov bot commented Mar 30, 2025 • edited Loading

Codecov Report

ogenstad left a comment

Choose a reason for hiding this comment

ogenstad Mar 31, 2025

Choose a reason for hiding this comment

dgarros Apr 1, 2025

Choose a reason for hiding this comment

ajtmccarty Apr 1, 2025

Choose a reason for hiding this comment

ajtmccarty Apr 1, 2025

Choose a reason for hiding this comment

ajtmccarty Apr 1, 2025

Choose a reason for hiding this comment

ajtmccarty Apr 1, 2025

Choose a reason for hiding this comment

Add `infrahubctl object` command #328

Add `infrahubctl object` command #328

dgarros commented Mar 30, 2025 •

edited

Loading

codecov bot commented Mar 30, 2025 •

edited

Loading