Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

docs: Add flow versioning guide for NiFi 2 #761

Merged
merged 10 commits into from
Mar 14, 2025
2 changes: 2 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,7 @@ All notable changes to this project will be documented in this file.
- Run a `containerdebug` process in the background of each Nifi container to collect debugging information ([#730]).
- Support configuring JVM arguments ([#724]).
- Aggregate emitted Kubernetes events on the CustomResources ([#742]).
- Document flow versioning with NiFi 2 ([#761]).

### Changed

Expand All @@ -26,6 +27,7 @@ All notable changes to this project will be documented in this file.
[#741]: https://github.com/stackabletech/nifi-operator/pull/741
[#742]: https://github.com/stackabletech/nifi-operator/pull/742
[#758]: https://github.com/stackabletech/nifi-operator/pull/758
[#761]: https://github.com/stackabletech/nifi-operator/pull/761

## [24.11.1] - 2025-01-10

Expand Down
25 changes: 9 additions & 16 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

74 changes: 74 additions & 0 deletions docs/modules/nifi/pages/usage_guide/flow-versioning.adoc
Original file line number Diff line number Diff line change
@@ -0,0 +1,74 @@
= Flow Versioning

IMPORTANT: The way described in this document to version flows is only available in NiFi 2.0.0 and later versions. Previous versions of NiFi relied exclusively on the NiFi Registry service for flow versioning.

Starting with Apache NiFi 2.0.0, you can connect directly to Git repositories using either GitHub or GitLab as your flow registry without requiring a separate NiFi Registry service. This document explains how to configure and use these registry clients to version your NiFi flows.

== Setting up the Registry Client

=== Configuration Steps

1. In your NiFi UI, click on the hamburger menu (☰) in the top right corner
2. Select "Controller Settings" > "Registry Clients"
3. Click the "+" icon to add a new registry client
4. Select "GitHubFlowRegistryClient" or "GitLabFlowRegistryClient" as the client type and click "Add"
5. Click the three dots (⋮) on the right side of the client and select "Edit"
6. Fill in the following properties:

[cols="1,3"]
|===
|Property |Description

|API URL
|The API URL, like `https://api.github.com` for GitHub.com or a self-hosted GitLab URL

|Repository Owner
|The username or organization name that owns the repository

|Repository Name
|The name of the repository where flows will be stored

|Default Branch
|The default branch to use (e.g., `main`)

|Authentication Type
|The type of authentication to use. You can choose "None" for public repositories, "Personal Access Token" or "App Installation" are required for private repositories and are also helpful for public repositories in order to avoid rate limits.

|Repository Path
|Directory within the repository to store flows (e.g., `/flows`)
|===

Finally, click "Apply" to save the registry client configuration.

== Using Flow Registry Clients

Once you've configured a Flow Registry Client, you can use it to version your Process Groups:

1. Right-click on a Process Group in your NiFi canvas
2. Select "Version" > "Start version control"
3. Select your configured registry client, the branch to use, and the bucket (directory) to store flows in
4. Provide a Flow Name and optional comments
5. Click "Save" to store the current version in the repository

IMPORTANT: Values of sensitive properties are not stored in the repository, so you must re-enter them when restoring a versioned flow from scratch.

To update a versioned Process Group:

1. Make changes to your Process Group
2. Right-click the Process Group
3. Select "Version" > "Commit local changes"
4. Add a comment describing your changes
5. Click "Save" to commit the changes

To restore a previous version of a Process Group:

1. Right-click the Process Group
2. Select "Version" > "Show version history"
3. Select the version you want to restore
4. Click "Restore" to revert to that version

To import a flow from the registry:

1. Drag the "Import from Registry" icon from the toolbar to the canvas
2. Select your configured registry client, the branch, the bucket (directory) and the flow to import
3. Click "Import" to add the flow to the canvas
1 change: 1 addition & 0 deletions docs/modules/nifi/partials/nav.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@
** xref:nifi:usage_guide/updating.adoc[]
** xref:nifi:usage_guide/overrides.adoc[]
** xref:nifi:usage_guide/writing-to-iceberg-tables.adoc[]
** xref:nifi:usage_guide/flow-versioning.adoc[]
** xref:nifi:usage_guide/custom-components/index.adoc[]
*** xref:nifi:usage_guide/custom-components/custom-nars.adoc[]
*** xref:nifi:usage_guide/custom-components/custom-python-processors.adoc[]
Expand Down