|
| 1 | += Loading custom components |
| 2 | +:description: Load custom NiFi components for enhanced functionality. |
| 3 | +:nifi-docs-developers-guide: https://nifi.apache.org/docs/nifi-docs/html/developer-guide.html |
| 4 | +:nifi-docs-python-developers-guide: https://nifi.apache.org/nifi-docs/python-developer-guide.html |
| 5 | +:git-sync-docs: https://github.com/kubernetes/git-sync/tree/v4.2.4#manual |
| 6 | + |
| 7 | +You can develop or use custom components for Apache NiFi, typically custom processors, to extend its functionality. |
| 8 | + |
| 9 | +There are currently two types of custom components: |
| 10 | + |
| 11 | +1. Custom NiFi Archives (NARs) + |
| 12 | + The {nifi-docs-developers-guide}[NiFi Developer’s Guide] provides further information on how to develop custom NARs. |
| 13 | +2. Starting with NiFi 2.0, also custom Python extensions can be used. + |
| 14 | + In {nifi-docs-python-developers-guide}, the development of custom components using Python is described. |
| 15 | +
|
| 16 | +The Stackable image contains the required tooling for both types. |
| 17 | +For instance, a supported Python version is included. |
| 18 | +NARs are only loaded once, but for Python scripts, hot-reloading is supported. |
| 19 | + |
| 20 | +TIP: You might need to refresh your browser window to see the new or changed Python components. |
| 21 | + |
| 22 | +Several options are described below, to add custom components to NiFi. |
| 23 | + |
| 24 | +== git-sync |
| 25 | + |
| 26 | +Custom NiFi components can be synchronized from a Git repository directly into the NiFi pods with git-sync. |
| 27 | +The {crd-docs}/nifi.stackable.tech/nificluster/v1alpha1[NifiCluster CRD] allows the specification of one or multiple Git repositories: |
| 28 | + |
| 29 | +[source,yaml] |
| 30 | +---- |
| 31 | +--- |
| 32 | +apiVersion: nifi.stackable.tech/v1alpha1 |
| 33 | +kind: NifiCluster |
| 34 | +spec: |
| 35 | + clusterConfig: |
| 36 | + customComponentsGitSync: # <1> |
| 37 | + - repo: https://example.com/git/custom-nifi-components # <2> |
| 38 | + branch: main # <3> |
| 39 | + gitFolder: path/to/the/components # <4> |
| 40 | + depth: 10 # <5> |
| 41 | + wait: 10s # <6> |
| 42 | + credentialsSecret: git-credentials # <7> |
| 43 | + gitSyncConf: # <8> |
| 44 | + --git-config: http.sslCAInfo:/tmp/ca-cert/ca.crt |
| 45 | + - repo: https://example.com/git/other-nifi-components # <9> |
| 46 | + nodes: |
| 47 | + config: |
| 48 | + logging: |
| 49 | + enableVectorAgent: true |
| 50 | + containers: |
| 51 | + git-sync: # <10> |
| 52 | + console: |
| 53 | + level: INFO |
| 54 | + file: |
| 55 | + level: INFO |
| 56 | + loggers: |
| 57 | + ROOT: |
| 58 | + level: INFO |
| 59 | +--- |
| 60 | +apiVersion: v1 |
| 61 | +kind: Secret |
| 62 | +metadata: |
| 63 | + name: git-credentials |
| 64 | +type: Opaque |
| 65 | +data: |
| 66 | + user: ... |
| 67 | + password: ... |
| 68 | +---- |
| 69 | +<1> If the optional field `customComponentsGitSync` is defined, then containers running git-sync are deployed to synchronize the specified repositories. |
| 70 | +<2> The git repository URL that will be cloned |
| 71 | +<3> The branch to clone; defaults to `main` |
| 72 | +<4> Location in the Git repository containing the NiFi components; defaults to the root folder |
| 73 | +<5> The depth of synchronizing, i.e. the number of commits to clone; defaults to 1 |
| 74 | +<6> The synchronization interval, e.g. `20s` or `5m`; defaults to `20s` |
| 75 | +<7> The name of the Secret used to access the repository if it is not public. + |
| 76 | + The referenced Secret must include two fields: `user` and `password`. |
| 77 | + The `password` field can either be an actual password (not recommended) or a GitHub token, as described in the {git-sync-docs}[git-sync documentation]. |
| 78 | +<8> A map of optional configuration settings that are listed in the {git-sync-docs}[git-sync documentation]. + |
| 79 | + These settings are not verified. |
| 80 | +<9> Multiple repositories can be defined. Only the `repo` field is mandatory. |
| 81 | +<10> Logging can be configured as described in xref:concepts:logging.adoc[]. |
| 82 | + As git-sync is a command-line tool, just its output is logged and no fine-grained log configuration is possible. |
| 83 | + All git-sync containers are configured via the one `git-sync` field. |
| 84 | + |
| 85 | +It cannot be specified, if a repository contains NiFi Archives or Python components. |
| 86 | +The operator just configures each repository for both types. |
| 87 | +In particular, the parameters `nifi.nar.library.directory` and `nifi.python.extensions.source.directory` are set. |
0 commit comments