v0.13.3

Changelog

• ae871a9: Add openai embeddings processor (#2182) (@alarbada)
• ecf40af: Add openai textgen processor (#2169) (@alarbada)
• 3efa052: Builtin Cohere Command Processor (#2165) (@parikshitg)
• eacc1c0: Builtin Cohere's Rerank processor (#2181) (@parikshitg)
• 53b1832: Bump built-in connectors (#2212) (@raulb)
• 76c9adf: [automated] Update Buf dependencies (#2198) (@github-actions[bot])
• 0e0ced9: update conduit version (#2186) (@raulb)
• dfbac9c: update readme (#2213) (@raulb)

Release Artifacts

OSX Darwin

• 📦 conduit_0.13.3_Darwin_arm64.tar.gz
• 📦 conduit_0.13.3_Darwin_x86_64.tar.gz

Linux

• 📦 conduit_0.13.3_Linux_arm64.deb
• 📦 conduit_0.13.3_Linux_arm64.rpm
• 📦 conduit_0.13.3_Linux_arm64.tar.gz
• 📦 conduit_0.13.3_Linux_i386.deb
• 📦 conduit_0.13.3_Linux_i386.rpm
• 📦 conduit_0.13.3_Linux_i386.tar.gz
• 📦 conduit_0.13.3_Linux_x86_64.deb
• 📦 conduit_0.13.3_Linux_x86_64.rpm
• 📦 conduit_0.13.3_Linux_x86_64.tar.gz

Windows

• 📦 conduit_0.13.3_Windows_arm64.zip
• 📦 conduit_0.13.3_Windows_x86_64.zip

Docker images

To pull the image, you can execute the following command:

docker pull conduit.docker.scarf.sh/conduitio/conduit:v0.13.3

v0.13.2

Changelog

• b5f31f1: Dockerfile: don't install git, not needed (#2160) (@hariso)
• 6aae01d: Dockerfile: use OS and arch from the OS of the build image (#2155) (@hariso)
• 9792e64: Fix parsing of the preview.pipeline-arch-v2 flag (#2177) (@hariso)
• 2e16bbf: Support writing custom Conduit apps with the CLI included (#2180) (@hariso)
• 7f341b2: [automated] Update Buf dependencies (#2147) (@github-actions[bot])
• 851b79c: [automated] Update Buf dependencies (#2158) (@github-actions[bot])
• b145fc4: chore: update conduit version (#2136) (@raulb)
• a2137fe: feat: add docker image for arm64 (#2152) (@raulb)

Release Artifacts

OSX Darwin

• 📦 conduit_0.13.2_Darwin_arm64.tar.gz
• 📦 conduit_0.13.2_Darwin_x86_64.tar.gz

Linux

• 📦 conduit_0.13.2_Linux_arm64.deb
• 📦 conduit_0.13.2_Linux_arm64.rpm
• 📦 conduit_0.13.2_Linux_arm64.tar.gz
• 📦 conduit_0.13.2_Linux_i386.deb
• 📦 conduit_0.13.2_Linux_i386.rpm
• 📦 conduit_0.13.2_Linux_i386.tar.gz
• 📦 conduit_0.13.2_Linux_x86_64.deb
• 📦 conduit_0.13.2_Linux_x86_64.rpm
• 📦 conduit_0.13.2_Linux_x86_64.tar.gz

Windows

• 📦 conduit_0.13.2_Windows_arm64.zip
• 📦 conduit_0.13.2_Windows_x86_64.zip

Docker images

To pull the image, you can execute the following command:

docker pull conduit.docker.scarf.sh/conduitio/conduit:v0.13.2

v0.13.1

Changelog

• de1579d: add usage parameters (#2125) (@maha-hajja)
• 432528b: chore: link to docs (#2124) (@raulb)
• 7f54a6c: chore: update github.com/conduitio/ecdysis (#2131) (@raulb)
• 5266985: chore: update issue for the release template (#2132) (@raulb)
• 75cbcd7: make pipeline name optional (#2130) (@raulb)

Release Artifacts

OSX Darwin

• 📦 conduit_0.13.1_Darwin_arm64.tar.gz
• 📦 conduit_0.13.1_Darwin_x86_64.tar.gz

Linux

• 📦 conduit_0.13.1_Linux_arm64.deb
• 📦 conduit_0.13.1_Linux_arm64.rpm
• 📦 conduit_0.13.1_Linux_arm64.tar.gz
• 📦 conduit_0.13.1_Linux_i386.deb
• 📦 conduit_0.13.1_Linux_i386.rpm
• 📦 conduit_0.13.1_Linux_i386.tar.gz
• 📦 conduit_0.13.1_Linux_x86_64.deb
• 📦 conduit_0.13.1_Linux_x86_64.rpm
• 📦 conduit_0.13.1_Linux_x86_64.tar.gz

Windows

• 📦 conduit_0.13.1_Windows_arm64.zip
• 📦 conduit_0.13.1_Windows_x86_64.zip

Docker images

To pull the image, you can execute the following command:

docker pull conduit.docker.scarf.sh/conduitio/conduit:v0.13.1

v0.13.0

A more automated method for keeping your connector documentation up to date. For more information, check out the original discussion, and the updated Connector Specification documentation page.

Conduit doesn't include a User Interface anymore, but it comes with a Command Line Interface that we think will fit better the needs of our users. Check out its dedicated page in our docs here where we'll continue documenting its capabilities. Here's where the original discussion took place.

📝 See the changelog post for more information.

Changelog

• 582be55: CLI: Add pipelines describe command (#2078) (@hariso)
• 2515cf5: Fix CLI flag naming (#2070) (@hariso)
• 717c033: Remove make ui-dist from goreleaser config (#2053) (@lovromazgon)
• 7cd3414: Update SDK to v0.13, bump builtin connectors (#2123) (@hariso)
• 4623062: Update latest version in README (#2119) (@hariso)
• 9265675: [automated] Update Buf dependencies (#2002) (@github-actions[bot])
• 897279e: [automated] Update Buf dependencies (#2044) (@github-actions[bot])
• 9ae2dc5: chore: add tests to ExecuteWithClient for pipelines cmds (#2111) (@raulb)
• e852011: chore: update ecdysis (#2030) (@raulb)
• 643a331: describe connector plugins command (#2086) (@maha-hajja)
• 8c8e3b7: feat(CLI): add connector plugins ls (#2079) (@raulb)
• ac8fff0: feat(CLI): add connectors describe (#2082) (@raulb)
• 0f0e5db: feat(CLI): add connectors ls (#2076) (@raulb)
• 84c87ed: feat(CLI): add processors ls (#2077) (@raulb)
• 603f313: feat(cli): add processor plugins ls (#2081) (@raulb)
• 48917b5: feat: add conduit config (#2033) (@raulb)
• 1cf0a29: feat: add conduit run (#2038) (@raulb)
• 3268d78: feat: add conduit version (#2035) (@raulb)
• 99517c7: feat: add docs command (#2098) (@raulb)
• 5360931: feat: add ecdysis to Conduit CLI (#1983) (@raulb)
• 1745afd: feat: add pipelines ls cmd (+client API) (#2063) (@raulb)
• 873e9e3: feat: add processor describe (#2089) (@raulb)
• fb39b12: feat: improve pipelines desc (#2099) (@raulb)
• d60e671: feat: show connector type on ls (#2096) (@raulb)
• acad1a9: feat: use output decorator, adding tests (#2104) (@raulb)
• 5a1710b: fix empty author bug (#2100) (@maha-hajja)
• 82a0391: fix init bug & docs (#2117) (@maha-hajja)
• 2c80621: fix(CLI): not found errors (#2114) (@raulb)
• 24f4475: fix: example pipeline with processor (#2060) (@raulb)
• 8d3395b: fix: include run (#2041) (@raulb)
• 3a8bb08: fix: required conduit.yaml (#2031) (@raulb)
• d8169e0: processor plugin desc (#2088) (@maha-hajja)
• fdb4f49: refactor: remove UI (#2036) (@raulb)
• 170fa09: refactor: update type on processors ls (#2097) (@raulb)
• 6183101: some more changes (#2042) (@raulb)

Release Artifacts

OSX Darwin

• 📦 conduit_0.13.0_Darwin_arm64.tar.gz
• 📦 conduit_0.13.0_Darwin_x86_64.tar.gz

Linux

• 📦 conduit_0.13.0_Linux_arm64.deb
• 📦 conduit_0.13.0_Linux_arm64.rpm
• 📦 conduit_0.13.0_Linux_arm64.tar.gz
• 📦 conduit_0.13.0_Linux_i386.deb
• 📦 conduit_0.13.0_Linux_i386.rpm
• 📦 conduit_0.13.0_Linux_i386.tar.gz
• 📦 conduit_0.13.0_Linux_x86_64.deb
• 📦 conduit_0.13.0_Linux_x86_64.rpm
• 📦 conduit_0.13.0_Linux_x86_64.tar.gz

Windows

• 📦 conduit_0.13.0_Windows_arm64.zip
• 📦 conduit_0.13.0_Windows_x86_64.zip

Docker images

To pull the image, you can execute the following command:

docker pull conduit.docker.scarf.sh/conduitio/conduit:v0.13.0

v0.12.3

Changelog

• 5c42c30: Add more unit tests (#1970) (@hariso)
• 3ffaa18: Conduit release template: add command for re-generating processor docs (#1965) (@hariso)
• 95ab7da: Fix GH action for running unit tests (#1992) (@hariso)
• 43fe0a1: Fix: filtered records holding up pipeline with destination batching (#1987) (@hariso)
• 1378b8a: [automated] Update Buf dependencies (#1969) (@github-actions[bot])
• f751508: [automated] Update Buf dependencies (#1982) (@github-actions[bot])
• 3100bc4: fix: ensure deprecated flags are correctly hidden (#1974) (@raulb)
• ed6ea11: rely on conduit.io/docs (#1972) (@raulb)

Release Artifacts

OSX Darwin

• 📦 conduit_0.12.3_Darwin_arm64.tar.gz
• 📦 conduit_0.12.3_Darwin_x86_64.tar.gz

Linux

• 📦 conduit_0.12.3_Linux_arm64.deb
• 📦 conduit_0.12.3_Linux_arm64.rpm
• 📦 conduit_0.12.3_Linux_arm64.tar.gz
• 📦 conduit_0.12.3_Linux_i386.deb
• 📦 conduit_0.12.3_Linux_i386.rpm
• 📦 conduit_0.12.3_Linux_i386.tar.gz
• 📦 conduit_0.12.3_Linux_x86_64.deb
• 📦 conduit_0.12.3_Linux_x86_64.rpm
• 📦 conduit_0.12.3_Linux_x86_64.tar.gz

Windows

• 📦 conduit_0.12.3_Windows_arm64.zip
• 📦 conduit_0.12.3_Windows_x86_64.zip

Docker images

To pull the image, you can execute the following command:

docker pull conduit.docker.scarf.sh/conduitio/conduit:v0.12.3

v0.12.2

Changelog

• b1dc373: Add installation script (#1921) (@hariso)
• 977b037: Allow a connector's or processor's plugin name to be updated (#1938) (@hariso)
• a60c36a: Basic conduit init and conduit pipelines init commands (#1927) (@hariso)
• 7028b67: Configure SonarCloud (#1944) (@hariso)
• 9a216a4: Exclude generated code from SonarCloud scans (#1945) (@hariso)
• 4d8d53f: Fix formatting in CLI design doc (#1934) (@hariso)
• 55ebed9: Load configs and directories from the working directory (#1952) (@hariso)
• 58284dc: Pipeline architecture v2 (preview) (#1913) (@lovromazgon)
• 1989bb4: Update Buf dependencies (#1933) (@github-actions[bot])
• dba04d5: adds a mutex to ctx cancel (#1909) (@raulb)
• 322c078: allow multiple acks to be returned from connector (#1912) (@lovromazgon)
• 1a558d1: chore: remove automatic npm updates (#1916) (@raulb)
• 4b66347: chore: remove codeowners (#1917) (@raulb)
• 0c74ada: doc: Add GitHub issue template for releasing Conduit (#1947) (@raulb)
• 3610610: doc: add cli design document for 0.13 (#1931) (@raulb)
• 5f5d4c9: doc: add links to the release issue template (#1954) (@raulb)
• 9faa30f: doc: update links (#1923) (@raulb)
• f1d2e43: fix: conduit release issue template (#1948) (@raulb)
• 048a99c: fix: link and tweak issue template title (#1949) (@raulb)
• 2a742ba: fix: processor links (#1928) (@raulb)
• 552a613: update pipeline_recovering metric on recovery (#1932) (@lovromazgon)

Release Artifacts

OSX Darwin

• 📦 conduit_0.12.2_Darwin_arm64.tar.gz
• 📦 conduit_0.12.2_Darwin_x86_64.tar.gz

Linux

• 📦 conduit_0.12.2_Linux_arm64.deb
• 📦 conduit_0.12.2_Linux_arm64.rpm
• 📦 conduit_0.12.2_Linux_arm64.tar.gz
• 📦 conduit_0.12.2_Linux_i386.deb
• 📦 conduit_0.12.2_Linux_i386.rpm
• 📦 conduit_0.12.2_Linux_i386.tar.gz
• 📦 conduit_0.12.2_Linux_x86_64.deb
• 📦 conduit_0.12.2_Linux_x86_64.rpm
• 📦 conduit_0.12.2_Linux_x86_64.tar.gz

Windows

• 📦 conduit_0.12.2_Windows_arm64.zip
• 📦 conduit_0.12.2_Windows_x86_64.zip

Docker images

To pull the image, you can execute the following command:

docker pull conduit.docker.scarf.sh/conduitio/conduit:v0.12.2

v0.12.1

Changelog

• 3df0511: Update version in README.md (#1904) (@hariso)
• 0c3d647: add conduit release view pixel to the release template (#1906) (@maha-hajja)

Release Artifacts

OSX Darwin

• 📦 conduit_0.12.1_Darwin_arm64.tar.gz
• 📦 conduit_0.12.1_Darwin_x86_64.tar.gz

Linux

• 📦 conduit_0.12.1_Linux_arm64.deb
• 📦 conduit_0.12.1_Linux_arm64.rpm
• 📦 conduit_0.12.1_Linux_arm64.tar.gz
• 📦 conduit_0.12.1_Linux_i386.deb
• 📦 conduit_0.12.1_Linux_i386.rpm
• 📦 conduit_0.12.1_Linux_i386.tar.gz
• 📦 conduit_0.12.1_Linux_x86_64.deb
• 📦 conduit_0.12.1_Linux_x86_64.rpm
• 📦 conduit_0.12.1_Linux_x86_64.tar.gz

Windows

• 📦 conduit_0.12.1_Windows_arm64.zip
• 📦 conduit_0.12.1_Windows_x86_64.zip

Docker images

To pull the image, you can execute the following command:

docker pull conduit.docker.scarf.sh/conduitio/conduit:v0.12.1

v0.12.0

Changelog

• 0e3280d: [Recovery] Make backoff parameters globally configurable (#1818) (@hariso)
• 6b2381e: Add 'recovering' status (#1797) (@hariso)
• 6b3a2ad: Adds: Recovery not triggered - user force-stopped pipeline. (#1877) (@hariso)
• c9bb34f: Correct comment in dependabot config (#1763) (@hariso)
• 468f998: Fix Go version (#1841) (@hariso)
• 579a71b: Fix scripts/get-compare-link.sh (#1890) (@hariso)
• 28cdc80: Guidelines for writing a connector (#1802) (@hariso)
• 38ca22d: Handle concurrent access to a pipeline's status field (#1794) (@hariso)
• 35343c5: Make logger configurable (#1829) (@lovromazgon)
• 2be6a5e: Makefile: mark targets as .PHONY (#1785) (@hariso)
• fe22032: Register prometheus metrics only once (#1842) (@lovromazgon)
• d4e6177: Regularly check for updates in Buf dependencies (#1772) (@hariso)
• 292193e: Track recovery attempts using leaky bucket (#1881) (@lovromazgon)
• 515adc6: Typo and some doc enhacements (#1889) (@FarukhS52)
• 891178b: Update Buf actions (#1739) (@hariso)
• d439290: [Design] Recover from pipeline errors (#1765) (@lovromazgon)
• c284938: [Recovery] Metrics (#1828) (@hariso)
• 7e52888: [automated] Update Buf dependencies (#1824) (@github-actions[bot])
• bab1afb: chore: remove docker warnings (#1816) (@raulb)
• 54bf050: chore: update constructor name (#1817) (@raulb)
• b429365: do not use a sandbox for Run (#1886) (@lovromazgon)
• af47e0f: don't enrich id if it's not provided (#1845) (@lovromazgon)
• 7edb34d: feat: add fatal errors (#1811) (@raulb)
• d7417fd: feat: restart pipeline with recovery (#1853) (@raulb)
• 8f2d5d1: fix: go (#1848) (@raulb)
• 0072ae1: refactor: Add Lifecycle service (#1835) (@raulb)
• 7e7bb8b: small adjustments to fatal errors (#1830) (@lovromazgon)

Release Artifacts

OSX Darwin

• 📦 conduit_0.12.0_Darwin_arm64.tar.gz
• 📦 conduit_0.12.0_Darwin_x86_64.tar.gz

Linux

• 📦 conduit_0.12.0_Linux_arm64.deb
• 📦 conduit_0.12.0_Linux_arm64.rpm
• 📦 conduit_0.12.0_Linux_arm64.tar.gz
• 📦 conduit_0.12.0_Linux_i386.deb
• 📦 conduit_0.12.0_Linux_i386.rpm
• 📦 conduit_0.12.0_Linux_i386.tar.gz
• 📦 conduit_0.12.0_Linux_x86_64.deb
• 📦 conduit_0.12.0_Linux_x86_64.rpm
• 📦 conduit_0.12.0_Linux_x86_64.tar.gz

Windows

• 📦 conduit_0.12.0_Windows_arm64.zip
• 📦 conduit_0.12.0_Windows_x86_64.zip

Docker images

To pull the image, you can execute the following command:

docker pull conduit.docker.scarf.sh/conduitio/conduit:v0.12.0

v0.11.1

Changelog

• 8670c8a: commit provisioning transaction before starting pipeline (#1759) (@lovromazgon)

Release Artifacts

OSX Darwin

• 📦 conduit_0.11.1_Darwin_arm64.tar.gz
• 📦 conduit_0.11.1_Darwin_x86_64.tar.gz

Linux

• 📦 conduit_0.11.1_Linux_arm64.deb
• 📦 conduit_0.11.1_Linux_arm64.rpm
• 📦 conduit_0.11.1_Linux_arm64.tar.gz
• 📦 conduit_0.11.1_Linux_i386.deb
• 📦 conduit_0.11.1_Linux_i386.rpm
• 📦 conduit_0.11.1_Linux_i386.tar.gz
• 📦 conduit_0.11.1_Linux_x86_64.deb
• 📦 conduit_0.11.1_Linux_x86_64.rpm
• 📦 conduit_0.11.1_Linux_x86_64.tar.gz

Windows

• 📦 conduit_0.11.1_Windows_arm64.zip
• 📦 conduit_0.11.1_Windows_x86_64.zip

Docker images

To pull the image, you can execute the following command:

docker pull conduit.docker.scarf.sh/conduitio/conduit:v0.11.1

v0.11.0

Schema Support:

• Introduced comprehensive schema support, allowing schemas to be attached to records.
• Integrated a built-in schema registry, with the option to use an external schema registry.
• Enhanced standalone connectors to retain type information.

Connector Improvements:

• Introduced middleware for automatic schema handling in built-in connectors and processors.
• Postgres Connector:
  • Added support for incremental snapshots in logical replication mode.
  • Improved management of logical replication slots.

Processor Improvements:

• error
  • Introduced an error processor to send records to DLQ or fail the pipeline based on conditions.
• webhook.http
  • Added the ability to specify headers in the webhook.http processor.
• field.convert
  • Extended to support conversion to Go time.Time objects, handling unix nano timestamps and RFC3339 formatted dates.
• avro.encode and avro.decode
  • Updated to use Conduit’s built-in schema registry.

Internal Improvements:

• Fixed bug causing a panic when trying to run a partial pipeline.

Changelog

• ab0ecac: Mention storage options in README (#1730) (@hariso)
• 428e651: Database implementation for SQLite3 (#1682) (@lyuboxa)
• bf9b0c7: Include conduit-schema-registry in release instructions (#1713) (@hariso)
• 840a650: More info about Postgres connection string (#1726) (@hariso)
• 7339241: Replace panic with error message when not declaring a source node in .yml config (#1695) (@AdamHaffar)
• 1397d5d: Replace pkg/foundation/database with commons, use schema registry with DB (#1720) (@hariso)
• 930afae: Schema support (#1702) (@lovromazgon)
• 3a9d244: Tests and examples for converting time using field.convert (#1745) (@lovromazgon)
• 5c956e1: Update SDK, protocol, Kafka connector (#1731) (@hariso)
• 7601e06: Update dependencies for v0.11.0 (#1756) (@lovromazgon)
• f53d710: Update linter config (#1675) (@hariso)
• d6e83ce: Update processor SDK, add middleware (#1742) (@lovromazgon)
• 06bd6e5: Use connector protocol v2 (#1622) (@lovromazgon)
• 0d5929d: Use gofumpt (#1741) (@hariso)
• d370cd5: Use interpreter runtime in wasm tests (#1746) (@lovromazgon)
• 306defe: Wrap builtin processors in middleware in examples (#1748) (@lovromazgon)
• 76c25f1: [Schemas] Handle name conflicts (#1718) (@hariso)
• 203aeb7: [Schemas] Limit schema service usage to connectors (#1701) (@hariso)
• 817d52c: [Schemas] Use configured schema service in Avro processors (#1743) (@hariso)
• a9812b7: add getSchema and createSchema for wasm processors (#1683) (@maha-hajja)
• 76151f4: change keep-n-most-recent to cut-off (required parameter) (#1689) (@lovromazgon)
• 6b9dfa2: doc: update releases template (#1749) (@raulb)
• fd9f261: don't release win/i386 (#1703) (@lovromazgon)
• 7bb8cd8: enable container-retention-policy action (#1693) (@lovromazgon)
• a8d76dc: feat: sets environment variable based on log level (#1740) (@raulb)
• 2da3a0b: refactor: architecture documentation (#1732) (@raulb)
• b6fa5be: support empty .payload.After / .payload.Before in avro.decode (#1678) (@samirketema)
• fab3ab9: update swagger ui (#1711) (@lovromazgon)

Release Artifacts

OSX Darwin

• 📦 conduit_0.11.0_Darwin_arm64.tar.gz
• 📦 conduit_0.11.0_Darwin_x86_64.tar.gz

Linux

• 📦 conduit_0.11.0_Linux_arm64.deb
• 📦 conduit_0.11.0_Linux_arm64.rpm
• 📦 conduit_0.11.0_Linux_arm64.tar.gz
• 📦 conduit_0.11.0_Linux_i386.deb
• 📦 conduit_0.11.0_Linux_i386.rpm
• 📦 conduit_0.11.0_Linux_i386.tar.gz
• 📦 conduit_0.11.0_Linux_x86_64.deb
• 📦 conduit_0.11.0_Linux_x86_64.rpm
• 📦 conduit_0.11.0_Linux_x86_64.tar.gz

Windows

• 📦 conduit_0.11.0_Windows_arm64.zip
• 📦 conduit_0.11.0_Windows_x86_64.zip

Docker images

To pull the image, you can execute the following command:

docker pull conduit.docker.scarf.sh/conduitio/conduit:v0.11.0