fs2-kafka v2.4.0

This release adds several new features to the fs2-kafka 2.x line, which uses cats-effect 3 and Kafka 2. It also fixes a concurrency-related bug in TransactionalKafkaProducer.

What's Changed

Enhancements

• Add support for using CooperativeStickyAssignor with KafkaConsumer by @biochimia in #844
• Allow transaction without upstream offsets by @janstenpickle in #883
• Add attempt method to RecordDeserializer by @bplommer in #891

Bugfixes

• Threadsafe transactions by @janstenpickle in #892

Doc improvements

• Add a basic version matrix to the readme by @bplommer in #841
• Slightly improve docs for stream methods by @LMnet in #872
• Publish certs docs by @bplommer in #881
• Update transactional producer docs by @janstenpickle in #890

Dependency updates

• Update cats-effect, cats-effect-laws, ... to 3.3.4 by @scala-steward in #835
• Update scala-library to 2.13.8 by @scala-steward in #838
• Update testcontainers-scala-kafka, ... to 0.40.0 in series/1.x by @scala-steward in #846
• Update fs2-core to 3.2.5 by @scala-steward in #868
• Update sbt-buildinfo to 0.11.0 by @scala-steward in #865
• Update sbt-header to 5.6.5 by @scala-steward in #862
• Update sbt to 1.6.2 by @scala-steward in #854
• Update scala3-library to 3.1.1 by @scala-steward in #857
• Update cats-effect-laws, ... to 3.3.6 by @scala-steward in #882
• Update vulcan version to 1.8.0 by @bplommer in #896

Housekeeping

• Update copyright notices by @bplommer in #833

New Contributors

• @biochimia made their first contribution in #844

Full Changelog: v2.3.0...v2.4.0

v1.10.0

This is a maintenance release for the 1.x series, which is based on cats-effect 2 and is no longer receiving new features. It fixes a concurrency-related bug in TransactionalKafkaProducer (see #888) and updates various dependencies.

What's Changed

• Update scala-library to 2.13.8 in series/1.x by @scala-steward in #839
• Update testcontainers-scala-kafka, ... to 0.40.0 in series/1.x by @scala-steward in #846
• Update sbt-buildinfo to 0.11.0 in series/1.x by @scala-steward in #866
• Update sbt-header to 5.6.5 in series/1.x by @scala-steward in #863
• Update sbt to 1.6.2 in series/1.x by @scala-steward in #855
• Improve scala-steward config for 1.x series by @LMnet in #871
• Publish certs docs by @bplommer in #881
• Update testcontainers-scala-kafka, ... to 0.40.1 in series/1.x by @scala-steward in #860
• Update logback-classic to 1.2.11 in series/1.x by @scala-steward in #884
• Update sbt-mima-plugin to 1.0.1 in series/1.x by @scala-steward in #711
• Threadsafe transactions by @janstenpickle in #888
• Update vulcan version to 1.8.0 in series/1.x by @bplommer in #895
• Don't publish docs when releasing 1.x by @bplommer in #893

Full Changelog: v1.9.0...v1.10.0

fs2-kafka v3.0.0-M4

This is the first milestone release of fs2-kafka following the release of 2.0.0. (We have previously released v3.0.0-M1 through v3.0.0-M3 but these became the 2.x series following a change in our roadmap.)

This is the first release of fs2-kafka to use version 3.0.0 of the Apache Kafka client library, which is not binary-compatible with previous versions. It is available for Scala 2.13 and Scala 3.1; Scala 2.12 has been dropped for the 3.x series but will continue to be supported in the 1.x and 2.x series.

fs2-kafka v2.3.0

This release brings new functionality and dependency updates to the fs2-kafka 2.x series, which is based on cats-effect 2. These include an update to kafka-clients that resolves a security vulnerability (see #821)

What's Changed

Functionality

• Add deleteConsumerGroups method by @keirlawson in #730
• Add avro settings register schema method by @agustafson in #687
• Add producer convenience methods by @keirlawson in #632

Updates

• Update sbt-mima-plugin to 1.0.1 by @scala-steward in #698
• Update kind-projector to 0.13.2 by @scala-steward in #682
• Update kafka-avro-serializer to 6.2.1 by @scala-steward in #695
• Update munit to 0.7.29 by @scala-steward in #679
• Update kafka-clients to 2.8.1 by @scala-steward in #690
• Update sbt-mdoc to 2.2.23 by @scala-steward in #675
• Update scala-library to 2.12.15 by @scala-steward in #688
• Update sbt-ci-release to 1.5.10 by @scala-steward in #700
• Update fs2-core to 3.1.6 by @scala-steward in #704
• Update logback-classic to 1.2.10 by @scala-steward in #804
• Update sbt to 1.6.1 by @scala-steward in #814
• Update scala-library to 2.13.7 by @scala-steward in #746
• Update sbt-unidoc to 0.5.0 by @scala-steward in #736
• Update sbt-scalafmt to 2.4.6 by @scala-steward in #807
• Update kafka-avro-serializer to 6.2.2 by @scala-steward in #781
• Update testcontainers-scala-kafka, ... to 0.39.12 by @scala-steward in #759
• Update scala3-library to 3.1.0 by @scala-steward in #705
• Update cats-effect, cats-effect-laws, ... to 3.3.3 by @scala-steward in #819
• Update fs2-core to 3.2.4 by @scala-steward in #810
• Update sbt-mdoc to 2.2.24 by @scala-steward in #727

Other

• Update code to use modern cats import by @Daenyth in #765

New Contributors

• @keirlawson made their first contribution in #632

Full Changelog: v2.2.0...v2.3.0

fs2-kafka v1.9.0

This is a maintenance release for the fs2-kafka 1.x series, which is based on cats-effect 2. It includes various dependency updates, including an update to kafka-clients that resolves a security vulnerability (see #821), and adds docs for the munit module. There are no other user-facing changes.

An update to the fs2-kafka 2.x series, which is based on cats-effect 3, will be released soon.

What's Changed

Docs

• Add docs for munit module by @keirlawson in #672

Dependency updates

• Update munit to 0.7.29 in series/1.x by @scala-steward in #716
• Update sbt-ci-release to 1.5.9 in series/1.x by @scala-steward in #710
• Update sbt-mdoc to 2.2.24 in series/1.x by @scala-steward in #728
• Update logback-classic to 1.2.10 in series/1.x by @scala-steward in #805
• Update sbt to 1.6.1 in series/1.x by @scala-steward in #815
• Update kind-projector to 0.13.2 in series/1.x by @scala-steward in #720
• Update kafka-avro-serializer to 6.2.2 in series/1.x by @scala-steward in #782
• Update sbt-scalafmt to 2.4.6 in series/1.x by @scala-steward in #808
• Update testcontainers-scala-kafka, ... to 0.39.12 in series/1.x by @scala-steward in #760
• Update scala-library to 2.12.15 in series/1.x by @scala-steward in #714
• Update kafka-clients to 2.8.1 in series/1.x by @scala-steward in #713
• Update sbt-unidoc to 0.5.0 in series/1.x by @scala-steward in #737
• Update cats-effect, cats-effect-laws to 2.5.4 in series/1.x by @scala-steward in #719
• Update scala-library to 2.13.7 in series/1.x by @scala-steward in #748
• Update fs2-core to 2.5.10 in series/1.x by @scala-steward in #707

Housekeeping

• Update copyright notices by @bplommer in #833

Full Changelog: v1.8.0...v1.9.0

fs2-kafka v2.2.0

This is the third release in the fs2-kafka 2.x series, based on cats-effect 3. If focuses on usability, bringing the same improvements as v1.8.0. It also includes various dependency updates.

fs2-kafka v1.8.0

This release focuses on improving usability, with new helper methods for working with Kafka producers and consumers. This release is for users stuck on cats-effect 2. For cats-effect 3 users, v2.2.0 will be coming shortly.

New KafkaProducer syntax (#632 @keirlawson)

We've added several new helper methods for KafkaProducer to simplify common use cases.

Producing one record and getting the resulting record metadata - before:

producer.produce(ProducerRecords.one(ProducerRecord(topic, key, value))).map(_.map(_.records.head.get._2))

After:

producer.produceOne_(topic, key, value)

New KafkaConsumer syntax (#662 #664 @bplommer )

We've also added new extension methods for KafkaConsumer in the Stream context (no new imports needed!), so the most common operations no longer require explicit use of flatMap and evalTap. The stream and partitionedStream methods on KafkaConsumer can now be invoked as records and partitionedRecords respectively, for greater clarity.

Allocating a consumer, subscribing it to a topic, and streaming records - before:

KafkaConsumer.stream(consumerSettings)
  .evalTap(_.subscribeTo("topic"))
  .flatMap(_.stream)

After:

KafkaConsumer.stream(consumerSettings)
  .subscribeTo("topic")
  .records

Vulcan testkit (#629 @keirlawson)

A new module, fs2-kafka-vulcan-testkit-munit, makes it easier to test the schema compatibility of Vulcan codecs against a live schema registry instance.

Other changes

Various dependency updates

fs2-kafka v2.1.0

Scala 3.0.0

The main feature of this release is support for Scala 3.0.0 (and dropping support for 3.0.0-RC3).

Updates

• Scala 3.0.0
• Scala 2.13.6
• Vulcan 1.7.1
• Cats-effect 3.1.1
• Cats 2.6.1
• fs2 3.0.4
• Discipline-scalatest 2.1.5
• Kind-projector 0.13.0
• testcontainers-scala 0.39.4

fs2-kafka v1.7.0

This release brings support for Scala 3.0.0 to the fs2-kafka 1.x series, which uses cats-effect 2.

Updates

• Scala 3.0.0
• Scala 2.13.6
• Vulcan 1.7.1
• Cats-effect 2.5.1
• Cats 2.6.1
• fs2 2.5.6
• Discipline-scalatest 2.1.5
• Kind-projector 0.13.0
• testcontainers-scala 0.39.4

fs2-kafka v2.0.0

This is the first release in the fs2-kafka 2.x series. Its main difference with previous releases is that it uses cats-effect 3 rather than cats-effect 2, but there are some other changes and improvements. It is identical to v2.0.0-RC3. See release notes for previous release candidates for further details.