These release notes cover a series of updates across our differential privacy libraries and tools. A key highlight is the introduction of PipelineDP4j, a new end-to-end differential privacy solution for the JVM.
PipelineDP4j
PipelineDP4j is a new end-to-end differential privacy solution for JVM that supports various frameworks for distributed data processing such as Apache Spark and Apache Beam. It is the JVM implementation of PipelineDP (https://pipelinedp.io/), and is conceptually similar to Privacy on Beam.
New Features
- Initial Release: Published PipelineDP4j, the JVM version of PipelineDP.
- New Stable Public API: Released a new stable public API, replacing the previous experimental one.
- The following frameworks (backends) are supported: Apache Spark (both Dataset and DataFrame APIs), Apache Beam, local execution.
- The following metrics are supported: PRIVACY_ID_COUNT, COUNT, SUM, VECTOR_SUM, MEAN, VARIANCE, QUANTILES.
- Private groups (partitions) selection is supported.
- Public groups are supported as well.
- A usage example is provided with instructions how to run locally with Beam and Spark and how to run on Google Cloud with Beam and Spark.
C++ DP Library
Breaking Changes
- Removal of Deprecated Functions: Removed unused, deprecated functions in approx bounds and partition selection pre-thresholding.
New Features
- BoundsProvider Framework: Refactored code and introduced
BoundsProvider
, making it easier to implement new bounding algorithms. AddedApproxBoundsAsBoundsProvider
wrapper for usingApproxBounds
with the new interface.
Bug Fixes
- Normalized NaN floating point values in
SetValue<T>
for architecture-independent floating point semantics. - Fixed float cast overflow in quantile tree.
- Simplified
UniformDouble()
implementation and made it match its description.
Other
- ApproxBounds Deprecation: Refactored
ApproxBoundsAsBoundsProvider
, replacing the wrapped class withApproxBoundsProvider
and deprecatingApproxBounds
. - Serialization Updates:
- Updated serialization (still backwards compatible). Added new serialization for
BoundsProvider
. Backwards compatability will be removed in the next release. - Changed serialization for sum, mean, and variance (remains fully backwards compatible for now). Backwards compatability will be removed in the next release.
- Updated serialization (still backwards compatible). Added new serialization for
- API Enhancements: Changed return types of
NumericalMechanism::AddNoise
fromT
todouble
orint64_t
to reduce confusion with unsigned types (not a breaking change due to implicit conversion). - Code Quality & Dependencies:
- Used
std::optional
andstd::make_unique
in favor ofabsl::*
. - General code cleanup, refined comments, and updated absl dependency.
- Moved reconstruction logic for clamped bounds into a separate class.
- Used
Java DP Library
New Features
- Added a method to add Gaussian noise with a given L2 sensitivity.
- Added support for noise addition with zCDP in Java Gaussian Noise.
Bug Fixes
- Changed
maxContributions
fromInteger
toint
inApproximateBounds
.
Other
- Updated dependencies.
- Loaded java rules from
@rules_java
.
Go DP Library
Breaking Changes
- Removed Bazel build support for Go code (library, examples, and Privacy on Beam). Building should now be done with the go tool. Building with Bazel could still work via gazelle, but is not officially supported.
- Standard Deviation Deprecated: Marked StandardDeviation as deprecated and updated README. Please use Variance instead.
Bug Fixes
- Handled unspecified noise & infinite lower/upper bounds for Variance.
Other
- Updated README to reflect that pre-thresholding is available in Go.
- Go examples now depends on the local version of the Go DP Library with the go tool (go.mod)
Privacy on Beam (PoB)
Breaking Changes
- Removed Bazel build support for Go code (library, examples, and Privacy on Beam). Building should now be done with the go tool. Building with Bazel could still work via gazelle, but is not officially supported.
New Features
- Implemented
VariancePerKey
, which returns variance; andVarianceStatisticsPerKey
, which returns variance, mean, and count together. - Implemented the
MeanStatisticsPerKey
API which returns count and mean together.
Bug Fixes
- Fixed inconsistent test case parameter in
TestDistinctPrivacyIDThresholdLeavesSomeEntries
. - Fixed tolerance in
TestMeanPerKeyCrossPartitionContributionBounding
.
Other
- Privacy on Beam now depends on the local version of the Go DP Library with the go tool (go.mod).
- Updated
pre_thresholding.md
to include links to Go Library & PoB.
General/Other
Breaking Changes
- Proto changes: Deprecated
bounds_summary
field in all protos (still populated).