Skip to content

DP Lib v4.0.0

Latest
Latest
Compare
Choose a tag to compare
@RamSaw RamSaw released this 08 May 08:21
v4.0.0
a7cc26a

These release notes cover a series of updates across our differential privacy libraries and tools. A key highlight is the introduction of PipelineDP4j, a new end-to-end differential privacy solution for the JVM.

PipelineDP4j

PipelineDP4j is a new end-to-end differential privacy solution for JVM that supports various frameworks for distributed data processing such as Apache Spark and Apache Beam. It is the JVM implementation of PipelineDP (https://pipelinedp.io/), and is conceptually similar to Privacy on Beam.

New Features

  • Initial Release: Published PipelineDP4j, the JVM version of PipelineDP.
  • New Stable Public API: Released a new stable public API, replacing the previous experimental one.
  • The following frameworks (backends) are supported: Apache Spark (both Dataset and DataFrame APIs), Apache Beam, local execution.
  • The following metrics are supported: PRIVACY_ID_COUNT, COUNT, SUM, VECTOR_SUM, MEAN, VARIANCE, QUANTILES.
  • Private groups (partitions) selection is supported.
  • Public groups are supported as well.
  • A usage example is provided with instructions how to run locally with Beam and Spark and how to run on Google Cloud with Beam and Spark.

C++ DP Library

Breaking Changes

  • Removal of Deprecated Functions: Removed unused, deprecated functions in approx bounds and partition selection pre-thresholding.

New Features

  • BoundsProvider Framework: Refactored code and introduced BoundsProvider, making it easier to implement new bounding algorithms. Added ApproxBoundsAsBoundsProvider wrapper for using ApproxBounds with the new interface.

Bug Fixes

  • Normalized NaN floating point values in SetValue<T> for architecture-independent floating point semantics.
  • Fixed float cast overflow in quantile tree.
  • Simplified UniformDouble() implementation and made it match its description.

Other

  • ApproxBounds Deprecation: Refactored ApproxBoundsAsBoundsProvider, replacing the wrapped class with ApproxBoundsProvider and deprecating ApproxBounds.
  • Serialization Updates:
    • Updated serialization (still backwards compatible). Added new serialization for BoundsProvider. Backwards compatability will be removed in the next release.
    • Changed serialization for sum, mean, and variance (remains fully backwards compatible for now). Backwards compatability will be removed in the next release.
  • API Enhancements: Changed return types of NumericalMechanism::AddNoise from T to double or int64_t to reduce confusion with unsigned types (not a breaking change due to implicit conversion).
  • Code Quality & Dependencies:
    • Used std::optional and std::make_unique in favor of absl::*.
    • General code cleanup, refined comments, and updated absl dependency.
    • Moved reconstruction logic for clamped bounds into a separate class.

Java DP Library

New Features

  • Added a method to add Gaussian noise with a given L2 sensitivity.
  • Added support for noise addition with zCDP in Java Gaussian Noise.

Bug Fixes

  • Changed maxContributions from Integer to int in ApproximateBounds.

Other

  • Updated dependencies.
  • Loaded java rules from @rules_java.

Go DP Library

Breaking Changes

  • Removed Bazel build support for Go code (library, examples, and Privacy on Beam). Building should now be done with the go tool. Building with Bazel could still work via gazelle, but is not officially supported.
  • Standard Deviation Deprecated: Marked StandardDeviation as deprecated and updated README. Please use Variance instead.

Bug Fixes

  • Handled unspecified noise & infinite lower/upper bounds for Variance.

Other

  • Updated README to reflect that pre-thresholding is available in Go.
  • Go examples now depends on the local version of the Go DP Library with the go tool (go.mod)

Privacy on Beam (PoB)

Breaking Changes

  • Removed Bazel build support for Go code (library, examples, and Privacy on Beam). Building should now be done with the go tool. Building with Bazel could still work via gazelle, but is not officially supported.

New Features

  • Implemented VariancePerKey, which returns variance; and VarianceStatisticsPerKey, which returns variance, mean, and count together.
  • Implemented the MeanStatisticsPerKey API which returns count and mean together.

Bug Fixes

  • Fixed inconsistent test case parameter in TestDistinctPrivacyIDThresholdLeavesSomeEntries.
  • Fixed tolerance in TestMeanPerKeyCrossPartitionContributionBounding.

Other

  • Privacy on Beam now depends on the local version of the Go DP Library with the go tool (go.mod).
  • Updated pre_thresholding.md to include links to Go Library & PoB.

General/Other

Breaking Changes

  • Proto changes: Deprecated bounds_summary field in all protos (still populated).
Assets 2
Loading