variadics_macro v0.5.5

New Features

• generalized hash trie indexes for relational tuples
  Generalized Hash Tries are part of the SIGMOD '23 FreeJoin
  paper by
  Wang/Willsey/Suciu. They provide a compressed ("factorized")
  representation of relations. By operating in the factorized domain, join
  algorithms can defer cross-products and achieve asymptotically optimal
  performance.

  ---

Commit Statistics

• 1 commit contributed to the release.
• 1 commit was understood as conventional.
• 1 unique issue was worked on: #1503

Commit Details

view details

• #1503
  • Generalized hash trie indexes for relational tuples (f7e740f)

variadics v0.0.7

Chore

• update pinned rust version, clippy lints, remove some dead code

New Features

• generalized hash trie indexes for relational tuples
  Generalized Hash Tries are part of the SIGMOD '23 FreeJoin
  paper by
  Wang/Willsey/Suciu. They provide a compressed ("factorized")
  representation of relations. By operating in the factorized domain, join
  algorithms can defer cross-products and achieve asymptotically optimal
  performance.

  ---

• additions to variadics including collection types
  adds a number of features:

  collection types for variadics (sets, multisets) that allow search via
  RefVars (variadic of refs)
  into_option (convert a variadic to a variadic of options)
  into_vec (convert a variadic to a variadic of vecs)

• additions to variadics including collection types
  adds a number of features:

  • collection types for variadics (sets, multisets) that allow search via
    RefVars (variadic of refs)
  • into_option (convert a variadic to a variadic of options)
  • into_vec (convert a variadic to a variadic of vecs)

Style

• fixes for nightly clippy
  a couple few spurious too_many_arguments and a spurious
  zombie_processes still on current nightly (clippy 0.1.84 (4392847410 2024-10-21))

Test

• ignore trybuild tests inconsistent on latest nightly

Commit Statistics

• 7 commits contributed to the release.
• 69 days passed between releases.
• 6 commits were understood as conventional.
• 6 unique issues were worked on: #1444, #1473, #1474, #1475, #1503, #1505

Commit Details

view details

• #1444
  • Update pinned rust version, clippy lints, remove some dead code (d567760)
• #1473
  • Additions to variadics including collection types (8afd326)
• #1474
  • Revert "feat: additions to variadics including collection types" (08c2af5)
• #1475
  • Additions to variadics including collection types (1c28259)
• #1503
  • Generalized hash trie indexes for relational tuples (f7e740f)
• #1505
  • Fixes for nightly clippy (47cb703)
• Uncategorized
  • Ignore trybuild tests inconsistent on latest nightly (656ee32)

stageleft_tool v0.4.0

Chore

• update pinned rust version, clippy lints, remove some dead code

New Features

• add ability to have staged flows inside unit tests
  Whenever a Hydroflow+ program is compiled, it depends on a generated
  __staged module, which contains the entire contents of the crate but
  with every type / function made pub and exported, so that the compiled
  UDFs can resolve local references appropriately.

  Previously, we would not do this for #[cfg(test)] modules, since they
  may use dev-dependencies and therefore the generated module may fail
  to compile when not in test mode. To solve this, when running a unit
  test (marked with hydroflow_plus::deploy::init_test()) that uses
  trybuild, we emit a version of the __staged module with #[cfg(test)]
  modules included into the generated trybuild sources because we can
  guarantee via trybuild that the appropriate dev-dependencies are
  available.

  This by itself allows crates depending on hydroflow_plus to have local
  unit tests with Hydroflow+ logic inside them. But we also want to use
  this support for unit tests inside hydroflow_plus itself. To enable
  that, we eliminate the hydroflow_plus_deploy crate and move its
  contents directly to hydroflow_plus itself so that we can access the
  trybuild machinery without incurring a circular dependency.

  Also fixes #1408

Bug Fixes

• properly handle crate:: imports

Refactor

• complete split into leader election and sequencing phases

Commit Statistics

• 4 commits contributed to the release.
• 69 days passed between releases.
• 4 commits were understood as conventional.
• 4 unique issues were worked on: #1444, #1450, #1486, #1527

Commit Details

view details

• #1444
  • Update pinned rust version, clippy lints, remove some dead code (d567760)
• #1450
  • Add ability to have staged flows inside unit tests (afe78c3)
• #1486
  • Complete split into leader election and sequencing phases (8b7b1c6)
• #1527
  • Properly handle crate:: imports (2faffdb)

stageleft_macro v0.4.0

Chore

• update pinned rust version, clippy lints, remove some dead code

New Features

• add ability to have staged flows inside unit tests
  Whenever a Hydroflow+ program is compiled, it depends on a generated
  __staged module, which contains the entire contents of the crate but
  with every type / function made pub and exported, so that the compiled
  UDFs can resolve local references appropriately.

  Previously, we would not do this for #[cfg(test)] modules, since they
  may use dev-dependencies and therefore the generated module may fail
  to compile when not in test mode. To solve this, when running a unit
  test (marked with hydroflow_plus::deploy::init_test()) that uses
  trybuild, we emit a version of the __staged module with #[cfg(test)]
  modules included into the generated trybuild sources because we can
  guarantee via trybuild that the appropriate dev-dependencies are
  available.

  This by itself allows crates depending on hydroflow_plus to have local
  unit tests with Hydroflow+ logic inside them. But we also want to use
  this support for unit tests inside hydroflow_plus itself. To enable
  that, we eliminate the hydroflow_plus_deploy crate and move its
  contents directly to hydroflow_plus itself so that we can access the
  trybuild machinery without incurring a circular dependency.

  Also fixes #1408

• splice UDFs with type hints to avoid inference failures

Bug Fixes

• support tuple patterns

Commit Statistics

• 4 commits contributed to the release.
• 69 days passed between releases.
• 4 commits were understood as conventional.
• 4 unique issues were worked on: #1434, #1444, #1445, #1450

Commit Details

view details

• #1434
  • Splice UDFs with type hints to avoid inference failures (60d9bec)
• #1444
  • Update pinned rust version, clippy lints, remove some dead code (d567760)
• #1445
  • Support tuple patterns (486dfbe)
• #1450
  • Add ability to have staged flows inside unit tests (afe78c3)

stageleft v0.5.0

Chore

• update pinned rust version, clippy lints, remove some dead code

New Features

• add API for external network inputs
  This is a key step towards being able to unit-test HF+ graphs, by being
  able to have controlled inputs. Outputs next.
• splice UDFs with type hints to avoid inference failures

Style

• fixes for nightly clippy
  a couple few spurious too_many_arguments and a spurious
  zombie_processes still on current nightly (clippy 0.1.84 (4392847410 2024-10-21))

Commit Statistics

• 4 commits contributed to the release.
• 69 days passed between releases.
• 4 commits were understood as conventional.
• 4 unique issues were worked on: #1434, #1444, #1449, #1505

Commit Details

view details

• #1434
  • Splice UDFs with type hints to avoid inference failures (60d9bec)
• #1444
  • Update pinned rust version, clippy lints, remove some dead code (d567760)
• #1449
  • Add API for external network inputs (8a80931)
• #1505
  • Fixes for nightly clippy (47cb703)

pusherator v0.0.9

Chore

• update pinned rust version, clippy lints, remove some dead code

Commit Statistics

• 1 commit contributed to the release.
• 69 days passed between releases.
• 1 commit was understood as conventional.
• 1 unique issue was worked on: #1444

Commit Details

view details

• #1444
  • Update pinned rust version, clippy lints, remove some dead code (d567760)

multiplatform_test v0.3.0

Chore

• update pinned rust version, clippy lints, remove some dead code

Commit Statistics

• 1 commit contributed to the release.
• 69 days passed between releases.
• 1 commit was understood as conventional.
• 1 unique issue was worked on: #1444

Commit Details

view details

• #1444
  • Update pinned rust version, clippy lints, remove some dead code (d567760)

lattices_macro v0.5.7

Chore

• update pinned rust version, clippy lints, remove some dead code

Commit Statistics

• 1 commit contributed to the release.
• 69 days passed between releases.
• 1 commit was understood as conventional.
• 1 unique issue was worked on: #1444

Commit Details

view details

• #1444
  • Update pinned rust version, clippy lints, remove some dead code (d567760)

lattices v0.5.8

Chore

• update pinned rust version, clippy lints, remove some dead code

New Features

• generalized hash trie indexes for relational tuples
  Generalized Hash Tries are part of the SIGMOD '23 FreeJoin
  paper by
  Wang/Willsey/Suciu. They provide a compressed ("factorized")
  representation of relations. By operating in the factorized domain, join
  algorithms can defer cross-products and achieve asymptotically optimal
  performance.

  ---

Style

• fixes for nightly clippy
  a couple few spurious too_many_arguments and a spurious
  zombie_processes still on current nightly (clippy 0.1.84 (4392847410 2024-10-21))

Commit Statistics

• 3 commits contributed to the release.
• 69 days passed between releases.
• 3 commits were understood as conventional.
• 3 unique issues were worked on: #1444, #1503, #1505

Commit Details

view details

• #1444
  • Update pinned rust version, clippy lints, remove some dead code (d567760)
• #1503
  • Generalized hash trie indexes for relational tuples (f7e740f)
• #1505
  • Fixes for nightly clippy (47cb703)

hydroflow_plus v0.10.0

Chore

• update pinned rust version, clippy lints, remove some dead code

New Features

• improve quickstart ergonomics

• add utility to dedup tees when debugging IR

• add decouple and simple test and two_pc

• implement support for external network outputs

• add ability to have staged flows inside unit tests
  Whenever a Hydroflow+ program is compiled, it depends on a generated
  __staged module, which contains the entire contents of the crate but
  with every type / function made pub and exported, so that the compiled
  UDFs can resolve local references appropriately.

  Previously, we would not do this for #[cfg(test)] modules, since they
  may use dev-dependencies and therefore the generated module may fail
  to compile when not in test mode. To solve this, when running a unit
  test (marked with hydroflow_plus::deploy::init_test()) that uses
  trybuild, we emit a version of the __staged module with #[cfg(test)]
  modules included into the generated trybuild sources because we can
  guarantee via trybuild that the appropriate dev-dependencies are
  available.

  This by itself allows crates depending on hydroflow_plus to have local
  unit tests with Hydroflow+ logic inside them. But we also want to use
  this support for unit tests inside hydroflow_plus itself. To enable
  that, we eliminate the hydroflow_plus_deploy crate and move its
  contents directly to hydroflow_plus itself so that we can access the
  trybuild machinery without incurring a circular dependency.

  Also fixes #1408

• add API for external network inputs
  This is a key step towards being able to unit-test HF+ graphs, by being
  able to have controlled inputs. Outputs next.

• splice UDFs with type hints to avoid inference failures

Bug Fixes

• properly handle crate:: imports

• be more careful about which parts of proposer and acceptor have to be maintained atomically

• adjust default features to allow compilation to musl targets
  Previously, the default deploy feature would pull in Hydro Deploy and
  its transitive native dependencies.

  Also sets up examples/paxos.rs with CLI flags to deploy to GCP.

• add missing sample_every for singletons
  Discovered during a live-coding demo, we only had it for optionals
  before.

Refactor

• move rewrites to a submodule
• move HfCompiled and friends to a module
• use location.flow_state() to avoid clone
• deduplicate some error messages and drop unused Interval IR node
• dedup signatures for Stream operators
• clean up traits for cycles and forward references
• split up location module and store locations directly in streams
• use usize for slot numbers
• make Paxos-KV generic
• simplify latency calculations
• complete split into leader election and sequencing phases

• start splitting out leader election into a separate module

  Stack created with Sapling. Best reviewed
  with
  ReviewStack.
  • #1493
  • #1492
  • #1489
  • #1488
  • #1487
  • #1486
  • -> #1485
• simplify persist_pullup code
  Instead of matching on &mut and juggling ownership, instead match on
  the owned node and always replaced *node = new_node (sometimes itself)
• use max and min in Paxos and make client generic over ballots

Style

• fixes for nightly clippy
  a couple few spurious too_many_arguments and a spurious
  zombie_processes still on current nightly (clippy 0.1.84 (4392847410 2024-10-21))

New Features (BREAKING)

• implicitly apply default optimizations
  This also changes the behavior of with_default_optimize to be
  terminal, if users want to apply optimizations after these they should
  explicitly invoke the optimizations.
• add an explicit API for creating tick contexts
  Previously, each location had a (semantic) global clock that drives
  ticks, and so all streams in a tick domain were all in the same atomic
  block. For future optimizations, we'd like developers to be able to
  place streams on the same location into different clocks to eliminate
  synchronization between them, which in turn would allow the computations
  in those separate clocks to be potentially decoupled across machines.
• strongly-typed runtime cluster IDs
  Instead of u32s everywhere, we now have a ClusterId<C> type that
  ensures that cluster IDs are not misused.
• provide an API for creating cycles across tick iterations
  Towards making it more clear which parts of a program depend on ticks
  versus don't.

Refactor (BREAKING)

• eliminate remaining Hf name prefixes
• location type parameter before boundedness
  When looking at a prefix in an IDE, the location type argument is
  generally more useful.
• dedup signatures for Singleton and Optional
  Also renames cross_singleton to zip when both sides are
  singleton-like.
• fold Tick vs NoTick into the location type parameter
  Now, when the location is a top-level Process or Cluster that
  corresponds to a NoTick, and for streams inside a tick we wrap the
  location type (e.g. Tick<Process<...>>). This simplifies type
  signatures for a lot of our example code.
• simplify intervals and split Paxos-KV into separate module
• move input APIs back to being on locations
• move self_id and members to be APIs on cluster instead of builder

Commit Statistics

• 38 commits contributed to the release.
• 69 days passed between releases.
• 38 commits were understood as conventional.
• 38 unique issues were worked on: #1434, #1441, #1443, #1444, #1449, #1450, #1451, #1453, #1455, #1461, #1464, #1468, #1471, #1477, #1485, #1486, #1488, #1491, #1505, #1515, #1516, #1517, #1519, #1521, #1523, #1524, #1525, #1526, #1527, #1540, #1541, #1542, #1543, #1550, #1551, #1553, #1554, #1557

Commit Details

view details

• #1434
  • Splice UDFs with type hints to avoid inference failures (60d9bec)
• #1441
  • Provide an API for creating cycles across tick iterations (4f3b51b)
• #1443
  • Use max and min in Paxos and make client generic over ballots (c752aff)
• #1444
  • Update pinned rust version, clippy lints, remove some dead code (d567760)
• #1449
  • Add API for external network inputs (8a80931)
• #1450
  • Add ability to have staged flows inside unit tests (afe78c3)
• #1451
  • Implement support for external network outputs (074f2cf)
• #1453
  • Add decouple and simple test and two_pc (2141c5f)
• #1455
  • Simplify persist_pullup code (1b18b35)
• #1461
  • Add missing sample_every for singletons (d4320e3)
• #1464
  • Adjust default features to allow compilation to musl targets (87a6834)
• #1468
  • Move self_id and members to be APIs on cluster instead of builder (8ad997b)
• #1471
  • Move input APIs back to being on locations (30c4f70)
• #1477
  • Strongly-typed runtime cluster IDs (edd8649)
• #1485
  • Start splitting out leader election into a separate module (dff2a40)
• #1486
  • Complete split into leader election and sequencing phases (8b7b1c6)
• #1488
  • Be more careful about which parts of proposer and acceptor have to be maintained atomically (275a0ed)
• #1491
  • Add utility to dedup tees when debugging IR (98a21e3)
• #1505
  • Fixes for nightly clippy (47cb703)
• #1515
  • Simplify latency calculations (38b17cd)
• #1516
  • Simplify intervals and split Paxos-KV into separate module (e5b456b)
• #1517
  • Make Paxos-KV generic (0a5abab)
• #1519
  • Fold Tick vs NoTick into the location type parameter (5657563)
• #1521
  • Use usize for slot numbers (534fe97)
• #1523
  • Split up location module and store locations directly in streams (d9634f2)
• #1524
  • Clean up traits for cycles and forward references (bf9dcd5)
• #1525
  • Dedup signatures for Stream operators (244207c)
• #1526
  • Dedup signatures for Singleton and Optional (919099e)
• #1527
  • Properly handle crate:: imports (2faffdb)
• #1540
  • Deduplicate some error messages and drop unused Interval IR node (5b819a2)
• #1541
  • Use location.flow_state() to avoid clone (9f74405)
• #1542
  • Move HfCompiled and friends to a module (e9d05bf)
• #1543
  • Move rewrites to a submodule (a1b4520)
• #1550
  • Add an explicit API for creating tick contexts (5d5209b)
• #1551
  • Location type parameter before boundedness (9107841)
• #1553
  • Improve quickstart ergonomics (baedf23)
• #1554
  • Eliminate remaining Hf name prefixes (0bd3a2d)
• #1557
  • Implicitly apply default optimizations (8d8b4b2)