Skip to content

Conversation

@triceo
Copy link
Collaborator

@triceo triceo commented Oct 30, 2025

Previously, Neighborhoods depended on joins on the left-hand side - these joins are, however, possibly massive. With large datasets, joining 100k entities with themselves would result in billions of entries per move. This is clearly not acceptable and would never scale.

The new approach only uses joins within pick(), and these joins will be enumerated on-demand. Therefore the memory use of this mechanism will be minimized. (Although possibly still significant, especially for random selection.)

The extra benefit of this approach is that it actually respects the order of entities and facts that was given by the user.

Further improvements to scalability are likely still possible, but will be introduced when the situation calls for it. (Which I expect it will.)

@triceo triceo marked this pull request as ready for review October 30, 2025 09:03
@triceo triceo requested review from Copilot and zepfred October 30, 2025 09:03
@triceo triceo added this to the v1.28.0 milestone Oct 30, 2025
@triceo triceo self-assigned this Oct 30, 2025
@triceo triceo requested review from Copilot and removed request for Copilot October 30, 2025 09:09
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR refactors the move stream framework by replacing the bi-dataset implementation with a joining iterator approach for improved performance. The changes eliminate BiDataset and associated classes, replacing them with iterator-based joining logic that performs filtering on-demand during iteration.

  • Removed BiDataset and replaced with UniDataset-based joining using JoiningIterator
  • Renamed BiEnumeratingFilter to BiEnumeratingPredicate for consistency
  • Updated move definitions to use pick().pick() pattern with joiners instead of join() operations
  • Fixed test assertions to reflect new iteration order (left-to-right instead of cached ordering)

Reviewed Changes

Copilot reviewed 52 out of 52 changed files in this pull request and generated 5 comments.

Show a summary per file
File Description
DefaultBiSamplingStream.java Renamed from DefaultBiFromUnisSamplingStream; updated to use BiEnumeratingJoinerComber instead of BiPredicate
DefaultBiFromBiSamplingStream.java Deleted as BiDataset implementation removed
BiMoveStream.java New implementation replacing FromUniBiMoveStream and FromBiUniMoveStream
BiOriginalMoveIterator.java New iterator for sequential bi-move enumeration using JoiningIterator
BiRandomMoveIterator.java New iterator for randomized bi-move selection with caching
JoiningIterator.java New core iterator implementing joiner and filter logic
BiEnumeratingPredicate.java Renamed from BiEnumeratingFilter for consistency
AbstractDataJoiner.java Deleted; functionality moved to AbstractJoiner in bavet package
BiDataset.java, BiDatasetInstance.java, TerminalBiEnumeratingStream.java Deleted as bi-dataset approach removed
SwapMoveDefinition.java Refactored to use pick().pick() with lessThan joiner for uniqueness
ChangeMoveDefinition.java Refactored to use pick().pick() with differentValueFilter and valueInRangeFilter
ListChangeMoveDefinition.java Updated parameter order in isValidChange predicate
Test files Updated assertions to match new left-to-right iteration order
Comments suppressed due to low confidence (1)

core/src/main/java/ai/timefold/solver/core/impl/neighborhood/maybeapi/stream/enumerating/function/BiEnumeratingPredicate.java:30

  • The lambda implementation doesn't cast to BiEnumeratingPredicate<Solution_, A, B> as the return type specifies. The previous implementation used an explicit cast. Without the cast, this may cause type inference issues in some contexts.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

Copilot reviewed 56 out of 56 changed files in this pull request and generated 3 comments.

Comments suppressed due to low confidence (1)

core/src/main/java/ai/timefold/solver/core/impl/neighborhood/maybeapi/stream/enumerating/function/BiEnumeratingPredicate.java:31

  • [nitpick] The and method now creates a lambda instead of casting TriPredicate.super.and(other). While this works, it loses the potential optimization that the default implementation in TriPredicate might have. Consider whether the explicit implementation is necessary or if the original approach with casting was preferable.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@sonarqubecloud
Copy link

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant