Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added Map Reduce Design Pattern #3184

Open
wants to merge 4 commits into
base: master
Choose a base branch
from
Open

Conversation

hvgh88
Copy link

@hvgh88 hvgh88 commented Jan 31, 2025

Description:
Implemented MapReduce design pattern

Issue: Close #2927

Copy link

github-actions bot commented Jan 31, 2025

PR Summary

This PR implements the MapReduce design pattern, including mapper, shuffler, and reducer components, along with unit tests and comprehensive documentation. It processes large datasets in parallel, improving efficiency.

Changes

File Summary
map-reduce/README.md This file provides a comprehensive guide to the MapReduce design pattern in Java. It includes a detailed explanation, real-world examples, a programmatic example with code snippets, and a discussion of benefits and trade-offs.
map-reduce/etc/map-reduce.png New file: Diagram showing the MapReduce pattern.
map-reduce/etc/map-reduce.urm.puml New file: UML diagram for the MapReduce classes.
map-reduce/pom.xml This file configures the Maven project for the MapReduce module, including dependencies and build settings.
map-reduce/src/main/java/com/iluwatar/Main.java This is the main class that runs the MapReduce process and prints the results to the console.
map-reduce/src/main/java/com/iluwatar/MapReduce.java This class orchestrates the MapReduce process by calling the mapper, shuffler, and reducer components.
map-reduce/src/main/java/com/iluwatar/Mapper.java This class implements the map function, processing input strings and generating key-value pairs.
map-reduce/src/main/java/com/iluwatar/Reducer.java This class implements the reduce function, aggregating values for each key.
map-reduce/src/main/java/com/iluwatar/Shuffler.java This class implements the shuffle function, grouping values by key.
map-reduce/src/test/java/com/iluwatar/MapReduceTest.java This file contains unit tests for the MapReduce class, verifying the correctness of the overall process.
map-reduce/src/test/java/com/iluwatar/MapperTest.java This file contains unit tests for the Mapper class, ensuring the correct mapping of input strings.
map-reduce/src/test/java/com/iluwatar/ReducerTest.java This file contains unit tests for the Reducer class, validating the aggregation of values.
map-reduce/src/test/java/com/iluwatar/ShufflerTest.java This file contains unit tests for the Shuffler class, verifying the correct grouping of values by key.
pom.xml The map-reduce module was added to the root pom.xml file.

autogenerated by presubmit.ai

Copy link

@github-actions github-actions bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

Review Summary

Commits Considered (2)
Files Processed (13)
  • map-reduce/README.md (1 hunk)
  • map-reduce/etc/map-reduce.png (0 hunks)
  • map-reduce/etc/map-reduce.urm.puml (1 hunk)
  • map-reduce/pom.xml (1 hunk)
  • map-reduce/src/main/java/com/iluwatar/Main.java (1 hunk)
  • map-reduce/src/main/java/com/iluwatar/MapReduce.java (1 hunk)
  • map-reduce/src/main/java/com/iluwatar/Mapper.java (1 hunk)
  • map-reduce/src/main/java/com/iluwatar/Reducer.java (1 hunk)
  • map-reduce/src/main/java/com/iluwatar/Shuffler.java (1 hunk)
  • map-reduce/src/test/java/com/iluwatar/MapReduceTest.java (1 hunk)
  • map-reduce/src/test/java/com/iluwatar/MapperTest.java (1 hunk)
  • map-reduce/src/test/java/com/iluwatar/ReducerTest.java (1 hunk)
  • map-reduce/src/test/java/com/iluwatar/ShufflerTest.java (1 hunk)
Actionable Comments (0)
Skipped Comments (0)

Copy link
Owner

@iluwatar iluwatar left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The new module should be added to parent pom.xml. Otherwise it won't be built by CI.

Copy link

@github-actions github-actions bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

Review Summary

Commits Considered (1)
  • 8fc6130: added module to parent pom
Files Processed (2)
  • map-reduce/README.md (1 hunk)
  • pom.xml (1 hunk)
Actionable Comments (0)
Skipped Comments (0)

Copy link

@github-actions github-actions bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

Review Summary

Commits Considered (1)
  • 30f97de: Merge branch 'master' into map-reduce
Files Processed (1)
  • pom.xml (1 hunk)
Actionable Comments (0)
Skipped Comments (0)

Copy link

Quality Gate Failed Quality Gate failed

Failed conditions
67.2% Coverage on New Code (required ≥ 80%)

See analysis details on SonarQube Cloud

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

MapReduce design pattern
2 participants