V5.0 - Adding Integration of the Google Gemini API to filter the candidates from the substantial decrease CSV File

Release 5.0 - Adding Integration of the Google Gemini API

This release introduces the integration of the Google Gemini API, enhancing the filtering process for candidates from the substantial decrease CSV file. Several significant improvements and features have been added, alongside various refactoring updates to enhance code quality and maintainability.

Notable Changes

• Feature Additions:

  • Integrated Google Gemini API to filter candidates from the substantial decrease CSV file.
  • Added multiple runs for the Gemini API Python code, allowing for better data validation.
  • Implemented similarity calculations for outputs from multiple runs.
  • Added threading support to execute multiple runs concurrently.
  • Introduced a retry mechanism with exponential backoff to enhance API reliability.
  • Included interval of confidence calculation for the Gemini API integration.
  • Implemented a semaphore in the Gemini Python code to prevent multiple requests and avoid being blocked by the API.

• Refactoring and Improvements:

  • Improved the refactoring types dictionary conversion to string format.
  • Updated CSV headers and corresponding documentation for clarity.
  • Enhanced the CK JAR file generation process and documentation.
  • Updated the Gemini Python code structure, including various utility functions for improved readability and organization.
  • Added verbose outputs and improved logging for better debugging.

• Documentation Updates:

  • Comprehensive updates to the documentation for both the substantial changes CSV header and the CK JAR file generation process.
  • Improved function documentation within the Gemini Python code for better clarity.

Full Changelog: v4.2...v5.0

V4.2 - Adding detection of Refactoring from RefactoringMiner in PyDriller/metrics_changes.py

Release 4.2 - Adding Detection of Refactoring from RefactoringMiner in PyDriller/metrics_changes.py

Summary of Changes

Release 4.2 introduces significant improvements to the PyDriller tool by adding detection capabilities for refactoring changes using data from RefactoringMiner. This release focuses on enhancing the CSV file output related to substantial changes and improving overall functionality.

Key Updates

• Refactoring Enhancements:

  • Implemented logic to generate refactoring information specifically for the after state of substantial decreases, allowing for more accurate tracking of code changes.
  • Improved the format and structure of the substantial changes CSV file, including modifications to the header and string formats for better readability.

• Repository Management:

  • Integrated the setup_repository function to streamline the process of cloning or updating the zookeeper repository for generating refactoring files.

• CSV Output Improvements:

  • Updated the order of CSV fields generated by the metrics changes Python code to ensure a logical flow of information.
  • Enhanced the refactoring types dictionary to count occurrences and avoid duplication of refactoring types for the same file path.

• Cross-Platform Compatibility:

  • Improved the handling of file paths across different operating systems (Linux, Mac, and Windows) to ensure consistent functionality.

• Documentation Updates:

  • Updated the documentation for generated outputs and the PyDriller metrics changes workflow to reflect the latest changes and improve user understanding.

Conclusion

With these enhancements, release 4.2 provides a more robust and informative experience for users analyzing code metrics in PyDriller, particularly focusing on the detection and reporting of refactoring changes.

Full Changelog: v4.1...v4.2

V4.1 - Improving the Substantial Metrics Change PyDriller Feature CSV File Output

Release 4.1 - Improving the Substantial Metrics Change PyDriller Feature CSV File Output

Summary of Changes

The transition from release 4.0 to 4.1 introduced several enhancements and refinements to the PyDriller tool. Key improvements include:

1. Submodule Management:

   • Integrated my forked CK project as a submodule in the Worked-Example-Miner tool, ensuring a seamless update process for the CK tool.
   • Implemented a GitHub Actions workflow to automatically update the CK submodule every hour.

2. Code Refactoring:

   • Updated various directory and branch names within the workflow to maintain consistency.
   • Refactored functions in the PyDriller code to improve readability and structure, including updating the return values and order of function calls to follow a bottom-up standard.

3. Verbose Output Functionality:

   • Introduced a verbose mode function that enhances debugging and tracking by printing detailed outputs in the PyDriller code.
   • Replaced standard print statements with the new verbose output function across multiple Python files for consistency.

4. Metrics Enhancements:

   • Added an occurrences counter to the metrics changes code, enhancing the output files generated by PyDriller.
   • Improved the CSV file output of the substantial metrics change feature, ensuring clearer and more comprehensive data representation.

5. Documentation Updates:

   • Updated documentation and docstring conventions to reflect changes in the codebase and improve clarity for future developers.

6. Dependency Management:

   • Upgraded the pillow library dependency from version 10.2.0 to 10.3.0, ensuring compatibility and incorporating the latest features and fixes.

7. Fixes:

   • Addressed issues with handling inner class names from the RefactoringMiner file path in JSON outputs, improving the robustness of the tool.

Conclusion

These updates aim to enhance the functionality and maintainability of the PyDriller tool, providing a more effective environment for analyzing and processing code metrics.

What's Changed

• Build(deps): Bump pillow from 10.2.0 to 10.3.0 in /PyDriller by @dependabot in #1

Full Changelog: v4.0...v4.1

V4.0 - Adding my Forked CK Project as a Submodule of the Worked-Example-Miner Tool

Release 4.0 - Adding my Forked CK Project as a Submodule of the Worked-Example-Miner Tool

Key Changes:

Refactoring Improvements:

• Enhanced the calculation of the Percentual Change Value to range between 0-100 with two decimal precision.
• Improved the overall structure of the Metrics Changes and Code Metrics Python files for better readability and maintainability.
• Eliminated redundant calculations and optimized function definitions to enhance performance.

Feature Additions:

• Implemented the deletion of the old substantial changes file during the first substantial change verification.
• Introduced new constants for paths related to RefactoringMiner refactorings.
• Added functions to generate refactoring files and validate paths for the RefactoringMiner tool.
• Integrated a mechanism for PyDriller to automatically retrieve commit counts for repositories.
• Added the VERBOSE output constant across various Python files for better debugging information.

Documentation Updates:

• Transitioned from JavaDoc to Python docstrings in multiple files for improved documentation practices.
• Enhanced comments throughout the codebase to provide clearer explanations of functionalities and processes.
• Introduced a CONTRIBUTING file to guide new contributors on how to engage with the project.

Makefile Enhancements:

• Updated the Makefile to utilize a Requirements.txt for managing dependencies.
• Added rules for generating the Requirements.txt for both PyDriller and RefactoringMiner directories.

License Addition:

• Included the Apache-2.0 license file to formalize the project’s licensing structure.

Refactoring:

• Various files and functions were renamed to follow Python naming conventions, enhancing clarity (e.g., renaming extractZipFiles to extract_zip_files).
• Enhanced output messages for all major functions to improve user feedback during execution.

Version Control and Recovery:

• Recovered missed files through the use of Git rebase to ensure all relevant changes are incorporated.

Summary of Commits:

• Refactors focused on code quality, readability, and efficiency.
• Features added to improve functionality and user interaction.
• Documentation updates for clarity and contribution guidelines.
• Licensing formalities to ensure compliance and openness.

This release marks a significant step forward in integrating the forked CK project into the Worked-Example-Miner tool, providing enhanced metrics capabilities and improved usability for contributors and users alike.

Full Changelog: v3.3...v4.0

V3.3 - Sorting the candidates by Percentual Variation of the metric between iterations

Release 3.3 - Sorting the Candidates by Percentual Variation of the Metric Between Iterations

The 3.3 release introduces significant enhancements aimed at improving the code structure and functionality of the PyDriller project. This release focuses on refining the metrics calculation and sorting mechanisms, allowing for a more efficient analysis of candidate variations over iterations. Below are the key changes made in this release:

Key Changes

Refactoring

• Progress Output Messages: The resuming progress output messages in PyDriller/Code_Metrics.py have been updated for better clarity and user experience. (Commit: 2023-11-03)

• Code Style Improvements: Standardization of string delimiters was implemented by updating single quotes to double quotes across the PyDriller/code_metrics.py file. (Commits: 2023-11-06)

• Background Color Updates: The background colors for various components, including the Python algorithm and scientific research scripts, have been updated to enhance visual consistency. (Commits: 2023-11-11)

• Play Sound Feature: The code for the "Play Sound At Exit" feature was refined for better functionality. (Commit: 2023-11-12)

• README Updates: The Skills List, License Section, and several new sections (Directories, Tools, Repositories, and Methodology) were added and updated in the README.md to provide clearer documentation and guidance for users. (Commits: 2023-12-12)

• Makefile Improvements: The Makefile has been updated to ensure correct functionality and to remove unnecessary dependencies. (Commits: 2024-01-30, 2024-03-06)

New Features

• Commit Tracking Enhancements: The addition of commit order numbers and hashes to the CSV for the Substantial Changes Detector allows for better tracking of changes over time. (Commit: 2024-03-11)

• Ignore Keywords Implementation: The ability to ignore certain keywords for class names or variable attributes was added to enhance the Substantial Changes Identifier functionality. (Commit: 2024-03-11)

• Percentual Change Calculation: A new feature was introduced to improve the percentual change value calculation, ensuring it falls within a 0-100 range with two decimal places, enhancing accuracy in metrics evaluation. (Commit: 2024-03-11)

Bug Fixes

• Commit Number Retrieval: Resolved an issue with retrieving the commit number from the commit hash, ensuring accurate data representation. (Commit: 2024-03-11)

Documentation Improvements

• Metrics_Track_Record Documentation: Comprehensive explanations of the Metrics_Track_Record dictionary values and structure were added to improve understanding and usability. (Commit: 2024-03-05)

Conclusion

With these updates, release 3.3 not only improves the overall code quality and documentation of the PyDriller project but also enhances the user experience by refining metrics calculations and providing better tracking capabilities. These changes pave the way for more effective analysis and utilization of metrics in Python projects.

Full Changelog: v3.2...v3.3

V3.2 - Adding a file to resume stopped executions of the extraction of the CK Metrics in PyDriller/code_metrics.py

V3.2 - Adding a file to resume stopped executions of the extraction of the CK Metrics in PyDriller/code_metrics.py

Refactorings

ck_metrics.py Updates

• Updating to create modified files diff.
• Updating to work with multiple repositories.
• Import adjustments and unused function removals.
• Updates to outputs and commits list CSV file.
• Renaming to code_metrics.py.
• Adding comments for better code understanding.

metrics_changes.py Updates

• General refactoring for clarity and performance.
• Importing existing functions from ck_metrics.py.
• Enhancements to paths, outputs, and error handling.
• Adding progress bars for user feedback during execution.

Shell Script Updates

• Improvements to generateZipFiles.sh and moveExtractedFiles.sh.
• Adding new ShellScripts (generateShortZipFiles, extractZipFiles).

Other Files

• Updating README.md multiple times for better documentation.
• Deleting generate_images.py.

Fixes

• Fixed errors in metrics_changes.py related to traversing CK metrics directory.
• Corrected the order of commit hashes in the metrics evolution.

Features

• Added RefactoringMiner for improved refactoring analysis.
• Introduced background color handling and macros in RefactoringMiner/main.py.
• Concurrent processing functions for repository analysis.

Notes

• Continuous emphasis on code legibility, maintainability, and reusability throughout all updates.

Summary

The focus of the commits in this release was primarily on refactoring existing code for clarity and efficiency, alongside introducing new features to enhance functionality, particularly in the area of repository processing and metrics analysis. The documentation has been thoroughly updated to support these changes.

Full Changelog: v3.1...v3.2

V3.1 - Adding the Modified Files (Diff) for each commit

V3.1 - Adding the Modified Files (Diff) for each commit

Summary of Commit Messages from v3.0 to v3.1

Major Changes:

• Refactorings:

  • Numerous updates to improve existing code structure, particularly in scripts like generate_images.py, metrics_changes.py, and file_diffs.py.
  • Enhancements to output logs and variable management.
  • Refactoring of iteration processes for image generation and IDs validation.

• Features Added:

  • Addition of new scripts including moveExtractedFiles.sh and diff.py.
  • Implementation of image generation criteria (e.g., MINIMUM CHANGES for images).

• Fixes:

  • Several fixes addressing issues in generate_images.py, particularly concerning labels and paths.

• Documentation:

  • Updates to documentation for metrics_changes.py and generate_images.py, ensuring clarity on the new changes and workflows.

• File Management:

  • Deletion and update of generated files and constants, including updates to .gitignore to manage ignored files better.

Detailed Breakdown:

• Refactoring Activities:

  • Improving save directory paths and output information.
  • Enhancements to method definitions and variable names for clarity and consistency.

• Feature Additions:

  • New shell scripts for managing file operations, contributing to streamlined processes.
  • Multiple updates to commit history management scripts.

• Fixes:

  • Addressed behavior inconsistencies in image generation scripts and ensured correct variable references.

Conclusion:

This series of commits reflects a concentrated effort on improving code quality, enhancing functionality through new features, and addressing issues from previous versions, contributing to a more robust and maintainable codebase.

Full Changelog: v3.0...v3.1

V3.0 - Adding Linear Regression Predictions

Release 3.0 - Adding Linear Regression Predictions

This release introduces significant enhancements and new features, primarily focused on integrating linear regression predictions into the metrics analysis workflow. Below are the key updates between v2.1 and v3.0:

• Linear Regression Prediction: Added a new function for linear regression predictions, allowing for better forecasting of metric changes based on historical data.
• Notification Sound: Introduced a notification sound that plays at the end of execution, providing users with enhanced feedback on process completion.
• Elapsed Execution Time: Implemented a feature to display the elapsed execution time for key processes, offering performance insights to users.
• Code Organization: Updated the metrics_changes.py file to include new directories for constants and libraries, ensuring a more organized structure.
• Function Improvements: Refactored multiple functions, including minor changes to improve readability and efficiency, particularly in the write_metrics_evolution_to_csv function.
• Prediction Directory Updates: Enhanced the linear prediction save directory structure to accommodate growing metrics data.
• Documentation Updates: Revised comments throughout the codebase to improve clarity and guidance for future developers.

This release marks a pivotal enhancement in our metrics analysis capabilities, empowering users with advanced prediction functionalities while maintaining a clear and organized code structure.

Full Changelog: v2.0...v3.0

V2.0 - Calculating Metrics Statistics (Min, Max, AVG and Q3) for the Classes and Methods.

Release 2.0 - Calculating Metrics Statistics (Min, Max, AVG and Q3) for the Classes and Methods.

This release introduces significant enhancements and new features, primarily focused on the analysis of method metrics within PyDriller. Below are the key updates between v1.1 and v2.0:

• Statistics calculation: Implemented a feature for calculating various statistics metrics, improving the analytical capabilities of the tool.
• Algorithm development: Introduced an algorithm to identify methods that have changed the most, offering valuable information for code reviews and refactoring.
• Refactoring and cleanup: A comprehensive refactoring of the codebase, including multiple updates to PyDriller/main.py, improving maintainability and performance.
• Documentation updates: The README file has been extensively updated to reflect new features and provide clearer guidance.
• Whitespace handling: Enhanced code functionality to handle whitespaces more effectively, ensuring accurate metric calculations.
• Pandas integration: Added support for the pandas library to facilitate data manipulation and analysis.

This release marks a major milestone in the development of PyDriller, enhancing its ability to analyze method-level metrics and providing users with valuable insights into code quality and changes.

Full Changelog: v1.1...v2.0

V1.1 - Adding the Metrics Evolution Files Generated using PyDriller.

Release 1.1 - Finishes metrics evolution files generated on PyDriller

This release includes significant improvements and new features, primarily focusing on metrics evolution within PyDriller. Below are the key updates between v1.0 and v1.1:

• Major PyDriller integration for enhanced metrics tracking and evolution.
• Directory path improvements: Paths are now dynamically set using the current working directory (PWD), streamlining file management.
• Legibility improvements: Output formatting has been refined for clearer and more readable results.
• Whitespace and formatting checks: Additional validation for whitespaces was added to maintain code quality.
• Gitignore updates to better handle files generated during the process.
• Processed data inclusion: Added support for processed data from the Apache Commons Lang library.
• Metrics evolution functionality: A new function to track and visualize the evolution of various metrics has been completed, marking a major enhancement to the tool.

This release finalizes the integration of metrics evolution within PyDriller, offering new insights and capabilities for repository analysis.

Full Changelog: v1.0...v1.1