Skip to content

Add review coverage model #601

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 4 commits into from
May 21, 2025
Merged

Conversation

gonzaM
Copy link
Contributor

@gonzaM gonzaM commented May 19, 2025

What does this PR do?

This PR introduces a new code review metric that measures the percentage of files in a pull request that have received comments during the review process.

Changes

  • Add ReviewCoverage model with associations and validations
  • Create necessary database migrations and schema updates
  • Extend GithubClient to fetch PR comments
  • Modify PullRequest model to build review coverage metrics upon merge
  • Add new tests and adjust existent ones.
  • Implement backfill rake task for existing merged PRs

@gonzaM gonzaM requested a review from a team as a code owner May 19, 2025 20:50
@gonzaM gonzaM requested review from lauperalti and brunoparma88 and removed request for a team May 19, 2025 20:50
@gonzaM gonzaM force-pushed the add_review_coverage_model branch from 0845656 to fa09744 Compare May 19, 2025 20:56
@gonzaM gonzaM requested review from rodrieiz and santib and removed request for lauperalti and brunoparma88 May 19, 2025 21:37
validates :total_files_changed, :files_with_comments_count,
numericality: { greater_than_or_equal_to: 0 }

before_save :calculate_coverage_percentage
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it's necessary to be callback or can be calculated when creating/updating the object? Just asking because I think is a good practice to avoid this type of callbacks

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Moved! 💪

@@ -18,6 +18,11 @@ def files
get_all_paginated_items(url, MAX_FILES_PER_PAGE)
end

def comments
url = "repos/#{repository.full_name}/pulls/#{pull_request.number}/comments"
get_all_paginated_items(url, MAX_FILES_PER_PAGE)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

mm should we handle or log possible errors here?

Comment on lines +65 to +75
after(:create) do |pull_request, evaluator|
create(
:merge_time,
pull_request: pull_request,
value: evaluator.merge_time
)
create(
:review_coverage,
pull_request: pull_request
)
end
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this should be created manually in each test that requires it. Maybe using a trait isn’t too bad, as long as we don’t overuse it, wdyt?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Given that this logic is used in many tests, I thought it was a good idea to have the trait to avoid repeating all the logic. Also, it's a simple logic and not a complex trait. Do you think it's ok in this case?

require 'rails_helper'
require 'rake'

# rubocop:disable RSpec/DescribeClass
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I way to find to avoid this is the task only calls another file review_coverage.rb
that have all the logic. feel free to ignore

Comment on lines +1 to +18
class AddReviewCoverageToMetricNameType < ActiveRecord::Migration[7.1]
disable_ddl_transaction!

def up
execute <<-SQL
ALTER TYPE metric_name ADD VALUE 'review_coverage';
SQL
end

def down
execute <<-SQL
ALTER TYPE metric_name RENAME TO metric_name_old;
CREATE TYPE metric_name AS ENUM ('review_turnaround', 'blog_visits', 'merge_time', 'blog_post_count', 'open_source_visits', 'defect_escape_rate', 'pull_request_size', 'development_cycle');
ALTER TABLE metrics ALTER COLUMN name TYPE metric_name USING name::text::metric_name;
DROP TYPE metric_name_old;
SQL
end
end
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this looks very strange to me. How this enum defined ? is not enough to add the new enum options in the model?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This enum is defined as a type at a database level, it's not an integer, and that's why I need to add the value with a migration.

ALTER TYPE metric_name RENAME TO metric_name_old;
CREATE TYPE metric_name AS ENUM ('review_turnaround', 'blog_visits', 'merge_time', 'blog_post_count', 'open_source_visits', 'defect_escape_rate', 'pull_request_size', 'development_cycle');
ALTER TABLE metrics ALTER COLUMN name TYPE metric_name USING name::text::metric_name;
DROP TYPE metric_name_old;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

aren't we losing all the old values?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I understand the previous line is transferring the values.

end

private

attr_reader :pull_request

def handle_exception(exception)
Honeybadger.notify(exception)
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I added Honeybadger because it is the tool being used, but I don't have access to it. I think in a future PR, we can move to Sentry or something similar. I added it anyway to keep track of this error notification.

@gonzaM gonzaM requested a review from rodrieiz May 21, 2025 18:12
Copy link
Contributor

@rodrieiz rodrieiz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks great!

@gonzaM gonzaM merged commit a6fc86e into feature/review_coverage May 21, 2025
2 checks passed
@gonzaM gonzaM deleted the add_review_coverage_model branch May 21, 2025 19:37
@gonzaM gonzaM mentioned this pull request May 27, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants