Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Disallow creating or updating scheduler if invalid #2214

Open
wants to merge 6 commits into
base: master
Choose a base branch
from

Conversation

jhs-panda
Copy link
Contributor

Description

When creating or updating a scheduler, tests visual run to ensure action can be loaded and works.
If visual run test works, allow scheduler creation/update.
If visual run test doesn't work on creation, delete created scheduler.
If visual run test doesn't work on update, reinstate previous scheduler prior to update attempt.

Motivation and Context

Resolves #2187

How Has This Been Tested?

Before:
Allows creation as long as inputs to Scheduler Creation filled. However, when running scheduler, gets many errors.
Screen Shot 2024-09-22 at 2 08 09 PM

After:
Notes error if action does not exist or is not readable:
Screen Shot 2024-09-22 at 2 13 12 PM

Notes error if action exists and is readable but cannot execute properly:
Screen Shot 2024-09-22 at 2 15 31 PM

If update is not successful, notes error and reverts to previous state:
Screen Shot 2024-09-22 at 2 16 42 PM

For successful actions that load properly, work when run:
Screen Shot 2024-09-22 at 2 17 58 PM

Types of changes

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to change)

Checklist:

  • I have run rubocop and erblint for style check. If you haven't, run overcommit --install && overcommit --sign to use pre-commit hook for linting
  • My change requires a change to the documentation, which is located at Autolab Docs
  • I have updated the documentation accordingly, included in this PR

Other issues / help required

Copy link
Contributor

coderabbitai bot commented Sep 22, 2024

📝 Walkthrough

Walkthrough

The changes made to the SchedulersController focus on improving the create, update, and visual_run actions by adding validations for the existence and readability of action files specified in the scheduler parameters. If these validations fail, users receive error messages and are redirected to the appropriate forms. New methods for validating and executing action files have also been introduced, enhancing error handling and allowing for rollback in case of failures during visual runs.

Changes

Files Change Summary
app/controllers/schedulers_controller.rb Enhanced create, update, and visual_run actions with validations for action files. Added methods for validating, running, and executing action files with improved error handling.

Assessment against linked issues

Objective Addressed Explanation
Validations could require the action to be an existing file (Issue #2187)
The controller could require that a successful visual run occur before enabling a job (Issue #2187)
Scheduler could auto-disable or do exponential back-off for failing jobs (Issue #2187) No implementation for auto-disable or back-off logic.
Scheduler errors that are likely to be course errors could be caught and logged but not passed to exception notifier (Issue #2187) Unclear if specific logging mechanisms were implemented.

Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media?

❤️ Share
🪧 Tips

Chat

There are 3 ways to chat with CodeRabbit:

  • Review comments: Directly reply to a review comment made by CodeRabbit. Example:
    • I pushed a fix in commit <commit_id>, please review it.
    • Generate unit testing code for this file.
    • Open a follow-up GitHub issue for this discussion.
  • Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query. Examples:
    • @coderabbitai generate unit testing code for this file.
    • @coderabbitai modularize this function.
  • PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
    • @coderabbitai gather interesting stats about this repository and render them as a table. Additionally, render a pie chart showing the language distribution in the codebase.
    • @coderabbitai read src/utils.ts and generate unit testing code.
    • @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.
    • @coderabbitai help me debug CodeRabbit configuration file.

Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments.

CodeRabbit Commands (Invoked using PR comments)

  • @coderabbitai pause to pause the reviews on a PR.
  • @coderabbitai resume to resume the paused reviews.
  • @coderabbitai review to trigger an incremental review. This is useful when automatic reviews are disabled for the repository.
  • @coderabbitai full review to do a full review from scratch and review all the files again.
  • @coderabbitai summary to regenerate the summary of the PR.
  • @coderabbitai resolve resolve all the CodeRabbit review comments.
  • @coderabbitai configuration to show the current CodeRabbit configuration for the repository.
  • @coderabbitai help to get help.

Other keywords and placeholders

  • Add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.
  • Add @coderabbitai summary to generate the high-level summary at a specific location in the PR description.
  • Add @coderabbitai anywhere in the PR title to generate the title automatically.

CodeRabbit Configuration File (.coderabbit.yaml)

  • You can programmatically configure CodeRabbit by adding a .coderabbit.yaml file to the root of your repository.
  • Please see the configuration documentation for more information.
  • If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation: # yaml-language-server: $schema=https://coderabbit.ai/integrations/schema.v2.json

Documentation and Community

  • Visit our Documentation for detailed information on how to use CodeRabbit.
  • Join our Discord Community to get help, request features, and share feedback.
  • Follow us on X/Twitter for updates and announcements.

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 5

Outside diff range and nitpick comments (2)
app/controllers/schedulers_controller.rb (2)

29-30: Correct error message formatting to avoid unintended newlines

The error message string spans multiple lines without proper concatenation, which can include unintended newline characters in the displayed message. To ensure the error message is formatted correctly, combine the string into a single line or use string concatenation.

Apply this diff to fix the formatting:

- flash[:error] = "Scheduler create failed. Action file does not exist or is
-   not readable at #{action_path}."
+ flash[:error] = "Scheduler create failed. Action file does not exist or is " \
+   "not readable at #{action_path}."

109-110: Correct error message formatting to avoid unintended newlines

Similar to the issue in the create method, the error message here spans multiple lines without proper concatenation. To ensure the message is displayed correctly, combine the string into a single line or use string concatenation.

Apply this diff to fix the formatting:

- flash[:error] = "Scheduler update failed. Action file does not exist or is
-   not readable at #{action_path}."
+ flash[:error] = "Scheduler update failed. Action file does not exist or is " \
+   "not readable at #{action_path}."
Review details

Configuration used: CodeRabbit UI
Review profile: CHILL

Commits

Files that changed from the base of the PR and between be48fe4 and 352afba.

Files selected for processing (1)
  • app/controllers/schedulers_controller.rb (3 hunks)
Additional comments not posted (2)
app/controllers/schedulers_controller.rb (2)

35-44: Robust error handling added to ensure scheduler validity

The addition of error handling in the create method ensures that if the test_run_visual_scheduler fails, the created scheduler is destroyed, and an appropriate error message is displayed. This enhances the reliability of the scheduler creation process.


106-113: Refactor repeated file existence checks into a private method

As mentioned earlier, the file existence and readability check here is duplicated from the create method. Extracting this logic into a private method will reduce duplication and improve code maintainability.

app/controllers/schedulers_controller.rb Outdated Show resolved Hide resolved
app/controllers/schedulers_controller.rb Outdated Show resolved Hide resolved
app/controllers/schedulers_controller.rb Outdated Show resolved Hide resolved
app/controllers/schedulers_controller.rb Outdated Show resolved Hide resolved
app/controllers/schedulers_controller.rb Outdated Show resolved Hide resolved
@jhs-panda jhs-panda requested review from a team and 20wildmanj and removed request for a team September 27, 2024 21:50
@@ -23,9 +23,25 @@ def new; end
action_auth_level :create, :instructor
def create
@scheduler = @course.scheduler.new(scheduler_params)
# Check if the action file exists and is readable
action_path = Rails.root.join(scheduler_params[:action]).to_path
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you also check that the action is within the assessment directory? See Archive.in_dir? which for example is used in handout.rb.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi! Do you mean the course directory? From what I can tell schedulers are for courses, not specific assessments.

Copy link
Contributor

@20wildmanj 20wildmanj left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Functionally this works pretty nicely! But I think there should be some refactoring of the code so that it's a bit more maintainable.

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 6

📜 Review details

Configuration used: CodeRabbit UI
Review profile: CHILL

📥 Commits

Reviewing files that changed from the base of the PR and between 352afba and 25d78cf.

📒 Files selected for processing (1)
  • app/controllers/schedulers_controller.rb (3 hunks)

app/controllers/schedulers_controller.rb Show resolved Hide resolved
app/controllers/schedulers_controller.rb Show resolved Hide resolved
app/controllers/schedulers_controller.rb Outdated Show resolved Hide resolved
app/controllers/schedulers_controller.rb Show resolved Hide resolved
app/controllers/schedulers_controller.rb Show resolved Hide resolved
Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🧹 Outside diff range and nitpick comments (1)
app/controllers/schedulers_controller.rb (1)

Line range hint 26-182: Consider architectural improvements for better maintainability

The current implementation could benefit from the following architectural improvements:

  1. Extract scheduler validation and execution logic into a service class to improve separation of concerns
  2. Move allowed paths configuration to a central configuration file
  3. Implement a more robust error handling strategy with custom exceptions

Example service class structure:

class SchedulerService
  def initialize(course)
    @course = course
  end

  def validate_and_run(scheduler)
    validate_path(scheduler.action)
    validate_file(scheduler.action)
    run_visual_test(scheduler)
  end

  private

  def validate_path(action_path)
    # Path validation logic
  end

  def validate_file(action_path)
    # File validation logic
  end

  def run_visual_test(scheduler)
    # Visual test logic
  end
end
📜 Review details

Configuration used: CodeRabbit UI
Review profile: CHILL

📥 Commits

Reviewing files that changed from the base of the PR and between 25d78cf and 5f48195.

📒 Files selected for processing (1)
  • app/controllers/schedulers_controller.rb (3 hunks)
🧰 Additional context used
📓 Learnings (1)
app/controllers/schedulers_controller.rb (1)
Learnt from: jhs-panda
PR: autolab/Autolab#2214
File: app/controllers/schedulers_controller.rb:131-141
Timestamp: 2024-11-13T17:02:19.398Z
Learning: The error detection logic in the `run_visual_scheduler` method in `app/controllers/schedulers_controller.rb` has changed, and string matching is no longer used.
🔇 Additional comments (1)
app/controllers/schedulers_controller.rb (1)

57-59: ⚠️ Potential issue

Add error handling to visual_run action

The action should handle potential exceptions from execute_action and provide appropriate feedback to users.

 def visual_run
   @scheduler = Scheduler.find(params[:scheduler_id])
-  @log = execute_action(@scheduler)
+  begin
+    @log = execute_action(@scheduler)
+  rescue StandardError => e
+    @log = "Error executing scheduler: #{e.message}"
+    flash[:error] = "Visual test failed"
+  end
   render partial: "visual_test"
 end

Likely invalid or redundant comment.

Comment on lines +26 to +42
action_path = Rails.root.join(scheduler_params[:action]).to_path
# Check if the action file exists, is readable, and compiles
if validate_compile_action_file(action_path)
if @scheduler.save
# Ensure visual run is successful
if run_visual_scheduler(@scheduler)
flash[:success] = "Scheduler created and executed successfully!"
redirect_to(course_schedulers_path(@course)) and return
else
@scheduler.destroy
flash[:error] = "Scheduler creation failed during execution."
end
else
flash[:error] = "Scheduler create failed. Please check all fields."
end
end
redirect_to(new_course_scheduler_path(@course))
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue

Add path validation to prevent directory traversal

The current implementation using Rails.root.join with user input could be vulnerable to path traversal attacks. Consider adding validation to ensure the action file is within the allowed course directory.

 def create
   @scheduler = @course.scheduler.new(scheduler_params)
+  course_dir = Rails.root.join('courses', @course.name).to_path
   action_path = Rails.root.join(scheduler_params[:action]).to_path
+  
+  unless action_path.start_with?(course_dir)
+    flash[:error] = "Action file must be within the course directory"
+    redirect_to(new_course_scheduler_path(@course)) and return
+  end

   # Check if the action file exists, is readable, and compiles
   if validate_compile_action_file(action_path)

Committable suggestion skipped: line range outside the PR's diff.

Comment on lines +65 to +82
action_path = Rails.root.join(scheduler_params[:action]).to_path
# Check if the action file exists, is readable, and compiles
if validate_compile_action_file(action_path)
previous_state = @scheduler.attributes
if @scheduler.update(scheduler_params)
# Ensure visual run is successful
if run_visual_scheduler(@scheduler)
flash[:success] = "Scheduler updated and executed successfully!"
redirect_to(course_schedulers_path(@course)) and return
else
@scheduler.update(previous_state) # If error, revert to previous state.
flash[:error] = "Scheduler update failed during execution. Reverted to previous state."
end
else
flash[:error] = "Scheduler update failed! Please check your fields."
end
end
redirect_to(edit_course_scheduler_path(@course, @scheduler))
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🛠️ Refactor suggestion

Use database transaction for atomic updates

The current implementation manually manages state rollback. Using a database transaction would ensure atomic updates and proper rollback of all changes.

 def update
   @scheduler = Scheduler.find_by(id: params[:id])
-  action_path = Rails.root.join(scheduler_params[:action]).to_path
-  # Check if the action file exists, is readable, and compiles
-  if validate_compile_action_file(action_path)
-    previous_state = @scheduler.attributes
-    if @scheduler.update(scheduler_params)
-      # Ensure visual run is successful
-      if run_visual_scheduler(@scheduler)
-        flash[:success] = "Scheduler updated and executed successfully!"
-        redirect_to(course_schedulers_path(@course)) and return
-      else
-        @scheduler.update(previous_state) # If error, revert to previous state.
-        flash[:error] = "Scheduler update failed during execution. Reverted to previous state."
-      end
+  ActiveRecord::Base.transaction do
+    action_path = Rails.root.join(scheduler_params[:action]).to_path
+    
+    if validate_compile_action_file(action_path)
+      if @scheduler.update(scheduler_params)
+        unless run_visual_scheduler(@scheduler)
+          raise ActiveRecord::Rollback, "Visual run failed"
+        end
+        flash[:success] = "Scheduler updated and executed successfully!"
+        redirect_to(course_schedulers_path(@course)) and return
+      end
     end
-  end
+    raise ActiveRecord::Rollback
+  end
+  flash[:error] ||= "Scheduler update failed"
   redirect_to(edit_course_scheduler_path(@course, @scheduler))
 end
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
action_path = Rails.root.join(scheduler_params[:action]).to_path
# Check if the action file exists, is readable, and compiles
if validate_compile_action_file(action_path)
previous_state = @scheduler.attributes
if @scheduler.update(scheduler_params)
# Ensure visual run is successful
if run_visual_scheduler(@scheduler)
flash[:success] = "Scheduler updated and executed successfully!"
redirect_to(course_schedulers_path(@course)) and return
else
@scheduler.update(previous_state) # If error, revert to previous state.
flash[:error] = "Scheduler update failed during execution. Reverted to previous state."
end
else
flash[:error] = "Scheduler update failed! Please check your fields."
end
end
redirect_to(edit_course_scheduler_path(@course, @scheduler))
def update
@scheduler = Scheduler.find_by(id: params[:id])
ActiveRecord::Base.transaction do
action_path = Rails.root.join(scheduler_params[:action]).to_path
if validate_compile_action_file(action_path)
if @scheduler.update(scheduler_params)
unless run_visual_scheduler(@scheduler)
raise ActiveRecord::Rollback, "Visual run failed"
end
flash[:success] = "Scheduler updated and executed successfully!"
redirect_to(course_schedulers_path(@course)) and return
end
end
raise ActiveRecord::Rollback
end
flash[:error] ||= "Scheduler update failed"
redirect_to(edit_course_scheduler_path(@course, @scheduler))
end

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Missing/invalid schedulers generate many exceptions
2 participants