Skip to content
This repository was archived by the owner on Nov 30, 2022. It is now read-only.

Add asynchrone job handler #752

Merged
merged 7 commits into from
Sep 7, 2020
Merged

Conversation

Alexander-Dubrawski
Copy link
Collaborator

@Alexander-Dubrawski Alexander-Dubrawski commented Sep 1, 2020

This PR is the first of four which aims to refactor the handling of Hyrise jobs (it aims to split #664). Hyrise jobs are interacting directly with the Hyrise to get the meta information and managing the Hyrise settings. We have three kinds of jobs:

  1. jobs that need to be continuously in the background (for example: get the system information)
  2. jobs that will be done asynchronously (for example: loading the tables)
  3. jobs that are done synchronously (for example: sending a SQL query over the SQL endpoint)

Both continuous and asynchronous jobs are executed via the apscheduler (https://apscheduler.readthedocs.io/en/stable/). Esspacillay with the asynchronous jobs that's not necessary and increases the complexity (we could just use normal python threads). All three kinds of jobs are scattered in the database and background_scheduler component. The goal of the refactoring is now to have a clear separation for the jobs in the three categories (continuous, asynchronous, and synchronous) and define handlers for that. Moreover, we can stop the continuous jobs while loading/deleting tables (hyrise/hyrise#2180).

This PR implements the handler for the asynchronous jobs. It uses normal python threads (https://docs.python.org/3/library/threading.html#threading.Thread) for that.

ToDo:

  • Adjust execute_queries_parallel.py to use Threads instead of processe



class AsynchronousJobHandler:
"""Handles asynchronous jobs."""
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great doc. :D

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I tried to improve the docstrings by using google style docstrings (https://github.com/google/styleguide/blob/gh-pages/pyguide.md#38-comments-and-docstrings). I would be very happy about your feedback. If it is now an improvement.

Copy link
Contributor

@Bouncner Bouncner left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mini typos. The code looks good, but I am not too familiar with the whole backend. So I am not really the one to judge. I am trusting you here, that everything is well tested both from an automated test site as well as tested with a complete run through of the cockpit.

@Alexander-Dubrawski Alexander-Dubrawski merged commit 5a5b213 into dev Sep 7, 2020
@Alexander-Dubrawski Alexander-Dubrawski deleted the B/Add_asynchrone_job_handler branch September 7, 2020 15:59
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants