Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adding page for CBO #24736

Open
wants to merge 10 commits into
base: master
Choose a base branch
from
Open

Adding page for CBO #24736

wants to merge 10 commits into from

Conversation

premkumr
Copy link
Contributor

@premkumr premkumr commented Oct 30, 2024

@netlify /preview/architecture/query-layer/planner-optimizer
DOC-383

Copy link

netlify bot commented Oct 30, 2024

Deploy Preview for infallible-bardeen-164bc9 ready!

Built without sensitive environment variables

Name Link
🔨 Latest commit 12c099d
🔍 Latest deploy log https://app.netlify.com/sites/infallible-bardeen-164bc9/deploys/67523d82f074e60008549714
😎 Deploy Preview https://deploy-preview-24736--infallible-bardeen-164bc9.netlify.app/preview/architecture/query-layer/planner-optimizer
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify site configuration.

@gauravk-in
Copy link
Contributor

It generates various plans of execution and determines the optimal path by taking into consideration the costs associated various factors like index lookups, scans, CPU usage, network latency, and so on.

Is missing ... costs associated *with* various factors...

Could also be rephrased as,

It computes the cost of various execution plans by taking into consideration factors such as index lookup, CPU usage, network throughput, and so on and picks the plan with the cheapest cost.

@gauravk-in
Copy link
Contributor

The primary component that calculates these values is the cost-based optimizer (CBO).

can be

The query planner, also known as CBO comprises primarily of selectivity estimation and cost modeling. We have implemented a new cost model for YugabyteDB which improves the accuracy of the CBO. This can be enabled as follows,

@gauravk-in
Copy link
Contributor

These statistics are essential for estimating the costs of various query plans accurately.

can be

These statistics are essential for estimating the selectivity of filters and costs of various query plans accurately.

@gauravk-in
Copy link
Contributor

To estimate the cost of fetching a tuple from DocDB, factors such as the number of SST files that may need to be read, and the estimated number of seeks, previous, and next operations that may be executed in the LSM subsystem, are taken into account.

can be

To estimate the cost of fetching a tuple from DocDB, we take into account factors such as the number of SST files that may need to be read, and the estimated number of seeks, previous, and next operations that may be executed in the LSM subsystem.

@ddhodge
Copy link
Contributor

ddhodge commented Oct 31, 2024

To estimate the cost of fetching a tuple from DocDB, we take into account factors such as the number of SST files that may need to be read, and the estimated number of seeks, previous, and next operations that may be executed in the LSM subsystem.

'we' - you mean the CBO? eg "the CBO takes into account..."

@ddhodge
Copy link
Contributor

ddhodge commented Oct 31, 2024

The query planner, also known as CBO comprises primarily of selectivity estimation and cost modeling. We have implemented a new cost model for YugabyteDB which improves the accuracy of the CBO. This can be enabled as follows,

Oh so the query planner and the CBO are the same?
How should we distinguish the new CBO from the old (PostgreSQL?) CBO?
What is 'selectivity estimation' and 'selectivity of filters'

@gauravk-in
Copy link
Contributor

The query planner, also known as CBO comprises primarily of selectivity estimation and cost modeling. We have implemented a new cost model for YugabyteDB which improves the accuracy of the CBO. This can be enabled as follows,

Oh so the query planner and the CBO are the same? How should we distinguish the new CBO from the old (PostgreSQL?) CBO? What is 'selectivity estimation' and 'selectivity of filters'

  • It may be better to remove the query planner and just use CBO. CBO maybe a part of the query planner which also includes the RBO (Rules Based Optimizer).
  • The old one was also technically a cost based optimizer, it was just not accurately modeling the execution pipeline. We can call it heuristics based optimizer, because we were assigning some heuristic costs to the number of result rows depending on the type of the scan.
  • Selectivity of a filter tells the number of rows that the filter will select, or the number of rows that will pass the filter.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/documentation Documentation needed
Projects
Status: In progress
Development

Successfully merging this pull request may close these issues.

3 participants