Skip to content

PLUGIN-1800: Adding read timeout to BQ Execute Config with default 120s #1430

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Aug 22, 2024

Conversation

masivesky
Copy link
Contributor

@masivesky masivesky commented Aug 19, 2024

https://cdap.atlassian.net/browse/PLUGIN-1800

Retries were already configured on the BigQuery Client by default, 5xx errors are being retried 6 times before failing:

this.retrySettings = (RetrySettings)MoreObjects.firstNonNull(builder.retrySettings, getDefaultRetrySettings());
private static RetrySettings.Builder getDefaultRetrySettingsBuilder() {
    return RetrySettings.newBuilder().setMaxAttempts(6).setInitialRetryDelay(Duration.ofMillis(1000L)).setMaxRetryDelay(Duration.ofMillis(32000L)).setRetryDelayMultiplier(2.0).setTotalTimeout(Duration.ofMillis(50000L)).setInitialRpcTimeout(Duration.ofMillis(50000L)).setRpcTimeoutMultiplier(1.0).setMaxRpcTimeout(Duration.ofMillis(50000L));
  }

Hence, increasing the read timeout for BigQuery Execute action plugin (where we are validating SQL) to a default of 120s. Earlier timeout was the BigQuery default of 20s.

Confirmed via local sandbox testing that the default timeout for BigQuery Execute is 120s.

@masivesky masivesky added the build Trigger unit test build label Aug 19, 2024
@masivesky masivesky marked this pull request as ready for review August 19, 2024 09:43
Copy link
Member

@itsankit-google itsankit-google left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please update the markdown as well.

@masivesky masivesky force-pushed the bugfix/PLUGIN-1800-bq-execute-timeout-increase branch from 1a5a0b0 to 440689c Compare August 21, 2024 09:03
@masivesky
Copy link
Contributor Author

Please update the markdown as well.

As discussed offline, we will be keeping this new parameter hidden, hence not adding it to the markdown files.

@itsankit-google itsankit-google force-pushed the bugfix/PLUGIN-1800-bq-execute-timeout-increase branch from 440689c to 13d88cf Compare August 21, 2024 16:23
@itsankit-google itsankit-google force-pushed the bugfix/PLUGIN-1800-bq-execute-timeout-increase branch from 13d88cf to 4cb7244 Compare August 22, 2024 09:21
@masivesky masivesky merged commit 6ce1e5d into develop Aug 22, 2024
4 of 16 checks passed
@masivesky masivesky deleted the bugfix/PLUGIN-1800-bq-execute-timeout-increase branch August 22, 2024 09:32
@masivesky masivesky self-assigned this Jan 16, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
build Trigger unit test build
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants