Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix unobserved exceptions with retries #2255

Merged
merged 12 commits into from
Aug 29, 2023

Conversation

JamesNK
Copy link
Member

@JamesNK JamesNK commented Aug 25, 2023

This PR stops unobserved exceptions in the client when using the retry feature. It's similar to #1660, except this PR is for calls with retry enabled.

  • Suppress ResponseAsync exception if ResponseHeadersAsync is called. If someone accesses headers with ResponseHeadersAsync then they're unlike to call ResponseAsync to see the same error.
  • Always suppress exceptions when a call ends with a Cancel or Deadline status. This is consistent with the automatic suppression of OperationCanceledException.
  • Update most tests in Grpc.Net.Client to always pass a Method<TRequest, TResponse> with the right type. This is important because retry/hedging calls look at the method type.

Feel free to ignore the test changes.

@JamesNK JamesNK force-pushed the jamesnk/retry-unobservedexceptions branch from 82372ac to ab2c2f1 Compare August 26, 2023 14:18
@JamesNK JamesNK marked this pull request as ready for review August 26, 2023 14:21
Copy link
Contributor

@amcasey amcasey left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems reasonable.

// If there was an error fetching response headers then it's likely the same error is reported
// by response TCS. The user is unlikely to observe both errors.
// Observed exception to avoid unobserved exception event.
_responseTask?.ObserveException();
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The comment makes it seem like the desirability of this behavior depends on the outcome of the header task, but that's not obviously reflected in the code. Will this observe/suppress the response exception if there's no header exception?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will this observe/suppress the response exception if there's no header exception?

Potentially. No error getting headers should mean there isn't an error getting the body. Observing like this is a pragmatic balance.

@JamesNK JamesNK merged commit b421751 into grpc:master Aug 29, 2023
2 checks passed
oguzhand95 referenced this pull request in cerbos/cerbos-sdk-net Sep 11, 2023
[![Mend
Renovate](https://app.renovatebot.com/images/banner.svg)](https://renovatebot.com)

This PR contains the following updates:

| Package | Type | Update | Change |
|---|---|---|---|
| [Google.Protobuf](https://togithub.com/protocolbuffers/protobuf) |
nuget | patch | `3.24.2` -> `3.24.3` |
| [Grpc.Net.Client](https://togithub.com/grpc/grpc-dotnet) | nuget |
minor | `2.56.0` -> `2.57.0` |
| [Testcontainers](https://dotnet.testcontainers.org/)
([source](https://togithub.com/testcontainers/testcontainers-dotnet)) |
nuget | minor | `3.4.0` -> `3.5.0` |

---

### Release Notes

<details>
<summary>protocolbuffers/protobuf (Google.Protobuf)</summary>

###
[`v3.24.3`](https://togithub.com/protocolbuffers/protobuf/compare/v3.24.2...v3.24.3)

</details>

<details>
<summary>grpc/grpc-dotnet (Grpc.Net.Client)</summary>

###
[`v2.57.0`](https://togithub.com/grpc/grpc-dotnet/releases/tag/v2.57.0)

#### What's Changed

- Start 2.57.x development cycle by
[@&#8203;jtattermusch](https://togithub.com/jtattermusch) in
[https://github.com/grpc/grpc-dotnet/pull/2227](https://togithub.com/grpc/grpc-dotnet/pull/2227)
- Validate Windows version when using WinHttpHandler by
[@&#8203;JamesNK](https://togithub.com/JamesNK) in
[https://github.com/grpc/grpc-dotnet/pull/2229](https://togithub.com/grpc/grpc-dotnet/pull/2229)
- Support infinite idle connection timeout values by
[@&#8203;JamesNK](https://togithub.com/JamesNK) in
[https://github.com/grpc/grpc-dotnet/pull/2231](https://togithub.com/grpc/grpc-dotnet/pull/2231)
- Improve BalancerAttributes debugging by
[@&#8203;JamesNK](https://togithub.com/JamesNK) in
[https://github.com/grpc/grpc-dotnet/pull/2235](https://togithub.com/grpc/grpc-dotnet/pull/2235)
- Update Grpc.Net.Client to remove ValueTask usage by
[@&#8203;JamesNK](https://togithub.com/JamesNK) in
[https://github.com/grpc/grpc-dotnet/pull/2222](https://togithub.com/grpc/grpc-dotnet/pull/2222)
- Update Newtonsoft.Json to 13.0.3 by
[@&#8203;JamesNK](https://togithub.com/JamesNK) in
[https://github.com/grpc/grpc-dotnet/pull/2219](https://togithub.com/grpc/grpc-dotnet/pull/2219)
- Consistently don't log message errors on cancellation by
[@&#8203;JamesNK](https://togithub.com/JamesNK) in
[https://github.com/grpc/grpc-dotnet/pull/2221](https://togithub.com/grpc/grpc-dotnet/pull/2221)
- Fix load balancing flaky test by
[@&#8203;JamesNK](https://togithub.com/JamesNK) in
[https://github.com/grpc/grpc-dotnet/pull/2223](https://togithub.com/grpc/grpc-dotnet/pull/2223)
- Improve comment in GrpcChannel for WinHttpHandler + OS validation by
[@&#8203;JamesNK](https://togithub.com/JamesNK) in
[https://github.com/grpc/grpc-dotnet/pull/2237](https://togithub.com/grpc/grpc-dotnet/pull/2237)
- Update OS version detection to get version directly from Windows by
[@&#8203;JamesNK](https://togithub.com/JamesNK) in
[https://github.com/grpc/grpc-dotnet/pull/2239](https://togithub.com/grpc/grpc-dotnet/pull/2239)
- Update implementation_comparison.md by
[@&#8203;JamesNK](https://togithub.com/JamesNK) in
[https://github.com/grpc/grpc-dotnet/pull/2240](https://togithub.com/grpc/grpc-dotnet/pull/2240)
- Update to use .NET 8 by
[@&#8203;JamesNK](https://togithub.com/JamesNK) in
[https://github.com/grpc/grpc-dotnet/pull/2021](https://togithub.com/grpc/grpc-dotnet/pull/2021)
- Bump semver from 6.3.0 to 6.3.1 in
/testassets/InteropTestsGrpcWebWebsite/Tests by
[@&#8203;dependabot](https://togithub.com/dependabot) in
[https://github.com/grpc/grpc-dotnet/pull/2243](https://togithub.com/grpc/grpc-dotnet/pull/2243)
- Force yielding after awaiting CallTask to avoid holding onto locks by
[@&#8203;JamesNK](https://togithub.com/JamesNK) in
[https://github.com/grpc/grpc-dotnet/pull/2245](https://togithub.com/grpc/grpc-dotnet/pull/2245)
- Update Grpc.Net.Client to use ActivitySource by
[@&#8203;JamesNK](https://togithub.com/JamesNK) in
[https://github.com/grpc/grpc-dotnet/pull/2244](https://togithub.com/grpc/grpc-dotnet/pull/2244)
- Clear IAsyncStreamReader<T>.Current value before reading next value by
[@&#8203;JamesNK](https://togithub.com/JamesNK) in
[https://github.com/grpc/grpc-dotnet/pull/2248](https://togithub.com/grpc/grpc-dotnet/pull/2248)
- Upgrade QpsWorker to net8 by
[@&#8203;jtattermusch](https://togithub.com/jtattermusch) in
[https://github.com/grpc/grpc-dotnet/pull/2250](https://togithub.com/grpc/grpc-dotnet/pull/2250)
- Change subchannel ID to include a channel ID by
[@&#8203;JamesNK](https://togithub.com/JamesNK) in
[https://github.com/grpc/grpc-dotnet/pull/2253](https://togithub.com/grpc/grpc-dotnet/pull/2253)
- Update Grpc.Tools dependency to 2.57.0 by
[@&#8203;jtattermusch](https://togithub.com/jtattermusch) in
[https://github.com/grpc/grpc-dotnet/pull/2257](https://togithub.com/grpc/grpc-dotnet/pull/2257)
- Add transport status to subchannel picked log by
[@&#8203;JamesNK](https://togithub.com/JamesNK) in
[https://github.com/grpc/grpc-dotnet/pull/2261](https://togithub.com/grpc/grpc-dotnet/pull/2261)
- Reduce logger allocations by not using generic CreateLogger by
[@&#8203;JamesNK](https://togithub.com/JamesNK) in
[https://github.com/grpc/grpc-dotnet/pull/2256](https://togithub.com/grpc/grpc-dotnet/pull/2256)
- Update call debugger display to show status code by
[@&#8203;JamesNK](https://togithub.com/JamesNK) in
[https://github.com/grpc/grpc-dotnet/pull/2259](https://togithub.com/grpc/grpc-dotnet/pull/2259)
- Log socket lifetime when closing unusable sockets by
[@&#8203;JamesNK](https://togithub.com/JamesNK) in
[https://github.com/grpc/grpc-dotnet/pull/2258](https://togithub.com/grpc/grpc-dotnet/pull/2258)
- Fix unobserved exceptions with retries by
[@&#8203;JamesNK](https://togithub.com/JamesNK) in
[https://github.com/grpc/grpc-dotnet/pull/2255](https://togithub.com/grpc/grpc-dotnet/pull/2255)
- Change subchannel BalancerAddress when attributes change by
[@&#8203;JamesNK](https://togithub.com/JamesNK) in
[https://github.com/grpc/grpc-dotnet/pull/2228](https://togithub.com/grpc/grpc-dotnet/pull/2228)
- Fix connection bugs from BalancerAddress changes by
[@&#8203;JamesNK](https://togithub.com/JamesNK) in
[https://github.com/grpc/grpc-dotnet/pull/2265](https://togithub.com/grpc/grpc-dotnet/pull/2265)
- \[v2.57.x] Update version to 2.57.0-pre1 by
[@&#8203;jtattermusch](https://togithub.com/jtattermusch) in
[https://github.com/grpc/grpc-dotnet/pull/2266](https://togithub.com/grpc/grpc-dotnet/pull/2266)
- \[2.57.x] Update version to 2.57.0 by
[@&#8203;jtattermusch](https://togithub.com/jtattermusch) in
[https://github.com/grpc/grpc-dotnet/pull/2272](https://togithub.com/grpc/grpc-dotnet/pull/2272)

**Full Changelog**:
grpc/grpc-dotnet@v2.56.0...v2.57.0

</details>

<details>
<summary>testcontainers/testcontainers-dotnet (Testcontainers)</summary>

###
[`v3.5.0`](https://togithub.com/testcontainers/testcontainers-dotnet/releases/tag/3.5.0)

[Compare
Source](https://togithub.com/testcontainers/testcontainers-dotnet/compare/3.4.0...3.5.0)

### What's Changed

#### 🚀 Features

- feat: Allow MongoDb module configuration without credentials
([#&#8203;983](https://togithub.com/testcontainers/testcontainers-dotnet/issues/983))
[@&#8203;the-avid-engineer](https://togithub.com/the-avid-engineer)
- feat: Add support for RSA private key (RsaPrivateCrtKeyParameters) TLS
authentication with protected Docker daemon sockets
([#&#8203;978](https://togithub.com/testcontainers/testcontainers-dotnet/issues/978))
[@&#8203;zuntio](https://togithub.com/zuntio)
- feat: Add InfluxDb module
([#&#8203;975](https://togithub.com/testcontainers/testcontainers-dotnet/issues/975))
[@&#8203;MelomanG](https://togithub.com/MelomanG)

#### 🐛 Bug Fixes

- fix: Do not pre pull Dockerfile build stages that do not correspond to
base images
([#&#8203;979](https://togithub.com/testcontainers/testcontainers-dotnet/issues/979))
[@&#8203;HofmeisterAn](https://togithub.com/HofmeisterAn)

#### 📖 Documentation

- docs: Add documentation on enabling debug log messages for the default
logger
([#&#8203;991](https://togithub.com/testcontainers/testcontainers-dotnet/issues/991))
[@&#8203;HofmeisterAn](https://togithub.com/HofmeisterAn)
- docs: Add global Testcontainers header
([#&#8203;967](https://togithub.com/testcontainers/testcontainers-dotnet/issues/967))
[@&#8203;leocross](https://togithub.com/leocross)

#### 🧹 Housekeeping

- chore: Improve error message when Docker is not running
([#&#8203;987](https://togithub.com/testcontainers/testcontainers-dotnet/issues/987))
[@&#8203;0xced](https://togithub.com/0xced)
- chore: Update BouncyCastle.Cryptography to 2.2.1 (previous
Portable.BouncyCastle)
([#&#8203;985](https://togithub.com/testcontainers/testcontainers-dotnet/issues/985))
[@&#8203;jcmrva](https://togithub.com/jcmrva)
- chore: Add User-Agent HTTP header to Docker.DotNet client
([#&#8203;970](https://togithub.com/testcontainers/testcontainers-dotnet/issues/970))
[@&#8203;eddumelendez](https://togithub.com/eddumelendez)
- chore: Remove `CLICKHOUSE_DEFAULT_ACCESS_MANAGEMENT` env var
([#&#8203;971](https://togithub.com/testcontainers/testcontainers-dotnet/issues/971))
[@&#8203;eddumelendez](https://togithub.com/eddumelendez)

</details>

---

### Configuration

📅 **Schedule**: Branch creation - "before 4am on Monday" (UTC),
Automerge - At any time (no schedule defined).

🚦 **Automerge**: Disabled by config. Please merge this manually once you
are satisfied.

♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the
rebase/retry checkbox.

👻 **Immortal**: This PR will be recreated if closed unmerged. Get
[config help](https://togithub.com/renovatebot/renovate/discussions) if
that's undesired.

---

- [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check
this box

---

This PR has been generated by [Mend
Renovate](https://www.mend.io/free-developer-tools/renovate/). View
repository job log
[here](https://developer.mend.io/github/cerbos/cerbos-sdk-net).

<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzNi44My4wIiwidXBkYXRlZEluVmVyIjoiMzYuODMuMCIsInRhcmdldEJyYW5jaCI6Im1haW4ifQ==-->

Signed-off-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Signed-off-by: Oğuzhan Durgun <[email protected]>
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants