Skip to content

System.Net.Quic tests timeout #107761

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
ericstj opened this issue Sep 12, 2024 · 9 comments
Open

System.Net.Quic tests timeout #107761

ericstj opened this issue Sep 12, 2024 · 9 comments
Labels
area-System.Net.Quic blocking-clean-ci Blocking PR or rolling runs of 'runtime' or 'runtime-extra-platforms' Known Build Error Use this to report build issues in the .NET Helix tab os-mac-os-x macOS aka OSX
Milestone
10.0.0

Comments

@ericstj
Copy link
Member

ericstj commented Sep 12, 2024 •
edited by build-analysis bot
Loading

Build Information

Build: https://dev.azure.com/dnceng-public/cbb18261-c48f-4abb-8651-8cdcb5474649/_build/results?buildId=805750
Build error leg or test failing: System.Net.Quic.Functional.Tests.WorkItemExecution
Pull request: #107724

Error Message

Fill the error message using step by step known issues guidance.

{
  "ErrorMessage": "",
  "ErrorPattern": "'System.Net.Quic.Functional.Tests' END OF WORK ITEM LOG: Command timed out, and was killed]",
  "BuildRetry": false,
  "ExcludeConsoleLog": false
}

Known issue validation

Build: 🔎 https://dev.azure.com/dnceng-public/public/_build/results?buildId=805750
Error message validated: ['System.Net.Quic.Functional.Tests' END OF WORK ITEM LOG: Command timed out, and was killed]]
Result validation: ✅ Known issue matched with the provided build.
Validation performed at: 9/12/2024 7:56:17 PM UTC

Report

Build Definition Test Pull Request
999726 dotnet/runtime System.Net.Quic.Functional.Tests.WorkItemExecution #113876
1003177 dotnet/runtime System.Net.Quic.Functional.Tests.WorkItemExecution #114172
1004767 dotnet/runtime System.Net.Quic.Functional.Tests.WorkItemExecution #114048
1005882 dotnet/runtime System.Net.Quic.Functional.Tests.WorkItemExecution
1005854 dotnet/runtime System.Net.Quic.Functional.Tests.WorkItemExecution #114194
1005817 dotnet/runtime System.Net.Quic.Functional.Tests.WorkItemExecution #113899
1005775 dotnet/runtime System.Net.Quic.Functional.Tests.WorkItemExecution #114180
1005698 dotnet/runtime System.Net.Quic.Functional.Tests.WorkItemExecution #100729
996660 dotnet/runtime System.Net.Quic.Functional.Tests.WorkItemExecution #113864
1005655 dotnet/runtime System.Net.Quic.Functional.Tests.WorkItemExecution #113896
1005617 dotnet/runtime System.Net.Quic.Functional.Tests.WorkItemExecution #113286
1005589 dotnet/runtime System.Net.Quic.Functional.Tests.WorkItemExecution #114267
1005541 dotnet/runtime System.Net.Quic.Functional.Tests.WorkItemExecution #114263
1005527 dotnet/runtime System.Net.Quic.Functional.Tests.WorkItemExecution #114003
1004699 dotnet/runtime System.Net.Quic.Functional.Tests.WorkItemExecution #114227
1005438 dotnet/runtime System.Net.Quic.Functional.Tests.WorkItemExecution #114180
1005249 dotnet/runtime System.Net.Quic.Functional.Tests.WorkItemExecution #114150
1005219 dotnet/runtime System.Net.Quic.Functional.Tests.WorkItemExecution #114080
1005183 dotnet/runtime System.Net.Quic.Functional.Tests.WorkItemExecution #114148
1005171 dotnet/runtime System.Net.Quic.Functional.Tests.WorkItemExecution
1005174 dotnet/runtime System.Net.Quic.Functional.Tests.WorkItemExecution #114257
1005163 dotnet/runtime System.Net.Quic.Functional.Tests.WorkItemExecution #113924
1005153 dotnet/runtime System.Net.Quic.Functional.Tests.WorkItemExecution #114249
1005140 dotnet/runtime System.Net.Quic.Functional.Tests.WorkItemExecution #114228
1005089 dotnet/runtime System.Net.Quic.Functional.Tests.WorkItemExecution #114080
1005040 dotnet/runtime System.Net.Quic.Functional.Tests.WorkItemExecution #114137
1004925 dotnet/runtime System.Net.Quic.Functional.Tests.WorkItemExecution #111408
1004916 dotnet/runtime System.Net.Quic.Functional.Tests.WorkItemExecution #114249
1004888 dotnet/runtime System.Net.Quic.Functional.Tests.WorkItemExecution #113976
1004820 dotnet/runtime System.Net.Quic.Functional.Tests.WorkItemExecution #114245
1004806 dotnet/runtime System.Net.Quic.Functional.Tests.WorkItemExecution #114244
1004763 dotnet/runtime System.Net.Quic.Functional.Tests.WorkItemExecution #111408
2678845 dotnet-runtime System.Net.Quic.Functional.Tests.WorkItemExecution
1004726 dotnet/runtime System.Net.Quic.Functional.Tests.WorkItemExecution #114157
1003296 dotnet/runtime System.Net.Quic.Functional.Tests.WorkItemExecution #113286
1004711 dotnet/runtime System.Net.Quic.Functional.Tests.WorkItemExecution #114240
1004708 dotnet/runtime System.Net.Quic.Functional.Tests.WorkItemExecution #114234
1004695 dotnet/runtime System.Net.Quic.Functional.Tests.WorkItemExecution #114187
1004454 dotnet/runtime System.Net.Quic.Functional.Tests.WorkItemExecution #114233
1004391 dotnet/runtime System.Net.Quic.Functional.Tests.WorkItemExecution #114227
1004397 dotnet/runtime System.Net.Quic.Functional.Tests.WorkItemExecution #114228
1004313 dotnet/runtime System.Net.Quic.Functional.Tests.WorkItemExecution #113313
1004288 dotnet/runtime System.Net.Quic.Functional.Tests.WorkItemExecution #114173
1004267 dotnet/runtime System.Net.Quic.Functional.Tests.WorkItemExecution #111072
1004249 dotnet/runtime System.Net.Quic.Functional.Tests.WorkItemExecution
1004186 dotnet/runtime System.Net.Quic.Functional.Tests.WorkItemExecution #114194
1004163 dotnet/runtime System.Net.Quic.Functional.Tests.WorkItemExecution #114214
1004159 dotnet/runtime System.Net.Quic.Functional.Tests.WorkItemExecution #114139
1004145 dotnet/runtime System.Net.Quic.Functional.Tests.WorkItemExecution #114211
1004125 dotnet/runtime System.Net.Quic.Functional.Tests.WorkItemExecution #114210
1004074 dotnet/runtime System.Net.Quic.Functional.Tests.WorkItemExecution #114107
1004083 dotnet/runtime System.Net.Quic.Functional.Tests.WorkItemExecution #114003
1004087 dotnet/runtime System.Net.Quic.Functional.Tests.WorkItemExecution #114003
1003904 dotnet/runtime System.Net.Quic.Functional.Tests.WorkItemExecution #100729
1003950 dotnet/runtime System.Net.Quic.Functional.Tests.WorkItemExecution #114080
1003836 dotnet/runtime System.Net.Quic.Functional.Tests.WorkItemExecution #114201
1003898 dotnet/runtime System.Net.Quic.Functional.Tests.WorkItemExecution #114003
1003811 dotnet/runtime System.Net.Quic.Functional.Tests.WorkItemExecution #114194
1003791 dotnet/runtime System.Net.Quic.Functional.Tests.WorkItemExecution #113689
1003768 dotnet/runtime System.Net.Quic.Functional.Tests.WorkItemExecution #113924
1003757 dotnet/runtime System.Net.Quic.Functional.Tests.WorkItemExecution #113313
1003735 dotnet/runtime System.Net.Quic.Functional.Tests.WorkItemExecution
1003668 dotnet/runtime System.Net.Quic.Functional.Tests.WorkItemExecution #114080
1002966 dotnet/runtime System.Net.Quic.Functional.Tests.WorkItemExecution #114164
1003501 dotnet/runtime System.Net.Quic.Functional.Tests.WorkItemExecution #113977
1003404 dotnet/runtime System.Net.Quic.Functional.Tests.WorkItemExecution #114139
1003439 dotnet/runtime System.Net.Quic.Functional.Tests.WorkItemExecution #114191
1003346 dotnet/runtime System.Net.Quic.Functional.Tests.WorkItemExecution #114057
1003328 dotnet/runtime System.Net.Quic.Functional.Tests.WorkItemExecution #114170
1003262 dotnet/runtime System.Net.Quic.Functional.Tests.WorkItemExecution #114016
1003254 dotnet/runtime System.Net.Quic.Functional.Tests.WorkItemExecution #114040
1003236 dotnet/runtime System.Net.Quic.Functional.Tests.WorkItemExecution #113907
1003168 dotnet/runtime System.Net.Quic.Functional.Tests.WorkItemExecution #101024
1003120 dotnet/runtime System.Net.Quic.Functional.Tests.WorkItemExecution #114080
1003107 dotnet/runtime System.Net.Quic.Functional.Tests.WorkItemExecution
1002768 dotnet/runtime System.Net.Quic.Functional.Tests.WorkItemExecution #114130
1002980 dotnet/runtime System.Net.Quic.Functional.Tests.WorkItemExecution #113313
1002946 dotnet/runtime System.Net.Quic.Functional.Tests.WorkItemExecution #114136
1002831 dotnet/runtime System.Net.Quic.Functional.Tests.WorkItemExecution #110818
1002643 dotnet/runtime System.Net.Quic.Functional.Tests.WorkItemExecution #113765
1002772 dotnet/runtime System.Net.Quic.Functional.Tests.WorkItemExecution #114085
1002251 dotnet/runtime System.Net.Quic.Functional.Tests.WorkItemExecution #114040
1002664 dotnet/runtime System.Net.Quic.Functional.Tests.WorkItemExecution #114046
1002688 dotnet/runtime System.Net.Quic.Functional.Tests.WorkItemExecution #114158
1002647 dotnet/runtime System.Net.Quic.Functional.Tests.WorkItemExecution #114157
1002636 dotnet/runtime System.Net.Quic.Functional.Tests.WorkItemExecution #113576
1001192 dotnet/runtime System.Net.Quic.Functional.Tests.WorkItemExecution #114111
1002606 dotnet/runtime System.Net.Quic.Functional.Tests.WorkItemExecution #114150
1002599 dotnet/runtime System.Net.Quic.Functional.Tests.WorkItemExecution #113924
1002592 dotnet/runtime System.Net.Quic.Functional.Tests.WorkItemExecution #114146
1002545 dotnet/runtime System.Net.Quic.Functional.Tests.WorkItemExecution #113689
1002524 dotnet/runtime System.Net.Quic.Functional.Tests.WorkItemExecution #114003
1002521 dotnet/runtime System.Net.Quic.Functional.Tests.WorkItemExecution #114003
1002454 dotnet/runtime System.Net.Quic.Functional.Tests.WorkItemExecution #114149
1002450 dotnet/runtime System.Net.Quic.Functional.Tests.WorkItemExecution #114148
1002364 dotnet/runtime System.Net.Quic.Functional.Tests.WorkItemExecution #114049
1002415 dotnet/runtime System.Net.Quic.Functional.Tests.WorkItemExecution #113576
1002378 dotnet/runtime System.Net.Quic.Functional.Tests.WorkItemExecution #114080
1002293 dotnet/runtime System.Net.Quic.Functional.Tests.WorkItemExecution #114142
1002283 dotnet/runtime System.Net.Quic.Functional.Tests.WorkItemExecution #113576
Displaying 100 of 431 results

Summary

24-Hour Hit Count 7-Day Hit Count 1-Month Count
0 1 431
@ericstj ericstj added blocking-clean-ci Blocking PR or rolling runs of 'runtime' or 'runtime-extra-platforms' Known Build Error Use this to report build issues in the .NET Helix tab labels Sep 12, 2024
@dotnet-policy-service dotnet-policy-service bot added the untriaged New issue has not been triaged by the area owner label Sep 12, 2024
@dotnet-policy-service Dotnet Policy Service
Copy link
Contributor

Tagging subscribers to this area: @dotnet/area-infrastructure-libraries
See info in area-owners.md if you want to be subscribed.

@dotnet-policy-service Dotnet Policy Service
Copy link
Contributor

Tagging subscribers to this area: @dotnet/ncl
See info in area-owners.md if you want to be subscribed.

@liveans liveans added the os-mac-os-x macOS aka OSX label Sep 14, 2024
@liveans liveans added this to the 10.0.0 milestone Sep 14, 2024
@liveans liveans removed the untriaged New issue has not been triaged by the area owner label Sep 14, 2024
@liveans
Copy link
Member

liveans commented Sep 14, 2024

Related to #103482

Log:

Console log: 'System.Net.Quic.Functional.Tests' from job 670b66ae-06e5-4b53-92f3-603cb67faae9 workitem c9d3b0a1-5903-476a-af61-2c2df727e696 (osx.1200.amd64.open) executed on machine dci-mac-build-193 running macOS-12.7.2
+ export __TestArchitecture=x64
+ __TestArchitecture=x64
+ find /tmp/helix/working/A7D00979/p -name createdump
+ xargs -n 1 codesign -s - -f --preserve-metadata=entitlements
/tmp/helix/working/A7D00979/p/shared/Microsoft.NETCore.App/9.0.0/createdump: replacing existing signature
+ ./RunTests.sh --runtime-path /tmp/helix/working/A7D00979/p
========================= Begin custom configuration settings ==============================
export __IsXUnitLogCheckerSupported=1
export XUNIT_HIDE_PASSING_OUTPUT_DIAGNOSTICS=1
========================== End custom configuration settings ===============================
----- start Wed Sep 11 20:47:08 EDT 2024 =============== To repro directly: =====================================================
pushd .
/tmp/helix/working/A7D00979/p/dotnet exec --runtimeconfig System.Net.Quic.Functional.Tests.runtimeconfig.json --depsfile System.Net.Quic.Functional.Tests.deps.json xunit.console.dll System.Net.Quic.Functional.Tests.dll -xml testResults.xml -nologo -nocolor -notrait category=IgnoreForCI -notrait category=OuterLoop -notrait category=failing 
popd
===========================================================================================================
/private/tmp/helix/working/A7D00979/w/ACE20948/e /private/tmp/helix/working/A7D00979/w/ACE20948/e
  Discovering: System.Net.Quic.Functional.Tests (method display = ClassAndMethod, method display options = None)
  Discovered:  System.Net.Quic.Functional.Tests (found 138 of 152 test cases)
  Starting:    System.Net.Quic.Functional.Tests (parallel test collections = on [12 threads], stop on fail = off)
MsQuic supported and using 'libmsquic.dylib 2.4.0.0 (e0385b047d7bd622319e9f6c4c00184a1294eea8)'.
    System.Net.Quic.Tests.MsQuicPlatformDetectionTests.UnsupportedPlatforms_ThrowsPlatformNotSupportedException [SKIP]
      Condition(s) not met: "IsQuicUnsupported"
   System.Net.Quic.Functional.Tests: [Long Running Test] 'System.Net.Quic.Tests.QuicStreamConformanceTests.ConcurrentBidirectionalReadsWrites_Success', Elapsed: 00:02:06
   System.Net.Quic.Functional.Tests: [Long Running Test] 'System.Net.Quic.Tests.QuicStreamConformanceTests.ConcurrentBidirectionalReadsWrites_Success', Elapsed: 00:04:07
   System.Net.Quic.Functional.Tests: [Long Running Test] 'System.Net.Quic.Tests.QuicStreamConformanceTests.ConcurrentBidirectionalReadsWrites_Success', Elapsed: 00:06:09
   System.Net.Quic.Functional.Tests: [Long Running Test] 'System.Net.Quic.Tests.QuicStreamConformanceTests.ConcurrentBidirectionalReadsWrites_Success', Elapsed: 00:08:10
   System.Net.Quic.Functional.Tests: [Long Running Test] 'System.Net.Quic.Tests.QuicStreamConformanceTests.ConcurrentBidirectionalReadsWrites_Success', Elapsed: 00:10:12
   System.Net.Quic.Functional.Tests: [Long Running Test] 'System.Net.Quic.Tests.QuicStreamConformanceTests.ConcurrentBidirectionalReadsWrites_Success', Elapsed: 00:12:13
['System.Net.Quic.Functional.Tests' END OF WORK ITEM LOG: Command timed out, and was killed]

@dotnet-policy-service Dotnet Policy Service
Copy link
Contributor

Tagging subscribers to this area: @dotnet/ncl
See info in area-owners.md if you want to be subscribed.

@jeffschwMSFT
Copy link
Member

@karelz can y'all take a look? this is hitting just about every PR'ish. has something gotten much worse here?

@liveans
Copy link
Member

liveans commented Mar 28, 2025

Seems like something is wrong with azurelinux3 queue in Azure because this is the either third or fourth report that we got from different suites.

@mdh1418 mdh1418 mentioned this issue Apr 3, 2025
Merged
@karelz
Copy link
Member

karelz commented Apr 4, 2025

Duplicate of dotnet/dnceng#5329

@karelz karelz marked this as a duplicate of dotnet/dnceng#5329 Apr 4, 2025
@karelz karelz closed this as completed Apr 4, 2025
@ManickaP
Copy link
Member

ManickaP commented Apr 4, 2025

Dupe of dotnet/dnceng#5329

@ManickaP ManickaP reopened this Apr 4, 2025
@ManickaP
Copy link
Member

ManickaP commented Apr 4, 2025 •
edited
Loading

So originally this had nothing to do with AzureLinux 3 (it was an OSX issue) but now it catches the AZL3 timeouts here 🤦 So this will need to get cleaned up once we're done with dotnet/dnceng#5329. As the original was waiting for us to update msquic in OSX helix machines.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
area-System.Net.Quic blocking-clean-ci Blocking PR or rolling runs of 'runtime' or 'runtime-extra-platforms' Known Build Error Use this to report build issues in the .NET Helix tab os-mac-os-x macOS aka OSX
Projects
None yet
Milestone
10.0.0
Development

No branches or pull requests
5 participants
@karelz @ericstj @jeffschwMSFT @ManickaP @liveans