Skip to content

Conversation

pingsutw
Copy link
Member

@pingsutw pingsutw commented Aug 8, 2025

Tracking issue

fsspec/gcsfs#659

Why are the changes needed?

Avoid GCSFileSystem reusing an HTTP session from a different event loop.

5-08-08T00:08:10Z │ /opt/conda/lib/python3.10/weakref.py:667 in _exitfunc                        │
2025-08-08T00:08:10Z │                                                                              │
2025-08-08T00:08:10Z │ ❱ 667 │   │   │   │   │   │   f()                                            │
2025-08-08T00:08:10Z │                                                                              │
2025-08-08T00:08:10Z │ /opt/conda/lib/python3.10/weakref.py:591 in __call__                         │
2025-08-08T00:08:10Z │                                                                              │
2025-08-08T00:08:10Z │ ❱ 591 │   │   │   return info.func(*info.args, **(info.kwargs or {}))        │
2025-08-08T00:08:10Z │                                                                              │
2025-08-08T00:08:10Z │ /opt/conda/lib/python3.10/site-packages/gcsfs/core.py:380 in close_session   │
2025-08-08T00:08:10Z │                                                                              │
2025-08-08T00:08:10Z │ ❱  380 │   │   │   │   asyn.sync(asyn.loop[0], session.close, timeout=0.1)   │
2025-08-08T00:08:10Z │                                                                              │
2025-08-08T00:08:10Z │ /opt/conda/lib/python3.10/site-packages/fsspec/asyn.py:103 in sync           │
2025-08-08T00:08:10Z │                                                                              │
2025-08-08T00:08:10Z │ ❱  103 │   │   raise return_result                                           │
2025-08-08T00:08:10Z │                                                                              │
2025-08-08T00:08:10Z │ /opt/conda/lib/python3.10/site-packages/fsspec/asyn.py:56 in _runner         │
2025-08-08T00:08:10Z │                                                                              │
2025-08-08T00:08:10Z │ ❱   56 │   │   result[0] = await coro                                        │
2025-08-08T00:08:10Z │                                                                              │
2025-08-08T00:08:10Z │ /opt/conda/lib/python3.10/asyncio/tasks.py:445 in wait_for                   │
2025-08-08T00:08:10Z │                                                                              │
2025-08-08T00:08:10Z │ ❱ 445 │   │   │   return fut.result()                                        │
2025-08-08T00:08:10Z │                                                                              │
2025-08-08T00:08:10Z │ /opt/conda/lib/python3.10/site-packages/aiohttp/client.py:1327 in close      │
2025-08-08T00:08:10Z │                                                                              │
2025-08-08T00:08:10Z │ ❱ 1327 │   │   │   │   await self._connector.close()                         │
2025-08-08T00:08:10Z │                                                                              │
2025-08-08T00:08:10Z │ /opt/conda/lib/python3.10/site-packages/aiohttp/connector.py:1044 in close   │
2025-08-08T00:08:10Z │                                                                              │
2025-08-08T00:08:10Z │ ❱ 1044 │   │   await super().close(abort_ssl=abort_ssl or self._ssl_shutdown 

What changes were proposed in this pull request?

How was this patch tested?

from click.testing import CliRunner

from flytekit import task, workflow, ImageSpec
from flytekit.clis.sdk_in_container import pyflyte

new_flytekit = "git+https://github.com/flyteorg/flytekit.git@0bdf91dc3e67bb4f44f1d2c997a8bd58b6e2c71b"

image = ImageSpec(
    registry="ghcr.io/flyteorg",
    packages=[new_flytekit, "gcsfs==2025.7.0", "fsspec==2025.7.0"],
    apt_packages=["git"],
)


@task(retries=5, container_image=image)
def say_hello1() -> int:
    print(123)
    return 3


@workflow
def wf():
    say_hello1()

Setup process

Screenshots

Check all the applicable boxes

  • I updated the documentation accordingly.
  • All new and existing tests passed.
  • All commits are signed-off.

Signed-off-by: Kevin Su <[email protected]>
Signed-off-by: Kevin Su <[email protected]>
Signed-off-by: Kevin Su <[email protected]>
Signed-off-by: Kevin Su <[email protected]>
Copy link

codecov bot commented Aug 8, 2025

Codecov Report

❌ Patch coverage is 11.11111% with 8 lines in your changes missing coverage. Please review.
✅ Project coverage is 46.58%. Comparing base (eb5a67f) to head (0bdf91d).

Files with missing lines Patch % Lines
flytekit/core/data_persistence.py 11.11% 8 Missing ⚠️

❗ There is a different number of reports uploaded between BASE (eb5a67f) and HEAD (0bdf91d). Click for more details.

HEAD has 96 uploads less than BASE
Flag BASE (eb5a67f) HEAD (0bdf91d)
99 3
Additional details and impacted files
@@             Coverage Diff             @@
##           master    #3308       +/-   ##
===========================================
- Coverage   85.26%   46.58%   -38.68%     
===========================================
  Files         386      215      -171     
  Lines       30276    22454     -7822     
  Branches     2969     2949       -20     
===========================================
- Hits        25814    10460    -15354     
- Misses       3615    11476     +7861     
+ Partials      847      518      -329     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Signed-off-by: Kevin Su <[email protected]>
@pingsutw pingsutw marked this pull request as draft August 8, 2025 16:09
@flyte-bot
Copy link
Contributor

Bito Automatic Review Skipped - Draft PR

Bito didn't auto-review because this pull request is in draft status.
No action is needed if you didn't intend for the agent to review it. Otherwise, to manually trigger a review, type /review in a comment and save.
You can change draft PR review settings here, or contact your Bito workspace admin at [email protected].

Signed-off-by: Kevin Su <[email protected]>
Signed-off-by: Kevin Su <[email protected]>
Signed-off-by: Kevin Su <[email protected]>
@flyte-bot
Copy link
Contributor

Bito Automatic Review Skipped - Draft PR

Bito didn't auto-review because this pull request is in draft status.
No action is needed if you didn't intend for the agent to review it. Otherwise, to manually trigger a review, type /review in a comment and save.
You can change draft PR review settings here, or contact your Bito workspace admin at [email protected].

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants