Fix Pydantic transformer to support from __future__ import annotations

[gverkes]

Tracking issue

Closes flyteorg/flyte#6694

Why are the changes needed?

When using from __future__ import annotations (PEP 563), the __annotations__ dictionary contains string literals instead of actual type objects. The current implementation directly accesses __annotations__ in the PydanticTransformer.get_literal_type() method, which causes standard Python types (str, int, float, etc.) in Pydantic BaseModel fields to be treated as unsupported types. This results in fallback to PickleFile serialization, causing warnings and inefficient serialization.

This fix is important for compatibility with Python 3.14, where PEP 563 future annotations will become the default behavior.

What changes were proposed in this pull request?

Updated PydanticTransformer.get_literal_type() in flytekit/extras/pydantic_transformer/transformer.py to use Pydantic's built-in field APIs instead of directly accessing __annotations__:

• model_fields for Pydantic v2
• __fields__ for Pydantic v1 (backwards compatibility)

These APIs properly resolve string annotations to actual type objects, regardless of whether future annotations are enabled.

Added comprehensive test suite in test_future_annotations_bug.py with 4 tests covering:

• Simple types (str, int, float, bool)
• Complex types (List, Dict, nested structures)
• Nested BaseModels
• Literal type structure validation (regression test to ensure types are not treated as PickleFile)

How was this patch tested?

Setup process

1. Created reproduction script with from __future__ import annotations
2. Verified the bug (types treated as strings causing PickleFile fallback)
3. Applied fix to transformer.py
4. Verified fix resolves the issue (no more warnings)
5. Added comprehensive test suite with 4 new tests
6. Ran all pydantic transformer tests: 29 tests passed (25 existing + 4 new)

Screenshots

Before fix:

WARNING - Unsupported Type str found, Flyte will default to use PickleFile as the transport.
WARNING - Unsupported Type int found, Flyte will default to use PickleFile as the transport.

After fix: No warnings, types correctly identified.

Check all the applicable boxes

• I updated the documentation accordingly.
• All new and existing tests passed.
• All commits are signed-off.

Related PRs

N/A

Docs link

N/A (code fix only, no user-facing documentation changes needed)

Summary by Bito

• Updates the get_literal_type method in the Pydantic transformer to correctly handle type annotations with 'from __future__ import annotations'.
• Prevents fallback to inefficient PickleFile serialization, ensuring compatibility with future Python versions.
• Adds a comprehensive test suite to validate functionality across different scenarios, including simple and complex types, nested models, and literal type structure validation.
• Overall summary: touches Pydantic transformer, type annotations, and serialization; introduces improvements in handling type annotations and testing.

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 79.53%. Comparing base (0663d8e) to head (e02c0cf).
⚠️ Report is 6 commits behind head on master.

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #3343      +/-   ##
==========================================
- Coverage   81.40%   79.53%   -1.87%     
==========================================
  Files         369      216     -153     
  Lines       29749    22697    -7052     
  Branches     2971     2974       +3     
==========================================
- Hits        24217    18052    -6165     
+ Misses       4738     3809     -929     
- Partials      794      836      +42     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[machichima]

Can we just use fields = typing.get_type_hints(t).items() here?

[machichima]

Thank you for this PR! Could you please run make lint to solve the lint error? Thank you!

[machichima]

LGTM. Thank you!

[welcome]

Congrats on merging your first pull request! 🎉