FXC-3378 Pydantic v2

[yaugenst-flex]

This PR completes the migration of our codebase from Pydantic V1 to Pydantic V2.

1. Imports & Dependency Updates:

• Standardized all Pydantic imports to use the pydantic namespace (and pydantic_settings for BaseSettings), removing pydantic.v1 compatibility imports.
• Switched to direct component imports (e.g., from pydantic import Field, model_validator).
• Modernized typing imports (e.g., Optional, Union, Self, TypeAlias) and other general Python type hinting.
• Updated project-specific imports, including removal of the skip_if_fields_missing decorator and replacing annotate_type with discriminated_union.
• Incorporated schema utilities from pydantic_core and pydantic.json_schema.

2. Field Definitions & Type-Hinting:

• Migrated pydantic.v1.Field usage to the new pydantic.Field syntax.
• Made field optionality explicit using Optional[T] and appropriate Field defaults (e.g., default=None, default_factory).
• Updated constrained types (e.g., PositiveFloat, NonNegativeInt) and replaced confloat with Annotated and Field.
• Standardized on default_factory for mutable defaults (e.g., dict, list, custom classes).
• Converted list defaults/assignments to tuples for immutability where appropriate.
• Defined fields with Unions of 'Traced' types for autograd compatibility.

3. Validation Logic:

• Replaced @pydantic.v1.validator with @pydantic.field_validator, adjusting signatures and return values.
• Migrated @pydantic.v1.root_validator to @pydantic.model_validator, specifying mode='before' or mode='after' and updating logic for field access (e.g., self.field in mode='after').
• Adjusted validator signatures, utilized the info object, and used direct attribute access/setting (e.g. object.__setattr__). Added @classmethod where needed.
• Removed the custom skip_if_fields_missing decorator, integrating its logic into model_validator or relying on V2's improved validation flow.
• Consolidated custom _post_init_validators logic into @model_validator(mode='after').

4. Configuration & Settings:

• Replaced the inner Config class with model_config = ConfigDict(...).
• Migrated pydantic.v1.BaseSettings to pydantic_settings.BaseSettings.
• Removed obsolete V1-specific configurations like allow_population_by_field_name.

5. Serialization & Parsing:

• Replaced .dict() with .model_dump() and .json() (or ._json()) with .model_dump_json().
• Updated data parsing from cls.parse_obj() to cls.model_validate().
• Streamlined serialization logic, removing custom encoders (e.g., ndarray_encoder) in favor of V2's built-in capabilities or Annotated serializers.
• Standardized exception handling to pydantic.ValidationError and PydanticSerializationError.

6. Schema & Custom Types:

• Replaced __get_validators__ with __get_pydantic_core_schema__ and __modify_schema__ with __get_pydantic_json_schema__ for custom type definitions.
• Refactored custom types using Pydantic V2's Annotated type along with BeforeValidator/AfterValidator and PlainSerializer/WrapSerializer.
• Replaced custom constrained_array with a new array_alias factory function using Annotated.
• Updated custom complex number types to Pydantic V2-compliant versions.
• Updated autograd utilities (e.g., TracedDict, get_static, hasbox).
• Utilized pydantic.PrivateAttr for non-schema model attributes.

7. Model Methods & Internal Refactors:

• Refactored copy() and updated_copy() methods to use model_copy(update=...).
• Adapted __init_subclass__ for V2 and updated docstring generation to use model_fields.
• Introduced new model traversal utilities (find_paths, find_submodels).
• Reworked __eq__ methods for V2 compatibility and updated/added __str__, __repr__, and __rich_repr__ for improved representations.
• Replaced __fields__ access with model_fields for introspection.

[momchil-flex]

It seems like fundamentally there's just a few things that change? Maybe I'm missing something important/tricky. Could you explain if there's anything like that, and what changes going forward?

Also very important - we will certainly need to update the backend python code too and might need to update denormalizer, metadata api, and who knows what else.

[yaugenst-flex]

Yes the biggest changes are related to custom types and serialization, everything else is relatively straightforward. I'll write something up.