chore: ED-1722 Add typed dicts #1034

clinton-encord · 2025-11-26T21:49:51Z

Introduction and Explanation

Part of preparatory work for adding spaces. Add more types to SDK.

JIRA

Link ticket(s)

Documentation

There should be enough internal documentation for a product owner to write customer-facing documentation or a separate PR linked if writing the customer documentation directly. Link all that are relevant below.

Internal: notion link
Customer docs PR: link
OpenAPI/SDK
- Generated docs: link to example if possible
- Command to generate: here

Tests

Make a quick statement and post any relevant links of CI / test results. If the testing infrastructure isn’t yet in-place, note that instead.

What are the critical unit tests?
Explain the Integration Tests such that it’s clear Correctness is satisfied. Link to test results if possible.

Known issues

If there are any known issues with the solution, make a statement about what they are and why they are Ok to leave unsolved for now. Make tickets for the known issues linked to the original ticket linked above

gemini-code-assist · 2025-11-26T21:50:11Z

Summary of Changes

Hello @clinton-encord, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request significantly refactors the SDK's data handling by introducing and consistently applying TypedDict definitions across core modules. This effort aims to enhance type safety, improve code readability, and provide clearer data structure contracts for developers. It also lays essential groundwork for the upcoming 'spaces' feature, ensuring that the SDK's internal representations are robust and well-defined for future expansions.

Highlights

Introduction of Typed Dictionaries: A comprehensive set of TypedDict definitions has been introduced across the SDK, particularly within the encord.objects.types module, to provide explicit type hints for various data structures used in the label blob, such as answers, frame objects, and classifications.
Enhanced Type Safety and Readability: Existing functions and methods throughout the encord.objects module, including answers.py, classification_instance.py, coordinates.py, ontology_labels_impl.py, and ontology_object_instance.py, have been updated to leverage these new TypedDicts, significantly improving type safety, code clarity, and maintainability.
New Utility Functions for Label Handling: A new file, encord/objects/label_utils.py, has been added, containing helper functions create_frame_object_dict and create_frame_classification_dict to streamline the creation of typed dictionaries for frame objects and classifications. Additionally, encord/objects/coordinates.py now includes add_coordinates_to_frame_object_dict and get_coordinates_from_frame_object_dict.
Preparatory Work for 'Spaces' Feature: This change is explicitly noted as preparatory work for adding 'spaces' functionality, with the LabelRowMetadata in encord/orm/label_row.py now including a spaces field to support this upcoming feature.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature	Command	Description
Code Review	`/gemini review`	Performs a code review for the current pull request in its current state.
Pull Request Summary	`/gemini summary`	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help	`/gemini help`	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

github-actions · 2025-11-26T21:50:33Z

Unit test report (Python 3.9.24, Pydantic 1.10.22)

272 tests 272 ✅ 7s ⏱️
1 suites 0 💤
1 files 0 ❌

Results for commit f7908aa.

♻️ This comment has been updated with latest results.

github-actions · 2025-11-26T21:50:36Z

Unit test report (Python 3.9.24, Pydantic 2.12.3)

272 tests 272 ✅ 8s ⏱️
1 suites 0 💤
1 files 0 ❌

Results for commit f7908aa.

♻️ This comment has been updated with latest results.

clinton-encord · 2025-11-26T21:51:41Z

encord/objects/ontology_object_instance.py

                confidence=d["confidence"],
                manual_annotation=d.get("manualAnnotation", True),
-                reviews=d.get("reviews"),
+                reviews=None,  # To be deprecated. Always set to None.


If you look at LabelRowV2._to_encord_object, this is never sent out anyways.

gemini-code-assist

Code Review

This pull request introduces typed dictionaries across the SDK, which is a significant step towards improving type safety and code clarity. The changes are comprehensive and well-executed. I've identified a few areas for improvement, primarily concerning code duplication and minor simplifications, which I've detailed in the review comments. Overall, this is a valuable contribution to the codebase.

encord/objects/coordinates.py

gemini-code-assist · 2025-11-26T21:51:59Z

encord/objects/coordinates.py

+def get_coordinates_from_frame_object_dict(frame_object_dict: FrameObject) -> Coordinates:
+    if frame_object_dict["shape"] == Shape.BOUNDING_BOX:
+        return BoundingBoxCoordinates.from_dict(frame_object_dict)
+    elif frame_object_dict["shape"] == Shape.ROTATABLE_BOUNDING_BOX:
+        return RotatableBoundingBoxCoordinates.from_dict(frame_object_dict)
+    elif frame_object_dict["shape"] == Shape.POLYGON:
+        return PolygonCoordinates.from_dict(frame_object_dict)
+    elif frame_object_dict["shape"] == Shape.POINT:
+        coords = frame_object_dict["point"]["0"]
+        if "x" in coords and "y" in coords and "z" in coords:
+            return PointCoordinate3D.from_dict(frame_object_dict)  # type: ignore
+        elif "x" in coords and "y" in coords:
+            return PointCoordinate.from_dict(frame_object_dict)  # type: ignore
+        else:
+            raise ValueError(f"Invalid point coordinates in {frame_object_dict}")
+    elif frame_object_dict["shape"] == Shape.POLYLINE:
+        return PolylineCoordinates.from_dict(frame_object_dict)
+    elif "skeleton" in frame_object_dict:
+
+        def _with_visibility_enum(point: dict):
+            if point.get(Visibility.INVISIBLE.value):
+                point["visibility"] = Visibility.INVISIBLE
+            elif point.get(Visibility.OCCLUDED.value):
+                point["visibility"] = Visibility.OCCLUDED
+            elif point.get(Visibility.SELF_OCCLUDED.value):
+                point["visibility"] = Visibility.SELF_OCCLUDED
+            elif point.get(Visibility.VISIBLE.value):
+                point["visibility"] = Visibility.VISIBLE
+            return point
+
+        values = [_with_visibility_enum(pnt) for pnt in frame_object_dict["skeleton"].values()]
+        skeleton_frame_object_label = {
+            "name": frame_object_dict["name"],
+            "values": values,
+        }
+        return SkeletonCoordinates.from_dict(skeleton_frame_object_label)
+    elif "bitmask" in frame_object_dict:
+        return BitmaskCoordinates.from_dict(frame_object_dict)
+    elif "cuboid" in frame_object_dict:
+        return CuboidCoordinates.from_dict(frame_object_dict)
+    else:
+        raise NotImplementedError(f"Getting coordinates for `{frame_object_dict}` is not supported yet.")
+
+
+def get_geometric_coordinates_from_frame_object_dict(
+    frame_object_dict: FrameObject,
+) -> GeometricCoordinates:
+    if frame_object_dict["shape"] == Shape.BOUNDING_BOX:
+        return BoundingBoxCoordinates.from_dict(frame_object_dict)
+    elif frame_object_dict["shape"] == Shape.ROTATABLE_BOUNDING_BOX:
+        return RotatableBoundingBoxCoordinates.from_dict(frame_object_dict)
+    elif frame_object_dict["shape"] == Shape.POLYGON:
+        return PolygonCoordinates.from_dict(frame_object_dict)
+    elif frame_object_dict["shape"] == Shape.POINT:
+        coords = frame_object_dict["point"]["0"]
+        if "x" in coords and "y" in coords and "z" in coords:
+            return PointCoordinate3D.from_dict(frame_object_dict)  # type: ignore
+        elif "x" in coords and "y" in coords:
+            return PointCoordinate.from_dict(frame_object_dict)  # type: ignore
+        else:
+            raise ValueError(f"Invalid point coordinates in {frame_object_dict}")
+    elif frame_object_dict["shape"] == Shape.POLYLINE:
+        return PolylineCoordinates.from_dict(frame_object_dict)
+    elif "skeleton" in frame_object_dict:
+
+        def _with_visibility_enum(point: dict):
+            if point.get(Visibility.INVISIBLE.value):
+                point["visibility"] = Visibility.INVISIBLE
+            elif point.get(Visibility.OCCLUDED.value):
+                point["visibility"] = Visibility.OCCLUDED
+            elif point.get(Visibility.SELF_OCCLUDED.value):
+                point["visibility"] = Visibility.SELF_OCCLUDED
+            elif point.get(Visibility.VISIBLE.value):
+                point["visibility"] = Visibility.VISIBLE
+            return point
+
+        values = [_with_visibility_enum(pnt) for pnt in frame_object_dict["skeleton"].values()]
+        skeleton_frame_object_label = {
+            "name": frame_object_dict["name"],
+            "values": values,
+        }
+        return SkeletonCoordinates.from_dict(skeleton_frame_object_label)
+    elif "bitmask" in frame_object_dict:
+        return BitmaskCoordinates.from_dict(frame_object_dict)
+    elif "cuboid" in frame_object_dict:
+        raise NotImplementedError("Cuboid is not a two dimensional coordinate.")
+    else:
+        raise NotImplementedError(f"Getting coordinates for `{frame_object_dict}` is not supported yet.")


There is significant code duplication between get_coordinates_from_frame_object_dict and get_geometric_coordinates_from_frame_object_dict. This can be refactored to improve maintainability. get_geometric_coordinates_from_frame_object_dict could call get_coordinates_from_frame_object_dict and then perform its specific checks.

For example:

def get_geometric_coordinates_from_frame_object_dict( frame_object_dict: FrameObject, ) -> GeometricCoordinates: coordinates = get_coordinates_from_frame_object_dict(frame_object_dict) if isinstance(coordinates, CuboidCoordinates): raise NotImplementedError("Cuboid is not a two dimensional coordinate.") # The cast is safe because we've excluded the non-geometric types. return cast(GeometricCoordinates, coordinates)

@clinton-encord sounds like a good idea?

definitely!

encord/objects/label_utils.py

clinton-encord · 2025-11-26T21:53:42Z

encord/objects/ontology_object_instance.py

-                for answer_dict in answers_list:
-                    feature_hashes: Set[str] = {answer["featureHash"] for answer in answer_dict["answers"]}
-                    all_feature_hashes.update(feature_hashes)
-                    for frame_range in ranges_list_to_ranges(answer_dict["range"]):
-                        ranges.append((frame_range, feature_hashes))


This seems to me like a bug. Answers could also be text, but here we seem to assume its a list?

github-actions · 2025-11-26T21:55:43Z

SDK integration test report

285 tests ±0 277 ✅ ±0 12m 37s ⏱️ - 5m 35s
1 suites ±0 4 💤 ±0
1 files ±0 4 ❌ ±0

For more details on these failures, see this check.

Results for commit f7908aa. ± Comparison against base commit dbabc87.

♻️ This comment has been updated with latest results.

arthur-encord · 2025-11-27T11:48:34Z

encord/objects/coordinates.py


    @staticmethod
-    def from_dict(d: Dict) -> "PolygonCoordinates":
+    def from_dict(d: PolygonFrameObject) -> "PolygonCoordinates":


technically this is a public method, so I think we should narrow the type more here. If I do
PolygonCoordinates.from_dict({"polygon": {"0": {"x": 0, "y": 0}}}) then it'll complain that featureHash is missing; but I don't care, I just care about the coordinates. So it should probably just be that?

class PolygonCoordsDict(TypedDict): polygon: LegacyPolygonDict polygons: Optional[PolygonDict]

Oh true! Gotcha!

arthur-encord · 2025-11-27T11:49:42Z

encord/objects/coordinates.py

    return [coord for point in coordinates for coord in [point.x, point.y]]


+PolylineDict = Union[Dict[str, PointDict], list[PointDict], Dict[str, PointDict3D], list[PointDict3D]]


nit: can we put all the coordinates dict types together at the top?

arthur-encord · 2025-11-27T11:51:22Z

encord/objects/coordinates.py

+def get_coordinates_from_frame_object_dict(frame_object_dict: FrameObject) -> Coordinates:
+    if frame_object_dict["shape"] == Shape.BOUNDING_BOX:
+        return BoundingBoxCoordinates.from_dict(frame_object_dict)
+    elif frame_object_dict["shape"] == Shape.ROTATABLE_BOUNDING_BOX:
+        return RotatableBoundingBoxCoordinates.from_dict(frame_object_dict)
+    elif frame_object_dict["shape"] == Shape.POLYGON:
+        return PolygonCoordinates.from_dict(frame_object_dict)
+    elif frame_object_dict["shape"] == Shape.POINT:
+        coords = frame_object_dict["point"]["0"]
+        if "x" in coords and "y" in coords and "z" in coords:
+            return PointCoordinate3D.from_dict(frame_object_dict)  # type: ignore
+        elif "x" in coords and "y" in coords:
+            return PointCoordinate.from_dict(frame_object_dict)  # type: ignore
+        else:
+            raise ValueError(f"Invalid point coordinates in {frame_object_dict}")
+    elif frame_object_dict["shape"] == Shape.POLYLINE:
+        return PolylineCoordinates.from_dict(frame_object_dict)
+    elif "skeleton" in frame_object_dict:
+
+        def _with_visibility_enum(point: dict):
+            if point.get(Visibility.INVISIBLE.value):
+                point["visibility"] = Visibility.INVISIBLE
+            elif point.get(Visibility.OCCLUDED.value):
+                point["visibility"] = Visibility.OCCLUDED
+            elif point.get(Visibility.SELF_OCCLUDED.value):
+                point["visibility"] = Visibility.SELF_OCCLUDED
+            elif point.get(Visibility.VISIBLE.value):
+                point["visibility"] = Visibility.VISIBLE
+            return point
+
+        values = [_with_visibility_enum(pnt) for pnt in frame_object_dict["skeleton"].values()]
+        skeleton_frame_object_label = {
+            "name": frame_object_dict["name"],
+            "values": values,
+        }
+        return SkeletonCoordinates.from_dict(skeleton_frame_object_label)
+    elif "bitmask" in frame_object_dict:
+        return BitmaskCoordinates.from_dict(frame_object_dict)
+    elif "cuboid" in frame_object_dict:
+        return CuboidCoordinates.from_dict(frame_object_dict)
+    else:
+        raise NotImplementedError(f"Getting coordinates for `{frame_object_dict}` is not supported yet.")
+
+
+def get_geometric_coordinates_from_frame_object_dict(
+    frame_object_dict: FrameObject,
+) -> GeometricCoordinates:
+    if frame_object_dict["shape"] == Shape.BOUNDING_BOX:
+        return BoundingBoxCoordinates.from_dict(frame_object_dict)
+    elif frame_object_dict["shape"] == Shape.ROTATABLE_BOUNDING_BOX:
+        return RotatableBoundingBoxCoordinates.from_dict(frame_object_dict)
+    elif frame_object_dict["shape"] == Shape.POLYGON:
+        return PolygonCoordinates.from_dict(frame_object_dict)
+    elif frame_object_dict["shape"] == Shape.POINT:
+        coords = frame_object_dict["point"]["0"]
+        if "x" in coords and "y" in coords and "z" in coords:
+            return PointCoordinate3D.from_dict(frame_object_dict)  # type: ignore
+        elif "x" in coords and "y" in coords:
+            return PointCoordinate.from_dict(frame_object_dict)  # type: ignore
+        else:
+            raise ValueError(f"Invalid point coordinates in {frame_object_dict}")
+    elif frame_object_dict["shape"] == Shape.POLYLINE:
+        return PolylineCoordinates.from_dict(frame_object_dict)
+    elif "skeleton" in frame_object_dict:
+
+        def _with_visibility_enum(point: dict):
+            if point.get(Visibility.INVISIBLE.value):
+                point["visibility"] = Visibility.INVISIBLE
+            elif point.get(Visibility.OCCLUDED.value):
+                point["visibility"] = Visibility.OCCLUDED
+            elif point.get(Visibility.SELF_OCCLUDED.value):
+                point["visibility"] = Visibility.SELF_OCCLUDED
+            elif point.get(Visibility.VISIBLE.value):
+                point["visibility"] = Visibility.VISIBLE
+            return point
+
+        values = [_with_visibility_enum(pnt) for pnt in frame_object_dict["skeleton"].values()]
+        skeleton_frame_object_label = {
+            "name": frame_object_dict["name"],
+            "values": values,
+        }
+        return SkeletonCoordinates.from_dict(skeleton_frame_object_label)
+    elif "bitmask" in frame_object_dict:
+        return BitmaskCoordinates.from_dict(frame_object_dict)
+    elif "cuboid" in frame_object_dict:
+        raise NotImplementedError("Cuboid is not a two dimensional coordinate.")
+    else:
+        raise NotImplementedError(f"Getting coordinates for `{frame_object_dict}` is not supported yet.")


@clinton-encord sounds like a good idea?

encord/objects/ontology_labels_impl.py

encord/objects/types.py

encord/objects/label_utils.py

clinton-encord added 2 commits November 26, 2025 21:38

Add typing

53396af

Abit more typing

ada0c56

clinton-encord commented Nov 26, 2025

View reviewed changes

More types

af080e7

gemini-code-assist bot reviewed Nov 26, 2025

View reviewed changes

clinton-encord commented Nov 26, 2025

View reviewed changes

clinton-encord and others added 3 commits November 26, 2025 21:57

Fix test

e38ba05

Reverted reviews

5e3eb2e

cleanup: remove spaceId, remove unused file

401e846

arthur-encord reviewed Nov 27, 2025

View reviewed changes

encord/objects/label_utils.py Outdated Show resolved Hide resolved

rad-cord approved these changes Nov 27, 2025

View reviewed changes

clinton-encord added 3 commits November 28, 2025 14:59

Fix some typing

9d94e24

Fix typing

4030339

Fix stuff

f7908aa

clinton-encord merged commit 6060568 into master Nov 28, 2025
5 of 7 checks passed

clinton-encord deleted the clinton/ed-1722/add-typed-dicts branch November 28, 2025 23:05

		return [coord for point in coordinates for coord in [point.x, point.y]]


		PolylineDict = Union[Dict[str, PointDict], list[PointDict], Dict[str, PointDict3D], list[PointDict3D]]

chore: ED-1722 Add typed dicts #1034

chore: ED-1722 Add typed dicts #1034

Conversation

clinton-encord commented Nov 26, 2025

Introduction and Explanation

JIRA

Documentation

Tests

Known issues

Uh oh!

gemini-code-assist bot commented Nov 26, 2025

Summary of Changes

Highlights

Footnotes

Uh oh!

github-actions bot commented Nov 26, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Unit test report (Python 3.9.24, Pydantic 1.10.22)

Uh oh!

github-actions bot commented Nov 26, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Unit test report (Python 3.9.24, Pydantic 2.12.3)

Uh oh!

Choose a reason for hiding this comment

Uh oh!

gemini-code-assist bot left a comment

Choose a reason for hiding this comment

Code Review

Uh oh!

Uh oh!

gemini-code-assist bot Nov 26, 2025

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

github-actions bot commented Nov 26, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

SDK integration test report

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

github-actions bot commented Nov 26, 2025 •

edited

Loading

github-actions bot commented Nov 26, 2025 •

edited

Loading

github-actions bot commented Nov 26, 2025 •

edited

Loading