[feature-v3] Move rest of OTX under native backend. Change structure of the folders.

[kprokofi]

Summary

• Move current OTX modules under Native backend
• Refactor "core" and "algo" folder
• Refine all imports
• Move unit tests

New structure:

New structure:

# New structure:

otx/
├── backend/  # Contains all backend implementations
│   ├── native/
│   │   ├── callbacks/              # Custom OTX callbacks
│   │   ├── cli/                    # CLI functionality and utilities
│   │   ├── exporter/               # OTX export logic; might be moved to a shared base folder to orchestrate exports across all backends
│   │   ├── lightning/              # Custom Lightning-specific components (e.g., accelerators, plugins, strategies)
│   │   ├── models/                 # Task-specific and shared model implementations
│   │   │   ├── anomaly/            # Models for anomaly detection
│   │   │   ├── classification/     # Models for image classification
│   │   │   ├── common/             # Shared components used across tasks
│   │   │   ├── detection/          # Models for object detection (base model resides in task folder now)
│   │   │   ├── instance_segmentation/  # Models for instance segmentation
│   │   │   ├── keypoint_detection/     # Models for keypoint detection
│   │   │   ├── modules/            # Internal modules for model building
│   │   │   ├── samplers/           # Samplers used in training pipelines
│   │   │   ├── segmentation/       # Models for semantic segmentation
│   │   │   ├── utils/              # Utilities specific to models
│   │   │   ├── __init__.py
│   │   │   └── base.py             # Base OTXModel class used by all tasks
│   │   ├── optimizers/             # Custom optimizers for training
│   │   ├── schedulers/             # Custom learning rate schedulers
│   │   ├── tools/                  # Additional utilities such as batch size search, explainability tools, and AutoConfigurator
│   │   ├── utils/                  # General-purpose utilities for the native backend
│   │   ├── __init__.py
│   │   └── engine.py               # Core training and inference logic for the native backend
│   └── openvino/
│       ├── models/                 # OpenVINO-compatible model wrappers
│       ├── __init__.py
│       └── engine.py               # Inference engine for OpenVINO backend
├── cli/                            # Entry point and shared CLI logic
├── config/                         # DTO configurations
├── data/                           # All data-related components
│   ├── dataset/                    # Dataset implementations and wrappers
│   ├── entity/                     # Data entity
│   │   ├── torch/                  # Torch-based OTX data entities (OTXDataItem)
│   │   ├── __init__.py
│   │   ├── base.py                 # Base class for data entities
│   │   ├── tile.py                 # Tiling-related data entity
│   │   └── utils.py                # Utility functions for entities
│   ├── transform_libs/            # Libraries for data transformations
│   ├── utils/                      # Shared utilities for data module
│   ├── __init__.py
│   ├── factory.py                  # Data factory for loading/preparing datasets
│   ├── mem_cache.py                # Caching mechanism for dataset loading
│   └── module.py                   # OTXDataModule implementation
├── engine/                         # Training and inference engine logic (abstract)
├── metrics/                        # Evaluation metrics and metric calculators 
├── recipe/                         # Training recipes and hyperparameter configurations
├── tools/                          # Integration tools for Geti + Geti templates (will be removed)
├── types/                          # Type definitions
└── utils/                          # Global utility functions used across the project

# API example (models and dataset import):

from otx.backend.native.models import ATSS, EfficientNetMulticlassCls, SegNext
from otx.backend.openvino.models import OVDetectionModel

from otx.data.datasets import OTXInstanceSegDataset, OTXAnomalyDataset, OTXUltaliticsDataset
from otx.data.entity import OTXDataItem

How to test

Checklist

• I have added unit tests to cover my changes.​
• I have added integration tests to cover my changes.​
• I have ran e2e tests and there is no issues.
• I have added the description of my changes into CHANGELOG in my target branch (e.g., CHANGELOG in develop).​
• I have updated the documentation in my target branch accordingly (e.g., documentation in develop).
• I have linked related issues.

License

• I submit my code changes under the same Apache License that covers the project.
  Feel free to contact the maintainers if that's a concern.
• I have updated the license header for each file (see an example below).

# Copyright (C) 2025 Intel Corporation
# SPDX-License-Identifier: Apache-2.0

[ashwinvaidya17]

Thanks for the efforts! I have some minor comments.

I couldn't comment on the file but I have concerns about src/otx/types/label.py. The types here are specific to the native backend. Maybe we can have backend.native.types?

[kprokofi]

Thanks for the efforts! I have some minor comments.

I couldn't comment on the file but I have concerns about src/otx/types/label.py. The types here are specific to the native backend. Maybe we can have backend.native.types?

We use LableInfo for our datumaro datasets. Our datasets will be shared across all backends. So, I would rather keep labels.py in the base "types" folder

[ashwinvaidya17]

Some minor picky comments but feel free to ignore

[rajeshgangireddy]

│ │ ├── exporter/ # OTX export logic; might be moved to a shared base folder to orchestrate exports across all backends

I agree. I think we should move code from exporter outside specific backends to something like model_converter.

I have a different question.

│ │ │ ├── samplers/ # Samplers used in training pipelines

Why is samplers in models directory ? Wouldn't it be common for all backends which can perform training ?

[kprokofi]

│ │ ├── exporter/ # OTX export logic; might be moved to a shared base folder to orchestrate exports across all backends

I agree. I think we should move code from exporter outside specific backends to something like model_converter.

I have a different question.

│ │ │ ├── samplers/ # Samplers used in training pipelines

Why is samplers in models directory ? Wouldn't it be common for all backends which can perform training ?

1. We will move it later depending on the Exporter design. For now we use it only for Native.
2. Let's place it under the data folder. "otx.data.samplers". What do you think?

[sovrasov]

samplers/ # Samplers used in training

Agree that samplers can be moved out from models

[kprokofi]

Samplers are moved under data folder

[Copilot]

Pull Request Overview

This PR reorganizes the OTX repository by moving existing OTX modules under a new backend/native folder, refactoring core and algo directories, and updating all import paths accordingly. It also removes outdated developer guides and example notebooks.

• Moves OTX modules into otx/backend/native and adjusts folder hierarchy
• Removes legacy for_developers guides and Jupyter notebook
• Updates import statements across documentation and code to reflect new module paths

Reviewed Changes

Copilot reviewed 732 out of 732 changed files in this pull request and generated no comments.

Show a summary per file

File	Description
for_developers/dir_structure.md	Deleted outdated directory-structure guide
for_developers/contribution_guide.md	Deleted outdated contribution guide
for_developers/add_custom_model.ipynb	Deleted example notebook
docs/source/.../how_to_train/*.rst	Updated import paths for SubsetConfig and OTXDataModule
docs/source/.../advanced/*.rst	Updated imports to otx.backend.native for models, callbacks, schedulers
docs/source/.../get_started/*.rst	Updated imports from otx.core.* to new top-level and backend paths
docs/source/.../explanation/additional_features/*.rst	Updated imports for config, data, samplers, callbacks
docker/download_pretrained_weights.py	Updated instantiator import path

Comments suppressed due to low confidence (2)

for_developers/dir_structure.md:1

• The developer directory-structure guide was removed in this PR; please add or point to an updated guide reflecting the new otx/backend/native hierarchy so developers can reference the current layout.

entire file removed

for_developers/contribution_guide.md:1

• The contribution guide has been deleted; ensure there is a replacement that explains the proposal and review process under the new structure to onboard contributors smoothly.

entire file removed