Skip to content

Commit ad9a043

Browse files
JSabadinkozlov721
andauthored
Updated docs (#286)
Co-authored-by: Martin Kozlovský <[email protected]>
1 parent a16ae24 commit ad9a043

File tree

5 files changed

+208
-26
lines changed

5 files changed

+208
-26
lines changed

luxonis_ml/data/README.md

Lines changed: 78 additions & 26 deletions
Original file line numberDiff line numberDiff line change
@@ -50,7 +50,7 @@ Each of these steps will be explained in more detail in the following examples.
5050

5151
We will be using our toy dataset `parking_lot` in all examples. The dataset consists of images of cars and motorcycles in a parking lot. Each image has a corresponding annotation in the form of a bounding box, keypoints and several segmentation masks.
5252

53-
**Dataset Annotations:**
53+
**Dataset Information:**
5454

5555
| Task | Annotation Type | Classes |
5656
| --------------------------- | ----------------- | ------------------------------------------------------------------------------------------------------ |
@@ -74,12 +74,15 @@ You can create as many datasets as you want, each with a unique name.
7474

7575
Datasets can be stored locally or in one of the supported cloud storage providers.
7676

77+
> \[!NOTE\]
78+
> 📚 For a complete list of all parameters and methods of the `LuxonisDataset` class, see the [datasets README.md](datasets/README.md).
79+
7780
### Dataset Creation
7881

7982
First we import `LuxonisDataset` and create a dataset with the name `"parking_lot"`.
8083

8184
```python
82-
from luxonisml.data import LuxonisDataset
85+
from luxonis_ml.data import LuxonisDataset
8386

8487
dataset_name = "parking_lot"
8588

@@ -104,58 +107,93 @@ Each data entry should be a dictionary with the following structure:
104107
```python
105108
{
106109
"file": str, # path to the image file
107-
"annotation": Optional[dict] # annotation of the file
110+
"task_name": Optional[str], # task type for this annotation
111+
"annotation": Optional[dict] # annotation of the instance in the file
108112
}
109113
```
110114

111-
The content of the `"annotation"` field depends on the task type and follows the Annotation Format described later in this document.
115+
Luxonis Data Format supports **annotations optionally structured into different tasks** for improved organization. Tasks can be explicitly named or left unset - if none are specified, all annotations will be grouped under a single `task_name` set by default to `""` . The [example below](#adding-data-with-a-generator-function) demonstrates this with instance keypoints and segmentation tasks.
116+
117+
The content of the `"annotation"` field depends on the task type and follows the [Annotation Format](#annotation-format) described later in this document.
112118

113119
#### Adding Data with a Generator Function
114120

115121
The recommended approach for adding data is to create a generator function that yields data entries one by one.
116122

117-
Here's an example that loads object detection annotations:
123+
The following example demonstrates how to load **bounding box annotations** along with their corresponding **keypoints annotations**, which are linked via `"instance_id"`.
124+
125+
Additionally, we yield **segmentation masks** while ensuring a clear separation between task groups. To achieve this, we use the `"task_name"` field—assigning `"instance_keypoints_car"` and `"instance_keypoints_motorbike"` for instance-keypoint-related annotations, and `"segmentation"` for the semantic segmentation task.
118126

119127
```python
120128
import json
121129
from pathlib import Path
130+
import cv2
131+
import numpy as np
122132

123133
# path to the dataset, replace it with the actual path on your system
124134
dataset_root = Path("data/parking_lot")
125135

126136
def generator():
127137
for annotation_dir in dataset_root.iterdir():
128-
with open(annotation_dir / "annotations.json") as f:
129-
data = json.load(f)
130-
131-
# get the width and height of the image
132-
W = data["dimensions"]["width"]
133-
H = data["dimensions"]["height"]
138+
annotation_file = annotation_dir / "annotations.json"
139+
if not annotation_file.exists():
140+
continue
134141

135-
image_path = annotation_dir / data["filename"]
142+
with open(annotation_file) as f:
143+
data = json.load(f)
136144

137-
for instance_id, bbox in data["BoundingBoxAnnotation"].items():
145+
W, H = data.get("dimensions", {}).get("width", 1), data.get("dimensions", {}).get("height", 1)
146+
image_path = str(annotation_dir / data.get("filename", ""))
138147

139-
# get unnormalized bounding box coordinates
148+
# Process Bounding Box Annotations
149+
for instance_id, bbox in data.get("BoundingBoxAnnotation", {}).items():
140150
x, y = bbox["origin"]
141151
w, h = bbox["dimension"]
142-
143-
# get the class name of the bounding box
144-
class_name = bbox["labelName"]
145152
yield {
146153
"file": image_path,
154+
"task_name": "instance_keypoints" + "_" + bbox["labelName"],
147155
"annotation": {
148-
"class": class_name,
149-
156+
"class": bbox["labelName"],
157+
"instance_id": instance_id,
150158
"boundingbox": {
151-
# normalized bounding box
152-
"x": x / W,
153-
"y": y / H,
154-
"w": w / W,
155-
"h": h / H,
159+
"x": x / W, "y": y / H, "w": w / W, "h": h / H
156160
}
157161
},
158162
}
163+
164+
# Process Keypoints Annotations
165+
for instance_id, keypoints_data in data.get("KeypointsAnnotation", {}).items():
166+
keypoints = [
167+
(kp["location"][0] / W, kp["location"][1] / H, kp["visibility"])
168+
for kp in keypoints_data["keypoints"]
169+
]
170+
yield {
171+
"file": image_path,
172+
"task_name": "instance_keypoints" + "_" + keypoints_data["labelName"],
173+
"annotation": {
174+
"instance_id": instance_id,
175+
"keypoints": {"keypoints": keypoints},
176+
},
177+
}
178+
179+
# Process Segmentation Annotations
180+
segmentation_data = data.get("VehicleTypeSegmentation", {})
181+
if "filename" in segmentation_data:
182+
mask_path = annotation_dir / segmentation_data["filename"]
183+
mask_rgb = cv2.cvtColor(cv2.imread(str(mask_path)), cv2.COLOR_BGR2RGB)
184+
if mask_rgb is not None:
185+
for instance in segmentation_data.get("instances", []):
186+
label = instance["labelName"]
187+
color = np.array(instance["pixelValue"], dtype=np.uint8)
188+
binary_mask = (mask_rgb == color).all(axis=-1).astype(np.uint8)
189+
yield {
190+
"file": image_path,
191+
"task_name": "segmentation",
192+
"annotation": {
193+
"class": label,
194+
"segmentation": {"mask": binary_mask},
195+
},
196+
}
159197
```
160198

161199
The generator is then passed to the `add` method of the dataset.
@@ -264,6 +302,9 @@ This guide covers the loading of datasets using the `LuxonisLoader` class.
264302

265303
The `LuxonisLoader` class can also take care of data augmentation, for more info see [Augmentation](#augmentation).
266304

305+
> \[!NOTE\]
306+
> 📚 For a complete list of all parameters of the `LuxonisLoader` class, see the [loaders README.md](loaders/README.md).
307+
267308
### Dataset Loading
268309

269310
To load a dataset with `LuxonisLoader`, we need an instance of `LuxonisDataset`, and we need to specify what view of the dataset we want to load.
@@ -299,6 +340,7 @@ The supported formats are:
299340
- [**MT YOLOv6**](https://roboflow.com/formats/mt-yolov6)
300341
- [**CreateML JSON**](https://roboflow.com/formats/createml-json)
301342
- [**TensorFlow Object Detection CSV**](https://roboflow.com/formats/tensorflow-object-detection-csv)
343+
- [**SOLO**](https://docs.unity3d.com/Packages/[email protected]/manual/Schema/SoloSchema.html)
302344
- **Classification Directory** - A directory with subdirectories for each class
303345

304346
```plaintext
@@ -348,6 +390,12 @@ The dataset directory can either be a local directory or a directory in one of t
348390

349391
The directory can also be a zip file containing the dataset.
350392

393+
The `task_name` argument can be specified as a single string or as a dictionary. If a string is provided, it will be used as the task name for all records.
394+
Alternatively, you can provide a dictionary that maps class names to task names for better dataset organization. See the example below.
395+
396+
> \[!NOTE\]
397+
> 📚 For a complete list of all parameters of the `LuxonisParser` class, see the [parsers README.md](parsers/README.md).
398+
351399
```python
352400
from luxonisml.data import LuxonisParser
353401
from luxonis_ml.enums import DatasetType
@@ -357,8 +405,12 @@ dataset_dir = "path/to/dataset"
357405
parser = LuxonisParser(
358406
dataset_dir=dataset_dir,
359407
dataset_name="my_dataset",
360-
dataset_type=DatasetType.COCO
361-
)
408+
dataset_type=DatasetType.COCO,
409+
task_name={
410+
"semantic_segmentation": "TorsoLimbs",
411+
"semantic_segmentation": "HeadNeck",
412+
"instance_keypoints": "FullPersonBody"
413+
},
362414
```
363415

364416
After initializing the parser, you can parse the dataset to create a `LuxonisDataset` instance. The `LuxonisDataset` instance will contain the data from the dataset with splits for training, validation, and testing based on the dataset directory structure.

luxonis_ml/data/datasets/README.md

Lines changed: 73 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,73 @@
1+
# LuxonisML Dataset
2+
3+
The `LuxonisDataset` class provides functionality for creating, managing, and interacting with datasets.
4+
5+
## Table of Contents
6+
7+
- [LuxonisML Dataset](#luxonisml-dataset)
8+
- [Parameters](#parameters)
9+
- [Core Methods](#core-methods)
10+
- [Adding Data](#adding-data)
11+
- [Creating Splits](#creating-splits)
12+
- [Merging Datasets](#merging-datasets)
13+
- [Cloning the Dataset](#cloning-the-dataset)
14+
15+
## Parameters
16+
17+
### LuxonisDataset Constructor Parameters
18+
19+
| Parameter | Type | Default | Description |
20+
| ----------------- | --------------- | --------------------- | ----------------------------------------------------- |
21+
| `dataset_name` | `str` | Required | The unique name for the dataset |
22+
| `team_id` | `Optional[str]` | `None` | Optional team identifier for the cloud |
23+
| `bucket_type` | `BucketType` | `BucketType.INTERNAL` | Whether to use external cloud buckets |
24+
| `bucket_storage` | `BucketStorage` | `BucketStorage.LOCAL` | Underlying storage (local, GCS, S3, Azure) |
25+
| `delete_existing` | `bool` | `False` | Whether to delete existing dataset with the same name |
26+
| `delete_remote` | `bool` | `False` | Whether to delete remote data when deleting dataset |
27+
28+
## Core Methods
29+
30+
### Adding Data
31+
32+
The `add()` method is used to add data to a dataset.
33+
34+
#### Parameters
35+
36+
| Parameter | Type | Default | Description |
37+
| ------------ | ----------------- | ----------- | ----------------------------------------------------- |
38+
| `generator` | `DatasetIterator` | Required | Generator yielding dataset records |
39+
| `batch_size` | `int` | `1_000_000` | Number of annotation records to process in each batch |
40+
41+
### Creating Splits
42+
43+
The `make_splits()` method divides the dataset into separate splits (train/val/test) for machine learning workflows.
44+
45+
#### Parameters
46+
47+
| Parameter | Type | Default | Description |
48+
| -------------------- | --------------------------------------------------------------------------------------------- | ------- | --------------------------------------------- |
49+
| `splits` | `Mapping[str, float]` or<br>`Tuple[float, float, float]` or<br>`Mapping[str, List[PathType]]` | `None` | Proportions or explicit file paths for splits |
50+
| `replace_old_splits` | `bool` | `False` | Whether to replace existing splits |
51+
52+
### Merging Datasets
53+
54+
The `merge_with()` method combines data from another dataset into the current one.
55+
56+
#### Parameters
57+
58+
| Parameter | Type | Default | Description |
59+
| ------------------ | ---------------- | -------- | --------------------------------------------------------- |
60+
| `other` | `LuxonisDataset` | Required | Dataset to merge with |
61+
| `inplace` | `bool` | `True` | Whether to modify the current dataset or create a new one |
62+
| `new_dataset_name` | `str` | `None` | Name for the new dataset if `inplace=False` |
63+
64+
### Cloning the Dataset
65+
66+
The `clone()` method creates a complete copy of a dataset with a new name. It copies all data, metadata, and splits from the original dataset.
67+
68+
#### Parameters
69+
70+
| Parameter | Type | Default | Description |
71+
| ------------------ | ------ | -------- | ------------------------------------------------------------------------------------------ |
72+
| `new_dataset_name` | `str` | Required | Name for the cloned dataset |
73+
| `push_to_cloud` | `bool` | `True` | Whether to push the cloned dataset to cloud storage. Only if the current dataset is remote |

luxonis_ml/data/loaders/README.md

Lines changed: 26 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,26 @@
1+
# LuxonisML Loader
2+
3+
The `LuxonisLoader` class provides efficient access to dataset samples with configurable preprocessing options.
4+
5+
## Table of Contents
6+
7+
- [LuxonisML Loader](#luxonisml-loader)
8+
- [Parameters](#parameters)
9+
10+
## Parameters
11+
12+
### LuxonisLoader Constructor Parameters
13+
14+
| Parameter | Type | Default | Description |
15+
| ----------------------------- | ----------------------------------------- | ------------------- | -------------------------------------------------------------- |
16+
| `dataset` | `LuxonisDataset` | Required | The dataset to load data from |
17+
| `view` | `Union[str, List[str]]` | `"train"` | Dataset split to use ("train", "val", "test") |
18+
| `augmentation_engine` | `str` | `"albumentations"` | Augmentation engine to use |
19+
| `augmentation_config` | `Optional[Union[List[Params], PathType]]` | `None` | Configuration for the augmentations |
20+
| `height` | `Optional[int]` | `None` | Height of the output images |
21+
| `width` | `Optional[int]` | `None` | Width of the output images |
22+
| `keep_aspect_ratio` | `bool` | `True` | Whether to keep image aspect ratio |
23+
| `exclude_empty_annotations` | `bool` | `False` | Whether to exclude empty annotations |
24+
| `color_space` | `Literal["RGB", "BGR"]` | `"RGB"` | Color space of output images |
25+
| `keep_categorical_as_strings` | `bool` | `False` | Whether to keep categorical metadata as strings |
26+
| `update_mode` | `UpdateMode` | `UpdateMode.ALWAYS` | Whether to always download dataset from cloud or only if empty |

luxonis_ml/data/parsers/README.md

Lines changed: 30 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,30 @@
1+
# LuxonisML Parsers
2+
3+
The `LuxonisParser` class provides functionality for converting various dataset formats to the Luxonis Dataset Format (LDF).
4+
5+
## Table of Contents
6+
7+
- [LuxonisML Parsers](#luxonisml-parsers)
8+
- [Parameters](#parameters)
9+
- [Parse Method Parameters](#parse-method-parameters)
10+
11+
## Parameters
12+
13+
### LuxonisParser Constructor Parameters
14+
15+
| Parameter | Type | Default | Description |
16+
| ---------------- | -------------------------------------- | -------- | ------------------------------------------------------------------------------------------------------------------------------- |
17+
| `dataset_dir` | `str` | Required | Path or URL to dataset directory (local path, `gcs://`, `s3://` or `roboflow://`) |
18+
| `dataset_name` | `Optional[str]` | `None` | Name for the dataset (if None, derived from directory name) |
19+
| `save_dir` | `Optional[Union[Path, str]]` | `None` | Where to save downloaded datasets if remote URL is provided (if None, uses current directory) |
20+
| `dataset_plugin` | `Optional[str]` | `None` | Dataset plugin to use (if None, uses `LuxonisDataset`) |
21+
| `dataset_type` | `Optional[DatasetType]` | `None` | Force specific dataset format type instead of auto-detection |
22+
| `task_name` | `Optional[Union[str, Dict[str, str]]]` | `None` | Task name(s) for the dataset. Used to link the classes to the desired tasks, with class names as keys and task names as values. |
23+
24+
### Parse Method Parameters
25+
26+
| Parameter | Type | Default | Description |
27+
| -------------- | ---------------------------- | ------- | ------------------------------------------------------------------------------------------------------------------------------------------- |
28+
| `split` | `Optional[str]` | `None` | Split name if parsing a single split |
29+
| `random_split` | `bool` | `True` | Whether to create random splits |
30+
| `split_ratios` | `Optional[Dict[str, float]]` | `None` | Ratios for train/validation/test splits. If set to `None`, the default behavior of the `LuxonisDataset`'s `make_splits` method will be used |

luxonis_ml/data/requirements.txt

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -10,3 +10,4 @@ filelock~=3.0
1010
bidict~=0.21
1111
gdown~=4.7
1212
defusedxml~=0.7
13+
pillow-heif<0.22.0

0 commit comments

Comments
 (0)