Create Dataclasses and Builder for GPU Index Build Config

[Rajrahane]

Description

Implements

1. Base Interfaces for the IndexBuildService orchestrator and its workflow tasks - GPUIndexBuildService, GPUToCPUIndexConverter, CPUIndexWriter
2. Base Response and Config models for GPU and CPU Index.
3. Concrete Faiss specific data models. Response models for Response for GPU and CPU Index.
4. Faiss Index Build Service build_index method to create a GPU Index, transform to CPU index and write CPU Index to local disc filepath, deletes the indexes after writing to file.
5. Utils to create the params required for GPUIndexConfig from the API IndexBuildParameters.

Issues Resolved

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

[Rajrahane]

Still a WIP
Here's how you can initialize the config-

def create_index_config(**kwargs) -> GPUIndexBuildConfig:
builder = IndexConfigBuilder()
director = IndexConfigDirector(builder)
return director.construct_config(kwargs)

print(create_index_config(metric='cosinesimil', gpu_config={'graph_build_algo': 'NN_DECENT', 'ivf_pq_build_params': {'n_lists': 1040}}))

[rchitale7]

@Rajrahane For general feedback - I feel that the index_config_builder.py and index_config_director.py classes are unnecessary, and can be consolidated into faiss_index_builder.py and faiss_index_config_builder.py. It will help make the code more understandable.

[Rajrahane]

I've tried consolidating creation of the GPUIndexBuildConfig to a simpler one function, rather than the elaborate index_config_builder and index_config_director as suggest by Rohan

The issue is that it was Difficult to parse a complex multilevel dict with
GPUIndexBuildConfig
-> GPUIndexCagraConfig ( with IVFPQBuild and IVFPQSearch configs)
-> IndexHNSWConfig

Because the dict sent is complex with missing fields it threw a lot of NULL issues,
it required me to create the inner objects manually using helpers.
Eg: ivf_pq_build_config = (
IVFPQBuildCagraConfig(**ivf_pq_build_params)
if ivf_pq_build_params
else IVFPQBuildCagraConfig()
)

And i used the builder pattern in a different filejust to make it more structured, else the single function would be very large.

Also had to maintain the single responsibility of the main function build_gpu_index of the FaissIndexBuilder.py file to only call all the steps ->

1. build the core dataclass from dict
2. build the faiss binding config version from the dataclass
3. create the gpu index
4. create the cpu index and write to disk

[Rajrahane]

Also the problem to solve in a gist
Goal is to completely configure the GPUIndexConfig from the IndexBuildParameters and convert it into Faiss IndexConfig bindings

before feeding it to the _create_gpu_index and _create_and_write_cpu_index_to_file helper methods

[Rajrahane]

Updated Design

[jed326]

Thanks @Rajrahane , this PR mostly LGTM, just a few spots of discussion/nits