Enhance Covalent's ConfigManager to become more centralized

[venkatBala]

ConfigManager class handles the core logic of managing Covalent's configuration (in-memory/on disk). It gets instantiated upon importing the module in any environment can the number of times the configuration gets either overwritten or opened during workflow execution is slightly convoluted. This discussion is aimed towards enhancing the ConfigManager to become a bit more centralized and consistent.

Few issues encountered with the current way of doing configuration management

• Assuming one has covalent self-hosted in a remote environment and the user has access to the dispatcher address and port
• From the client' machine (assuming only Covalent core SDK) is installed and no covalent server is running locally, the user can create any workflow using an IDE of their choosing
• Consider the following sample workflow

import covalent as ct

ct.set_config({"dispatcher.address": <addr>})
ct.set_config({"dispatcher.port": <port>})

@ct.lattice
@ct.electron
def identity(x):
   return x

dispatch_id = ct.dispatch(identity)(1)
result = ct.get_result(dispatch_id, wait=True)
print(result)

The very first time the dispatch is sent, the error noticed is that it fails and raises a connection error that dispatcher not found at localhost:48008. But if one is to re-dispatch the workflow again everything works properly and a user can expect their workflow to have been submitted to their self-hosted covalent server.

This behaviour is non-intutive and warrants a thorough dissection of the way covalent is performing configuration updates.

Few ideas for enhancement:

• Covalent's config CLI command can be augmented by providing set and get methods for various configuration options i.e.

covalent config set dispatcher.address <addr>
covalent config set dispatcher.port <port>
covalent config set executor.batch.s3_bucket_name <bucket _name>
covalent config get executor.batch

[venkatBala]

@cjao @santoshkumarradha @kessler-frost @wjcunningham7 @AlejandroEsquivel @FyzHsn