DESIGN ERROR: Dependency Injection and settings_customize_sources

[danizen]

I've done a lot of work to migrate to pydantic settings, but as I get started with unit testing the code, I see the potential for all sorts of side effects when I do things like this:

    app_config = CommonsConfig(
        env='dev',
        region='us-east-1',
        job_name='test-job',
        job_run_id='test-job-id',
        vault_addr="https://chamber-qa.clouddqt.capitalone.com",
        vault_role="891892889hsds09090s9d0s9090",
        vault_paths=[CLIENT_ID_PATH, CLIENT_SECRET_PATH]
    )

Essentially, I want to disable the behavior of settings_customize_sources most of the time, and only use those sources when resolving. Is there a way to easily override the loading behavior and get the model to behave more like a pydantic BaseModel?

[danizen]

I've deleted my previous comments as I strive to articulate the problem here. It is likely that a BaseSettings loads from env and CLI and potentially other places to meet the Liskov object substitution principle. There is no new API or convention to call to get a BaseSettings to do its job, and once done, it can substitute for any other pydantic model.

However, in so doing, the method settings_customize_sources violates the Dependency Injection principle of SOLID, in that I cannot alter the sources without subclassing, which was one of my examples to solve this. I think it would be appropriate to consider how to address this in version 3 and ask that this issue be so labeled.

[danizen]

I should also add that this violation is emergent - the only default source other than init settings are environment variables, and that can be effectively disabled using _env_prefix. Parsing Cli parameters has no side effects outside the program and doesn't open resources, which was my rubric for what not to do in C++. This really becomes problematic only when loading from external cloud resources, such as Azure, Google Secrets Manager, Glue Workflow Parameters (what I am doing).

[hramezani]

Thanks @danizen for reporting this issue.

By default pydantic-settings has init, env, dotenv, file secret sources enable. if you include the init source when you customize sources, you always can pass values to the model and initialize the model like a normal Pydantic model.

BTW, we are open to changes and I think we can change it in V3. Feel free to put an example code here if you have any suggestions. it doesn't need to be a perfect working code.

[danizen]

Here is what I am concerned about in terms of the principle of least surprise and dependency injection:

https://gist.github.com/danizen/61de10c4c601fa36c97a65546b785248

[danizen]

As you can see, I found a solution which works for me. I'm fooling around with model_dump and so on to get a true copy going.

[hramezani]

Thanks @danizen for providing the example code.

I assume you meant we need an easy way to determine the required setting sources for the settings model when we initialize the settings object.

right now, we need to define a fixed set of sources in settings_customize_sources and do some hacks if we want to change them e.g. in the test.