Conventions around schema used for test cases.

[silentninja]

There are multiple fixtures used in our test cases that return different types of schema, some of that schema(db.tests.conftest.engine_with_roster) have custom types installed while others don't(db.tests.conftest.engine_with_schema). This has been causing some confusion when writing E2E tests, for example

• Add E2E test for converting boolean type to text type #1182
• Install mathesar_types to full-fill some of the important db functions to work with e2e integ Testing #1168

Question

• When should a test case use a schema that has custom types installed?
• When should it use a schema without custom types installed?

[mathemancer]

We need to organize our fixtures in general. Currently, they're a bit ad-hoc. This would make it quicker and easier to write tests (especially if the fixtures were documented somewhere). I'd rather have the documentation be at the fixture level, rather than test-by-test. (I.e., the fixture should somehow be named so that it's obvious whether custom types are involved or not.)

[kgodey]

I'm not sure what feedback this discussion is asking for. I agree we should have better fixtures and @mathemancer's suggestions make sense to me.

[silentninja]

It would be good to mention when a test case should use a schema that has custom types installed.

The question is to know which test cases should use a schema with custom types installed. Should all test cases use a schema with custom types installed or should the usage of custom types installed schema be limited to very specific test cases?

I rephrased the descriptions to make questions more clear

[mathemancer]

I think the default should be to test using a schema with custom types installed, and an engine with custom type mapping loaded into ischema_names. To me, there are special cases we need to test where custom types aren't desired because (for example) we're testing installation behavior, or maybe testing to see what happens if a function is called without the custom types existing.

[kgodey]

@mathemancer's answer sounds good to me.