Not all tests are run in CI

Update:
#1135 solves this and fixes the failed test cases. Except for org.opensearch.flint.core.storage.OpenSearchClientUtilsSuite, which passes when run on its own, but fails when run by sbt test.
Temporarily @Ignore the test case. Needs further RCA.

---
Running test locally yields 7 failures in flint-core tests.
```
% sbt test

...

[error] Failed: Total 106, Failed 7, Errors 0, Passed 99
[error] Failed tests:
[error] 	org.opensearch.flint.core.storage.OpenSearchBulkWrapperTest
[error] 	org.opensearch.flint.core.http.RetryableHttpAsyncClientSuite
[error] 	org.opensearch.flint.core.storage.OpenSearchClientUtilsSuite
[error] (flintCore / Test / test) sbt.TestsFailedException: Tests unsuccessful
[error] Total time: 688 s (11:28), completed Apr 2, 2025, 10:58:05 AM
```

Upon checking recent CI workflow, test reports show that only `integ-test`, `flint-spark-integration`, `ppl-spark-integration`, and `sql-job` tests are run.
`flint-commons` and `flint-core` are both left out.
We removed running unit tests explicitly and merge them into the integtest run: https://github.com/opensearch-project/opensearch-spark/pull/501
But now some module's UT are missing, either introduced by this change or later changes in build.sbt

We need to
1. Fix workflow or build.sbt so all UT are run
2. Fix failing UT

Below is a test report directory content uploaded by our CI
```
.
├── flint-spark-integration
│  ├── css
│  ├── images
│  ├── js
│  ├── index.html
│  ├── org.apache.spark.opensearch.catalog.OpenSearchCatalogTest.html
│  ├── org.apache.spark.sql.flint.config.FlintSparkConfSuite.html
│  ├── org.apache.spark.sql.flint.datatype.FlintDataTypeSuite.html
│  ├── org.apache.spark.sql.flint.datatype.FlintMetadataExtensionsSuite.html
│  ├── org.apache.spark.sql.flint.storage.FlintQueryCompilerSuite.html
│  ├── org.apache.spark.sql.FlintCatalogSuite.html
│  ├── org.opensearch.flint.spark.covering.ApplyFlintSparkCoveringIndexSuite.html
│  ├── org.opensearch.flint.spark.covering.FlintSparkCoveringIndexSuite.html
│  ├── org.opensearch.flint.spark.FlintSparkCheckpointSuite.html
│  ├── org.opensearch.flint.spark.FlintSparkIndexBuilderSuite.html
│  ├── org.opensearch.flint.spark.FlintSparkIndexFactorySuite.html
│  ├── org.opensearch.flint.spark.FlintSparkIndexOptionsSuite.html
│  ├── org.opensearch.flint.spark.FlintSparkIndexRefreshSuite.html
│  ├── org.opensearch.flint.spark.FlintSparkIndexSuite.html
│  ├── org.opensearch.flint.spark.FlintSparkTransactionSupportSuite.html
│  ├── org.opensearch.flint.spark.function.TumbleFunctionSuite.html
│  ├── org.opensearch.flint.spark.metadatacache.FlintMetadataCacheSuite.html
│  ├── org.opensearch.flint.spark.mv.FlintSparkMaterializedViewSuite.html
│  ├── org.opensearch.flint.spark.scheduler.util.RefreshQueryGeneratorTest.html
│  ├── org.opensearch.flint.spark.skipping.ApplyFlintSparkSkippingIndexSuite.html
│  ├── org.opensearch.flint.spark.skipping.bloomfilter.BloomFilterMightContainSuite.html
│  ├── org.opensearch.flint.spark.skipping.bloomfilter.BloomFilterSkippingStrategySuite.html
│  ├── org.opensearch.flint.spark.skipping.FlintSparkSkippingFileIndexSuite.html
│  ├── org.opensearch.flint.spark.skipping.FlintSparkSkippingIndexSuite.html
│  ├── org.opensearch.flint.spark.skipping.minmax.MinMaxSkippingStrategySuite.html
│  ├── org.opensearch.flint.spark.skipping.partition.PartitionSkippingStrategySuite.html
│  ├── org.opensearch.flint.spark.skipping.valueset.ValueSetSkippingStrategySuite.html
│  └── org.opensearch.flint.spark.sql.FlintSparkSqlParserSuite.html
├── integ-test
│  ├── css
│  ├── images
│  ├── js
│  ├── index.html
│  ├── org.apache.spark.FlintDataSourceV2ITSuite.html
│  ├── org.apache.spark.opensearch.catalog.OpenSearchCatalogITSuite.html
│  ├── org.apache.spark.opensearch.table.OpenSearchDashboardITSuite.html
│  ├── org.apache.spark.opensearch.table.OpenSearchTableITSuite.html
│  ├── org.apache.spark.opensearch.table.OpenSearchTableQueryITSuite.html
│  ├── org.apache.spark.sql.FlintJobITSuite.html
│  ├── org.apache.spark.sql.FlintREPLITSuite.html
│  ├── org.opensearch.flint.core.FlintMetadataLogITSuite.html
│  ├── org.opensearch.flint.core.FlintOpenSearchClientSuite.html
│  ├── org.opensearch.flint.core.FlintOpenSearchIndexMetadataServiceITSuite.html
│  ├── org.opensearch.flint.core.FlintTransactionITSuite.html
│  ├── org.opensearch.flint.core.OpenSearchUpdaterSuite.html
│  ├── org.opensearch.flint.core.scheduler.OpenSearchAsyncQuerySchedulerITSuite.html
│  ├── org.opensearch.flint.spark.FlintSparkCoveringIndexITSuite.html
│  ├── org.opensearch.flint.spark.FlintSparkCoveringIndexSqlITSuite.html
│  ├── org.opensearch.flint.spark.FlintSparkIndexDescribeITSuite.html
│  ├── org.opensearch.flint.spark.FlintSparkIndexJobITSuite.html
│  ├── org.opensearch.flint.spark.FlintSparkIndexJobSqlITSuite.html
│  ├── org.opensearch.flint.spark.FlintSparkIndexMonitorITSuite.html
│  ├── org.opensearch.flint.spark.FlintSparkIndexNameITSuite.html
│  ├── org.opensearch.flint.spark.FlintSparkIndexSqlITSuite.html
│  ├── org.opensearch.flint.spark.FlintSparkIndexValidationITSuite.html
│  ├── org.opensearch.flint.spark.FlintSparkMaterializedViewIntegrationsITSuite.html
│  ├── org.opensearch.flint.spark.FlintSparkMaterializedViewITSuite.html
│  ├── org.opensearch.flint.spark.FlintSparkMaterializedViewSqlITSuite.html
│  ├── org.opensearch.flint.spark.FlintSparkSkippingIndexITSuite.html
│  ├── org.opensearch.flint.spark.FlintSparkSkippingIndexSqlITSuite.html
│  ├── org.opensearch.flint.spark.FlintSparkTransactionITSuite.html
│  ├── org.opensearch.flint.spark.FlintSparkUpdateIndexITSuite.html
│  ├── org.opensearch.flint.spark.FlintSparkWindowingFunctionITSuite.html
│  ├── org.opensearch.flint.spark.iceberg.FlintSparkIcebergMaterializedViewITSuite.html
│  ├── org.opensearch.flint.spark.iceberg.FlintSparkIcebergSkippingIndexITSuite.html
│  ├── org.opensearch.flint.spark.metadatacache.FlintOpenSearchMetadataCacheWriterITSuite.html
│  ├── org.opensearch.flint.spark.ppl.FlintSparkPPLAggregationsITSuite.html
│  ├── org.opensearch.flint.spark.ppl.FlintSparkPPLAggregationWithSpanITSuite.html
│  ├── org.opensearch.flint.spark.ppl.FlintSparkPPLAppendColITSuite.html
│  ├── org.opensearch.flint.spark.ppl.FlintSparkPPLBasicITSuite.html
│  ├── org.opensearch.flint.spark.ppl.FlintSparkPPLBetweenITSuite.html
│  ├── org.opensearch.flint.spark.ppl.FlintSparkPPLBuiltInDateTimeFunctionITSuite.html
│  ├── org.opensearch.flint.spark.ppl.FlintSparkPPLBuiltinFunctionITSuite.html
│  ├── org.opensearch.flint.spark.ppl.FlintSparkPPLCastITSuite.html
│  ├── org.opensearch.flint.spark.ppl.FlintSparkPPLCidrmatchITSuite.html
│  ├── org.opensearch.flint.spark.ppl.FlintSparkPPLCommentITSuite.html
│  ├── org.opensearch.flint.spark.ppl.FlintSparkPPLCorrelationITSuite.html
│  ├── org.opensearch.flint.spark.ppl.FlintSparkPPLDedupeITSuite.html
│  ├── org.opensearch.flint.spark.ppl.FlintSparkPPLEvalITSuite.html
│  ├── org.opensearch.flint.spark.ppl.FlintSparkPPLEventstatsITSuite.html
│  ├── org.opensearch.flint.spark.ppl.FlintSparkPPLExistsSubqueryITSuite.html
│  ├── org.opensearch.flint.spark.ppl.FlintSparkPPLExpandITSuite.html
│  ├── org.opensearch.flint.spark.ppl.FlintSparkPPLFieldSummaryITSuite.html
│  ├── org.opensearch.flint.spark.ppl.FlintSparkPPLFillnullITSuite.html
│  ├── org.opensearch.flint.spark.ppl.FlintSparkPPLFiltersITSuite.html
│  ├── org.opensearch.flint.spark.ppl.FlintSparkPPLFlattenITSuite.html
│  ├── org.opensearch.flint.spark.ppl.FlintSparkPPLGeoipITSuite.html
│  ├── org.opensearch.flint.spark.ppl.FlintSparkPPLGrokITSuite.html
│  ├── org.opensearch.flint.spark.ppl.FlintSparkPPLInSubqueryITSuite.html
│  ├── org.opensearch.flint.spark.ppl.FlintSparkPPLJoinITSuite.html
│  ├── org.opensearch.flint.spark.ppl.FlintSparkPPLJsonFunctionITSuite.html
│  ├── org.opensearch.flint.spark.ppl.FlintSparkPPLLambdaFunctionITSuite.html
│  ├── org.opensearch.flint.spark.ppl.FlintSparkPPLLookupITSuite.html
│  ├── org.opensearch.flint.spark.ppl.FlintSparkPPLNestedFieldsITSuite.html
│  ├── org.opensearch.flint.spark.ppl.FlintSparkPPLParseITSuite.html
│  ├── org.opensearch.flint.spark.ppl.FlintSparkPPLPatternsITSuite.html
│  ├── org.opensearch.flint.spark.ppl.FlintSparkPPLRenameITSuite.html
│  ├── org.opensearch.flint.spark.ppl.FlintSparkPPLScalarSubqueryITSuite.html
│  ├── org.opensearch.flint.spark.ppl.FlintSparkPPLTimeWindowITSuite.html
│  ├── org.opensearch.flint.spark.ppl.FlintSparkPPLTopAndRareITSuite.html
│  ├── org.opensearch.flint.spark.ppl.FlintSparkPPLTrendlineITSuite.html
│  ├── org.opensearch.flint.spark.ppl.tpch.TPCHQueryITSuite.html
│  ├── org.opensearch.flint.spark.sessioncatalog.FlintSparkSessionCatalogMaterializedViewITSuite.html
│  └── org.opensearch.flint.spark.sessioncatalog.FlintSparkSessionCatalogSkippingIndexITSuite.html
├── ppl-spark-integration
│  ├── css
│  ├── images
│  ├── js
│  ├── index.html
│  ├── org.opensearch.flint.spark.ppl.PPLLogicalPlanAggregationQueriesTranslatorTestSuite.html
│  ├── org.opensearch.flint.spark.ppl.PPLLogicalPlanAppendColCommandTranslatorTestSuite.html
│  ├── org.opensearch.flint.spark.ppl.PPLLogicalPlanBasicQueriesTranslatorTestSuite.html
│  ├── org.opensearch.flint.spark.ppl.PPLLogicalPlanBetweenExpressionTranslatorTestSuite.html
│  ├── org.opensearch.flint.spark.ppl.PPLLogicalPlanCastTestSuite.html
│  ├── org.opensearch.flint.spark.ppl.PPLLogicalPlanCorrelationQueriesTranslatorTestSuite.html
│  ├── org.opensearch.flint.spark.ppl.PPLLogicalPlanCryptographicFunctionsTranslatorTestSuite.html
│  ├── org.opensearch.flint.spark.ppl.PPLLogicalPlanDateTimeFunctionsTranslatorTestSuite.html
│  ├── org.opensearch.flint.spark.ppl.PPLLogicalPlanDedupeTranslatorTestSuite.html
│  ├── org.opensearch.flint.spark.ppl.PPLLogicalPlanEvalTranslatorTestSuite.html
│  ├── org.opensearch.flint.spark.ppl.PPLLogicalPlanEventstatsTranslatorTestSuite.html
│  ├── org.opensearch.flint.spark.ppl.PPLLogicalPlanExistsSubqueryTranslatorTestSuite.html
│  ├── org.opensearch.flint.spark.ppl.PPLLogicalPlanExpandCommandTranslatorTestSuite.html
│  ├── org.opensearch.flint.spark.ppl.PPLLogicalPlanFieldSummaryTranslatorTestSuite.html
│  ├── org.opensearch.flint.spark.ppl.PPLLogicalPlanFillnullCommandTranslatorTestSuite.html
│  ├── org.opensearch.flint.spark.ppl.PPLLogicalPlanFiltersTranslatorTestSuite.html
│  ├── org.opensearch.flint.spark.ppl.PPLLogicalPlanFlattenCommandTranslatorTestSuite.html
│  ├── org.opensearch.flint.spark.ppl.PPLLogicalPlanGeoipFunctionTranslatorTestSuite.html
│  ├── org.opensearch.flint.spark.ppl.PPLLogicalPlanGrokTranslatorTestSuite.html
│  ├── org.opensearch.flint.spark.ppl.PPLLogicalPlanInSubqueryTranslatorTestSuite.html
│  ├── org.opensearch.flint.spark.ppl.PPLLogicalPlanJoinTranslatorTestSuite.html
│  ├── org.opensearch.flint.spark.ppl.PPLLogicalPlanJsonFunctionsTranslatorTestSuite.html
│  ├── org.opensearch.flint.spark.ppl.PPLLogicalPlanLambdaFunctionsTranslatorTestSuite.html
│  ├── org.opensearch.flint.spark.ppl.PPLLogicalPlanMathFunctionsTranslatorTestSuite.html
│  ├── org.opensearch.flint.spark.ppl.PPLLogicalPlanNestedQueriesTranslatorTestSuite.html
│  ├── org.opensearch.flint.spark.ppl.PPLLogicalPlanParenthesizedConditionTestSuite.html
│  ├── org.opensearch.flint.spark.ppl.PPLLogicalPlanParseCidrmatchTestSuite.html
│  ├── org.opensearch.flint.spark.ppl.PPLLogicalPlanParseTranslatorTestSuite.html
│  ├── org.opensearch.flint.spark.ppl.PPLLogicalPlanPatternsTranslatorTestSuite.html
│  ├── org.opensearch.flint.spark.ppl.PPLLogicalPlanRenameTranslatorTestSuite.html
│  ├── org.opensearch.flint.spark.ppl.PPLLogicalPlanScalarSubqueryTranslatorTestSuite.html
│  ├── org.opensearch.flint.spark.ppl.PPLLogicalPlanStringFunctionsTranslatorTestSuite.html
│  ├── org.opensearch.flint.spark.ppl.PPLLogicalPlanTimeFunctionsTranslatorTestSuite.html
│  ├── org.opensearch.flint.spark.ppl.PPLLogicalPlanTopAndRareQueriesTranslatorTestSuite.html
│  └── org.opensearch.flint.spark.ppl.PPLLogicalPlanTrendlineCommandTranslatorTestSuite.html
└── sql-job
   ├── css
   ├── images
   ├── js
   ├── index.html
   ├── org.apache.spark.sql.FlintJobTest.html
   ├── org.apache.spark.sql.FlintREPLTest.html
   ├── org.apache.spark.sql.util.CleanerFactoryTest.html
   ├── org.apache.spark.sql.WarmpoolTest.html
   ├── org.opensearch.sql.FlintDelegatingSessionCatalogTest.html
   └── org.opensearch.sql.SQLJobTest.html
```


Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Not all tests are run in CI #1097

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Not all tests are run in CI #1097

Description

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions