Skip to content

Commit 8968adb

Browse files
committed
Make self-checks false by default in order to investigate performance impact.
1 parent 55eba12 commit 8968adb

File tree

2 files changed

+7
-7
lines changed

2 files changed

+7
-7
lines changed

README.md

+6-6
Original file line numberDiff line numberDiff line change
@@ -1619,12 +1619,12 @@ The output looks like this:
16191619

16201620
##### Debug helper options
16211621

1622-
| Option (usage example) | Description |
1623-
|----------------------------------------------------|:-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
1624-
| .option("pedantic", "false") | If 'true' Cobrix will throw an exception is an unknown option is encountered. If 'false' (default), unknown options will be logged as an error without failing Spark Application. |
1625-
| .option("debug_layout_positions", "true") | If 'true' Cobrix will generate and log layout positions table when reading data. |
1626-
| .option("debug_ignore_file_size", "true") | If 'true' no exception will be thrown if record size does not match file size. Useful for debugging copybooks to make them match a data file. |
1627-
| .option("enable_self_checks", "true") | If 'true' (default) Cobrix will run self-checks to validate internal consistency. Note: Enabling this option may impact performance, especially for large datasets. It is recommended to disable this option in performance-critical environments. The only check implemented so far is custom record extractor indexing compatibility check. |
1622+
| Option (usage example) | Description |
1623+
|----------------------------------------------------|:------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
1624+
| .option("pedantic", "false") | If 'true' Cobrix will throw an exception is an unknown option is encountered. If 'false' (default), unknown options will be logged as an error without failing Spark Application. |
1625+
| .option("debug_layout_positions", "true") | If 'true' Cobrix will generate and log layout positions table when reading data. |
1626+
| .option("debug_ignore_file_size", "true") | If 'true' no exception will be thrown if record size does not match file size. Useful for debugging copybooks to make them match a data file. |
1627+
| .option("enable_self_checks", "true") | If 'true' Cobrix will run self-checks to validate internal consistency. Note: Enabling this option may impact performance, especially for large datasets. It is recommended to disable this option in performance-critical environments. The only check implemented so far is custom record extractor indexing compatibility check. |
16281628

16291629
##### Currently supported EBCDIC code pages
16301630

spark-cobol/src/main/scala/za/co/absa/cobrix/spark/cobol/parameters/CobolParametersParser.scala

+1-1
Original file line numberDiff line numberDiff line change
@@ -285,7 +285,7 @@ object CobolParametersParser extends Logging {
285285
getDebuggingFieldsPolicy(recordFormat, params),
286286
params.getOrElse(PARAM_DEBUG_IGNORE_FILE_SIZE, "false").toBoolean,
287287
params.getOrElse(PARAM_DEBUG_LAYOUT_POSITIONS, "false").toBoolean,
288-
params.getOrElse(PARAM_ENABLE_SELF_CHECKS, "true").toBoolean,
288+
params.getOrElse(PARAM_ENABLE_SELF_CHECKS, "false").toBoolean,
289289
MetadataPolicy(params.getOrElse(PARAM_METADATA, "basic"))
290290
)
291291
validateSparkCobolOptions(params, recordFormat)

0 commit comments

Comments
 (0)