dex.ksy: warning about non-canonical encoding reported on a wrong field

[generalmimon]

KSC 0.11-SNAPSHOT at 91aeb53e compiles executable/dex.ksy with the following warnings:

$ kaitai-struct-compiler -- -I . -t python --verbose file executable/dex.ksy
parsing executable/dex.ksy...
reading executable/dex.ksy...
reading ./common/vlq_base128_le.ksy...
... compiling it for python...
.... writing dex.py
.... writing vlq_base128_le.py
executable/dex.ksy: /types/map_list/seq/0/id:
        warning: use `num_list` instead of `size`, (...)

executable/dex.ksy: /types/type_list/seq/0/id:
        warning: use `num_list` instead of `size`, (...)

executable/dex.ksy: /types/class_def_item/instances/type_name/encoding:
        warning: use canonical encoding name `ASCII` instead of `ascii` (see https://doc.kaitai.io/ksy_style_guide.html#encoding-name)

executable/dex.ksy: /types/header_item/seq/1/encoding:
        warning: use canonical encoding name `ASCII` instead of `ascii` (see https://doc.kaitai.io/ksy_style_guide.html#encoding-name)

The dex.ksy spec has 2 occurrences of encoding:

1. /types/header_item/seq/1/encoding - see executable/dex.ksy:143-151

   types:
     header_item:
       seq:
         - id: magic
           contents: "dex\n"
         - id: version_str
           size: 4
           type: strz
           encoding: ascii

2. /types/string_id_item/types/string_data_item/seq/1/encoding - see executable/dex.ksy:275-284

   types:
     # ...
     string_id_item:
       # ...
       types:
         string_data_item:
           -webide-representation: "{data}"
           seq:
             - id: utf16_size
               type: vlq_base128_le
             - id: data
               size: utf16_size.value
               type: str
               encoding: ascii

As you can see, the warning was correctly reported at /types/header_item/seq/1/encoding, i.e. on the 1st occurrence of encoding in the spec. However, it was not reported on the 2nd occurrence of encoding, i.e. at /types/string_id_item/types/string_data_item/seq/1/encoding.

Instead, it was reported at /types/class_def_item/instances/type_name/encoding:

executable/dex.ksy: /types/class_def_item/instances/type_name/encoding:
        warning: use canonical encoding name `ASCII` instead of `ascii` (see https://doc.kaitai.io/ksy_style_guide.html#encoding-name)

But /types/class_def_item/instances/type_name is a value instance that doesn't have the encoding key:

executable/dex.ksy:471-474

    instances:
      type_name:
        value: _root.type_ids[class_idx].type_name
        -webide-parse-mode: eager

[generalmimon]

The discussion in kaitai-io/kaitai_struct_compiler#278 (comment) is related and hints at the root cause of the problem. Apparently, we're trying to detect non-canonical encoding too late in the compiler's "processing pipeline": on the attribute data types, not in the YAML representation as we should. I believe that /types/class_def_item/instances/type_name probably has the non-canonical encoding: ascii encoded in its data type (it inherited the same data type as /types/string_id_item/types/string_data_item/seq/1, since it's ultimately just an identity function of it) and that's why the compiler reports the warning on it.

[generalmimon]

Another interesting behavior occurs on filesystem/apm_partition_table.ksy:

pp@DESKTOP-89OPGF3 MINGW64 /c/temp/kaitai_struct/formats (master)
$ kaitai-struct-compiler -- -I . -t python --verbose file filesystem/apm_partition_table.ksy
parsing filesystem/apm_partition_table.ksy...
reading filesystem/apm_partition_table.ksy...
... compiling it for python...
.... writing apm_partition_table.py
filesystem/apm_partition_table.ksy: /instances/sector_size/id:
        warning: use `len_partition_entries` instead of `sector_size`, (...)

filesystem/apm_partition_table.ksy: /instances/sector_size/id:
        warning: use `len_partition_lookup` instead of `sector_size`, (...)

filesystem/apm_partition_table.ksy: /types/partition_entry/seq/11/id:
        warning: use `len_boot_code` instead of `boot_code_size`, (...)

filesystem/apm_partition_table.ksy: /meta/encoding:
        warning: use canonical encoding name `ASCII` instead of `ascii` (see https://doc.kaitai.io/ksy_style_guide.html#encoding-name)

filesystem/apm_partition_table.ksy: /types/partition_entry/meta/encoding:
        warning: use canonical encoding name `ASCII` instead of `ascii` (see https://doc.kaitai.io/ksy_style_guide.html#encoding-name)

The problem is that /types/partition_entry doesn't have any meta/encoding, so that's a fabricated path - see filesystem/apm_partition_table.ksy:37-43:

types:
  partition_entry:
    seq:
      - id: magic
        contents: [ 0x50, 0x4d ]
      - id: reserved_1
        size: 0x2

Since the only occurrence of encoding in filesystem/apm_partition_table.ksy is /meta/encoding, this also means that kaitai-io/kaitai_struct_compiler#278, which was supposed to suppress warnings for inherited encoding like this, doesn't actually work reliably.