BJData optimized binary array type

[nebkat]

See NeuroJSON/bjdata#6 for further information.

Introduces a dedicated B marker for bytes.

This is used as the strong type marker in optimized array format to encode binary data such that it can also be decoded back to binary data (instead of wrongly decoding as an integer array).

Draft while awaiting the release of BJData draft 3.

Would a legacy_binary flag be desirable to continue supporting draft 2 expectations of uint8 typed arrays for binary?

---

Pull request checklist

Read the Contribution Guidelines for detailed information.

• Changes are described in the pull request, or an existing issue is referenced.
• The test suite compiles and runs without error.
• Code coverage is 100%. Test cases can be added by editing the test suite.
• The source code is amalgamated; that is, after making changes to the sources in the include/nlohmann directory, run make amalgamate to create the single-header files single_include/nlohmann/json.hpp and single_include/nlohmann/json_fwd.hpp. The whole process is described here.

[coveralls]

coverage: 99.649%. remained the same
when pulling 7f12cd6 on nebkat:develop
into a006a7a on nlohmann:develop.

[github-actions]

🔴 Amalgamation check failed! 🔴

The source code has not been amalgamated. @nebkat
Please read and follow the Contribution Guidelines.

[nlohmann]

Please run make amalgamate with AStyle 3.1.

[nebkat]

draft3_binary parameter added to to_bjdata, defaulting to false, along with tests for both versions.

from_bjdata previously wasn't able to parse uint8 binary arrays so it has just gained the ability to parse byte arrays as binary with no parameter.

Would it make sense to introduce a to_bjdata_draft3() function and deprecate to_bjdata() to encourage use of the newer version?

[nlohmann]

Thanks a lot! Feels much better without a breaking change!

Regarding the draft version - would it make sense to add an int parameter draft to the function and defaulting it to 2 and allow to set it to 3 instead of the boolean? Then we have some room for the future there.

Regarding deprecation - adding a new function to the API just for the sake of deprecating another feels odd. I think we should rather deprecate the default parameter for the mentioned draft parameter and force client to choose the value in the future.

What do you think?

[nebkat]

I like it, didn't think of that, much better solution!

So perhaps something like this?

/// how to encode BJData
enum class bjdata_version_t
{
    draft2 JSON_HEDLEY_DEPRECATED_FOR(3.12.0, draft3),
    draft3,
};

...

static std::vector<std::uint8_t> to_bjdata(const basic_json& j,
        const bool use_size = false,
        const bool use_type = false,
        const bjdata_version_t version = bjdata_version_t::draft2)
{
    std::vector<std::uint8_t> result;
    to_bjdata(j, result, use_size, use_type, version);
    return result;
}

// warning: 'nlohmann::json_abi_v3_11_3::detail::bjdata_version_t::draft2' is deprecated: Since 3.12.0; use draft3 [-Wdeprecated-declarations]
// 4315 |             const bjdata_version_t version = bjdata_version_t::draft2)

Could technically even put ubjson as a version as they are so close and use it internally.

[nlohmann]

Could technically even put ubjson as a version as they are so close and use it internally.

You mean using BJData as parameter for UBJSON output/input?

[nebkat]

Yes, internally.

json/include/nlohmann/detail/output/binary_writer.hpp:

    // json/include/nlohmann/detail/output/binary_writer.hpp
    void write_ubjson(const BasicJsonType& j, const bool use_count,
                      const bool use_type, const bool add_prefix = true,
-                      const bool use_bjdata = false)
+                      const bjdata_version_t version = bjdata_version_t::ubjson) // or "draft0", which could be argued is equivalent
    {

Then we can do:

include/nlohmann/json.hpp:

    /// @brief create a UBJSON serialization of a given JSON value
    /// @sa https://json.nlohmann.me/api/basic_json/to_ubjson/
    static void to_ubjson(const basic_json& j, detail::output_adapter<std::uint8_t> o,
                          const bool use_size = false, const bool use_type = false)
    {
-        binary_writer<std::uint8_t>(o).write_ubjson(j, use_size, use_type);
+         binary_writer<std::uint8_t>(o).write_bjdata(j, use_size, use_type, bjdata_version_t::ubjson); // Or defaulted as above
    }

    /// @brief create a BJData serialization of a given JSON value
    /// @sa https://json.nlohmann.me/api/basic_json/to_bjdata/
    static void to_bjdata(const basic_json& j, detail::output_adapter<std::uint8_t> o,
-                         const bool use_size = false, const bool use_type = false)
+                         const bool use_size = false, const bool use_type = false, const bjdata_version_t version = bjdata_version_t::draft2)
    {
-        binary_writer<std::uint8_t>(o).write_ubjson(j, use_size, use_type, true, true);
+        binary_writer<std::uint8_t>(o).write_ubjson(j, use_size, use_type, true, version);
    }

Would cut down on the amount of if (use_bjdata && bjdata_version == ...) and instead just be if (version == ...).

[nlohmann]

Yes, what I meant: would this be something that could - in the long run - be exposed to the customer? As in: deprecating to_bjdata in favor of to_ubjson with a version parameter.

[nebkat]

Ah right - yes, but if anything I would deprecate to_ubjson in favour of to_bjdata considering UBJSON has not seen any meaningful updates in ~8 years. If we can play our part in moving people towards BJData that seems like a positive thing in the long run! It is the closest thing we have to a 1-to-1 binary mapping of JSON without the extra fluff.

[nlohmann]

Well, I thought of BJData to be a dialect of UBJSON, so I don't think replacing the UBJSON with those of BJData is a good idea.