Byte Array Deserialization

[roofdiver]

Hi y'all I'm using the de-serialization method where you build out structs for the fields you want to extract. This works great, but I am running into an issue when deserializing byte arrays in avro.

I have a struct like this

#[derive(Debug,Deserialize)]
struct Services{
    port: i64,
    transport: Option<String>,
    service_name: Option<String>,
    banner: Option<Vec<u8>>,
    tls: TLS

}

and I'm de-serializing this way
`
...
let reader = Reader::new(file)?;

for value in reader {
    let value = value?;
    let record: Services = from_value::<Services>(&value)?;
    println!("{:#?}",record)
}

`

I've confirmed my avro schema from the file as well
{ 'default': None, 'name': 'banner', 'type': ['null', 'bytes'] }

No matter what I do I get this error.
Error: Error { details: Failed to deserialize Avro value into value: Expected an Array or Union, but got: Bytes([0, 91, 0, 0, 0, 0, 0, 0]) }

This leads me to believe that I'm supposed to use an avro built in type (Bytes?) to deserialize the byte array since it's not liking Vec. The issue there is that I'm seeing Bytes as an Enum of the apache_avro Value which doesn't implement Deserialize.

Would really appreciate anything pointing me in the right direction.

Thanks,

K

[martin-g]

Could you please send a PR with a failing unit test?
It would help us to debug and fix it faster.

[roofdiver]

Hi, this is for a super specific type of data, if I did a PR would I need to provide an example avro file that contains a byte array?

[roofdiver]

I do see something similar you worked on here:

https://lists.apache.org/thread/547f66q0l42yw5r5rxtb84b65osqg2bw.

I think it's the same issue that was occuring with deserializing Decimal, except in this case it's Bytes

[roofdiver]

Hi! @martin-g I've created this PR #293

This is my first open source contribution (if this could even be considered that), so please let me know if I did anything incorrectly.

[roofdiver]

Ok so I went looking in the source code for a solution and I found that you can use

#[serde(with = "apache_avro::serde_avro_bytes_opt")] in your struct and then de-serialization of byte arrays works.

Then you can do something like this

#[derive(Debug, Deserialize, Serialize, Encode, Decode, Clone)]
pub struct Example{
    #[serde(with = "apache_avro::serde_avro_bytes_opt")]
    pub data: Option<Vec<u8>>,
}

I would like to put it out that there that I searched pretty heavily and couldn't find this until I dug into the source code. I think it may be worth noting in the examples on the doc page.

[martin-g]

Great!
It's time for your second open source contribution! 🙃

[roofdiver]

Fair enough! What's the best preferred place to edit the documentation for this project? I can go ahead and add something in 😃

[roofdiver]

Ok! I updated my pull request to modify the readme file and put a section at the bottom for byte array deserialization. I also updated my unit tests with the solution so they don't fail and added some comments. Let me know if that works!