Downloading a binary file and saving it

[ayuhito]

I've been trying to write a downloaded binary file to a directory but either keep encountering errors or the file is incorrectly written differing from the source. This is within a Node environment and doesn't need to work in browsers. I've searched for hours regarding this topic but still without much luck.

How would we modify the below example to successfully download the WOFF file and write it within the directory?

const url = "https://fonts.gstatic.com/s/zillaslabhighlight/v10/gNMUW2BrTpK8-inLtBJgMMfbm6uNVDvRxiP0THTxaGR10A.woff2"
const dest = "./directory/file.woff2"

const options = {
    encoding: "binary",
    method: "GET",
  }
  
try {
    const response = await got(url, options).buffer()
    fs.writeFileSync(dest, response)
} catch (error) {
    console.log(error)
}

Thanks in advance!

[sindresorhus]

What errors are you getting?

Don't forget that you also have to create the directory before writing if it doesn't already exist. Also, drop the encoding option.

[sindresorhus]

But the resulting file will be far larger than the source (35kb vs 19kb).

How do you find the original file size? It sounds like the 19kb is the gzipped size and 35kb is the actual uncompressed size.

I've messed around with using different encoding types, like just using .text() and seeing how that goes too. But while it successfully writes, it's clearly different from the source.

You have to know whether the source is binary or text and use the appropriate method on Got. In this case, it's binary, so you need to use .buffer() without any encoding option as encoding is for text.

The "data" argument must be of type string or an instance of Buffer, TypedArray, or DataView. Received an instance of IncomingMessage

The error message is quite clear. You're passing the response instead of the response body.

[ayuhito]

Thanks for looking into this.

How do you find the original file size? It sounds like the 19kb is the gzipped size and 35kb is the actual uncompressed size.

The above example URL I provided downloads a 19kb file through a browser or the ShellJS script. Your hypothesis seems to make sense and it could be something that Google does intentionally, but when I passed decompress: false into options, the file size didn't actually change (35kb).

I do need it in its 19kb state even if it's compressed.

[szmarczak]

But the resulting file will be far larger than the source (35kb vs 19kb).

I have just checked and it is 19kb, not 35kb. You must be doing something wrong.

[ayuhito]

I appreciate the time you guys have spent on this, and also I have to apologize because this was entirely me overlooking another part of my project.

It seems the root cause of this was due to a file write from a previous function with the same filename. I could get away with it due to the ShellJS function being so slow and synchronous, but with got the second function which was supposed to overwrite the first file finished first due to the smaller filesize (19kb). It was then subsequently overwritten with the larger (35kb) file.

Entirely my bad on this one - I did manage to fix the underlying bug on my end though!

[szmarczak]

No worries. This is what discussions is for ;) I'm glad you solved the problem.