Image as Buffer

[joe-bell]

Summary

Provide a mechanism for outputting the results of Image/getImage as a Buffer (or Uint8Array)

Background & Motivation

In my Astro project, I have a wrapper around the official Image component that displays a Low Quality Image Placeholder (LQIP) whilst the image is loading – you can see this in action at joebell.co.uk

Prior to Astro 3.0, I stored my images in ./public and used plaiceholder to create a Base64 LQIP

In migration to Astro 3.0, I would like to use the new astro:assets strategy, but this involves some "hacking" around; by manually interpreting the ImageMetadata src path to extract the file, and then using sharp to manually create a Base64 image.

If Astro allowed me to access the image Buffer, I could skip the whole path hacking approach altogether

Goals

To provide a simple mechanism for consumers who wish to inline images as Buffer; for example, to aid the generation of LQIPs or to support easier integration of images with API routes

Example

Current Behaviour

---
import someImage from '../assets/images/test.jpg';
---

const { src: base64 } = await getImage({ src: publicSrc, width: 10 });

// => "/_image?href=<...>%2Fsrc%2Fassets%2Fimages%2Ftest.jpg%3ForigWidth%3D4032%26origHeight%3D3022%26origFormat%3Djpg&w=10&f=webp"

Proposed Behaviour

When specifying output: "buffer", a Buffer string (or Uint8Array) would be returned:

---
import someImage from '../assets/images/test.jpg';
---

const { src: base64 } = await getImage({ src: publicSrc, width: 10, output: "buffer" });

// => "<Buffer …>"

[Princesseuh]

That's unfortunately not really possible... The image generation is done separately from this call. This is especially relevant in SSR, where the image is only generated on request

Something we would be able to do is be able to get a buffer from the ESM import though, that way you could use the original image to generate your placeholder at least

[joe-bell]

Something we would be able to do is be able to get a buffer from the ESM import though, that way you could use the original image to generate your placeholder at least

This would be amazing – happy to rewrite my proposal if that's the direction Astro want to take? 🙏

[Princesseuh]

Sure, I will definitely be +1-ing it! I've wanted it from day one of working on astro:assets 😄

[joe-bell]

Big same! This would be particularly powerful when consuming images in API routes (e.g. for open-graph generation)

[joe-bell]

I've updated the proposal ✌️

[joe-bell]

Workaround

To follow up, here's the Node.js workaround I'm using in the meantime

// src/lib/image.ts
import type { LocalImageProps } from "astro:assets";
import fs from "node:fs/promises";

export const getImageAsBuffer = async (src: LocalImageProps["src"]) => {
  const ASSETS_DIR = "/src/assets";

  const imageMetadata = await src;

  return await fs.readFile(
    new URL(
      import.meta.env.PROD
        ? [
            "../../dist",
            // @ts-expect-error – 'Property 'src' does not exist on type'
            imageMetadata?.src,
          ].join("")
        : [
            "../..",
            ASSETS_DIR,
            // @ts-expect-error – 'Property 'src' does not exist on type'
            imageMetadata?.src.split("?orig")[0].split(ASSETS_DIR)[1],
          ].join(""),
      import.meta.url,
    ),
  );
};

---
// src/components/example.astro
import { getImageAsBuffer } from "@/lib/image";
import image from "@/assets/images/example.jpg";

const buffer = await getImageAsBuffer(image);
---

[halas]

thank you for this, I was hitting my head on the wall trying to solve a similar use case

[joe-bell]

This workaround is no longer suitable, but here's what I'm doing currently

It drastically increases build time, but it's the only workaround I have

import type { ComponentProps } from "astro/types";
import { Image } from "astro:assets";

const IMAGES_DIR = "/assets/images/";

const imagesAsMetadata = import.meta.glob("../assets/images/*", {
  import: "default",
  eager: true,
});

const imagesAsBuffer = import.meta.glob<Uint8Array>("../assets/images/*", {
  import: "default",
  query: "?uint8array",
});

// Create a map where each key is the `src` (Astro reference) and the value
// is the image buffer
const images = Object.keys(imagesAsMetadata).reduce(
  (acc, path) => ({
    ...acc,
    // @ts-expect-error – 'Property 'src' does not exist on type'
    [imagesAsMetadata[path]?.src]: imagesAsBuffer[path],
  }),
  {} as Record<string, () => Promise<Buffer>>,
);

export async function getImageAsBuffer(
  src: ComponentProps<typeof Image>["src"],
): Promise<Buffer> {
  const metadata = await src;

  const path =
    typeof metadata === "string"
      ? metadata
      : // @ts-expect-error – 'Property 'src' does not exist on type'
        metadata?.src;

  const image = images[path];

  if (typeof image == "undefined") {
    const pathWithinImages =
      path.split(IMAGES_DIR)[path.split(IMAGES_DIR).length - 1];

    // Although this does work with nested directories (i.e. `/assets/images/**/*`),
    // the build performance is significantly slower (almost 100% slower)
    if (pathWithinImages.length > 1) {
      throw new Error(
        `\`/src${IMAGES_DIR}\` must be a flat directory: ${path}`,
      );
    }

    throw new Error(`Unable to find image: ${path}`);
  }

  const buffer = await image();

  return Buffer.from(buffer);
}

[VityaSchel]

This workaround is no longer suitable, but here's what I'm doing currently

It drastically increases build time, but it's the only workaround I have

import type { ComponentProps } from "astro/types";
import { Image } from "astro:assets";

const IMAGES_DIR = "/assets/images/";

const imagesAsMetadata = import.meta.glob("../assets/images/*", {
  import: "default",
  eager: true,
});

const imagesAsBuffer = import.meta.glob<Uint8Array>("../assets/images/*", {
  import: "default",
  query: "?uint8array",
});

// Create a map where each key is the `src` (Astro reference) and the value
// is the image buffer
const images = Object.keys(imagesAsMetadata).reduce(
  (acc, path) => ({
    ...acc,
    // @ts-expect-error – 'Property 'src' does not exist on type'
    [imagesAsMetadata[path]?.src]: imagesAsBuffer[path],
  }),
  {} as Record<string, () => Promise<Buffer>>,
);

export async function getImageAsBuffer(
  src: ComponentProps<typeof Image>["src"],
): Promise<Buffer> {
  const metadata = await src;

  const path =
    typeof metadata === "string"
      ? metadata
      : // @ts-expect-error – 'Property 'src' does not exist on type'
        metadata?.src;

  const image = images[path];

  if (typeof image == "undefined") {
    const pathWithinImages =
      path.split(IMAGES_DIR)[path.split(IMAGES_DIR).length - 1];

    // Although this does work with nested directories (i.e. `/assets/images/**/*`),
    // the build performance is significantly slower (almost 100% slower)
    if (pathWithinImages.length > 1) {
      throw new Error(
        `\`/src${IMAGES_DIR}\` must be a flat directory: ${path}`,
      );
    }

    throw new Error(`Unable to find image: ${path}`);
  }

  const buffer = await image();

  return Buffer.from(buffer);
}

Any way to make this work when images are nested in content's directories? I.e. src/content/blog/foobar/baz.png (with dynamic/glob import)

[joe-bell]

Unfortunately wasn't able to, it significantly slows down the build process

[Princesseuh]

I'll also note that at the moment, it's possible to simply do this if you control the images:

---
import imageBuffer from "../image.png?raw";
---

See https://vitejs.dev/guide/assets.html#importing-asset-as-string

[joe-bell]

Update on this following withastro/astro#10380 (comment), I tried using ?raw but it seems that Vite spits out unusable data here

I was able to workaround the issue using vite-plugin-arraybuffer for static imports:
vitejs/vite#12366 (comment)

Still haven't quite figured out how to get images dynamically, but my current thinking is to glob import all the images and manually iterate through to find the relevant image (but this sounds a little intensive)

[joe-bell]

Here's the dynamic solution I landed on

#679 (reply in thread)

[amxmln]

I have recently ran into this as well and eventually fell back to a solution involving loading the images via node:fs – it works, but it has drawbacks, because basically I have to manually transform them to match the transformations done via getImage() and the image service in order to get accurate placeholder images.

In a conversation on Mastodon with @delucis, the idea was formed to perhaps introduce a new method getRawImage() that would immediately process in the assets pipeline and return the raw image data for the options object passed.

This would have some advantages over passing the raw data as part of the import, as the transformations would be the same (even when using a custom image service) and the API would be more clean, while at the same time allowing for a lot of flexibility in handling the resulting data, for example to generate a low quality placeholder image, extracting dominant colours on a smaller version of the image, or even generating a BlurHash or ThumbHash from the scaled-down version.

[Princesseuh]

Unfortunately doing it through a function like that wouldn't quite work for SSR, you'd end up blocking the page while you generate the placeholders, and due to serverless, you wouldn't be able to cache them

[amxmln]

I’m not quite sure I follow – how is generating them while handling the request more blocking than handling the request for the full-sized image at the _image endpoint? 🤔

And regarding caching, wouldn’t the placeholders just be cached with the response? They’re meant to be embedded directly in the HTML after all. 🤔

I apologise if I’m misunderstanding something here, my expertise is in static sites. 😅

[Princesseuh]

The image endpoint is a separate, non-blocking, request. While rendering the page, the only thing the Image component (or getImage, etc) does is generate a link. For the placeholder you'd want to block the rendering of the page so that the placeholder can be inlined in the initial request.

And yes, it'd be cached on the client once It's loaded, assuming the page is cacheable. First load are quite important too, and there's also a fair amount of server load that comes with it. In an ideal scenario, you'd be able to cache the placeholders locally so that your server does not process the images again for every uncached requests

SSR in general makes this clumsy and annoying, in SSG you don't care about blocking (apart from it making building the page slower, of course) so you'd be able to do a perfect implementation with support for local and remote images, multiple algorithm per image etc, alas, SSR.

[amxmln]

Ah, got it! Thank you for explaining. 😊