Proper Pixel Art

Summary

• Converts noisy, high resolution pixel-art-style images such as those produced by generative models to true pixel resolution assets.

• Clean screenshots or low-quality web uploads of sprites.

Installation

From Source

Clone the Repository

git clone [email protected]:KennethJAllen/proper-pixel-art.git
cd proper-pixel-art

Create Virtual Environment

• Install uv if not already installed.
• Sync environments:
  • uv sync

From Pip

pip install proper-pixel-art

Or with uv:

uv add proper-pixel-art

Usage

First, obtain a source pixel-art-style image (e.g. a pixel-art-style image generated by GPT-4o or a screenshot of pixel-art).

CLI

uv run ppa <input_path> -o <output_path> -c <num_colors> -s <result_scale> [-t]
# or directly using uvx
uvx --from "proper-pixel-art" ppa <input_path> -o <output_path> -c <num_colors> -s <result_scale> [-t]

Flags

Flag	Description
INPUT (positional)	Source file in pixel-art-style
-o, --output <path>	Output directory or file path for result. (default: '.')
-c, --colors <int>	Number of colors for output. May need to try a few different values. (default 16)
-s, --scale-result <int>	Width/height of each "pixel" in the output. (default: 1)
-t, --transparent <bool>	Output with transparent background. (default: off)
-u, --initial-upscale <int>	Initial image upscale factor. Increasing this may help detect pixel edges. (default 2)
-w, --pixel-width <int>	Width of the pixels in the input image. If not set, it will be determined automatically. (default: None)

Example

uv run ppa assets/blob/blob.png -c 16 -s 20 -t
# or directly using uvx
uvx --from "proper-pixel-art" ppa path/to/input.png -o . -c 16 -s 5 -t

Python

from PIL import Image
from proper_pixel_art.pixelate import pixelate

image = Image.open('path/to/input.png')
result = pixelate(image, num_colors=16)
result.save('path/to/output.png')

Parameters

• image : PIL.Image.Image

  • A PIL image to pixelate.

• num_colors : int

  • The number of colors in result.
  • May need to try a few values if the colors don't look right.
  • 8, 16, 32, or 64 typically works.

• initial_upscale : int

  • Upscale result after algorithm is complete if not None.

• scale_result : int

  • Upscale initial image. This may help detect lines.

• transparent_background : bool

  • If True, flood fills each corner of the result with transparent alpha.

• intermediate_dir : Path | None

  • Directory to save images visualizing intermediate steps of algorithm. Useful for development.

• pixel_width : int | None

  • Width of the pixels in the input image. If not set, it will be determined automatically. It may be helpful to increase this parameter if not enough pixel edges are being detected.

Returns

A PIL image with true pixel resolution and quantized colors.

Web Interface

Local:

uv sync --extra web
uv run ppa-web
# Opens http://127.0.0.1:7860

Without cloning:

uvx --from "proper-pixel-art[web]" ppa-web

Examples

The algorithm is robust. It performs well for images that are already approximately alligned to a grid.

Here are a few examples. A mesh is computed, where each cell corresponds to one pixel.

Bat

• Generated by GPT-4o.

Noisy, High Resolution

Mesh

True Pixel Resolution

Ash

• Screenshot from Google images of Pokemon asset.

Noisy, High Resolution

Mesh

True Pixel Resolution

Demon

• Original image generated by GPT-4o.

Noisy, High Resolution

Mesh

True Pixel Resolution

Pumpkin

• Screenshot from Google Images of Stardew Valley asset. This is an adversarial example as the source image is both low quality and the object is round.

Noisy, High Resolution

Mesh

True Pixel Resolution

Real Images To Pixel Art

• This tool can also be used to convert real images to pixel art by first requesting a pixelated version of the original image from GPT-4o, then using the tool to get the true pixel-resolution image.

• Consider this image of a mountain

• Here are the results of first requesting a pixelated version of the mountain, then using the tool to get a true resolution pixel art version.

Noisy, High Resolution

Mesh

True Pixel Resolution

Challenges

The result of pixel-art style images from LLMs are noisy, high resolution images with a non-uniform grid and random artifacts. Due to these issues, standard downsampling techniques do not work. How can we recover the pixel art with "true" resolution and colors?

The current approach to turning pixel art into useable assets for games are either

1. Use naive downsampling which does not give a result that is faithful to the original image.
2. Manually re-create the image in the appropriate resolution pixel by pixel.

Algorithm

• The main algorithm solves these challenges. Here is a high level overview. We will apply it step by step on this example image of blob pixel art that was generated from GPT-4o.

• Note that this image is high resolution and noisy.

1. Trim the edges of the image and zero out pixels with more than 50% alpha.

   • This is to work around some issues with models such as GPT-4o not giving a perfectly transparent background.

2. Upscale by a factor of 2 using nearest neighbor.

   • This can help identify the correct pixel mesh.

3. Find edges of the pixel art using Canny edge detection.

4. Close small gaps in edges with a morphological closing.

5. Take the probabalistic Hough transform to get the coordinates of lines in the detected edges. Only keep lines that are close to vertical or horizontal giving some grid coordinates. Cluster lines that are closeby together.

6. Find the grid spacing by filtering outliers and taking the median of the spacings, then complete the mesh.

7. Quantize the original image to a small number of colors.

   • Note: The result is sensitive to the number of colors chosen.
   • The parameter is not difficult to tune, but the script may need to be re-run if the colors don't look right.
   • 8, 16, 32, or 64 typically works.

8. In each cell specified by the mesh, choose the most common color in the cell as the color for the pixel. Recreate the original image with one pixel per cell.

   • Result upscaled by a factor of $20 \times$ using nearest neighbor.

Testing

To test algorithm changes and verify output quality:

uv run pytest -s

The tests pixelate all assets in assets/{name}/{name}.png and save outputs to tests/outputs/ for manual visual inspection.