Table of Contents

Memory and threading

The context

// Everything that is not specific to one call lives in an immutable EgamiContext. Change it with `with`.
EgamiContext context = EgamiContext.Default with
{
    MaxParallelism = 4,                                         // threads one operation may use; 1 is sequential
    Limits = SafetyLimits.Default with { MaxPixels = 50_000_000 },
};

using Raster image = Raster.Load("duck.png", context);
image.Apply(ops => ops.Blur(2));                                // uses the raster's context: 4 threads at most

A raster remembers the context it was loaded or created with, and its operations use it. EgamiContext.Default serves every call that passes none; EgamiContext.SetDefault replaces it once, at startup, before anything reads it.

Threads

Operations split the image into bands of rows and run them on up to MaxParallelism threads (the processor count by default). Small images run on one thread, where splitting would cost more than it saves. Decoding is mostly sequential by nature; JPEG splits files at their restart markers, pipelines progressive scans and converts in parallel bands, and animated formats map frames in parallel. PNG decompresses a large image on a second thread while the first reverses the filters and converts the rows, and compresses large images on several threads when PngWriterOptions.ParallelCompression is set, with the same bytes at any thread count.

Rasters are not thread-safe: do not run two chains on the same raster at once. Separate rasters, contexts and loads are independent, and readers, writers and contexts can be shared freely.

Streaming

RasterDecoder decodes a band of rows or a frame at a time, so an image never has to exist whole:

// A band of rows at a time: memory is the band plus the format's working set, whatever the image size.
using RasterDecoder decoder = RasterDecoder.Open("egret.png");
Console.WriteLine($"{decoder.Info.Width}×{decoder.Info.Height}, {decoder.Info.Access}, holding {decoder.Info.WorkingBytes:N0} bytes");
using Raster<Gray8> band = Raster.Create<Gray8>(decoder.Info.Width, 64);
int rows;
while ((rows = decoder.ReadRows(band)) > 0)
{
    for (int y = 0; y < rows; y++)
    {
        foreach (Gray8 pixel in band.Row(y))
        {
            sum += pixel.L;
        }
    }
}

ReaderOptions.Region decodes only a rectangle and MinimumSize a reduced size, in Load, Probe and the decoder alike. RasterInfo.Access says whether a format streams its rows (RasterAccess.Streamed) or is decoded into a buffer first (RasterAccess.Buffered), and WorkingBytes how much the decode holds besides your band:

// Only a rectangle, at a quarter of its size or a little more.
ReaderOptions options = new() { Region = new Rect(100, 200, 256, 256), MinimumSize = new Size(64, 64) };
RasterInfo info = Raster.Probe("egret.png", options: options);   // Access and WorkingBytes before decoding
using Raster tile = Raster.Load("egret.png", options: options);

Raster.LoadInto decodes into a raster you already have, and ReadFrame gives an animation one composed frame at a time:

// Decode into a raster you already have: video frames and pooled images reuse their memory.
using Raster<Rgb8> reused = Raster.Create<Rgb8>(480, 640);
Raster.LoadInto("egret.png", reused);

// An animation one composed frame at a time, into one reused canvas.
using RasterDecoder gif = RasterDecoder.Open("pan.gif");
using Raster<Rgba8> canvas = Raster.Create<Rgba8>(gif.Info.Width, gif.Info.Height);
while (gif.ReadFrame(canvas) is Metadata.FrameMetadata frame)
{
    Console.WriteLine($"frame {count++}, shown for {frame.Delay.TotalMilliseconds} ms");
}

RasterDecoder.OpenIncremental decodes while the bytes are still arriving: Append each chunk as it comes, read the rows finished so far, and call EndOfInput after the last. Status (a DecodeStatus) says whether rows are ready, more bytes are needed, or the frame or image is done, and RenderPreview draws an interlaced PNG, a progressive JPEG, a GIF frame, a layered AVIF or a JPEG XL still as far as it has arrived. Files that are decoded into a buffer first (such as bottom-up BMP and TGA) wait for the end.

// Decode while the bytes arrive, a chunk at a time, as from a download.
using RasterDecoder decoder = RasterDecoder.OpenIncremental();
Raster<Rgb8>? band = null;
foreach (ReadOnlyMemory<byte> chunk in file.Chunk(4096).Select(c => new ReadOnlyMemory<byte>(c)))
{
    decoder.Append(chunk.Span);
    if (decoder.HeaderRead)
    {
        band ??= Raster.Create<Rgb8>(decoder.Info.Width, 16);
        int rows;
        while ((rows = decoder.ReadRows(band)) > 0)
        {
            rowsSoFar += rows; // the rows finished so far; an interlaced file shows RenderPreview instead
        }
    }
}

decoder.EndOfInput();
while (decoder.ReadRows(band!) is int rest and > 0)
{
    rowsSoFar += rest;
}

band?.Dispose();

On a server, RasterDecoder.OpenAsync reads an asynchronous stream the same way, and LoadAsync from a stream that cannot seek, such as a request body, no longer holds the whole file while it decodes:

// On a server: read rows as the upload arrives, without holding the whole file.
await using Stream upload = File.OpenRead("egret.png");
using RasterDecoder decoder = await RasterDecoder.OpenAsync(upload, cancellationToken: TestContext.Current.CancellationToken);
using Raster<Gray8> band = Raster.Create<Gray8>(decoder.Info.Width, 32);
int rows;
while ((rows = await decoder.ReadRowsAsync(band, TestContext.Current.CancellationToken)) > 0)
{
    for (int y = 0; y < rows; y++)
    {
        sum += band.Row(y)[0].L;
    }
}

RasterEncoder writes the other way: a band of rows or a frame at a time, into a file or stream, with the bytes Save writes. Options that need every pixel before the first byte, such as an automatic palette, PNG's Reduce or choosing alpha by looking at the pixels, throw NotSupportedException at Create and say what to set instead. BMP and TGA store rows bottom-up: on a stream that cannot seek they are written top-down, as is run-length TGA, and an APNG there needs PngWriterOptions.FrameCount. An interlaced PNG (PngWriterOptions.Interlace) holds the rows until the last one, since its first pass needs the whole height.

// A huge image written a band at a time: only the band is ever in memory.
const int Width = 8_000, Height = 6_000;
using Raster<Rgb8> band = Raster.Create<Rgb8>(Width, 64);
using RasterEncoder<Rgb8> encoder = RasterEncoder.Create<Rgb8>("big.png", Width, Height);
for (int top = 0; top < Height; top += band.Height)
{
    int rows = Math.Min(band.Height, Height - top);
    for (int y = 0; y < rows; y++)
    {
        band.Row(y).Fill(new Rgb8((byte)(top + y), 128, 200));
    }

    encoder.WriteRows(band, rows);
}

encoder.Complete(); // writes IEND; without it the file is deleted on dispose

// An animation written a frame at a time, each frame from one reused canvas.
using Raster<Rgba8> canvas = Raster.Create<Rgba8>(120, 80);
using FileStream file = File.Create("spinner.gif");
using RasterEncoder<Rgba8> gif = RasterEncoder.Create<Rgba8>(file, GifFormat.Info, 120, 80);
for (int i = 0; i < 12; i++)
{
    for (int y = 0; y < canvas.Height; y++)
    {
        canvas.Row(y).Fill(new Rgba8(255, 255, 255, 255));
    }

    canvas.Row(40)[(i * 10)..((i * 10) + 10)].Fill(new Rgba8(200, 0, 0, 255));
    gif.WriteFrame(canvas, new Metadata.FrameMetadata { Delay = TimeSpan.FromMilliseconds(80) });
}

gif.Complete();

WriteRowsAsync and CompleteAsync write each band out asynchronously, and SaveAsync now does the same instead of encoding the whole file into memory first:

// On a server: send each band as it is encoded, without holding the file.
using RasterDecoder source = RasterDecoder.Open("egret.png");
using Raster<Rgb8> band = Raster.Create<Rgb8>(source.Info.Width, 32);
using RasterEncoder<Rgb8> encoder = RasterEncoder.Create<Rgb8>(response, JpegFormat.Info, source.Info.Width, source.Info.Height);
int rows;
while ((rows = source.ReadRows(band)) > 0)
{
    await encoder.WriteRowsAsync(band, rows, TestContext.Current.CancellationToken);
}

await encoder.CompleteAsync(TestContext.Current.CancellationToken);

Raster.Transcode joins the two: rows go from the decoder through a short chain of operations into the encoder, so a thumbnail of a 20,000 × 20,000 PNG holds well under a megabyte of pixels instead of 1.6 GB. When the chain starts with Fit, Cover or Resize, the decoder already shrinks by a whole factor (MinimumSize), and a leading Crop becomes Region. The chain may crop, resize, mirror left to right and adjust colors pixel by pixel; anything that needs rows out of order or the whole image, such as Blur or Rotate, throws NotSupportedException before anything is written.

// A thumbnail of a big file without ever holding the full image: the decoder already shrinks by a whole factor.
Raster.Transcode("egret.png", "thumb.jpg", x => x.Fit(256, 256).Saturation(1.1f));

// On a server: an upload to a 512-pixel PNG preview, streamed both ways.
await using Stream upload = File.OpenRead("egret.png");
await Raster.TranscodeAsync(upload, response, PngFormat.Info, x => x.Fit(512, 512), cancellationToken: TestContext.Current.CancellationToken);

// A progress handler sees the output's rows go out; the token stops the work between bands.
IProgress<SaveProgress> progress = new Progress<SaveProgress>(p => Console.WriteLine($"{p.Fraction:P0}, {p.BytesWritten} bytes"));
Raster.Transcode("egret.png", "egret.jpg", x => x.Fit(1024, 1024), null, null, null, progress, TestContext.Current.CancellationToken);

Pixel memory

Pixel memory never lives on the garbage-collected heap as large arrays. PooledPixelMemorySource.Shared hands out pooled arrays for small blocks and 64-byte-aligned unmanaged memory for large ones, keeps up to 128 MB of freed blocks for reuse, and splits very large images into blocks of whole rows. Trim() frees what it keeps.

You can plug in your own source through EgamiContext.Memory by implementing IPixelMemorySource.

Finding leaks

PixelMemoryUsage usage = PooledPixelMemorySource.Shared.Usage;
Console.WriteLine($"{usage.LiveBlocks} blocks, {usage.LiveBytes:N0} bytes in use");

// Looking for a raster someone forgot to dispose: track leaks for a while, with stack traces.
usage.CaptureStackTraces = true;
usage.LeakDetected += (_, leak) => Console.WriteLine($"{leak.Bytes:N0} bytes never disposed, allocated at\n{leak.StackTrace}");
usage.TrackLeaks = true;

LiveBlocks and LiveBytes are always counted, at the cost of one atomic addition per allocation. TrackLeaks gives each new block a small finalizable tracker, so turn it on only while you look; CaptureStackTraces makes allocations much slower still. LeakDetected runs on the finalizer thread. Leaks come only through LeakDetected, never through the diagnostics handler, which reports what one call found.

Your own memory source

// Every pixel buffer comes from the context's IPixelMemorySource; wrap the pooled one to count what it hands out.
CountingSource counting = new(PooledPixelMemorySource.Shared);
EgamiContext context = EgamiContext.Default with { Memory = counting };
using (Raster<Rgba8> image = Raster.Create<Rgba8>(512, 512, context))
{
    image.Apply(ops => ops.Blur(2));
}

Console.WriteLine($"{counting.Blocks} blocks, {counting.Bytes:N0} bytes");

private sealed class CountingSource(IPixelMemorySource inner) : IPixelMemorySource
{
    public int Blocks;
    public long Bytes;

    public int MaxBlockBytes => inner.MaxBlockBytes;

    public IMemoryOwner<byte> Allocate(int byteLength, ClearMode clear)
    {
        Interlocked.Increment(ref Blocks);
        Interlocked.Add(ref Bytes, byteLength);
        return inner.Allocate(byteLength, clear);   // ClearMode.Zero when the caller needs zeroed memory
    }
}

// In tests or debug builds: report a raster that was never disposed, and where it was allocated.
PooledPixelMemorySource memory = new();
memory.Usage.TrackLeaks = true;
memory.Usage.CaptureStackTraces = true;
memory.Usage.LeakDetected += (sender, e) =>
    Console.Error.WriteLine($"Leaked {e.Bytes:N0} bytes, allocated at:\n{e.StackTrace}");

Run it

samples/MemoryAndThreading is a console app that does what this guide shows on the bundled images: dotnet run --project samples/MemoryAndThreading writes its results to ./output.

Edit this page