Proposed improvement: support the new ImageData.pixelFormat colorType in SC

[KaliedaRik]

The new ImageData.pixelFormat work is not just an ImageData issue. It is part of a broader change in the HTML canvas / 2D Canvas API model: the 2D canvas backing store is becoming more explicitly configurable in terms of color space and color type.

In the current platform model, a 2D context can be created with settings including colorSpace, colorType, alpha, desynchronized, and willReadFrequently:

• colorType is now defined as either "unorm8" or "float16"

ImageData is following the same direction: ImageData.pixelFormat can now be "rgba-unorm8" or "rgba-float16", and ImageData.data may therefore be either a Uint8ClampedArray or a Float16Array.

Main conclusion

Scrawl-canvas does not need an urgent rewrite, but it does need a deliberate audit and a policy. The immediate risk is not "canvas is broken"; it is that parts of SC may have implicit unorm8 assumptions baked into code paths, especially where pixel data is read, written, packed, unpacked, filtered, or exposed through convenience APIs.

Recommended immediate policy for SC

For now, SC should probably continue to define a conservative 2D-canvas policy:

• alpha: true
• desynchronized: false
• colorSpace: continue current support model
• colorType: "unorm8" wherever supported
• treat float16 support as opt-in / experimental rather than default

This keeps SC stable while allowing later float-capable work to be added intentionally. The platform defaults still favour ordinary unorm8 behaviour, so this is a safe transitional stance.

Work areas to consider

Scrawl-canvas should treat the new ImageData.pixelFormat work as part of a wider 2D-canvas evolution toward explicit colorSpace + colorType backing stores, and should respond by hardening unorm8 assumptions now while leaving room for selective, experimental float16 support later.

1. Context creation and feature detection

SC should centralise 2D context creation so that context attributes are requested deliberately rather than inherited accidentally. That means adding explicit handling for colorType during getContext("2d", …) setup, and adding feature detection so SC can tell whether the browser actually honoured the requested settings. getContextAttributes() is now the obvious inspection point for that.

This is also where SC should decide what to do with browser-specific extras. Chrome currently exposes a toneMapping attribute in getContextAttributes(), but tone mapping is not part of the settled 2D canvas settings model in the HTML Standard yet; there is still active standards discussion about adding tone-mapping controls for floating-point / wide-gamut canvases. SC should therefore treat toneMapping as experimental / browser-specific metadata for now, not as a stable API commitment - thus out-of-scope for this issue.

2. ImageData audit

SC should audit every place where it assumes that ImageData.data is always byte-backed RGBA. The critical assumptions to look for are:

• instanceof Uint8ClampedArray
• packed 32-bit views over data.buffer
• channel values assumed to be integers in the range 0..255
• comments/docs that describe ImageData.data as always unorm8 RGBA

Those assumptions are valid only for "rgba-unorm8", not for "rgba-float16"

3. Filter engine

This is likely the biggest SC subsystem affected by the new model.

Most current filters appear to assume unorm8 RGBA input/output. That is fine for today, but SC should introduce format awareness into the filter pipeline so filters can dispatch by pixel format rather than silently assuming byte RGBA. The practical model is:

• keep existing fast unorm8 paths
• add float-capable paths only where they are useful
• centralise format dispatch rather than branching ad-hoc inside every filter

Filters based on packed integer tricks will remain unorm8-specific for now. Filters that already work internally in float side-buffers, especially OK-based or multi-pass filters, are much better candidates for future float16 support. The goal of a first float-capable phase would be to preserve more information at ingress/egress, not to solve "full HDR semantics" on day one.

4. Direct pixel manipulation APIs (cell.js)

The current getCellData / paintCellData function pair is really a unorm8-oriented experimental convenience API. It assumes byte RGBA storage, uses a packed Uint32Array view, and exposes per-pixel channel values as 8-bit integers. That should probably remain true.

If SC later supports float-capable direct pixel access, it would be better to add a separate experimental API function pair rather than forcing both models into the same abstraction. A float-aware pair would likely use different backing stores, different hooks, and a different public contract. It should be thought of as a more numeric / field-oriented API rather than merely "the same pixel objects, but float". That keeps both APIs cleaner and avoids contaminating the current one with too many conditional branches.

5. Color engine

The color engine looks relatively safe if canvas interaction is limited to resolving CSS named colors and SC does the rest of its color parsing/manipulation computationally. In that case, the main requirement is simply to ensure that the resolver canvas remains a boring, controlled srgb + unorm8 compatibility shim. This area looks lower risk than filters and direct pixel APIs. The main work here is documenting that assumption clearly.

6. Image asset ingestion / draw pipelines

Image files should not be thought of as "having canvas colorType". colorType belongs to the destination canvas backing store. When an image is drawn to a 2D canvas, the browser decodes the source image and converts it into the destination canvas representation. For SC, the practical implication is:

• a unorm8 canvas is a normalizing sink
• future float16 canvases could preserve more precision / range if the browser and source material support it

That means no urgent work is required here beyond understanding that source image richness can be lost when drawn into an ordinary unorm8 canvas.