ArrayTag Generic Types

[Offroaders123]

Looking into a few different ways to accomplish representing tuple types with TypedArray-based NBT key types.

Right now the standard library types don't provide anything like tuple support for TypedArray values. I'd like to be able to do something like Int8Array<[number, number]>, and only allow access of those two properties, just like the regular Array tuple type notation allows for ([number, number] on it's own).

I've encountered a few places where this would be a great help at type validation, namely things like player NBT UUID fields, which would be IntArrayTag<[number, number, number, number]>, as well as position tuple fields, which tend to be what would be described with IntArrayTag<[number, number, number]>.

[Offroaders123]

Worked out yesterday how to accomplish this with a few builder types! It does everything I wanted to accomplish with it, and it really does help ensure type-safety. The only downside I've noticed with it so far though is that you can't validate with this type when you are creating a given TypedArray value, and you have to cast the value to meet the type. So this may be a one-way type help, because it would more be for the reading and editing of existing key values, rather than when defining new ones. I will think of how I can possible make this work, I feel like it should be doable somehow.

typescript - How to remove index signature using mapped types - Stack Overflow

type RemoveIndex<T extends object> = {
  [K in keyof T as
    string extends K
      ? never
      : number extends K
      ? never
      : symbol extends K
      ? never
      : K
  ] : T[K];
};

type ExtractTuple<T extends any[]> = {
  [Entry in keyof T as Entry extends `${number}` ? Entry : never]: T[Entry];
};

type ArrayTuple<TypedArray extends object, Tuple extends any[]> = RemoveIndex<TypedArray> & ExtractTuple<Tuple>;

type ArrayTag<ArrayLike extends object, Tuple extends any[] | ArrayLike> =
  Tuple extends any[]
    ? ArrayTuple<ArrayLike, Tuple>
    : ArrayLike;

type ByteArrayTag<T extends Int8Array[number][] | Int8Array = Int8Array> = ArrayTag<Int8Array, T>;

type IntArrayTag<T extends Int32Array[number][] | Int32Array = Int32Array> = ArrayTag<Int32Array, T>;

type LongArrayTag<T extends BigInt64Array[number][] | BigInt64Array = BigInt64Array> = ArrayTag<BigInt64Array, T>;

type Nice = ByteArrayTag;
type Nicer = ByteArrayTag<[number, 5]>;

declare const nice: Nice;
const hi = nice[110];

declare const nicer: Nicer;
// @ts-expect-error - `Array` methods shouldn't be present on this custom derived tuple type, since it's an `Int8Array`.
nicer.push;

const hello = nicer[1];
// @ts-expect-error - property shouldn't be indexable, it's not defined in the tuple generic.
nicer[5];

// Demos pt.2

// player position NBT type
type Pos = IntArrayTag<[number, number, number]>;

declare let pos: Pos;
pos[0];
pos[1];
pos[2];
// @ts-expect-error
pos[3];

// @ts-expect-error - this shouldn't be assignable to the tuple type
pos = new Int32Array() as IntArrayTag;

// comparing to regular tuple
type Lol = [0, 1, 2, 3];

declare const lol: Lol;
// @ts-expect-error - just like this isn't accessible
lol[5];

// Next demos after rehashing

// @ts-expect-error - How would one validate this, and get automatic type handling, withough needing to use `as` on the value? Can you add generics to existing interfaces?
const hii: LongArrayTag<[5n]> = new BigInt64Array([5n]);

[Offroaders123]

Here's another demo I was just writing to try out if this really does work well when embedded directly into NBTify.

[SuperLlama88888]

+1, TypedArrays are much better for memory. When I tested in Chrome, storing an array of Int16()s from NBTify used 16 bytes/Int16 and an array overhead of around 5 bytes/element. (It was over double this in Firefox!) Using an Int16Array, however, clearly showed only 2 bytes per element in both browsers. This would be a 10x memory improvement in Chrome, and a 25x in Firefox.

[Offroaders123]

+1, TypedArrays are much better for memory. When I tested in Chrome, storing an array of Int16()s from NBTify used 16 bytes/Int16 and an array overhead of around 5 bytes/element. (It was over double this in Firefox!) Using an Int16Array, however, clearly showed only 2 bytes per element in both browsers. This would be a 10x memory improvement in Chrome, and a 25x in Firefox.

NBTify primitives use TypedArrays under the hood already, the only thing this would be changing is the ability to define tuple types around them too. Right now the types for these don't provide a way to generically define keys on those in that way.

[SuperLlama88888]

I was wondering if TypedArrays could also be used for NBT Lists with number types - the generic List ones which accept any other data type. There are TypedArray classes for all number types supported in NBT. I see that there are valueOf parameters in #readByte, #readInt, #readLong etc. in read.ts which directly return the numeric values. The issue is that individual elements would lose their type.

Possibly, could there be an option to drop all number types when reading for those of us who aren't concerned about the types? I'm not sure how this would work with TypeScript but it'd be better for memory. Even more possibly, the untyped reading could be done when the code is built/in production, which means that the developers get all the advantages of types and the users get lower memory usage.

[Offroaders123]

That's an interesting concept, I like it. It would need a new way to distinguish regular TypedArray NBT tags from that of an optimized TAG_List value with that new implementation.

My first thought of concern is that it wouldn't allow for differently-typed values inside of it, but NBT itself doesn't support that either, so it might actually work out okay.

I'll make a new issue for this idea, thanks for the input on that!