Skip to content

refactor(core): canonical DType primitive constants #147

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
dfa1 merged 1 commit into from
Jun 23, 2026
Merged

refactor(core): canonical DType primitive constants #147

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 10 additions & 0 deletions CHANGELOG.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,16 @@ All notable changes to **vortex-java** are documented here.
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).

## [Unreleased]

### Added

- Canonical non-nullable `DType` constants: primitives `DType.I8` … `DType.I64`, `DType.U8` … `DType.U64`, `DType.F16`/`F32`/`F64`, plus `DType.BOOL`, `DType.UTF8`, `DType.BINARY`, `DType.NULL`, `DType.VARIANT`. Shared immutable instances — prefer them over `new DType.Primitive(pt, false)` / `new DType.Utf8(false)` etc.; build a nullable column with `DType.I64.asNullable()`. The encoders/decoders that each declared private `*_DTYPE` fields now reference these.

### Removed

- **Breaking (minor):** the no-arg factory methods `DType.i8()` … `DType.f64()`, `DType.bool_()`, `DType.utf8()`, `DType.binary()`, `DType.null_()`, `DType.variant()` are replaced by the constants above (`DType.i64()` → `DType.I64`, `DType.utf8()` → `DType.UTF8`). The `DType.decimal(..)`/`DType.structBuilder()` factories and the record constructors are unchanged.

## [0.8.3] — 2026-06-23

A **Sonar-driven refactoring** release: no new file-format capability, but a focused pass using SonarCloud findings to drive cleanups — dead code removed, duplication factored out, and one hot-loop micro-optimisation. Each finding was triaged (lead, not verdict) so the changes preserve behaviour and the JIT vectorisation of the hot decode loops. The interpretation framework behind this is now documented in `docs/testing.md`.
Expand Down
8 changes: 4 additions & 4 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -97,10 +97,10 @@ try (VortexReader vf = VortexReader.open(Path.of("data/example.vortex"));

```java
DType.Struct schema = DType.structBuilder()
.field("timestamp", DType.i64())
.field("symbol", DType.utf8())
.field("price", DType.f64())
.field("volume", DType.i64().asNullable()) // boxed Long[] → nullable
.field("timestamp", DType.I64)
.field("symbol", DType.UTF8)
.field("price", DType.F64)
.field("volume", DType.I64.asNullable()) // boxed Long[] → nullable
.build();

try (var ch = FileChannel.open(Path.of("data/example.vortex"),
Expand Down
11 changes: 5 additions & 6 deletions cli/src/test/java/io/github/dfa1/vortex/cli/CliTestSupport.java
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,7 +1,6 @@
package io.github.dfa1.vortex.cli;

import io.github.dfa1.vortex.core.DType;
import io.github.dfa1.vortex.core.PType;
import io.github.dfa1.vortex.writer.VortexWriter;
import io.github.dfa1.vortex.writer.WriteOptions;

Expand All @@ -27,7 +26,7 @@ static Path writeSmallVortex(Path dir, String name) throws IOException {
Path file = dir.resolve(name);
DType.Struct schema = new DType.Struct(
List.of("id"),
List.of(new DType.Primitive(PType.I64, false)),
List.of(DType.I64),
false);
try (FileChannel ch = FileChannel.open(file, StandardOpenOption.CREATE, StandardOpenOption.WRITE);
VortexWriter writer = VortexWriter.create(ch, schema, WriteOptions.defaults())) {
Expand All @@ -44,10 +43,10 @@ static Path writeTypedVortex(Path dir, String name) throws IOException {
DType.Struct schema = new DType.Struct(
List.of("id", "qty", "price", "name"),
List.of(
new DType.Primitive(PType.I64, false),
new DType.Primitive(PType.I32, false),
new DType.Primitive(PType.F64, false),
new DType.Utf8(false)),
DType.I64,
DType.I32,
DType.F64,
DType.UTF8),
false);
try (FileChannel ch = FileChannel.open(file, StandardOpenOption.CREATE, StandardOpenOption.WRITE);
VortexWriter writer = VortexWriter.create(ch, schema, WriteOptions.defaults())) {
Expand Down
16 changes: 8 additions & 8 deletions cli/src/test/java/io/github/dfa1/vortex/cli/SchemaCommandTest.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -64,17 +64,17 @@ void validFile_printsStructSchemaAndReturnsOk(@TempDir Path tmp) throws IOExcept
}

static Stream<Arguments> dtypeCases() {
DType i64 = new DType.Primitive(PType.I64, false);
DType i64 = DType.I64;
return Stream.of(
arguments(new DType.Primitive(PType.I64, false), "I64"),
arguments(DType.I64, "I64"),
arguments(new DType.Primitive(PType.I32, true), "I32?"),
arguments(new DType.Utf8(false), "utf8"),
arguments(DType.UTF8, "utf8"),
arguments(new DType.Utf8(true), "utf8?"),
arguments(new DType.Binary(false), "binary"),
arguments(DType.BINARY, "binary"),
arguments(new DType.Binary(true), "binary?"),
arguments(new DType.Bool(false), "bool"),
arguments(DType.BOOL, "bool"),
arguments(new DType.Bool(true), "bool?"),
arguments(new DType.Null(false), "null"),
arguments(DType.NULL, "null"),
arguments(new DType.Decimal((byte) 10, (byte) 2, false), "decimal(10,2)"),
arguments(new DType.Decimal((byte) 10, (byte) 2, true), "decimal(10,2)?"),
arguments(new DType.List(i64, false), "list<I64>"),
Expand All @@ -83,9 +83,9 @@ static Stream<Arguments> dtypeCases() {
arguments(new DType.FixedSizeList(i64, 4, true), "list<I64>[4]?"),
arguments(new DType.Extension("vortex.uuid", i64, null, false), "ext<vortex.uuid>"),
arguments(new DType.Extension("vortex.uuid", i64, null, true), "ext<vortex.uuid>?"),
arguments(new DType.Variant(false), "variant"),
arguments(DType.VARIANT, "variant"),
arguments(new DType.Variant(true), "variant?"),
arguments(new DType.Struct(List.of("a", "b"), List.of(i64, new DType.Utf8(false)), false),
arguments(new DType.Struct(List.of("a", "b"), List.of(i64, DType.UTF8), false),
"struct<a: I64, b: utf8>"));
}

Expand Down
18 changes: 9 additions & 9 deletions cli/src/test/java/io/github/dfa1/vortex/cli/tui/ArrayFixtures.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -36,47 +36,47 @@ static LongArray longs(Arena arena, long... vs) {
for (int i = 0; i < vs.length; i++) {
seg.setAtIndex(ValueLayout.JAVA_LONG, i, vs[i]);
}
return new MaterializedLongArray(new DType.Primitive(PType.I64, false), vs.length, seg.asReadOnly());
return new MaterializedLongArray(DType.I64, vs.length, seg.asReadOnly());
}

static IntArray ints(Arena arena, int... vs) {
MemorySegment seg = arena.allocate(vs.length * 4L, 4);
for (int i = 0; i < vs.length; i++) {
seg.setAtIndex(ValueLayout.JAVA_INT, i, vs[i]);
}
return new MaterializedIntArray(new DType.Primitive(PType.I32, false), vs.length, seg.asReadOnly());
return new MaterializedIntArray(DType.I32, vs.length, seg.asReadOnly());
}

static ShortArray shorts(Arena arena, short... vs) {
MemorySegment seg = arena.allocate(vs.length * 2L, 2);
for (int i = 0; i < vs.length; i++) {
seg.setAtIndex(ValueLayout.JAVA_SHORT, i, vs[i]);
}
return new MaterializedShortArray(new DType.Primitive(PType.I16, false), vs.length, seg.asReadOnly());
return new MaterializedShortArray(DType.I16, vs.length, seg.asReadOnly());
}

static ByteArray bytes(Arena arena, byte... vs) {
MemorySegment seg = arena.allocate(vs.length, 1);
for (int i = 0; i < vs.length; i++) {
seg.set(ValueLayout.JAVA_BYTE, i, vs[i]);
}
return new MaterializedByteArray(new DType.Primitive(PType.I8, false), vs.length, seg.asReadOnly());
return new MaterializedByteArray(DType.I8, vs.length, seg.asReadOnly());
}

static DoubleArray doubles(Arena arena, double... vs) {
MemorySegment seg = arena.allocate(vs.length * 8L, 8);
for (int i = 0; i < vs.length; i++) {
seg.setAtIndex(ValueLayout.JAVA_DOUBLE, i, vs[i]);
}
return new MaterializedDoubleArray(new DType.Primitive(PType.F64, false), vs.length, seg.asReadOnly());
return new MaterializedDoubleArray(DType.F64, vs.length, seg.asReadOnly());
}

static FloatArray floats(Arena arena, float... vs) {
MemorySegment seg = arena.allocate(vs.length * 4L, 4);
for (int i = 0; i < vs.length; i++) {
seg.setAtIndex(ValueLayout.JAVA_FLOAT, i, vs[i]);
}
return new MaterializedFloatArray(new DType.Primitive(PType.F32, false), vs.length, seg.asReadOnly());
return new MaterializedFloatArray(DType.F32, vs.length, seg.asReadOnly());
}

static BoolArray bools(Arena arena, boolean... vs) {
Expand All @@ -89,7 +89,7 @@ static BoolArray bools(Arena arena, boolean... vs) {
seg.set(ValueLayout.JAVA_BYTE, byteIdx, (byte) (cur | (1 << (i & 7))));
}
}
return new MaterializedBoolArray(new DType.Bool(false), vs.length, seg.asReadOnly());
return new MaterializedBoolArray(DType.BOOL, vs.length, seg.asReadOnly());
}

/// Builds a UTF-8 [VarBinArray] (`OffsetMode`, I64 offsets) from the given strings.
Expand All @@ -98,12 +98,12 @@ static VarBinArray utf8(Arena arena, String... vs) {
for (int i = 0; i < vs.length; i++) {
rows[i] = vs[i].getBytes(StandardCharsets.UTF_8);
}
return varbin(arena, new DType.Utf8(false), rows);
return varbin(arena, DType.UTF8, rows);
}

/// Builds a binary [VarBinArray] (`OffsetMode`, I64 offsets) from the given byte rows.
static VarBinArray binary(Arena arena, byte[]... rows) {
return varbin(arena, new DType.Binary(false), rows);
return varbin(arena, DType.BINARY, rows);
}

private static VarBinArray varbin(Arena arena, DType dtype, byte[]... rows) {
Expand Down
23 changes: 11 additions & 12 deletions cli/src/test/java/io/github/dfa1/vortex/cli/tui/GridRenderTest.java
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,7 +1,6 @@
package io.github.dfa1.vortex.cli.tui;

import io.github.dfa1.vortex.core.DType;
import io.github.dfa1.vortex.core.PType;
import io.github.dfa1.vortex.encoding.TimeUnit;
import io.github.dfa1.vortex.reader.array.Array;
import io.github.dfa1.vortex.reader.array.ByteArray;
Expand All @@ -27,9 +26,9 @@
/// arrays without a terminal or fixture file.
class GridRenderTest {

private static final DType I64 = new DType.Primitive(PType.I64, false);
private static final DType F64 = new DType.Primitive(PType.F64, false);
private static final DType BOOL = new DType.Bool(false);
private static final DType I64 = DType.I64;
private static final DType F64 = DType.F64;
private static final DType BOOL = DType.BOOL;

@Test
void nullArrayRendersEmpty() {
Expand Down Expand Up @@ -77,7 +76,7 @@ void rendersDateExtensionFromIntStorage() {
try (Arena arena = Arena.ofConfined()) {
// Given — vortex.date over I32 epoch-day storage: day 0 = 1970-01-01
DType dateExt = new DType.Extension("vortex.date",
new DType.Primitive(PType.I32, false), null, false);
DType.I32, null, false);
IntArray storage = ArrayFixtures.ints(arena, 0, 1);

// When / Then
Expand Down Expand Up @@ -117,7 +116,7 @@ void rendersUuidExtensionFromFixedSizeListStorage() {
DType uuidExt = UuidExtensionDecoder.INSTANCE.dtype(false);
ByteArray elems = ArrayFixtures.bytes(arena, new byte[16]);
FixedSizeListArray storage = new FixedSizeListArray(
new DType.FixedSizeList(new DType.Primitive(PType.U8, false), 16, false), 1, elems);
new DType.FixedSizeList(DType.U8, 16, false), 1, elems);

// When / Then
assertThat(GridRender.formatCell(storage, 0, uuidExt))
Expand Down Expand Up @@ -145,8 +144,8 @@ void rendersUtf8VarBinAsRawString() {
Array sut = ArrayFixtures.utf8(arena, "héllo", "x");

// When / Then
assertThat(GridRender.formatCell(sut, 0, new DType.Utf8(false))).isEqualTo("héllo");
assertThat(GridRender.formatCell(sut, 1, new DType.Utf8(false))).isEqualTo("x");
assertThat(GridRender.formatCell(sut, 0, DType.UTF8)).isEqualTo("héllo");
assertThat(GridRender.formatCell(sut, 1, DType.UTF8)).isEqualTo("x");
}
}

Expand All @@ -157,7 +156,7 @@ void rendersBinaryVarBinAsHex() {
Array sut = ArrayFixtures.binary(arena, new byte[]{0x01, (byte) 0xab, 0x02});

// When / Then
assertThat(GridRender.formatCell(sut, 0, new DType.Binary(false))).isEqualTo("0x01ab02");
assertThat(GridRender.formatCell(sut, 0, DType.BINARY)).isEqualTo("0x01ab02");
}
}

Expand All @@ -169,7 +168,7 @@ void truncatesBinaryHexBeyond16Bytes() {
Array sut = ArrayFixtures.binary(arena, twenty);

// When
String cell = GridRender.formatCell(sut, 0, new DType.Binary(false));
String cell = GridRender.formatCell(sut, 0, DType.BINARY);

// Then — "0x" + 16*"00" + "..."
assertThat(cell).isEqualTo("0x" + "00".repeat(16) + "...");
Expand All @@ -181,7 +180,7 @@ void extensionDecodeFailureRendersDiagnostic() {
try (Arena arena = Arena.ofConfined()) {
// Given — vortex.date declared but storage is F64, which epochInteger rejects
DType dateExt = new DType.Extension("vortex.date",
new DType.Primitive(PType.I32, false), null, false);
DType.I32, null, false);
Array badStorage = ArrayFixtures.doubles(arena, 1.5);

// When — the decode throws and is caught
Expand All @@ -197,7 +196,7 @@ void unknownExtensionIdFallsThroughToStorageRendering() {
try (Arena arena = Arena.ofConfined()) {
// Given — an extension id this reader does not know: render the raw storage value
DType unknownExt = new DType.Extension("custom.thing",
new DType.Primitive(PType.I32, false), null, false);
DType.I32, null, false);
IntArray storage = ArrayFixtures.ints(arena, 42);

// When / Then — no extension formatting, falls to the IntArray branch
Expand Down
10 changes: 5 additions & 5 deletions cli/src/test/java/io/github/dfa1/vortex/cli/tui/InspectorRenderTest.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@
/// in-memory arrays — no terminal, worker, or encoded fixture required.
class InspectorRenderTest {

private static final DType I64 = new DType.Primitive(PType.I64, false);
private static final DType I64 = DType.I64;

@Nested
class FormatValue {
Expand Down Expand Up @@ -92,7 +92,7 @@ void rendersDateExtension() {
try (Arena arena = Arena.ofConfined()) {
// Given — vortex.date over I32 epoch-day storage
DType dateExt = new DType.Extension("vortex.date",
new DType.Primitive(PType.I32, false), null, false);
DType.I32, null, false);
IntArray storage = ArrayFixtures.ints(arena, 0);

// When / Then — day 0 = 1970-01-01
Expand All @@ -107,7 +107,7 @@ void rendersUtf8VarBinQuoted() {
Array sut = ArrayFixtures.utf8(arena, "hi");

// When / Then
assertThat(InspectorRender.formatValue(sut, 0, new DType.Utf8(false))).isEqualTo("\"hi\"");
assertThat(InspectorRender.formatValue(sut, 0, DType.UTF8)).isEqualTo("\"hi\"");
}
}

Expand All @@ -118,7 +118,7 @@ void rendersBinaryVarBinAsShortHex() {
Array sut = ArrayFixtures.binary(arena, new byte[]{0x01, (byte) 0xab});

// When / Then
assertThat(InspectorRender.formatValue(sut, 0, new DType.Binary(false))).isEqualTo("0x01ab");
assertThat(InspectorRender.formatValue(sut, 0, DType.BINARY)).isEqualTo("0x01ab");
}
}

Expand All @@ -127,7 +127,7 @@ void dateExtensionDecodeFailureFallsThroughToGeneric() {
try (Arena arena = Arena.ofConfined()) {
// Given — vortex.date declared but storage is F64; the decode throws and is swallowed
DType dateExt = new DType.Extension("vortex.date",
new DType.Primitive(PType.I32, false), null, false);
DType.I32, null, false);
Array badStorage = ArrayFixtures.doubles(arena, 1.5);

// When / Then — falls through to the generic DoubleArray rendering
Expand Down
17 changes: 8 additions & 9 deletions cli/src/test/java/io/github/dfa1/vortex/cli/tui/TuiTestSupport.java
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,7 +1,6 @@
package io.github.dfa1.vortex.cli.tui;

import io.github.dfa1.vortex.core.DType;
import io.github.dfa1.vortex.core.PType;
import io.github.dfa1.vortex.reader.VortexHandle;
import io.github.dfa1.vortex.reader.VortexReader;
import io.github.dfa1.vortex.writer.VortexWriter;
Expand Down Expand Up @@ -31,9 +30,9 @@ static Path writeGridVortex(Path dir, String name, int rows) throws IOException
DType.Struct schema = new DType.Struct(
List.of("a", "b", "c"),
List.of(
new DType.Primitive(PType.I64, false),
new DType.Primitive(PType.I64, false),
new DType.Primitive(PType.I64, false)),
DType.I64,
DType.I64,
DType.I64),
false);
long[] a = new long[rows];
long[] b = new long[rows];
Expand All @@ -58,10 +57,10 @@ static Path writeMultiTypeVortex(Path dir, String name) throws IOException {
DType.Struct schema = new DType.Struct(
List.of("i", "d", "flag", "name"),
List.of(
new DType.Primitive(PType.I64, false),
new DType.Primitive(PType.F64, false),
new DType.Bool(false),
new DType.Utf8(false)),
DType.I64,
DType.F64,
DType.BOOL,
DType.UTF8),
false);
try (FileChannel ch = FileChannel.open(file, StandardOpenOption.CREATE, StandardOpenOption.WRITE);
VortexWriter writer = VortexWriter.create(ch, schema, WriteOptions.defaults())) {
Expand Down Expand Up @@ -90,7 +89,7 @@ static Path writeRichVortex(Path dir, String name, int rows) throws IOException
Path file = dir.resolve(name);
DType.Struct schema = new DType.Struct(
List.of("label", "n"),
List.of(new DType.Utf8(false), new DType.Primitive(PType.I64, false)),
List.of(DType.UTF8, DType.I64),
false);
String[] labels = {"red", "green", "blue"};
String[] label = new String[rows];
Expand Down
Loading
Loading