Temporal Property Index (TPI)

The Temporal Property Index is HyperMesh DB’s primary storage structure. It is a read-optimised, immutable binary file pair that enables range queries over hyperedges to touch only the time buckets that overlap the query window.

---

On-disk layout

bucket_directory.bin
┌──────────────────────────────────────────────────────────┐
│  HmTpiHeader  (magic + version + metadata)               │
├──────────────────────────────────────────────────────────┤
│  BucketEntry[0]   { start_ts, record_count, file_offset }│
│  BucketEntry[1]   ...                                    │
│  ...                                                     │
│  BucketEntry[B-1]                                        │
└──────────────────────────────────────────────────────────┘

hyperedges.bin
┌──────────────────────────────────────────────────────────┐
│  HmRecord[0]   { event_ts, weight, mean_dist_m,          │
│                  formation[16], member_count,             │
│                  members[member_count] }                  │
│  HmRecord[1]   ...                                       │
│  ...                                                     │
└──────────────────────────────────────────────────────────┘

All integers are little-endian. Structs use __attribute__((packed)) to eliminate compiler-inserted padding.

---

Query execution

A TPI range query for [T1, T2] proceeds in three steps:

1. Binary search the bucket directory for the first bucket with start_ts >= T1. O(log B) comparisons on a ~700-byte file in L1 cache.

2. Read only the buckets whose [start_ts, start_ts + bucket_seconds) interval overlaps [T1, T2]. Each pread() call fetches exactly the records in one bucket.

3. Filter records: discard any whose event_ts falls outside [T1, T2] (occurs only at the boundary buckets).

The result is then merged with matching WAL entries. and returned sorted by event_ts.

---

Performance characteristics

Metric	Value (1,774-record, 86-bucket dataset)
Bucket directory size	~700 B (fits in one cache line cluster)
Single-bucket read	~1 pread()
Speedup vs full scan (1 % window)	~86×
Median range query latency	< 500µs
P95 range query latency	< 2ms

---

Compaction

After WAL entries accumulate (inserts + tombstones), the compact() operation:

1. Reads all TPI records into memory.
2. Reads all WAL entries.
3. Applies: final_set = (TPI ∪ WAL inserts) − WAL tombstones.
4. Re-sorts by event_ts.
5. Re-partitions into buckets of bucket_seconds width.
6. Writes new bucket_directory.bin and hyperedges.bin.
7. Truncates wal.bin to zero entries.
8. Reopens the TPI + FMI readers.

Compaction is O(N log N) in the number of records and is designed to be idempotent — running it on an already-compacted database is a no-op.