Case Study: Congressional Cosponsorship (end-to-end)

This walkthrough takes a real, public hypergraph from raw files all the way to a trained hypergraph neural network, exercising every layer of HyperMesh in a single script. It doubles as a reference for how the pieces fit together.

The complete, self-contained script is reproduced in full at the end of this page — copy it into a file and run it. (Unlike the AML case study, this one downloads a public dataset, so it needs network access the first time.)

pip install "hypermesh[analytics,interop,ml]" pandas
python case_study_congress_bills.py            # downloads the dataset
python case_study_congress_bills.py --limit 20000   # faster subset

The dataset

We use congress-bills from the Austin R. Benson data collection: a temporal higher-order network where

• nodes are US Congresspersons (1,718 of them),
• hyperedges are legislative bills — the set of a bill’s sponsor and co-sponsors (260,851 timestamped bills),
• timestamps are the day each bill was introduced.

This is a textbook hypergraph: a bill naturally connects many legislators at once, which a plain graph can only approximate with cliques. It ships in the standard Benson format (-nverts.txt, -simplices.txt, -times.txt, -node-labels.txt), which the script downloads and parses.

Citation

Benson, Abebe, Schaub, Jadbabaie, Kleinberg. Simplicial closure and higher-order link prediction. PNAS 2018. Cosponsorship data derived from James Fowler, Connecting the Congress, Political Analysis 2006.

1. Ingest — bulk-load timestamped hyperedges

Each bill becomes one hyperedge. We hand copy_from_df a frame with event_ts and a members list per row — the fast bulk path (no row-by-row inserts).

he_df = pd.DataFrame({
    "event_ts": times,
    "members":  [list(s) for s in simplices],   # variable-size sets
    "weight":   [1.0] * len(simplices),
})
db.execute("CREATE HYPEREDGE TABLE Cosponsorship () BUCKET_SECONDS 365")
db.copy_from_df(he_df, "Cosponsorship")

We also compute each legislator’s tenure (first/last active day) on the way in — used later as model features.

2. Query — temporal Cypher

The hyperedges are immediately queryable, including time windows and pagination:

db.execute(
    "MATCH HYPEREDGE (he:Cosponsorship) "
    "WHERE he.event_ts >= 2000 AND he.event_ts <= 4000 RETURN * LIMIT 1000"
)
db.execute("MATCH HYPEREDGE (he:Cosponsorship) RETURN * SKIP 5 LIMIT 3")

3. Analytics — who matters, and how it’s structured

The analytics engine runs directly on the stored hypergraph — degree, PageRank influence, density, spectral gap, and spectral communities:

an = db.analytics("Cosponsorship")
an.density()
an.spectral_gap()
degree = an.node_degree()       # {node_id: #bills}
pr     = an.pagerank()          # influence ranking
an.zhou_clustering()            # spectral communities

We then derive a reproducible supervised label for the modeling stage: legislators in the top quartile of cosponsorship degree are tagged "high" influence, everyone else "low". This label is written into a node table:

db.execute(
    "CREATE NODE TABLE Congressperson ("
    "  node_id INTEGER PRIMARY KEY, name TEXT, "
    "  first_q INTEGER, last_q INTEGER, tier TEXT)"
)
db.copy_from_df(nodes_df, "Congressperson")   # name + tenure + tier

4. Interop — export for visualisation

HyperMesh doesn’t render graphs itself, but interop bridges the hypergraph to the standard ecosystem. Here we project to NetworkX and write GraphML for Gephi/Cytoscape:

hg = db.to_hypergraph("Cosponsorship")
g  = hm.interop.to_networkx(hg, kind="clique")
hm.interop.to_graphml(hg, "congress.graphml", kind="clique")

5. Modeling — train an owned HGNN in one call

Now the payoff. We predict each legislator’s influence tier from their tenure features + co-sponsorship structure, using the modeling layer. Tenure is non-leaky with respect to the degree-derived label, so the model genuinely has to learn from the hypergraph.

fhg = hm.nn.featurize(
    db, "Cosponsorship",
    node_table="Congressperson",
    node_features=["first_q", "last_q"],
    label="tier",
)

# Framework-native tensors, if you'd rather bring your own model:
tensors = hm.nn.prepare(db, "Cosponsorship", framework="torch",
                        node_table="Congressperson",
                        node_features=["first_q", "last_q"], label="tier")

model = hm.nn.fit(fhg, epochs=150)
model.evaluate(fhg.y, "val")["accuracy"]
model.embed()                  # node embeddings for downstream use

hm.nn.fit builds the spectral propagation operator from the incidence matrix, standardises features, splits train/val, and returns a FittedHGNN you can predict, predict_proba, embed, and evaluate.

6. Temporal — reservoir computing (LSM)

Finally we treat each legislator as a time series of yearly activity and classify their influence tier with a reservoir / liquid-state model — no backprop through time:

seq = hm.nn.temporal_features(db, "Cosponsorship", window_seconds=365)

clf = hm.nn.ReservoirClassifier(n_reservoir=128, seed=0)
clf.fit(per_legislator_sequences, tiers)
clf.score(per_legislator_sequences, tiers)

What this demonstrates

In one script, with one connect():

Layer	API	What it did
Ingest	copy_from_df	260k variable-size bills as hyperedges
Query	Cypher MATCH HYPEREDGE	temporal windows + pagination
Analytics	db.analytics(...)	influence, density, communities, spectral gap
Interop	hm.interop.*	NetworkX / GraphML export
Modeling	hm.nn.fit	owned spectral HGNN, trained on the DB
Temporal	hm.nn.ReservoirClassifier	activity-trajectory classification

The same APIs run unchanged on the full dataset (--limit 0) and against a remote HyperMesh server.

Full script

The complete, self-contained source for this walkthrough. Save it as case_study_congress_bills.py and run it with python case_study_congress_bills.py. It downloads the public dataset on first run; optional backends (networkx, matplotlib, torch) degrade gracefully if absent.

"""
case_study_congress_bills.py — an end-to-end HyperMesh tour on a real dataset.

This single script exercises the whole HyperMesh stack on the congress-bills
temporal hypergraph (US Congresspersons = nodes, legislative bills = hyperedges
over their sponsor + co-sponsors, timestamped in days):

    1. Ingest     — bulk-load timestamped hyperedges + a node table.
    2. Query      — temporal Cypher (windows, membership, SKIP/LIMIT).
    3. Analytics  — degree, PageRank, density, communities, spectral gap.
    4. Interop    — export to NetworkX / GraphML for visualisation.
    5. Modeling   — featurize -> prepare (tensors) -> fit an owned HGNN.
    6. Temporal   — per-legislator activity sequences -> reservoir (LSM) model.

Dataset: Austin R. Benson, https://www.cs.cornell.edu/~arb/data/congress-bills/
Cite: "Simplicial closure and higher-order link prediction", Benson et al.,
PNAS 2018; and Fowler, "Connecting the Congress", Political Analysis 2006.

Usage
-----
    pip install "hypermesh[analytics,interop,ml]" pandas
    python case_study_congress_bills.py                 # downloads data
    python case_study_congress_bills.py --limit 20000   # faster subset
    python case_study_congress_bills.py --data-dir ./data/congress-bills --no-download

Optional backends (networkx, matplotlib, torch) degrade gracefully if absent.
"""

from __future__ import annotations

import argparse
import io
import sys
import tarfile
import tempfile
import urllib.request
from collections import defaultdict
from pathlib import Path

DATA_URL = "https://www.cs.cornell.edu/~arb/data/congress-bills/congress-bills.tar.gz"
NAME = "congress-bills"


# 0. Load the Benson "nverts / simplices / times" format.

def _download(data_dir: Path) -> None:
    if (data_dir / f"{NAME}-nverts.txt").exists():
        print(f"  data already present in {data_dir}")
        return
    data_dir.mkdir(parents=True, exist_ok=True)
    print(f"  downloading {DATA_URL} ...")
    with urllib.request.urlopen(DATA_URL) as resp:  # noqa: S310 - trusted host
        blob = resp.read()
    with tarfile.open(fileobj=io.BytesIO(blob), mode="r:gz") as tar:
        for member in tar.getmembers():
            if not member.isfile():
                continue
            # flatten any leading "congress-bills/" directory
            out = data_dir / Path(member.name).name
            with tar.extractfile(member) as src:  # type: ignore[union-attr]
                out.write_bytes(src.read())
    print(f"  extracted to {data_dir}")


def _read_ints(path: Path) -> list[int]:
    with path.open() as fh:
        return [int(line) for line in fh if line.strip()]


def _read_node_labels(path: Path) -> dict[int, str]:
    """`<id> <name...>` per line, 1-indexed. Falls back to line number for id."""
    labels: dict[int, str] = {}
    if not path.exists():
        return labels
    with path.open(encoding="utf-8", errors="replace") as fh:
        for i, line in enumerate(fh, start=1):
            line = line.rstrip("\n")
            if not line:
                continue
            parts = line.split(maxsplit=1)
            if parts and parts[0].isdigit():
                labels[int(parts[0])] = parts[1].strip() if len(parts) > 1 else parts[0]
            else:
                labels[i] = line.strip()
    return labels


def load_simplices(data_dir: Path, limit: int | None):
    """Return (simplices, times, node_labels). Each simplex is a list[int]."""
    nverts = _read_ints(data_dir / f"{NAME}-nverts.txt")
    flat = _read_ints(data_dir / f"{NAME}-simplices.txt")
    times = _read_ints(data_dir / f"{NAME}-times.txt")
    labels = _read_node_labels(data_dir / f"{NAME}-node-labels.txt")

    simplices: list[list[int]] = []
    cursor = 0
    n = len(nverts) if limit is None else min(limit, len(nverts))
    for k in nverts[:n]:
        simplices.append(flat[cursor:cursor + k])
        cursor += k
    return simplices, times[:n], labels


# 1. Ingest.

def ingest(db, simplices, times):
    """Bulk-load the hyperedge table and return per-node tenure (first/last day)."""
    import pandas as pd

    print("\n[1] INGEST — bulk-load timestamped hyperedges")
    he_df = pd.DataFrame(
        {
            "event_ts": times,
            "members": [list(s) for s in simplices],
            "weight": [1.0] * len(simplices),
        }
    )
    db.execute("CREATE HYPEREDGE TABLE Cosponsorship () BUCKET_SECONDS 365")
    res = db.copy_from_df(he_df, "Cosponsorship")
    row = res.fetchone() or {}
    print(f"    loaded {row.get('rows_loaded', len(he_df)):,} bills "
          f"(skipped {row.get('rows_skipped', 0)})")

    first_q: dict[int, int] = {}
    last_q: dict[int, int] = {}
    for s, t in zip(simplices, times, strict=False):
        for nid in s:
            first_q[nid] = min(t, first_q.get(nid, t))
            last_q[nid] = max(t, last_q.get(nid, t))
    return first_q, last_q


def build_node_table(db, first_q, last_q, labels, tier):
    """Create + populate the Congressperson node table (with the tier label)."""
    import pandas as pd

    nodes_df = pd.DataFrame(
        {
            "node_id": list(first_q),
            "name": [labels.get(nid, f"node-{nid}")[:80] for nid in first_q],
            "first_q": [first_q[nid] for nid in first_q],
            "last_q": [last_q[nid] for nid in first_q],
            "tier": [tier.get(nid, "low") for nid in first_q],
        }
    )
    db.execute(
        "CREATE NODE TABLE Congressperson ("
        "  node_id INTEGER PRIMARY KEY, name TEXT, first_q INTEGER, last_q INTEGER, tier TEXT"
        ")"
    )
    db.copy_from_df(nodes_df, "Congressperson")
    print(f"    registered {len(nodes_df):,} legislators in node table")


# 2. Query.

def query(db, times):
    print("\n[2] QUERY — temporal Cypher")
    lo, hi = min(times), max(times)
    mid = (lo + hi) // 2
    windowed = db.execute(
        f"MATCH HYPEREDGE (he:Cosponsorship) "
        f"WHERE he.event_ts >= {mid} AND he.event_ts <= {hi} RETURN * LIMIT 1000"
    )
    print(f"    bills in the second half of the timeline (capped at 1000): "
          f"{windowed.num_tuples}")

    page = db.execute(
        "MATCH HYPEREDGE (he:Cosponsorship) RETURN * SKIP 5 LIMIT 3"
    )
    print(f"    pagination (SKIP 5 LIMIT 3) returned {page.num_tuples} rows")


# 3. Analytics.

def analytics(db, labels):
    import numpy as np

    print("\n[3] ANALYTICS — structure of the cosponsorship hypergraph")
    an = db.analytics("Cosponsorship")
    print(f"    incidence density : {an.density():.5f}")
    print(f"    spectral gap (lambda2-lambda1): {an.spectral_gap():.4f}")

    degree = an.node_degree()                      # {node_id: #bills}
    pr = an.pagerank()                             # {node_id: influence}
    top = sorted(pr.items(), key=lambda kv: kv[1], reverse=True)[:10]
    print("    top-10 most influential legislators (PageRank):")
    for nid, score in top:
        print(f"      {score:.4f}  {labels.get(nid, nid)}")

    try:
        communities = an.zhou_clustering()         # {node_id: community}
        n_comm = len(set(communities.values()))
        print(f"    spectral communities detected: {n_comm}")
    except Exception as exc:  # pragma: no cover - optional/most graphs
        print(f"    community detection skipped: {exc}")

    # Derive a reproducible supervised label: top-quartile degree -> "high".
    cutoff = float(np.quantile(list(degree.values()), 0.75))
    tier = {nid: ("high" if d >= cutoff else "low") for nid, d in degree.items()}
    print(f"    labelled influence tier (degree >= {cutoff:.0f} -> 'high')")
    return degree, tier


# 4. Interop / visualisation.

def interop(db, out_dir: Path):
    print("\n[4] INTEROP — export for visualisation")
    import hypermesh as hm

    try:
        import networkx  # noqa: F401
    except ImportError:
        print("    networkx not installed — skipping (pip install 'hypermesh[interop]')")
        return
    hg = db.to_hypergraph("Cosponsorship")
    g = hm.interop.to_networkx(hg, kind="clique")
    print(f"    clique projection: {g.number_of_nodes()} nodes, "
          f"{g.number_of_edges()} edges")
    graphml = out_dir / "congress.graphml"
    hm.interop.to_graphml(hg, str(graphml), kind="clique")
    print(f"    wrote {graphml}  (open in Gephi / Cytoscape)")


# 5. Modeling — owned HGNN.

def modeling(db, epochs: int):
    print("\n[5] MODELING — train an owned hypergraph neural network")
    try:
        import torch  # noqa: F401
    except ImportError:
        print("    torch not installed — skipping (pip install 'hypermesh[ml]')")
        return None
    import hypermesh as hm

    # Predict influence tier from tenure features + co-sponsorship structure
    # (tenure is non-leaky w.r.t. the degree-derived label).
    fhg = hm.nn.featurize(
        db, "Cosponsorship",
        node_table="Congressperson",
        node_features=["first_q", "last_q"],
        label="tier",
    )
    print(f"    featurized: X={fhg.X.shape}, classes={fhg.classes_}")

    tensors = hm.nn.prepare(db, "Cosponsorship", framework="torch",
                            node_table="Congressperson",
                            node_features=["first_q", "last_q"], label="tier")
    print(f"    torch tensors: {sorted(tensors)}")

    model = hm.nn.fit(fhg, epochs=epochs, seed=0)
    y = fhg.y
    print(f"    HGNN accuracy — train {model.evaluate(y, 'train')['accuracy']:.3f}, "
          f"val {model.evaluate(y, 'val')['accuracy']:.3f}")
    print(f"    node embeddings: {model.embed().shape}")
    return model


# 6. Temporal — reservoir / LSM.

def temporal(db, simplices, times, tier):
    print("\n[6] TEMPORAL — reservoir computing (LSM) over activity trajectories")
    import numpy as np

    import hypermesh as hm

    # Graph-level per-year activity sequence (the provided bridge).
    seq = hm.nn.temporal_features(db, "Cosponsorship", window_seconds=365)
    print(f"    graph-level temporal feature sequence: {seq.shape}")

    # Per-legislator yearly activity trajectories, labelled by influence tier.
    lo, hi = min(times), max(times)
    n_windows = max(1, (hi - lo) // 365 + 1)
    activity: dict[int, np.ndarray] = defaultdict(lambda: np.zeros(n_windows))
    for s, t in zip(simplices, times, strict=False):
        w = min((t - lo) // 365, n_windows - 1)
        for nid in s:
            activity[nid][w] += 1.0

    seqs, labels = [], []
    for nid, traj in activity.items():
        seqs.append(traj.reshape(-1, 1))
        labels.append(tier.get(nid, "low"))
    if len(set(labels)) < 2:
        print("    only one tier present — skipping reservoir classification")
        return
    clf = hm.nn.ReservoirClassifier(n_reservoir=128, seed=0)
    clf.fit(seqs, labels)
    print(f"    reservoir classifies influence tier from activity trajectories: "
          f"train accuracy {clf.score(seqs, labels):.3f}")


# Orchestration.

def main() -> int:
    ap = argparse.ArgumentParser(description=__doc__)
    ap.add_argument("--data-dir", default=f"./data/{NAME}")
    ap.add_argument("--db-dir", default=None, help="DB directory (default: temp)")
    ap.add_argument("--limit", type=int, default=40000,
                    help="cap number of bills for a faster run (0 = all)")
    ap.add_argument("--epochs", type=int, default=150)
    ap.add_argument("--no-download", action="store_true")
    args = ap.parse_args()

    import hypermesh as hm
    print(f"HyperMesh {hm.__version__} — congress-bills case study")

    data_dir = Path(args.data_dir)
    if not args.no_download:
        _download(data_dir)
    simplices, times, labels = load_simplices(
        data_dir, None if args.limit == 0 else args.limit
    )
    print(f"  {len(simplices):,} bills, {len({n for s in simplices for n in s}):,} "
          f"legislators, days {min(times)}-{max(times)}")

    db_dir = args.db_dir or tempfile.mkdtemp(prefix="hm-congress-")
    out_dir = Path(db_dir)
    db = hm.connect(db_dir)
    try:
        first_q, last_q = ingest(db, simplices, times)
        query(db, times)
        _degree, tier = analytics(db, labels)
        build_node_table(db, first_q, last_q, labels, tier)
        interop(db, out_dir)
        modeling(db, args.epochs)
        temporal(db, simplices, times, tier)
    finally:
        db.close()
    print("\nDone. The same APIs scale to the full dataset (--limit 0).")
    return 0


if __name__ == "__main__":
    sys.exit(main())