Research · Public overview

T-JEPACollapse dynamics

A research project on how tabular self-supervised systems lose and recover row-specific signal.

Role
Independent research & implementation
Setting
Tabular self-supervised learning
Focus
Collapse and escape dynamics
01 · The question

What changes when the loss looks healthy but the representation does not?

I started from an open implementation of T-JEPA, a tabular Joint-Embedding Predictive Architecture. The study began by testing a register-token explanation for collapse, then moved below the aggregate loss to examine the information carried across rows.

The central idea was simple: make the suspected failure mode measurable before changing the model, then use controlled comparisons to decide which explanation remains credible.

02 · The approach

Instrument the mechanism, then let the controls decide.

  1. 01 · Observe

    Separate the signal hidden by the loss.

    I built fixed-task diagnostics to examine training dynamics, representation geometry, and the information captured by downstream probes.

  2. 02 · Intervene

    Change one part of the objective.

    I tested a targeted objective modification, with checks that the baseline behavior and evaluation remained comparable.

  3. 03 · Correct

    Revise explanations when controls disagree.

    Normalization, coordinate, and stochastic-control checks changed the direction of the work when an attractive result did not survive reconstruction.

03 · Qualitative findings

A clearer account of what the intervention changed.

  1. 01 · Representation dynamics

    The register token was not required for escape in the tested setting.

    The comparison shifted the project away from treating the token as a universal explanation and toward measuring the transition itself.

  2. 02 · Transfer behavior

    Earlier transition did not guarantee better final quality.

    The tested modification moved the observed transition earlier across the recorded transfer settings, while downstream quality remained dependent on dataset geometry and configuration.

  3. 03 · Scientific practice

    Geometry alone was not enough.

    Negative controls and corrected experimental setups mattered as much as positive curves. The final interpretation keeps timing, geometry, and representation quality separate.

04 · Contribution

A research software and analysis project.

I directed the experimental questions, mechanism design, controls, run comparisons, and interpretation. I extended the open implementation with diagnostics and objective experiments, then documented the corrections that bounded the claims.

The full methods, figures, benchmark records, and report are available to invited reviewers.

05 · Reviewer access

The full research package is available for invited review.

Reviewer access includes the detailed technical report, experiment figures, protocol manifests, and supporting records.

06 · Starting points

An open method, extended and tested.