Research · Public overview
T-JEPACollapse dynamics
A research project on how tabular self-supervised systems lose and recover row-specific signal.
- Role
- Independent research & implementation
- Setting
- Tabular self-supervised learning
- Focus
- Collapse and escape dynamics
What changes when the loss looks healthy but the representation does not?
I started from an open implementation of T-JEPA, a tabular Joint-Embedding Predictive Architecture. The study began by testing a register-token explanation for collapse, then moved below the aggregate loss to examine the information carried across rows.
The central idea was simple: make the suspected failure mode measurable before changing the model, then use controlled comparisons to decide which explanation remains credible.
Instrument the mechanism, then let the controls decide.
- 01 · Observe
Separate the signal hidden by the loss.
I built fixed-task diagnostics to examine training dynamics, representation geometry, and the information captured by downstream probes.
- 02 · Intervene
Change one part of the objective.
I tested a targeted objective modification, with checks that the baseline behavior and evaluation remained comparable.
- 03 · Correct
Revise explanations when controls disagree.
Normalization, coordinate, and stochastic-control checks changed the direction of the work when an attractive result did not survive reconstruction.
A clearer account of what the intervention changed.
- 01 · Representation dynamics
The register token was not required for escape in the tested setting.
The comparison shifted the project away from treating the token as a universal explanation and toward measuring the transition itself.
- 02 · Transfer behavior
Earlier transition did not guarantee better final quality.
The tested modification moved the observed transition earlier across the recorded transfer settings, while downstream quality remained dependent on dataset geometry and configuration.
- 03 · Scientific practice
Geometry alone was not enough.
Negative controls and corrected experimental setups mattered as much as positive curves. The final interpretation keeps timing, geometry, and representation quality separate.
A research software and analysis project.
I directed the experimental questions, mechanism design, controls, run comparisons, and interpretation. I extended the open implementation with diagnostics and objective experiments, then documented the corrections that bounded the claims.
The full methods, figures, benchmark records, and report are available to invited reviewers.
The full research package is available for invited review.
Reviewer access includes the detailed technical report, experiment figures, protocol manifests, and supporting records.
An open method, extended and tested.
- T-JEPA: Augmentation-Free Self-Supervised Learning for Tabular DataUpstream method and paper by Hugo Thimonier and collaborators.
- Upstream implementation on GitHubOpen-source codebase extended and instrumented for this investigation.
