On the fraud benchmark NVIDIA uses, the best score comes from the smallest model: Neospace's Cortex — ahead of NVIDIA's and Revolut's far larger ones

The models that set the bar

Cortex was measured against the work that defined this problem. Three reference points set the bar, all on the same public benchmark.

The score is the detector

Cortex reads each cardholder's raw transaction sequence and produces one fraud score per transaction. That score already carries the behavioral context that drives fraud: recency, merchant patterns, spending rhythm.

On the IBM TabFormer test — held out by time (train 1991–2017, validation 2018, test 2019–2020, 2.03M transactions, 2,068 fraud, 0.10%) — the Cortex score reaches AUPRC 0.99 and F1 0.96, versus 0.14 for an XGBoost over the dataset's 13 raw columns, 0.83 for PRAGMA-M, and 0.18 for NVIDIA's Transaction Foundation Model. The dataset and the target are held fixed; only the model changes — and Cortex's model is ~8M parameters, against 29M for NVIDIA's and 100M for PRAGMA-M.

The other models hand an embedding to a separate downstream classifier. With Cortex the score itself is the detector, so there is no second model to fuse features or maintain. Across every comparison on this page the dataset, the target, and the time-isolated protocol stay fixed, and only the feature changes.

Results on the full time-isolated test

The test set is the whole held-out split, isolated by time. We train on 1991 to 2017, validate on 2018, and test on 2019 to 2020: 2,034,720 transactions, of which 2,068 are fraud (0.102%) — not a sample. The decision threshold is set on a pre-2019 validation split and applied frozen to the test years. We report AUPRC and F1 only, because at this prevalence AUROC saturates near 1.0 and stops telling strong models apart.

The readout is how each model's output becomes a fraud prediction. NVIDIA's TFM fuses its embedding with the raw columns; its embedding alone falls below the raw baseline. PRAGMA-M appears twice — its embedding fused with the raw columns into XGBoost, and a LoRA fine-tune with a direct fraud head. Cortex's score is reported by itself. Raw, NVIDIA, and Cortex are scored on the identical 2.03M-transaction rows; NVIDIA's embedding fused with the raw columns lifts the raw baseline from 0.14 to 0.18. PRAGMA-M's fine-tune is scored on every fraud plus a sample of legitimate transactions and reweighted to the true 0.1% rate, since it labels one transaction per pass.

The Cortex fraud score is a standalone detector — AUPRC 0.99, F1 0.96, about 7× the raw baseline and far above PRAGMA-M (0.83) and NVIDIA's TFM (0.18). It is a calibrated per-transaction probability, fed straight to the metric with no downstream classifier. And it reaches this with the smallest model by a wide margin — ~8M parameters versus NVIDIA's 29M and PRAGMA-M's 100M: the result is the representation, not scale.

AUPRC is the cleaner axis at this prevalence (threshold-free). All metrics are shown rounded up to two decimals (Cortex's exact AUPRC is 0.989), except the raw-feature baseline, which is rounded to nearest so the bar everyone clears isn't flattered (0.142 → 0.14); the underlying figures are read from the committed results files.

Model sizes — Cortex is the smallest

All three models were run on the same IBM TabFormer dataset. Cortex delivers the strongest fraud detection while being ~4× smaller than NVIDIA's model and ~12× smaller than PRAGMA-M, and trains in well under half PRAGMA-M's time.

Training time is measured on 4×GB200, across pre-training and fine-tuning, for the two models we trained here; NVIDIA's blueprint ships a checkpoint, so its training was not re-run.

How this compares to Revolut's PRAGMA

PRAGMA is Revolut's transaction foundation model (arXiv:2604.08649): a 100M-parameter, encoder-only transformer trained with a self-supervised masked-modelling objective. We pretrained PRAGMA-M on the same 1991–2017 training data as Cortex and read it out two ways. Its frozen embedding fused with the raw columns into XGBoost reaches AUPRC 0.47 / F1 0.60; adapting it for fraud with LoRA — the fine-tuning recipe from the PRAGMA paper — and a direct head reaches 0.83 / 0.81. Both clear the raw baseline but fall short of Cortex's 0.99 / 0.96, at about 12× the size.

Fraud usually surfaces in combinations of fields: an amount, at a merchant type, hour, and location, taken together. PRAGMA has no direct feature interaction; it relates fields only implicitly, through general-purpose attention. Cortex models those interactions directly. This benchmark also labels every transaction, and PRAGMA's attention is bidirectional, so only the transaction at a window's end can be scored without seeing its own future — one per pass. We therefore score the fine-tune on every fraud plus a sample of legitimate transactions and reweight to the true 0.1% rate, recovering the metric PRAGMA would reach across the whole set; Cortex scores a whole window in one pass. PRAGMA-M is the 100M-parameter variant from the paper, which we pretrained and fine-tuned on IBM TabFormer ourselves (Ostroukhov et al., PRAGMA: Revolut Foundation Model).

Compared with NVIDIA's Transaction Foundation Model

NeoLDM grew up alongside NVIDIA's open Transaction Foundation Model developer blueprint — same dataset (IBM TabFormer), same idea of training a model on raw transactions and using its output for downstream fraud. NVIDIA's blueprint is a 29M-parameter Llama decoder; it pools the last-token hidden state into a 512-d embedding, reduces it to 64-d with PCA, and feeds XGBoost.

NVIDIA's repo ships no metrics in its notebook outputs, so we ran their blueprint end-to-end ourselves — their shipped checkpoint, their notebooks, the real IBM TabFormer data. Their own notebook calls Average Precision (AUPRC) “the operationally critical metric for fraud,” so we lead with it. On the full time-isolated 2019–2020 test (the same rows as Cortex), fusing the foundation-model embedding with the raw columns improves their fraud detection: the raw-feature baseline scores AUPRC 0.14, the embedding alone 0.01, and the two fused reach 0.18 — the combined number we report for NVIDIA.

Cortex's fraud score reaches 0.99 on the full time-isolated test — more than 5× higher, with a model ~4× smaller.

NVIDIA publishes no absolute numbers, so we ran their blueprint ourselves on the full test split — the same rows as Cortex and the raw baseline. The ranking: Cortex (0.99) far ahead, then Revolut's PRAGMA-M (0.83), then NVIDIA's embedding plus raw (0.18), just above the raw baseline (0.14).

What the result means in practice

The result is not only academic. Each advantage maps to something practical.

Reproduce the results

The notebook computes the Cortex score from the published per-transaction scores. Without them it falls back to the committed results, so it renders either way. You do not need a GPU, the dataset, or a checkpoint just to see the numbers.

Dataset & scores

The raw baseline feeds the transaction table's columns to XGBoost directly — no temporal aggregation, no engineered features — so the comparison is clean: a learned Cortex representation versus the raw fields. For reference, the original TabFormer paper reports its best fraud-detection result at F1 ≈ 0.86 (TabBERT features plus an LSTM). That number is not comparable to the results here: the paper predicts over a window of 10 transactions (a fraud window every 10 rows, so roughly 10× higher effective prevalence), on its own windowed split — an easier framing than per-transaction detection at 0.1%. It is a historical reference point on this dataset, not a like-for-like baseline.

References

• NVIDIA Transaction Foundation Model — developer blueprint (decoder-only LM on tokenized transactions, last-token embedding → XGBoost): github.com/NVIDIA-AI-Blueprints/transaction-foundation-model.
• Revolut PRAGMA — Ostroukhov et al. PRAGMA: Revolut Foundation Model. arXiv:2604.08649.
• IBM TabFormer — Padhi, Schiff, Melnyk, Rigotti, Mroueh, Dognin, Ross, Nair, Altman. Tabular Transformers for Modeling Multivariate Time Series. ICASSP 2021. arXiv:2011.01843.
• NV-Embed — Lee et al. NV-Embed: Improved Techniques for Training LLMs as Generalist Embedding Models. arXiv:2405.17428 — the last-token-pooling recipe that turns a decoder's hidden state into a usable embedding.