Smolnikov Research: DLRMv4 and Yambda temporal evaluation
Published study: https://smolnikovresearch.ru/publications/dlrmv4-yambda-temporal-evaluation/
Study execution: 2026-11-06. Research note, no external peer review.

WHAT THIS IS
A new deterministic baseline experiment on 881456 like events from Yambda-50M, and a separate secondary analysis of 60 public MLCommons RCP logs. No HSTU training, MLPerf submission, CapyAgent measurement, paid APIs, private data, or cross-vendor benchmark.

REPRODUCE
Use a research machine with Python 3.14, >=1.5 GiB available memory and network access. Code runs a single DuckDB thread, 700 MB DuckDB memory limit. Run:
  bash reproduce.sh
This creates an isolated venv, downloads public pinned data (~7.2 MB) and logs (~21 MB), verifies their hashes, evaluates, independently verifies, runs exact-duplicate sensitivity and creates plots. dependencies have been pinned from the execution environment. Package availability on other Python versions/OS is not promised.

The original execution used Fedora Linux, Python 3.14.8, one-CPU systemd scope with MemoryMax=1500M and MemorySwapMax=0, a shared machine lock, nice 19 and timeouts. GPU runtime in the article belongs to upstream RCP logs, NOT this CPU run.

FILES
protocol.txt: fixed before primary evaluation, local protocol (not externally preregistered).
evaluate.py: DuckDB evaluation and MLLOG analysis.
verify.py: independent Python-counter metrics and RCP threshold checks.
sensitivity.py: post-hoc exact duplicate removal, with original evaluator in an isolated directory.
plots.py: plot generation from recorded aggregate outputs.
fetch_logs.py, rcp-files.json: pinned upstream inputs verified using Git blob hashes.
requirements.txt, reproduce.sh: dependencies and commands.
results/: aggregate outputs, source hashes, environment, graphs, verification.

IMPORTANT INTERPRETATION
60/70/80% are fractions of the observed timestamp span, NOT training-event shares. A user with prior and future likes contributes the set of tracks at their first future timestamp. All eligible users are evaluated; unseen targets count as misses. Repeated recommendations are permitted. No random negative sampling. The future_control uses future events intentionally and is invalid for deployment. Global lists are not personalized. The three temporal cohorts overlap.

Exact duplicates are retained in the primary experiment. Deduplication is separately labelled post-hoc sensitivity, not proof that duplicates are erroneous. Organic flags define observational groups, not causal treatment.

GPU-minutes are upstream GPU counts times elapsed minutes; they are not energy or monetary cost. RCPs do not contain run_stop by upstream design. No official MLPerf result is claimed.

SOURCES AND RIGHTS
Yandex/Yambda: https://huggingface.co/datasets/yandex/yambda/tree/dd6f3a19eef5866e346c3270e098baa641a44948
Yambda paper: https://arxiv.org/abs/2505.22238v2
MLCommons: https://github.com/mlcommons/training/tree/e309244f7f81ecb00da434cae98355a73fc50ff0/recommendation
HSTU paper: https://arxiv.org/abs/2402.17152
Original analysis code: MIT (LICENSE.txt).
Yambda LICENSE: Apache 2.0; the dataset card also states scientific/research purposes. This package uses it for research. No raw user-event files are redistributed. Retain original attribution and review original terms for another use.
MLCommons recommendation code/log repository: Apache 2.0. This package fetches public log evidence; it does not redistribute or execute training code. Upstream license texts included for attribution.
