# AGENTS.md
The SelectArena leaderboard: a static Hugging Face Space (`sdk: static`) that serves files
generated in the [SelectArena](https://github.com/schaeferbasti/SelectArena) repo. Static Spaces
run no compute, so they are free; do not switch it to Gradio or Docker (those need a paid plan).
## Files
- `index.html`: the whole page: header (on screens wider than 640px it stays at the top together with
the tabs, see `initStickyHeader` in `app.js`), the four tab panels (Leaderboard, which opens first: the
Elo figure above the table; Performance: Performance-Time Trade-Off (Pareto front), performance per dataset characteristic, model, task type and budget, Budget- and Time-constraint Ablations and complete
results; Stability;
Validity),
all user-facing text (each explanation sits next to the figure or table it describes; there is no
About tab; the Paper and GitHub links are the header buttons), the citation (always visible
in the footer) and `` (not displayed; there is no version
history, the commit messages are the changelog).
- `assets/app.js`: plain JavaScript, no build step. Fetches `data/meta.json`, renders the selector
chips (task type, downstream model, budget; combinations that are not published are disabled;
shared by the Leaderboard and Performance Trade-offs tabs; task type, view and noise level of
the lasagna plots; each tab's chips sit in a collapsed `` whose "Customize"
button shows the current choices), sets the iframes' `src` and the figures'
`
` to the generated files and resizes the iframes when a page posts `{saFrameHeight}`.
Keeps the tab and the selections in the URL hash (tab keys `leaderboard`, `tradeoffs`,
`stability`, `validity`; old names such as `summary`, `performance`, `data` and `about` still work).
- `assets/downloads.js`: SVG / PNG / PDF downloads of the figures drawn in `index.html` (the
`.fig-dl` buttons of the 22-hour and Lasso/ElasticNet ablation figures, and of Figure D.3), in the
light paper look, and the PDFs of the generated figure pages, which send their SVG (message
`{saPdf}`, answered by `{saPdfAck}`). The PDFs are vector PDFs made with jsPDF and svg2pdf.js, with
the text in SelectArena Sans (a renamed subset of Liberation Sans) embedded; these files in
`assets/vendor/` (see its README) are loaded at the first PDF download only. Where they cannot be
loaded, or a figure page has no leaderboard around it, the PDF holds the figure rendered at 4x.
- `assets/style.css`: color tokens on `:root` (light) and for dark mode, which follows the system
setting unless the theme button (after "Paper", "Cite" and "Add your method") set `data-theme` on ``. The choice is kept in
`localStorage` (`selectarena-theme`) and passed on to the embedded pages, whose CSS
(SelectArena's `website/html.py`) honours the same `data-theme` attribute.
- `smoke_test.py`: standard-library check of `data/` and of the version tag (see below).
## Data contract
`data/` is SelectArena's `generated_website_artifacts/clean_website_artifacts/website_data/`:
- `meta.json`: counts, `models` (key -> name), `budgets`, `total_budget`, `task_types`,
`has_stability`, `has_validity`, `validity_noise`, `validity_tables`, `generated_at`, `subsets` (each with
`tasks`, `model`, `budget`, `path`, `n_methods`, `n_datasets`, `n_comparisons`, `pareto`),
`lasagna` (`variants`, `variant_labels`, and when present `stability` = `{path, task_types}` and
`validity` = a list of `{noise, path, task_types}`), `analysis` (`path`, `groupings` = a list of
`{key, label}`, `improvement_by_model`, `improvement_by_task_type`, `rank_by_size`, `elo_without_timeouts`)
and `has_results_explorer`.
- `subsets/tasks_/models_/budgets_/`: `website_leaderboard.csv`, `leaderboard_table.html`,
`elo_explorer.html`, and `pareto.html` / `pareto.csv` when the subset's `pareto` is true.
- `lasagna/stability/tasks_/` and `lasagna/validity/noise_/tasks_/`: one
`.html` per entry of `lasagna.variants` and `per_epv.csv` (the plotted values).
- `analysis/`: the figures of the Performance tab's sections per dataset characteristic, model, task type and budget and of its Ablations (all results, not
filtered by the chips): `heatmap_.html` per entry of `analysis.groupings` (chips "Group by",
hash key `group`), `improvement_by_model.html`, `improvement_by_task_type.html` and
`rank_by_size.html` when listed (`heatmap_budget.html` is also shown as a figure of its own), `elo_no_timeouts.html` (Elo on the datasets where no method ran out of time next to the default Elo, as in the paper's Figure D.3; leaderboard of those datasets in `leaderboard_no_timeouts.csv`) when listed, and their CSV files.
- `results.csv` and `results_explorer.html`: the results per dataset, method, downstream model and
budget (Performance Trade-offs tab, below the Pareto front; not filtered by the chips).
- `per_dataset.csv` (mean rank per dataset; data only, not shown), and optionally `stability.csv`,
`stability_table.html`, `validity.csv` and one `validity_noise__table.html` per noise level
(listed in `meta.json` as `validity_tables`).
- `website_leaderboard.csv` columns: `Rank`, `Method`, `Category` (Search / Non-search / Baseline),
`Subgroup` (the paper's taxonomy, e.g. Tree, Backward search), Elo and its interval, the metrics.
The generated pages are self-contained (inline CSS/JS, no network) and post their height to the
parent window. All figures (Elo, Pareto front, lasagna plots) are such pages with an inline SVG:
tooltips on hover or tap, colours that follow the page's theme, and SVG / PNG / PDF / CSV buttons
whose downloads always use the light paper look.
`subsetDir()` in `app.js` mirrors `selectarena.website.subsets.Subset.rel_path`; change both
together.
## Conventions
- Never hand-edit `data/`: refresh it with the `update-leaderboard` skill in the SelectArena repo
(`../SelectArena/.claude/skills/update-leaderboard/`), which deletes the old content before copying.
- Every refresh bumps `selectarena-version` in `index.html` (patch for a data refresh, minor for
site changes) and says what changed in the commit message. `smoke_test.py` checks the tag exists.
- No external scripts, fonts or CDNs; everything is served from this repo.
- Verification: `python smoke_test.py`, then `python -m http.server 8000` and a look at
http://localhost:8000 (all four tabs, a few selections, light and dark mode).
- Do not commit or push without an explicit request: pushing deploys the Space.