# AGENTS.md The SelectArena leaderboard: a static Hugging Face Space (`sdk: static`) that serves files generated in the [SelectArena](https://github.com/schaeferbasti/SelectArena) repo. Static Spaces run no compute, so they are free; do not switch it to Gradio or Docker (those need a paid plan). ## Files - `index.html`: the whole page: header (on screens wider than 640px it stays at the top together with the tabs, see `initStickyHeader` in `app.js`), the four tab panels (Leaderboard, which opens first: the Elo figure above the table; Performance: Performance-Time Trade-Off (Pareto front), performance per dataset characteristic, model, task type and budget, Budget- and Time-constraint Ablations and complete results; Stability; Validity), all user-facing text (each explanation sits next to the figure or table it describes; there is no About tab; the Paper and GitHub links are the header buttons), the citation (always visible in the footer) and `` (not displayed; there is no version history, the commit messages are the changelog). - `assets/app.js`: plain JavaScript, no build step. Fetches `data/meta.json`, renders the selector chips (task type, downstream model, budget; combinations that are not published are disabled; shared by the Leaderboard and Performance Trade-offs tabs; task type, view and noise level of the lasagna plots; each tab's chips sit in a collapsed `
` whose "Customize" button shows the current choices), sets the iframes' `src` and the figures' `` to the generated files and resizes the iframes when a page posts `{saFrameHeight}`. Keeps the tab and the selections in the URL hash (tab keys `leaderboard`, `tradeoffs`, `stability`, `validity`; old names such as `summary`, `performance`, `data` and `about` still work). - `assets/downloads.js`: SVG / PNG / PDF downloads of the figures drawn in `index.html` (the `.fig-dl` buttons of the 22-hour and Lasso/ElasticNet ablation figures, and of Figure D.3), in the light paper look, and the PDFs of the generated figure pages, which send their SVG (message `{saPdf}`, answered by `{saPdfAck}`). The PDFs are vector PDFs made with jsPDF and svg2pdf.js, with the text in SelectArena Sans (a renamed subset of Liberation Sans) embedded; these files in `assets/vendor/` (see its README) are loaded at the first PDF download only. Where they cannot be loaded, or a figure page has no leaderboard around it, the PDF holds the figure rendered at 4x. - `assets/style.css`: color tokens on `:root` (light) and for dark mode, which follows the system setting unless the theme button (after "Paper", "Cite" and "Add your method") set `data-theme` on ``. The choice is kept in `localStorage` (`selectarena-theme`) and passed on to the embedded pages, whose CSS (SelectArena's `website/html.py`) honours the same `data-theme` attribute. - `smoke_test.py`: standard-library check of `data/` and of the version tag (see below). ## Data contract `data/` is SelectArena's `generated_website_artifacts/clean_website_artifacts/website_data/`: - `meta.json`: counts, `models` (key -> name), `budgets`, `total_budget`, `task_types`, `has_stability`, `has_validity`, `validity_noise`, `validity_tables`, `generated_at`, `subsets` (each with `tasks`, `model`, `budget`, `path`, `n_methods`, `n_datasets`, `n_comparisons`, `pareto`), `lasagna` (`variants`, `variant_labels`, and when present `stability` = `{path, task_types}` and `validity` = a list of `{noise, path, task_types}`), `analysis` (`path`, `groupings` = a list of `{key, label}`, `improvement_by_model`, `improvement_by_task_type`, `rank_by_size`, `elo_without_timeouts`) and `has_results_explorer`. - `subsets/tasks_/models_/budgets_/`: `website_leaderboard.csv`, `leaderboard_table.html`, `elo_explorer.html`, and `pareto.html` / `pareto.csv` when the subset's `pareto` is true. - `lasagna/stability/tasks_/` and `lasagna/validity/noise_/tasks_/`: one `.html` per entry of `lasagna.variants` and `per_epv.csv` (the plotted values). - `analysis/`: the figures of the Performance tab's sections per dataset characteristic, model, task type and budget and of its Ablations (all results, not filtered by the chips): `heatmap_.html` per entry of `analysis.groupings` (chips "Group by", hash key `group`), `improvement_by_model.html`, `improvement_by_task_type.html` and `rank_by_size.html` when listed (`heatmap_budget.html` is also shown as a figure of its own), `elo_no_timeouts.html` (Elo on the datasets where no method ran out of time next to the default Elo, as in the paper's Figure D.3; leaderboard of those datasets in `leaderboard_no_timeouts.csv`) when listed, and their CSV files. - `results.csv` and `results_explorer.html`: the results per dataset, method, downstream model and budget (Performance Trade-offs tab, below the Pareto front; not filtered by the chips). - `per_dataset.csv` (mean rank per dataset; data only, not shown), and optionally `stability.csv`, `stability_table.html`, `validity.csv` and one `validity_noise__table.html` per noise level (listed in `meta.json` as `validity_tables`). - `website_leaderboard.csv` columns: `Rank`, `Method`, `Category` (Search / Non-search / Baseline), `Subgroup` (the paper's taxonomy, e.g. Tree, Backward search), Elo and its interval, the metrics. The generated pages are self-contained (inline CSS/JS, no network) and post their height to the parent window. All figures (Elo, Pareto front, lasagna plots) are such pages with an inline SVG: tooltips on hover or tap, colours that follow the page's theme, and SVG / PNG / PDF / CSV buttons whose downloads always use the light paper look. `subsetDir()` in `app.js` mirrors `selectarena.website.subsets.Subset.rel_path`; change both together. ## Conventions - Never hand-edit `data/`: refresh it with the `update-leaderboard` skill in the SelectArena repo (`../SelectArena/.claude/skills/update-leaderboard/`), which deletes the old content before copying. - Every refresh bumps `selectarena-version` in `index.html` (patch for a data refresh, minor for site changes) and says what changed in the commit message. `smoke_test.py` checks the tag exists. - No external scripts, fonts or CDNs; everything is served from this repo. - Verification: `python smoke_test.py`, then `python -m http.server 8000` and a look at http://localhost:8000 (all four tabs, a few selections, light and dark mode). - Do not commit or push without an explicit request: pushing deploys the Space.