Spaces:
Running
Running
Commit Β·
2efb236
1
Parent(s): ed6ab15
Leaderboard v0.2.0
Browse filesThis view is limited to 50 files because it contains too many changes. Β See raw diff
- .gitignore +5 -0
- AGENTS.md +6 -3
- LICENSE +201 -0
- assets/app.js +30 -7
- assets/style.css +1 -0
- data/meta.json +15 -1
- data/per_dataset_table.html +3 -1
- data/stability_table.html +3 -1
- data/subsets/tasks_all/models_all/budgets_0/elo_explorer.html +31 -31
- data/subsets/tasks_all/models_all/budgets_0/leaderboard_table.html +19 -17
- data/subsets/tasks_all/models_all/budgets_0/website_leaderboard.csv +16 -16
- data/subsets/tasks_all/models_all/budgets_1/elo_explorer.html +31 -31
- data/subsets/tasks_all/models_all/budgets_1/leaderboard_table.html +19 -17
- data/subsets/tasks_all/models_all/budgets_1/website_leaderboard.csv +16 -16
- data/subsets/tasks_all/models_all/budgets_2/elo_explorer.html +31 -31
- data/subsets/tasks_all/models_all/budgets_2/leaderboard_table.html +19 -17
- data/subsets/tasks_all/models_all/budgets_2/website_leaderboard.csv +16 -16
- data/subsets/tasks_all/models_all/budgets_3/elo_explorer.html +31 -31
- data/subsets/tasks_all/models_all/budgets_3/leaderboard_table.html +19 -17
- data/subsets/tasks_all/models_all/budgets_3/website_leaderboard.csv +16 -16
- data/subsets/tasks_all/models_all/budgets_4/elo_explorer.html +31 -31
- data/subsets/tasks_all/models_all/budgets_4/leaderboard_table.html +19 -17
- data/subsets/tasks_all/models_all/budgets_4/website_leaderboard.csv +16 -16
- data/subsets/tasks_all/models_all/budgets_all/elo_explorer.html +31 -31
- data/subsets/tasks_all/models_all/budgets_all/leaderboard_table.html +19 -17
- data/subsets/tasks_all/models_all/budgets_all/website_leaderboard.csv +16 -16
- data/subsets/tasks_all/models_lightgbm/budgets_0/elo_explorer.html +31 -31
- data/subsets/tasks_all/models_lightgbm/budgets_0/leaderboard_table.html +19 -17
- data/subsets/tasks_all/models_lightgbm/budgets_0/website_leaderboard.csv +16 -16
- data/subsets/tasks_all/models_lightgbm/budgets_1/elo_explorer.html +31 -31
- data/subsets/tasks_all/models_lightgbm/budgets_1/leaderboard_table.html +19 -17
- data/subsets/tasks_all/models_lightgbm/budgets_1/website_leaderboard.csv +16 -16
- data/subsets/tasks_all/models_lightgbm/budgets_2/elo_explorer.html +31 -31
- data/subsets/tasks_all/models_lightgbm/budgets_2/leaderboard_table.html +19 -17
- data/subsets/tasks_all/models_lightgbm/budgets_2/website_leaderboard.csv +16 -16
- data/subsets/tasks_all/models_lightgbm/budgets_3/elo_explorer.html +31 -31
- data/subsets/tasks_all/models_lightgbm/budgets_3/leaderboard_table.html +19 -17
- data/subsets/tasks_all/models_lightgbm/budgets_3/website_leaderboard.csv +16 -16
- data/subsets/tasks_all/models_lightgbm/budgets_4/elo_explorer.html +31 -31
- data/subsets/tasks_all/models_lightgbm/budgets_4/leaderboard_table.html +19 -17
- data/subsets/tasks_all/models_lightgbm/budgets_4/website_leaderboard.csv +16 -16
- data/subsets/tasks_all/models_lightgbm/budgets_all/elo_explorer.html +31 -31
- data/subsets/tasks_all/models_lightgbm/budgets_all/leaderboard_table.html +19 -17
- data/subsets/tasks_all/models_lightgbm/budgets_all/website_leaderboard.csv +16 -16
- data/subsets/tasks_all/models_linearmodel/budgets_0/elo_explorer.html +31 -31
- data/subsets/tasks_all/models_linearmodel/budgets_0/leaderboard_table.html +19 -17
- data/subsets/tasks_all/models_linearmodel/budgets_0/website_leaderboard.csv +16 -16
- data/subsets/tasks_all/models_linearmodel/budgets_1/elo_explorer.html +31 -31
- data/subsets/tasks_all/models_linearmodel/budgets_1/leaderboard_table.html +19 -17
- data/subsets/tasks_all/models_linearmodel/budgets_1/website_leaderboard.csv +16 -16
.gitignore
ADDED
|
@@ -0,0 +1,5 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
__pycache__/
|
| 2 |
+
*.py[cod]
|
| 3 |
+
.venv/
|
| 4 |
+
.ruff_cache/
|
| 5 |
+
.DS_Store
|
AGENTS.md
CHANGED
|
@@ -21,12 +21,15 @@ run no compute, so they are free; do not switch it to Gradio or Docker (those ne
|
|
| 21 |
`data/` is SelectArena's `generated_website_artifacts/clean_website_artifacts/website_data/`:
|
| 22 |
|
| 23 |
- `meta.json`: counts, `models` (key -> name), `budgets`, `total_budget`, `task_types`,
|
| 24 |
-
`has_stability`, `has_validity`, `validity_noise`, `generated_at`, and `subsets` (each with
|
| 25 |
`tasks`, `model`, `budget`, `path`, `n_methods`, `n_datasets`, `n_comparisons`).
|
| 26 |
- `subsets/tasks_<t>/models_<m>/budgets_<b>/`: `website_leaderboard.csv`, `leaderboard_table.html`,
|
| 27 |
`elo_explorer.html`.
|
| 28 |
-
- `per_dataset.csv`, `per_dataset_table.html`, and optionally `stability.csv`,
|
| 29 |
-
`stability_table.html`, `validity.csv`
|
|
|
|
|
|
|
|
|
|
| 30 |
|
| 31 |
The generated pages are self-contained (inline CSS/JS, no network) and post their height to the
|
| 32 |
parent window. `subsetDir()` in `app.js` mirrors `selectarena.website.subsets.Subset.rel_path`;
|
|
|
|
| 21 |
`data/` is SelectArena's `generated_website_artifacts/clean_website_artifacts/website_data/`:
|
| 22 |
|
| 23 |
- `meta.json`: counts, `models` (key -> name), `budgets`, `total_budget`, `task_types`,
|
| 24 |
+
`has_stability`, `has_validity`, `validity_noise`, `validity_tables`, `generated_at`, and `subsets` (each with
|
| 25 |
`tasks`, `model`, `budget`, `path`, `n_methods`, `n_datasets`, `n_comparisons`).
|
| 26 |
- `subsets/tasks_<t>/models_<m>/budgets_<b>/`: `website_leaderboard.csv`, `leaderboard_table.html`,
|
| 27 |
`elo_explorer.html`.
|
| 28 |
+
- `per_dataset.csv`, `per_dataset_table.html` (datasets by name), and optionally `stability.csv`,
|
| 29 |
+
`stability_table.html`, `validity.csv` and one `validity_noise_<level>_table.html` per noise level
|
| 30 |
+
(listed in `meta.json` as `validity_tables`).
|
| 31 |
+
- `website_leaderboard.csv` columns: `Rank`, `Method`, `Category` (Search / Non-search / Baseline),
|
| 32 |
+
`Subgroup` (the paper's taxonomy, e.g. Tree, Backward search), Elo and its interval, the metrics.
|
| 33 |
|
| 34 |
The generated pages are self-contained (inline CSS/JS, no network) and post their height to the
|
| 35 |
parent window. `subsetDir()` in `app.js` mirrors `selectarena.website.subsets.Subset.rel_path`;
|
LICENSE
ADDED
|
@@ -0,0 +1,201 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
Apache License
|
| 2 |
+
Version 2.0, January 2004
|
| 3 |
+
http://www.apache.org/licenses/
|
| 4 |
+
|
| 5 |
+
TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
|
| 6 |
+
|
| 7 |
+
1. Definitions.
|
| 8 |
+
|
| 9 |
+
"License" shall mean the terms and conditions for use, reproduction,
|
| 10 |
+
and distribution as defined by Sections 1 through 9 of this document.
|
| 11 |
+
|
| 12 |
+
"Licensor" shall mean the copyright owner or entity authorized by
|
| 13 |
+
the copyright owner that is granting the License.
|
| 14 |
+
|
| 15 |
+
"Legal Entity" shall mean the union of the acting entity and all
|
| 16 |
+
other entities that control, are controlled by, or are under common
|
| 17 |
+
control with that entity. For the purposes of this definition,
|
| 18 |
+
"control" means (i) the power, direct or indirect, to cause the
|
| 19 |
+
direction or management of such entity, whether by contract or
|
| 20 |
+
otherwise, or (ii) ownership of fifty percent (50%) or more of the
|
| 21 |
+
outstanding shares, or (iii) beneficial ownership of such entity.
|
| 22 |
+
|
| 23 |
+
"You" (or "Your") shall mean an individual or Legal Entity
|
| 24 |
+
exercising permissions granted by this License.
|
| 25 |
+
|
| 26 |
+
"Source" form shall mean the preferred form for making modifications,
|
| 27 |
+
including but not limited to software source code, documentation
|
| 28 |
+
source, and configuration files.
|
| 29 |
+
|
| 30 |
+
"Object" form shall mean any form resulting from mechanical
|
| 31 |
+
transformation or translation of a Source form, including but
|
| 32 |
+
not limited to compiled object code, generated documentation,
|
| 33 |
+
and conversions to other media types.
|
| 34 |
+
|
| 35 |
+
"Work" shall mean the work of authorship, whether in Source or
|
| 36 |
+
Object form, made available under the License, as indicated by a
|
| 37 |
+
copyright notice that is included in or attached to the work
|
| 38 |
+
(an example is provided in the Appendix below).
|
| 39 |
+
|
| 40 |
+
"Derivative Works" shall mean any work, whether in Source or Object
|
| 41 |
+
form, that is based on (or derived from) the Work and for which the
|
| 42 |
+
editorial revisions, annotations, elaborations, or other modifications
|
| 43 |
+
represent, as a whole, an original work of authorship. For the purposes
|
| 44 |
+
of this License, Derivative Works shall not include works that remain
|
| 45 |
+
separable from, or merely link (or bind by name) to the interfaces of,
|
| 46 |
+
the Work and Derivative Works thereof.
|
| 47 |
+
|
| 48 |
+
"Contribution" shall mean any work of authorship, including
|
| 49 |
+
the original version of the Work and any modifications or additions
|
| 50 |
+
to that Work or Derivative Works thereof, that is intentionally
|
| 51 |
+
submitted to Licensor for inclusion in the Work by the copyright owner
|
| 52 |
+
or by an individual or Legal Entity authorized to submit on behalf of
|
| 53 |
+
the copyright owner. For the purposes of this definition, "submitted"
|
| 54 |
+
means any form of electronic, verbal, or written communication sent
|
| 55 |
+
to the Licensor or its representatives, including but not limited to
|
| 56 |
+
communication on electronic mailing lists, source code control systems,
|
| 57 |
+
and issue tracking systems that are managed by, or on behalf of, the
|
| 58 |
+
Licensor for the purpose of discussing and improving the Work, but
|
| 59 |
+
excluding communication that is conspicuously marked or otherwise
|
| 60 |
+
designated in writing by the copyright owner as "Not a Contribution."
|
| 61 |
+
|
| 62 |
+
"Contributor" shall mean Licensor and any individual or Legal Entity
|
| 63 |
+
on behalf of whom a Contribution has been received by Licensor and
|
| 64 |
+
subsequently incorporated within the Work.
|
| 65 |
+
|
| 66 |
+
2. Grant of Copyright License. Subject to the terms and conditions of
|
| 67 |
+
this License, each Contributor hereby grants to You a perpetual,
|
| 68 |
+
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
|
| 69 |
+
copyright license to reproduce, prepare Derivative Works of,
|
| 70 |
+
publicly display, publicly perform, sublicense, and distribute the
|
| 71 |
+
Work and such Derivative Works in Source or Object form.
|
| 72 |
+
|
| 73 |
+
3. Grant of Patent License. Subject to the terms and conditions of
|
| 74 |
+
this License, each Contributor hereby grants to You a perpetual,
|
| 75 |
+
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
|
| 76 |
+
(except as stated in this section) patent license to make, have made,
|
| 77 |
+
use, offer to sell, sell, import, and otherwise transfer the Work,
|
| 78 |
+
where such license applies only to those patent claims licensable
|
| 79 |
+
by such Contributor that are necessarily infringed by their
|
| 80 |
+
Contribution(s) alone or by combination of their Contribution(s)
|
| 81 |
+
with the Work to which such Contribution(s) was submitted. If You
|
| 82 |
+
institute patent litigation against any entity (including a
|
| 83 |
+
cross-claim or counterclaim in a lawsuit) alleging that the Work
|
| 84 |
+
or a Contribution incorporated within the Work constitutes direct
|
| 85 |
+
or contributory patent infringement, then any patent licenses
|
| 86 |
+
granted to You under this License for that Work shall terminate
|
| 87 |
+
as of the date such litigation is filed.
|
| 88 |
+
|
| 89 |
+
4. Redistribution. You may reproduce and distribute copies of the
|
| 90 |
+
Work or Derivative Works thereof in any medium, with or without
|
| 91 |
+
modifications, and in Source or Object form, provided that You
|
| 92 |
+
meet the following conditions:
|
| 93 |
+
|
| 94 |
+
(a) You must give any other recipients of the Work or
|
| 95 |
+
Derivative Works a copy of this License; and
|
| 96 |
+
|
| 97 |
+
(b) You must cause any modified files to carry prominent notices
|
| 98 |
+
stating that You changed the files; and
|
| 99 |
+
|
| 100 |
+
(c) You must retain, in the Source form of any Derivative Works
|
| 101 |
+
that You distribute, all copyright, patent, trademark, and
|
| 102 |
+
attribution notices from the Source form of the Work,
|
| 103 |
+
excluding those notices that do not pertain to any part of
|
| 104 |
+
the Derivative Works; and
|
| 105 |
+
|
| 106 |
+
(d) If the Work includes a "NOTICE" text file as part of its
|
| 107 |
+
distribution, then any Derivative Works that You distribute must
|
| 108 |
+
include a readable copy of the attribution notices contained
|
| 109 |
+
within such NOTICE file, excluding those notices that do not
|
| 110 |
+
pertain to any part of the Derivative Works, in at least one
|
| 111 |
+
of the following places: within a NOTICE text file distributed
|
| 112 |
+
as part of the Derivative Works; within the Source form or
|
| 113 |
+
documentation, if provided along with the Derivative Works; or,
|
| 114 |
+
within a display generated by the Derivative Works, if and
|
| 115 |
+
wherever such third-party notices normally appear. The contents
|
| 116 |
+
of the NOTICE file are for informational purposes only and
|
| 117 |
+
do not modify the License. You may add Your own attribution
|
| 118 |
+
notices within Derivative Works that You distribute, alongside
|
| 119 |
+
or as an addendum to the NOTICE text from the Work, provided
|
| 120 |
+
that such additional attribution notices cannot be construed
|
| 121 |
+
as modifying the License.
|
| 122 |
+
|
| 123 |
+
You may add Your own copyright statement to Your modifications and
|
| 124 |
+
may provide additional or different license terms and conditions
|
| 125 |
+
for use, reproduction, or distribution of Your modifications, or
|
| 126 |
+
for any such Derivative Works as a whole, provided Your use,
|
| 127 |
+
reproduction, and distribution of the Work otherwise complies with
|
| 128 |
+
the conditions stated in this License.
|
| 129 |
+
|
| 130 |
+
5. Submission of Contributions. Unless You explicitly state otherwise,
|
| 131 |
+
any Contribution intentionally submitted for inclusion in the Work
|
| 132 |
+
by You to the Licensor shall be under the terms and conditions of
|
| 133 |
+
this License, without any additional terms or conditions.
|
| 134 |
+
Notwithstanding the above, nothing herein shall supersede or modify
|
| 135 |
+
the terms of any separate license agreement you may have executed
|
| 136 |
+
with Licensor regarding such Contributions.
|
| 137 |
+
|
| 138 |
+
6. Trademarks. This License does not grant permission to use the trade
|
| 139 |
+
names, trademarks, service marks, or product names of the Licensor,
|
| 140 |
+
except as required for reasonable and customary use in describing the
|
| 141 |
+
origin of the Work and reproducing the content of the NOTICE file.
|
| 142 |
+
|
| 143 |
+
7. Disclaimer of Warranty. Unless required by applicable law or
|
| 144 |
+
agreed to in writing, Licensor provides the Work (and each
|
| 145 |
+
Contributor provides its Contributions) on an "AS IS" BASIS,
|
| 146 |
+
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
|
| 147 |
+
implied, including, without limitation, any warranties or conditions
|
| 148 |
+
of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
|
| 149 |
+
PARTICULAR PURPOSE. You are solely responsible for determining the
|
| 150 |
+
appropriateness of using or redistributing the Work and assume any
|
| 151 |
+
risks associated with Your exercise of permissions under this License.
|
| 152 |
+
|
| 153 |
+
8. Limitation of Liability. In no event and under no legal theory,
|
| 154 |
+
whether in tort (including negligence), contract, or otherwise,
|
| 155 |
+
unless required by applicable law (such as deliberate and grossly
|
| 156 |
+
negligent acts) or agreed to in writing, shall any Contributor be
|
| 157 |
+
liable to You for damages, including any direct, indirect, special,
|
| 158 |
+
incidental, or consequential damages of any character arising as a
|
| 159 |
+
result of this License or out of the use or inability to use the
|
| 160 |
+
Work (including but not limited to damages for loss of goodwill,
|
| 161 |
+
work stoppage, computer failure or malfunction, or any and all
|
| 162 |
+
other commercial damages or losses), even if such Contributor
|
| 163 |
+
has been advised of the possibility of such damages.
|
| 164 |
+
|
| 165 |
+
9. Accepting Warranty or Additional Liability. While redistributing
|
| 166 |
+
the Work or Derivative Works thereof, You may choose to offer,
|
| 167 |
+
and charge a fee for, acceptance of support, warranty, indemnity,
|
| 168 |
+
or other liability obligations and/or rights consistent with this
|
| 169 |
+
License. However, in accepting such obligations, You may act only
|
| 170 |
+
on Your own behalf and on Your sole responsibility, not on behalf
|
| 171 |
+
of any other Contributor, and only if You agree to indemnify,
|
| 172 |
+
defend, and hold each Contributor harmless for any liability
|
| 173 |
+
incurred by, or claims asserted against, such Contributor by reason
|
| 174 |
+
of your accepting any such warranty or additional liability.
|
| 175 |
+
|
| 176 |
+
END OF TERMS AND CONDITIONS
|
| 177 |
+
|
| 178 |
+
APPENDIX: How to apply the Apache License to your work.
|
| 179 |
+
|
| 180 |
+
To apply the Apache License to your work, attach the following
|
| 181 |
+
boilerplate notice, with the fields enclosed by brackets "[]"
|
| 182 |
+
replaced with your own identifying information. (Don't include
|
| 183 |
+
the brackets!) The text should be enclosed in the appropriate
|
| 184 |
+
comment syntax for the file format. We also recommend that a
|
| 185 |
+
file or class name and description of purpose be included on the
|
| 186 |
+
same "printed page" as the copyright notice for easier
|
| 187 |
+
identification within third-party archives.
|
| 188 |
+
|
| 189 |
+
Copyright [yyyy] [name of copyright owner]
|
| 190 |
+
|
| 191 |
+
Licensed under the Apache License, Version 2.0 (the "License");
|
| 192 |
+
you may not use this file except in compliance with the License.
|
| 193 |
+
You may obtain a copy of the License at
|
| 194 |
+
|
| 195 |
+
http://www.apache.org/licenses/LICENSE-2.0
|
| 196 |
+
|
| 197 |
+
Unless required by applicable law or agreed to in writing, software
|
| 198 |
+
distributed under the License is distributed on an "AS IS" BASIS,
|
| 199 |
+
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
| 200 |
+
See the License for the specific language governing permissions and
|
| 201 |
+
limitations under the License.
|
assets/app.js
CHANGED
|
@@ -31,6 +31,9 @@
|
|
| 31 |
return s;
|
| 32 |
}
|
| 33 |
|
|
|
|
|
|
|
|
|
|
| 34 |
function budgetLabel(index, total) {
|
| 35 |
var label = 'Budget ' + (index + 1);
|
| 36 |
if (total && index === 0) return label + ' (smallest)';
|
|
@@ -194,13 +197,33 @@
|
|
| 194 |
setFrame($('frame-datasets'), 'data/per_dataset_table.html');
|
| 195 |
}
|
| 196 |
function loadRobustness() {
|
| 197 |
-
|
| 198 |
-
|
| 199 |
-
|
| 200 |
-
|
| 201 |
-
|
| 202 |
-
|
| 203 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 204 |
}
|
| 205 |
|
| 206 |
// ---- start ----
|
|
|
|
| 31 |
return s;
|
| 32 |
}
|
| 33 |
|
| 34 |
+
// 0.5 -> "0.5", 0.75 -> "0.75", 1 -> "1.0"
|
| 35 |
+
function noiseLabel(value) { return Number(value).toFixed(2).replace(/0$/, ''); }
|
| 36 |
+
|
| 37 |
function budgetLabel(index, total) {
|
| 38 |
var label = 'Budget ' + (index + 1);
|
| 39 |
if (total && index === 0) return label + ' (smallest)';
|
|
|
|
| 197 |
setFrame($('frame-datasets'), 'data/per_dataset_table.html');
|
| 198 |
}
|
| 199 |
function loadRobustness() {
|
| 200 |
+
var hasStability = !!(meta && meta.has_stability);
|
| 201 |
+
$('stability-missing').hidden = hasStability;
|
| 202 |
+
$('frame-stability').hidden = !hasStability;
|
| 203 |
+
if (hasStability) setFrame($('frame-stability'), 'data/stability_table.html');
|
| 204 |
+
|
| 205 |
+
// One validity table per noise level (meta.validity_tables); older data had a single table.
|
| 206 |
+
var tables = (meta && meta.validity_tables) || [];
|
| 207 |
+
if (!tables.length && meta && meta.has_validity) tables = [{ noise: null, path: 'validity_table.html' }];
|
| 208 |
+
$('validity-missing').hidden = tables.length > 0;
|
| 209 |
+
var container = $('validity-tables');
|
| 210 |
+
if (!container.childElementCount) {
|
| 211 |
+
tables.forEach(function (table) {
|
| 212 |
+
if (table.noise != null) {
|
| 213 |
+
var heading = document.createElement('h3');
|
| 214 |
+
heading.textContent = 'Noise level ' + noiseLabel(table.noise) +
|
| 215 |
+
(Number(table.noise) === Number(meta.validity_noise) ? ' (used on the leaderboard)' : '');
|
| 216 |
+
container.appendChild(heading);
|
| 217 |
+
}
|
| 218 |
+
var frame = document.createElement('iframe');
|
| 219 |
+
frame.className = 'frame sa-frame';
|
| 220 |
+
frame.title = 'Validity per method' + (table.noise != null ? ' at noise level ' + noiseLabel(table.noise) : '');
|
| 221 |
+
frame.style.height = '640px';
|
| 222 |
+
frame.src = 'data/' + table.path;
|
| 223 |
+
container.appendChild(frame);
|
| 224 |
+
});
|
| 225 |
+
}
|
| 226 |
+
if (meta && meta.validity_noise != null) $('validity-noise').textContent = noiseLabel(meta.validity_noise);
|
| 227 |
}
|
| 228 |
|
| 229 |
// ---- start ----
|
assets/style.css
CHANGED
|
@@ -95,6 +95,7 @@ a { color: var(--accent); }
|
|
| 95 |
color: var(--muted); margin: 8px 0 16px;
|
| 96 |
}
|
| 97 |
h2 { font-size: 1.2rem; margin: 26px 0 8px; }
|
|
|
|
| 98 |
.prose { max-width: 80ch; }
|
| 99 |
.prose li { margin: 4px 0; }
|
| 100 |
code { font-size: 0.88em; background: var(--surface); padding: 1px 5px; border-radius: 4px; }
|
|
|
|
| 95 |
color: var(--muted); margin: 8px 0 16px;
|
| 96 |
}
|
| 97 |
h2 { font-size: 1.2rem; margin: 26px 0 8px; }
|
| 98 |
+
h3 { font-size: 1rem; margin: 20px 0 6px; color: var(--muted); }
|
| 99 |
.prose { max-width: 80ch; }
|
| 100 |
.prose li { margin: 4px 0; }
|
| 101 |
code { font-size: 0.88em; background: var(--surface); padding: 1px 5px; border-radius: 4px; }
|
data/meta.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
| 1 |
{
|
| 2 |
"benchmark_name": "feature_selection_benchmark_2026_all",
|
| 3 |
-
"generated_at": "2026-10-14
|
| 4 |
"selectarena_version": "0.1.0",
|
| 5 |
"n_results": 79873,
|
| 6 |
"n_datasets": 71,
|
|
@@ -48,6 +48,20 @@
|
|
| 48 |
"validity_noise": 1.0,
|
| 49 |
"has_stability": true,
|
| 50 |
"has_validity": true,
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 51 |
"subsets": [
|
| 52 |
{
|
| 53 |
"tasks": "all",
|
|
|
|
| 1 |
{
|
| 2 |
"benchmark_name": "feature_selection_benchmark_2026_all",
|
| 3 |
+
"generated_at": "2026-10-14 11:34 UTC",
|
| 4 |
"selectarena_version": "0.1.0",
|
| 5 |
"n_results": 79873,
|
| 6 |
"n_datasets": 71,
|
|
|
|
| 48 |
"validity_noise": 1.0,
|
| 49 |
"has_stability": true,
|
| 50 |
"has_validity": true,
|
| 51 |
+
"validity_tables": [
|
| 52 |
+
{
|
| 53 |
+
"noise": 0.5,
|
| 54 |
+
"path": "validity_noise_0.5_table.html"
|
| 55 |
+
},
|
| 56 |
+
{
|
| 57 |
+
"noise": 0.75,
|
| 58 |
+
"path": "validity_noise_0.75_table.html"
|
| 59 |
+
},
|
| 60 |
+
{
|
| 61 |
+
"noise": 1.0,
|
| 62 |
+
"path": "validity_noise_1_table.html"
|
| 63 |
+
}
|
| 64 |
+
],
|
| 65 |
"subsets": [
|
| 66 |
{
|
| 67 |
"tasks": "all",
|
data/per_dataset_table.html
CHANGED
|
@@ -26,7 +26,9 @@ td.method { font-weight:600; }
|
|
| 26 |
td .sub { color:var(--muted); font-size:12px; font-weight:400; margin-left:6px; }
|
| 27 |
td.na { color:var(--muted); }
|
| 28 |
.pill { display:inline-block; padding:0 8px; border:1px solid var(--line); border-radius:999px;
|
| 29 |
-
font-size:12px; color:var(--
|
|
|
|
|
|
|
| 30 |
</style></head><body><div class='toolbar'><span class='hint'>Mean rank of each method per dataset (lower is better). Click a column to sort.</span><button onclick="saCsv('selectarena_per_dataset_ranks.csv')">CSV</button></div><div class='wrap'><table id='sa-table'><thead><tr><th class=''>Dataset</th><th class=''>Problem type</th><th class='num'>(R)ReliefF</th><th class='num'>CART</th><th class='num'>ElasticNet</th><th class='num'>F-Test</th><th class='num'>GainRatio</th><th class='num'>LOCO</th><th class='num'>LaplacianScore</th><th class='num'>Lasso</th><th class='num'>MI</th><th class='num'>MarkovBlanket</th><th class='num'>RFImportance</th><th class='num'>Random</th><th class='num'>SBE</th><th class='num'>SFS</th><th class='num'>mRMR</th></tr></thead><tbody><tr><td class='method' data-sort='Task-1087039797'>Task-1087039797</td><td class='' data-sort='binary'>binary</td><td class='num' data-sort='8.17'>8.17</td><td class='num' data-sort='3.48'>3.48</td><td class='num' data-sort='11.93'>11.93</td><td class='num' data-sort='11.27'>11.27</td><td class='num' data-sort='9.17'>9.17</td><td class='num' data-sort='5.76'>5.76</td><td class='num' data-sort='13.39'>13.39</td><td class='num' data-sort='11.76'>11.76</td><td class='num' data-sort='8.33'>8.33</td><td class='num' data-sort='5.29'>5.29</td><td class='num' data-sort='3.69'>3.69</td><td class='num' data-sort='10.23'>10.23</td><td class='num' data-sort='6.81'>6.81</td><td class='num' data-sort='6.16'>6.16</td><td class='num' data-sort='4.55'>4.55</td></tr><tr><td class='method' data-sort='Task-1119718077'>Task-1119718077</td><td class='' data-sort='regression'>regression</td><td class='num' data-sort='8.04'>8.04</td><td class='num' data-sort='7.27'>7.27</td><td class='num' data-sort='8.83'>8.83</td><td class='num' data-sort='10.8'>10.80</td><td class='num' data-sort='6.71'>6.71</td><td class='num' data-sort='7.96'>7.96</td><td class='num' data-sort='6.51'>6.51</td><td class='num' data-sort='8.83'>8.83</td><td class='num' data-sort='7.2'>7.20</td><td class='num' data-sort='7.94'>7.94</td><td class='num' data-sort='7.65'>7.65</td><td class='num' data-sort='8.23'>8.23</td><td class='num' data-sort='8.42'>8.42</td><td class='num' data-sort='7.88'>7.88</td><td class='num' data-sort='7.74'>7.74</td></tr><tr><td class='method' data-sort='Task-1190597780'>Task-1190597780</td><td class='' data-sort='binary'>binary</td><td class='num' data-sort='6.51'>6.51</td><td class='num' data-sort='9.95'>9.95</td><td class='num' data-sort='7.53'>7.53</td><td class='num' data-sort='6.69'>6.69</td><td class='num' data-sort='7.26'>7.26</td><td class='num' data-sort='8.39'>8.39</td><td class='num' data-sort='7.87'>7.87</td><td class='num' data-sort='7.0'>7.00</td><td class='num' data-sort='6.91'>6.91</td><td class='num' data-sort='8.5'>8.50</td><td class='num' data-sort='7.94'>7.94</td><td class='num' data-sort='9.58'>9.58</td><td class='num' data-sort='9.13'>9.13</td><td class='num' data-sort='9.08'>9.08</td><td class='num' data-sort='7.66'>7.66</td></tr><tr><td class='method' data-sort='Task-1429652751'>Task-1429652751</td><td class='' data-sort='binary'>binary</td><td class='num' data-sort='8.35'>8.35</td><td class='num' data-sort='7.29'>7.29</td><td class='num' data-sort='6.4'>6.40</td><td class='num' data-sort='8.08'>8.08</td><td class='num' data-sort='7.17'>7.17</td><td class='num' data-sort='10.55'>10.55</td><td class='num' data-sort='8.47'>8.47</td><td class='num' data-sort='7.01'>7.01</td><td class='num' data-sort='7.88'>7.88</td><td class='num' data-sort='7.71'>7.71</td><td class='num' data-sort='7.32'>7.32</td><td class='num' data-sort='9.98'>9.98</td><td class='num' data-sort='9.11'>9.11</td><td class='num' data-sort='7.92'>7.92</td><td class='num' data-sort='6.76'>6.76</td></tr><tr><td class='method' data-sort='Task-1458608432'>Task-1458608432</td><td class='' data-sort='binary'>binary</td><td class='num' data-sort='14.12'>14.12</td><td class='num' data-sort='6.07'>6.07</td><td class='num' data-sort='4.05'>4.05</td><td class='num' data-sort='2.73'>2.73</td><td class='num' data-sort='4.9'>4.90</td><td class='num' data-sort='9.72'>9.72</td><td class='num' data-sort='14.44'>14.44</td><td class='num' data-sort='4.06'>4.06</td><td class='num' data-sort='13.23'>13.23</td><td class='num' data-sort='5.53'>5.53</td><td class='num' data-sort='4.03'>4.03</td><td class='num' data-sort='11.31'>11.31</td><td class='num' data-sort='11.19'>11.19</td><td class='num' data-sort='9.89'>9.89</td><td class='num' data-sort='4.64'>4.64</td></tr><tr><td class='method' data-sort='Task-1470619609'>Task-1470619609</td><td class='' data-sort='binary'>binary</td><td class='num' data-sort='6.89'>6.89</td><td class='num' data-sort='6.53'>6.53</td><td class='num' data-sort='5.31'>5.31</td><td class='num' data-sort='7.21'>7.21</td><td class='num' data-sort='6.27'>6.27</td><td class='num' data-sort='11.73'>11.73</td><td class='num' data-sort='14.87'>14.87</td><td class='num' data-sort='4.36'>4.36</td><td class='num' data-sort='5.2'>5.20</td><td class='num' data-sort='6.28'>6.28</td><td class='num' data-sort='4.69'>4.69</td><td class='num' data-sort='11.83'>11.83</td><td class='num' data-sort='11.76'>11.76</td><td class='num' data-sort='11.1'>11.10</td><td class='num' data-sort='5.97'>5.97</td></tr><tr><td class='method' data-sort='Task-1537910476'>Task-1537910476</td><td class='' data-sort='multiclass'>multiclass</td><td class='num' data-sort='8.61'>8.61</td><td class='num' data-sort='8.28'>8.28</td><td class='num' data-sort='6.59'>6.59</td><td class='num' data-sort='6.53'>6.53</td><td class='num' data-sort='7.3'>7.30</td><td class='num' data-sort='8.38'>8.38</td><td class='num' data-sort='10.38'>10.38</td><td class='num' data-sort='6.51'>6.51</td><td class='num' data-sort='7.05'>7.05</td><td class='num' data-sort='6.5'>6.50</td><td class='num' data-sort='6.45'>6.45</td><td class='num' data-sort='9.79'>9.79</td><td class='num' data-sort='7.76'>7.76</td><td class='num' data-sort='12.46'>12.46</td><td class='num' data-sort='7.4'>7.40</td></tr><tr><td class='method' data-sort='Task-1548247551'>Task-1548247551</td><td class='' data-sort='multiclass'>multiclass</td><td class='num' data-sort='4.36'>4.36</td><td class='num' data-sort='10.63'>10.63</td><td class='num' data-sort='4.33'>4.33</td><td class='num' data-sort='5.45'>5.45</td><td class='num' data-sort='5.99'>5.99</td><td class='num' data-sort='11.75'>11.75</td><td class='num' data-sort='13.51'>13.51</td><td class='num' data-sort='5.03'>5.03</td><td class='num' data-sort='5.97'>5.97</td><td class='num' data-sort='6.49'>6.49</td><td class='num' data-sort='4.35'>4.35</td><td class='num' data-sort='11.96'>11.96</td><td class='num' data-sort='12.08'>12.08</td><td class='num' data-sort='12.08'>12.08</td><td class='num' data-sort='6.03'>6.03</td></tr><tr><td class='method' data-sort='Task-156413859'>Task-156413859</td><td class='' data-sort='binary'>binary</td><td class='num' data-sort='6.73'>6.73</td><td class='num' data-sort='5.13'>5.13</td><td class='num' data-sort='9.99'>9.99</td><td class='num' data-sort='10.83'>10.83</td><td class='num' data-sort='7.89'>7.89</td><td class='num' data-sort='4.34'>4.34</td><td class='num' data-sort='14.51'>14.51</td><td class='num' data-sort='9.67'>9.67</td><td class='num' data-sort='6.69'>6.69</td><td class='num' data-sort='8.0'>8.00</td><td class='num' data-sort='4.67'>4.67</td><td class='num' data-sort='11.84'>11.84</td><td class='num' data-sort='3.59'>3.59</td><td class='num' data-sort='5.82'>5.82</td><td class='num' data-sort='10.32'>10.32</td></tr><tr><td class='method' data-sort='Task-1713586811'>Task-1713586811</td><td class='' data-sort='binary'>binary</td><td class='num' data-sort='10.27'>10.27</td><td class='num' data-sort='9.29'>9.29</td><td class='num' data-sort='8.33'>8.33</td><td class='num' data-sort='6.31'>6.31</td><td class='num' data-sort='6.89'>6.89</td><td class='num' data-sort='4.46'>4.46</td><td class='num' data-sort='13.69'>13.69</td><td class='num' data-sort='8.33'>8.33</td><td class='num' data-sort='4.35'>4.35</td><td class='num' data-sort='6.85'>6.85</td><td class='num' data-sort='9.99'>9.99</td><td class='num' data-sort='13.23'>13.23</td><td class='num' data-sort='5.09'>5.09</td><td class='num' data-sort='4.86'>4.86</td><td class='num' data-sort='8.05'>8.05</td></tr><tr><td class='method' data-sort='Task-1737676527'>Task-1737676527</td><td class='' data-sort='regression'>regression</td><td class='num' data-sort='8.84'>8.84</td><td class='num' data-sort='3.17'>3.17</td><td class='num' data-sort='5.44'>5.44</td><td class='num' data-sort='8.93'>8.93</td><td class='num' data-sort='7.55'>7.55</td><td class='num' data-sort='3.75'>3.75</td><td class='num' data-sort='14.93'>14.93</td><td class='num' data-sort='5.43'>5.43</td><td class='num' data-sort='5.27'>5.27</td><td class='num' data-sort='7.62'>7.62</td><td class='num' data-sort='3.55'>3.55</td><td class='num' data-sort='13.31'>13.31</td><td class='num' data-sort='13.69'>13.69</td><td class='num' data-sort='8.0'>8.00</td><td class='num' data-sort='10.52'>10.52</td></tr><tr><td class='method' data-sort='Task-1747559729'>Task-1747559729</td><td class='' data-sort='multiclass'>multiclass</td><td class='num' data-sort='9.89'>9.89</td><td class='num' data-sort='5.05'>5.05</td><td class='num' data-sort='6.16'>6.16</td><td class='num' data-sort='9.97'>9.97</td><td class='num' data-sort='7.06'>7.06</td><td class='num' data-sort='4.71'>4.71</td><td class='num' data-sort='13.64'>13.64</td><td class='num' data-sort='6.14'>6.14</td><td class='num' data-sort='7.12'>7.12</td><td class='num' data-sort='5.66'>5.66</td><td class='num' data-sort='6.83'>6.83</td><td class='num' data-sort='12.35'>12.35</td><td class='num' data-sort='4.95'>4.95</td><td class='num' data-sort='7.87'>7.87</td><td class='num' data-sort='12.6'>12.60</td></tr><tr><td class='method' data-sort='Task-1767785790'>Task-1767785790</td><td class='' data-sort='binary'>binary</td><td class='num' data-sort='8.27'>8.27</td><td class='num' data-sort='8.17'>8.17</td><td class='num' data-sort='8.43'>8.43</td><td class='num' data-sort='8.26'>8.26</td><td class='num' data-sort='7.73'>7.73</td><td class='num' data-sort='7.31'>7.31</td><td class='num' data-sort='8.89'>8.89</td><td class='num' data-sort='7.9'>7.90</td><td class='num' data-sort='7.27'>7.27</td><td class='num' data-sort='6.38'>6.38</td><td class='num' data-sort='9.08'>9.08</td><td class='num' data-sort='7.97'>7.97</td><td class='num' data-sort='6.63'>6.63</td><td class='num' data-sort='10.03'>10.03</td><td class='num' data-sort='7.69'>7.69</td></tr><tr><td class='method' data-sort='Task-2209824369'>Task-2209824369</td><td class='' data-sort='binary'>binary</td><td class='num' data-sort='6.43'>6.43</td><td class='num' data-sort='5.28'>5.28</td><td class='num' data-sort='8.42'>8.42</td><td class='num' data-sort='12.05'>12.05</td><td class='num' data-sort='11.93'>11.93</td><td class='num' data-sort='5.47'>5.47</td><td class='num' data-sort='7.54'>7.54</td><td class='num' data-sort='8.45'>8.45</td><td class='num' data-sort='8.06'>8.06</td><td class='num' data-sort='9.03'>9.03</td><td class='num' data-sort='7.69'>7.69</td><td class='num' data-sort='12.4'>12.40</td><td class='num' data-sort='2.49'>2.49</td><td class='num' data-sort='2.61'>2.61</td><td class='num' data-sort='12.15'>12.15</td></tr><tr><td class='method' data-sort='Task-2263383072'>Task-2263383072</td><td class='' data-sort='binary'>binary</td><td class='num' data-sort='7.45'>7.45</td><td class='num' data-sort='7.36'>7.36</td><td class='num' data-sort='8.25'>8.25</td><td class='num' data-sort='7.35'>7.35</td><td class='num' data-sort='7.37'>7.37</td><td class='num' data-sort='8.13'>8.13</td><td class='num' data-sort='13.37'>13.37</td><td class='num' data-sort='8.03'>8.03</td><td class='num' data-sort='6.67'>6.67</td><td class='num' data-sort='4.32'>4.32</td><td class='num' data-sort='7.36'>7.36</td><td class='num' data-sort='13.23'>13.23</td><td class='num' data-sort='6.65'>6.65</td><td class='num' data-sort='8.27'>8.27</td><td class='num' data-sort='6.19'>6.19</td></tr><tr><td class='method' data-sort='Task-2269177443'>Task-2269177443</td><td class='' data-sort='multiclass'>multiclass</td><td class='num' data-sort='5.35'>5.35</td><td class='num' data-sort='5.23'>5.23</td><td class='num' data-sort='3.94'>3.94</td><td class='num' data-sort='5.99'>5.99</td><td class='num' data-sort='10.97'>10.97</td><td class='num' data-sort='6.72'>6.72</td><td class='num' data-sort='12.91'>12.91</td><td class='num' data-sort='4.66'>4.66</td><td class='num' data-sort='5.0'>5.00</td><td class='num' data-sort='9.85'>9.85</td><td class='num' data-sort='5.71'>5.71</td><td class='num' data-sort='12.24'>12.24</td><td class='num' data-sort='8.88'>8.88</td><td class='num' data-sort='9.88'>9.88</td><td class='num' data-sort='12.68'>12.68</td></tr><tr><td class='method' data-sort='Task-2582981881'>Task-2582981881</td><td class='' data-sort='binary'>binary</td><td class='num' data-sort='6.4'>6.40</td><td class='num' data-sort='8.39'>8.39</td><td class='num' data-sort='4.23'>4.23</td><td class='num' data-sort='8.16'>8.16</td><td class='num' data-sort='10.09'>10.09</td><td class='num' data-sort='10.05'>10.05</td><td class='num' data-sort='12.67'>12.67</td><td class='num' data-sort='5.93'>5.93</td><td class='num' data-sort='8.11'>8.11</td><td class='num' data-sort='8.95'>8.95</td><td class='num' data-sort='4.67'>4.67</td><td class='num' data-sort='7.91'>7.91</td><td class='num' data-sort='8.87'>8.87</td><td class='num' data-sort='5.49'>5.49</td><td class='num' data-sort='10.09'>10.09</td></tr><tr><td class='method' data-sort='Task-2807200459'>Task-2807200459</td><td class='' data-sort='binary'>binary</td><td class='num' data-sort='11.09'>11.09</td><td class='num' data-sort='9.55'>9.55</td><td class='num' data-sort='5.79'>5.79</td><td class='num' data-sort='6.59'>6.59</td><td class='num' data-sort='10.19'>10.19</td><td class='num' data-sort='5.12'>5.12</td><td class='num' data-sort='10.9'>10.90</td><td class='num' data-sort='5.77'>5.77</td><td class='num' data-sort='6.11'>6.11</td><td class='num' data-sort='7.91'>7.91</td><td class='num' data-sort='10.75'>10.75</td><td class='num' data-sort='10.58'>10.58</td><td class='num' data-sort='4.23'>4.23</td><td class='num' data-sort='5.73'>5.73</td><td class='num' data-sort='9.71'>9.71</td></tr><tr><td class='method' data-sort='Task-2824973323'>Task-2824973323</td><td class='' data-sort='multiclass'>multiclass</td><td class='num' data-sort='6.41'>6.41</td><td class='num' data-sort='4.27'>4.27</td><td class='num' data-sort='7.01'>7.01</td><td class='num' data-sort='5.09'>5.09</td><td class='num' data-sort='9.12'>9.12</td><td class='num' data-sort='7.48'>7.48</td><td class='num' data-sort='11.53'>11.53</td><td class='num' data-sort='7.3'>7.30</td><td class='num' data-sort='3.65'>3.65</td><td class='num' data-sort='7.95'>7.95</td><td class='num' data-sort='6.01'>6.01</td><td class='num' data-sort='13.27'>13.27</td><td class='num' data-sort='12.37'>12.37</td><td class='num' data-sort='10.87'>10.87</td><td class='num' data-sort='7.65'>7.65</td></tr><tr><td class='method' data-sort='Task-2922517334'>Task-2922517334</td><td class='' data-sort='regression'>regression</td><td class='num' data-sort='9.56'>9.56</td><td class='num' data-sort='5.04'>5.04</td><td class='num' data-sort='7.77'>7.77</td><td class='num' data-sort='8.19'>8.19</td><td class='num' data-sort='7.87'>7.87</td><td class='num' data-sort='7.5'>7.50</td><td class='num' data-sort='13.72'>13.72</td><td class='num' data-sort='7.89'>7.89</td><td class='num' data-sort='7.78'>7.78</td><td class='num' data-sort='8.11'>8.11</td><td class='num' data-sort='5.46'>5.46</td><td class='num' data-sort='13.79'>13.79</td><td class='num' data-sort='3.91'>3.91</td><td class='num' data-sort='4.59'>4.59</td><td class='num' data-sort='8.84'>8.84</td></tr><tr><td class='method' data-sort='Task-3274731682'>Task-3274731682</td><td class='' data-sort='regression'>regression</td><td class='num' data-sort='5.67'>5.67</td><td class='num' data-sort='2.97'>2.97</td><td class='num' data-sort='8.49'>8.49</td><td class='num' data-sort='10.79'>10.79</td><td class='num' data-sort='7.39'>7.39</td><td class='num' data-sort='10.15'>10.15</td><td class='num' data-sort='11.87'>11.87</td><td class='num' data-sort='8.45'>8.45</td><td class='num' data-sort='8.65'>8.65</td><td class='num' data-sort='6.31'>6.31</td><td class='num' data-sort='2.78'>2.78</td><td class='num' data-sort='11.85'>11.85</td><td class='num' data-sort='5.47'>5.47</td><td class='num' data-sort='8.63'>8.63</td><td class='num' data-sort='10.53'>10.53</td></tr><tr><td class='method' data-sort='Task-3294278573'>Task-3294278573</td><td class='' data-sort='binary'>binary</td><td class='num' data-sort='5.53'>5.53</td><td class='num' data-sort='10.19'>10.19</td><td class='num' data-sort='5.85'>5.85</td><td class='num' data-sort='5.77'>5.77</td><td class='num' data-sort='5.39'>5.39</td><td class='num' data-sort='11.27'>11.27</td><td class='num' data-sort='11.37'>11.37</td><td class='num' data-sort='6.59'>6.59</td><td class='num' data-sort='5.54'>5.54</td><td class='num' data-sort='6.97'>6.97</td><td class='num' data-sort='5.33'>5.33</td><td class='num' data-sort='12.23'>12.23</td><td class='num' data-sort='11.11'>11.11</td><td class='num' data-sort='11.19'>11.19</td><td class='num' data-sort='5.65'>5.65</td></tr><tr><td class='method' data-sort='Task-3421250263'>Task-3421250263</td><td class='' data-sort='binary'>binary</td><td class='num' data-sort='6.83'>6.83</td><td class='num' data-sort='9.0'>9.00</td><td class='num' data-sort='6.89'>6.89</td><td class='num' data-sort='7.27'>7.27</td><td class='num' data-sort='5.91'>5.91</td><td class='num' data-sort='9.03'>9.03</td><td class='num' data-sort='12.43'>12.43</td><td class='num' data-sort='7.73'>7.73</td><td class='num' data-sort='5.83'>5.83</td><td class='num' data-sort='8.26'>8.26</td><td class='num' data-sort='6.93'>6.93</td><td class='num' data-sort='8.74'>8.74</td><td class='num' data-sort='8.76'>8.76</td><td class='num' data-sort='8.15'>8.15</td><td class='num' data-sort='8.23'>8.23</td></tr><tr><td class='method' data-sort='Task-3562356624'>Task-3562356624</td><td class='' data-sort='binary'>binary</td><td class='num' data-sort='9.99'>9.99</td><td class='num' data-sort='8.96'>8.96</td><td class='num' data-sort='6.92'>6.92</td><td class='num' data-sort='6.59'>6.59</td><td class='num' data-sort='9.79'>9.79</td><td class='num' data-sort='8.55'>8.55</td><td class='num' data-sort='5.27'>5.27</td><td class='num' data-sort='6.92'>6.92</td><td class='num' data-sort='9.33'>9.33</td><td class='num' data-sort='8.43'>8.43</td><td class='num' data-sort='8.45'>8.45</td><td class='num' data-sort='6.43'>6.43</td><td class='num' data-sort='6.87'>6.87</td><td class='num' data-sort='6.39'>6.39</td><td class='num' data-sort='11.11'>11.11</td></tr><tr><td class='method' data-sort='Task-4023552040'>Task-4023552040</td><td class='' data-sort='binary'>binary</td><td class='num' data-sort='10.35'>10.35</td><td class='num' data-sort='5.77'>5.77</td><td class='num' data-sort='5.45'>5.45</td><td class='num' data-sort='5.37'>5.37</td><td class='num' data-sort='11.92'>11.92</td><td class='num' data-sort='9.79'>9.79</td><td class='num' data-sort='12.47'>12.47</td><td class='num' data-sort='5.27'>5.27</td><td class='num' data-sort='8.37'>8.37</td><td class='num' data-sort='12.52'>12.52</td><td class='num' data-sort='6.29'>6.29</td><td class='num' data-sort='12.76'>12.76</td><td class='num' data-sort='3.79'>3.79</td><td class='num' data-sort='4.36'>4.36</td><td class='num' data-sort='5.55'>5.55</td></tr><tr><td class='method' data-sort='Task-4265382138'>Task-4265382138</td><td class='' data-sort='binary'>binary</td><td class='num' data-sort='12.05'>12.05</td><td class='num' data-sort='4.87'>4.87</td><td class='num' data-sort='5.08'>5.08</td><td class='num' data-sort='8.73'>8.73</td><td class='num' data-sort='10.39'>10.39</td><td class='num' data-sort='9.77'>9.77</td><td class='num' data-sort='13.05'>13.05</td><td class='num' data-sort='5.99'>5.99</td><td class='num' data-sort='7.51'>7.51</td><td class='num' data-sort='4.93'>4.93</td><td class='num' data-sort='3.89'>3.89</td><td class='num' data-sort='8.03'>8.03</td><td class='num' data-sort='10.21'>10.21</td><td class='num' data-sort='7.19'>7.19</td><td class='num' data-sort='8.31'>8.31</td></tr><tr><td class='method' data-sort='Task-4507720433'>Task-4507720433</td><td class='' data-sort='multiclass'>multiclass</td><td class='num' data-sort='8.33'>8.33</td><td class='num' data-sort='1.79'>1.79</td><td class='num' data-sort='3.67'>3.67</td><td class='num' data-sort='8.2'>8.20</td><td class='num' data-sort='12.01'>12.01</td><td class='num' data-sort='12.4'>12.40</td><td class='num' data-sort='7.44'>7.44</td><td class='num' data-sort='4.68'>4.68</td><td class='num' data-sort='8.6'>8.60</td><td class='num' data-sort='3.29'>3.29</td><td class='num' data-sort='2.47'>2.47</td><td class='num' data-sort='13.33'>13.33</td><td class='num' data-sort='13.23'>13.23</td><td class='num' data-sort='13.41'>13.41</td><td class='num' data-sort='7.15'>7.15</td></tr><tr><td class='method' data-sort='Task-4532501802'>Task-4532501802</td><td class='' data-sort='binary'>binary</td><td class='num' data-sort='7.61'>7.61</td><td class='num' data-sort='9.05'>9.05</td><td class='num' data-sort='7.58'>7.58</td><td class='num' data-sort='7.8'>7.80</td><td class='num' data-sort='7.75'>7.75</td><td class='num' data-sort='8.52'>8.52</td><td class='num' data-sort='6.9'>6.90</td><td class='num' data-sort='8.49'>8.49</td><td class='num' data-sort='7.63'>7.63</td><td class='num' data-sort='6.77'>6.77</td><td class='num' data-sort='6.2'>6.20</td><td class='num' data-sort='9.53'>9.53</td><td class='num' data-sort='7.86'>7.86</td><td class='num' data-sort='9.63'>9.63</td><td class='num' data-sort='8.67'>8.67</td></tr><tr><td class='method' data-sort='Task-4548273248'>Task-4548273248</td><td class='' data-sort='multiclass'>multiclass</td><td class='num' data-sort='6.55'>6.55</td><td class='num' data-sort='7.97'>7.97</td><td class='num' data-sort='5.87'>5.87</td><td class='num' data-sort='8.57'>8.57</td><td class='num' data-sort='5.17'>5.17</td><td class='num' data-sort='9.76'>9.76</td><td class='num' data-sort='14.2'>14.20</td><td class='num' data-sort='8.55'>8.55</td><td class='num' data-sort='5.68'>5.68</td><td class='num' data-sort='6.17'>6.17</td><td class='num' data-sort='5.93'>5.93</td><td class='num' data-sort='9.96'>9.96</td><td class='num' data-sort='10.17'>10.17</td><td class='num' data-sort='9.46'>9.46</td><td class='num' data-sort='5.99'>5.99</td></tr><tr><td class='method' data-sort='Task-4591045950'>Task-4591045950</td><td class='' data-sort='binary'>binary</td><td class='num' data-sort='5.92'>5.92</td><td class='num' data-sort='6.0'>6.00</td><td class='num' data-sort='12.99'>12.99</td><td class='num' data-sort='4.73'>4.73</td><td class='num' data-sort='5.29'>5.29</td><td class='num' data-sort='3.57'>3.57</td><td class='num' data-sort='13.25'>13.25</td><td class='num' data-sort='13.05'>13.05</td><td class='num' data-sort='9.47'>9.47</td><td class='num' data-sort='4.41'>4.41</td><td class='num' data-sort='6.13'>6.13</td><td class='num' data-sort='12.49'>12.49</td><td class='num' data-sort='12.91'>12.91</td><td class='num' data-sort='4.01'>4.01</td><td class='num' data-sort='5.76'>5.76</td></tr><tr><td class='method' data-sort='Task-4613115627'>Task-4613115627</td><td class='' data-sort='binary'>binary</td><td class='num' data-sort='6.23'>6.23</td><td class='num' data-sort='3.36'>3.36</td><td class='num' data-sort='5.25'>5.25</td><td class='num' data-sort='5.96'>5.96</td><td class='num' data-sort='11.37'>11.37</td><td class='num' data-sort='11.37'>11.37</td><td class='num' data-sort='12.72'>12.72</td><td class='num' data-sort='4.23'>4.23</td><td class='num' data-sort='5.51'>5.51</td><td class='num' data-sort='10.48'>10.48</td><td class='num' data-sort='4.17'>4.17</td><td class='num' data-sort='11.24'>11.24</td><td class='num' data-sort='11.08'>11.08</td><td class='num' data-sort='5.67'>5.67</td><td class='num' data-sort='11.37'>11.37</td></tr><tr><td class='method' data-sort='Task-4673108304'>Task-4673108304</td><td class='' data-sort='binary'>binary</td><td class='num' data-sort='10.35'>10.35</td><td class='num' data-sort='8.08'>8.08</td><td class='num' data-sort='5.9'>5.90</td><td class='num' data-sort='10.15'>10.15</td><td class='num' data-sort='7.87'>7.87</td><td class='num' data-sort='8.28'>8.28</td><td class='num' data-sort='8.21'>8.21</td><td class='num' data-sort='6.31'>6.31</td><td class='num' data-sort='8.05'>8.05</td><td class='num' data-sort='7.87'>7.87</td><td class='num' data-sort='7.37'>7.37</td><td class='num' data-sort='8.09'>8.09</td><td class='num' data-sort='8.85'>8.85</td><td class='num' data-sort='6.71'>6.71</td><td class='num' data-sort='7.91'>7.91</td></tr><tr><td class='method' data-sort='Task-4699939988'>Task-4699939988</td><td class='' data-sort='binary'>binary</td><td class='num' data-sort='7.9'>7.90</td><td class='num' data-sort='5.36'>5.36</td><td class='num' data-sort='9.07'>9.07</td><td class='num' data-sort='7.43'>7.43</td><td class='num' data-sort='5.98'>5.98</td><td class='num' data-sort='7.15'>7.15</td><td class='num' data-sort='14.27'>14.27</td><td class='num' data-sort='9.07'>9.07</td><td class='num' data-sort='8.92'>8.92</td><td class='num' data-sort='7.61'>7.61</td><td class='num' data-sort='5.45'>5.45</td><td class='num' data-sort='9.41'>9.41</td><td class='num' data-sort='4.09'>4.09</td><td class='num' data-sort='9.05'>9.05</td><td class='num' data-sort='9.24'>9.24</td></tr><tr><td class='method' data-sort='Task-4867148065'>Task-4867148065</td><td class='' data-sort='binary'>binary</td><td class='num' data-sort='13.15'>13.15</td><td class='num' data-sort='4.73'>4.73</td><td class='num' data-sort='8.38'>8.38</td><td class='num' data-sort='4.19'>4.19</td><td class='num' data-sort='6.85'>6.85</td><td class='num' data-sort='7.36'>7.36</td><td class='num' data-sort='14.97'>14.97</td><td class='num' data-sort='10.65'>10.65</td><td class='num' data-sort='4.97'>4.97</td><td class='num' data-sort='7.4'>7.40</td><td class='num' data-sort='6.19'>6.19</td><td class='num' data-sort='8.01'>8.01</td><td class='num' data-sort='10.65'>10.65</td><td class='num' data-sort='4.64'>4.64</td><td class='num' data-sort='7.85'>7.85</td></tr><tr><td class='method' data-sort='Task-5047778267'>Task-5047778267</td><td class='' data-sort='binary'>binary</td><td class='num' data-sort='4.6'>4.60</td><td class='num' data-sort='8.95'>8.95</td><td class='num' data-sort='5.03'>5.03</td><td class='num' data-sort='5.43'>5.43</td><td class='num' data-sort='5.29'>5.29</td><td class='num' data-sort='12.49'>12.49</td><td class='num' data-sort='13.63'>13.63</td><td class='num' data-sort='5.43'>5.43</td><td class='num' data-sort='5.97'>5.97</td><td class='num' data-sort='7.02'>7.02</td><td class='num' data-sort='6.21'>6.21</td><td class='num' data-sort='12.15'>12.15</td><td class='num' data-sort='12.19'>12.19</td><td class='num' data-sort='10.11'>10.11</td><td class='num' data-sort='5.49'>5.49</td></tr><tr><td class='method' data-sort='Task-5061570515'>Task-5061570515</td><td class='' data-sort='binary'>binary</td><td class='num' data-sort='7.71'>7.71</td><td class='num' data-sort='7.22'>7.22</td><td class='num' data-sort='5.86'>5.86</td><td class='num' data-sort='4.45'>4.45</td><td class='num' data-sort='11.55'>11.55</td><td class='num' data-sort='7.25'>7.25</td><td class='num' data-sort='12.81'>12.81</td><td class='num' data-sort='5.55'>5.55</td><td class='num' data-sort='6.09'>6.09</td><td class='num' data-sort='12.29'>12.29</td><td class='num' data-sort='5.65'>5.65</td><td class='num' data-sort='11.33'>11.33</td><td class='num' data-sort='6.2'>6.20</td><td class='num' data-sort='7.33'>7.33</td><td class='num' data-sort='8.69'>8.69</td></tr><tr><td class='method' data-sort='Task-5079031577'>Task-5079031577</td><td class='' data-sort='regression'>regression</td><td class='num' data-sort='6.12'>6.12</td><td class='num' data-sort='6.69'>6.69</td><td class='num' data-sort='5.97'>5.97</td><td class='num' data-sort='8.65'>8.65</td><td class='num' data-sort='9.19'>9.19</td><td class='num' data-sort='8.87'>8.87</td><td class='num' data-sort='9.03'>9.03</td><td class='num' data-sort='5.97'>5.97</td><td class='num' data-sort='9.27'>9.27</td><td class='num' data-sort='9.22'>9.22</td><td class='num' data-sort='6.08'>6.08</td><td class='num' data-sort='12.32'>12.32</td><td class='num' data-sort='7.08'>7.08</td><td class='num' data-sort='6.63'>6.63</td><td class='num' data-sort='8.92'>8.92</td></tr><tr><td class='method' data-sort='Task-5168164060'>Task-5168164060</td><td class='' data-sort='binary'>binary</td><td class='num' data-sort='6.19'>6.19</td><td class='num' data-sort='11.7'>11.70</td><td class='num' data-sort='4.99'>4.99</td><td class='num' data-sort='6.31'>6.31</td><td class='num' data-sort='9.88'>9.88</td><td class='num' data-sort='9.07'>9.07</td><td class='num' data-sort='7.24'>7.24</td><td class='num' data-sort='6.73'>6.73</td><td class='num' data-sort='6.1'>6.10</td><td class='num' data-sort='10.4'>10.40</td><td class='num' data-sort='9.46'>9.46</td><td class='num' data-sort='11.45'>11.45</td><td class='num' data-sort='7.24'>7.24</td><td class='num' data-sort='6.28'>6.28</td><td class='num' data-sort='6.96'>6.96</td></tr><tr><td class='method' data-sort='Task-5416034620'>Task-5416034620</td><td class='' data-sort='binary'>binary</td><td class='num' data-sort='10.92'>10.92</td><td class='num' data-sort='10.48'>10.48</td><td class='num' data-sort='6.95'>6.95</td><td class='num' data-sort='5.63'>5.63</td><td class='num' data-sort='11.11'>11.11</td><td class='num' data-sort='4.23'>4.23</td><td class='num' data-sort='12.25'>12.25</td><td class='num' data-sort='7.09'>7.09</td><td class='num' data-sort='3.98'>3.98</td><td class='num' data-sort='10.55'>10.55</td><td class='num' data-sort='10.36'>10.36</td><td class='num' data-sort='8.87'>8.87</td><td class='num' data-sort='5.07'>5.07</td><td class='num' data-sort='6.12'>6.12</td><td class='num' data-sort='6.4'>6.40</td></tr><tr><td class='method' data-sort='Task-5622217616'>Task-5622217616</td><td class='' data-sort='binary'>binary</td><td class='num' data-sort='9.35'>9.35</td><td class='num' data-sort='8.57'>8.57</td><td class='num' data-sort='7.77'>7.77</td><td class='num' data-sort='8.96'>8.96</td><td class='num' data-sort='6.07'>6.07</td><td class='num' data-sort='9.43'>9.43</td><td class='num' data-sort='7.92'>7.92</td><td class='num' data-sort='7.77'>7.77</td><td class='num' data-sort='7.67'>7.67</td><td class='num' data-sort='8.33'>8.33</td><td class='num' data-sort='7.27'>7.27</td><td class='num' data-sort='8.91'>8.91</td><td class='num' data-sort='7.6'>7.60</td><td class='num' data-sort='6.87'>6.87</td><td class='num' data-sort='7.5'>7.50</td></tr><tr><td class='method' data-sort='Task-5745371303'>Task-5745371303</td><td class='' data-sort='binary'>binary</td><td class='num' data-sort='8.71'>8.71</td><td class='num' data-sort='9.07'>9.07</td><td class='num' data-sort='6.98'>6.98</td><td class='num' data-sort='7.93'>7.93</td><td class='num' data-sort='8.47'>8.47</td><td class='num' data-sort='7.07'>7.07</td><td class='num' data-sort='13.15'>13.15</td><td class='num' data-sort='6.95'>6.95</td><td class='num' data-sort='6.63'>6.63</td><td class='num' data-sort='5.76'>5.76</td><td class='num' data-sort='8.18'>8.18</td><td class='num' data-sort='12.63'>12.63</td><td class='num' data-sort='4.8'>4.80</td><td class='num' data-sort='7.75'>7.75</td><td class='num' data-sort='5.92'>5.92</td></tr><tr><td class='method' data-sort='Task-5763189736'>Task-5763189736</td><td class='' data-sort='binary'>binary</td><td class='num' data-sort='7.33'>7.33</td><td class='num' data-sort='8.01'>8.01</td><td class='num' data-sort='6.99'>6.99</td><td class='num' data-sort='6.82'>6.82</td><td class='num' data-sort='7.07'>7.07</td><td class='num' data-sort='6.27'>6.27</td><td class='num' data-sort='13.27'>13.27</td><td class='num' data-sort='8.77'>8.77</td><td class='num' data-sort='8.95'>8.95</td><td class='num' data-sort='5.37'>5.37</td><td class='num' data-sort='8.12'>8.12</td><td class='num' data-sort='10.56'>10.56</td><td class='num' data-sort='5.67'>5.67</td><td class='num' data-sort='8.82'>8.82</td><td class='num' data-sort='7.98'>7.98</td></tr><tr><td class='method' data-sort='Task-616857086'>Task-616857086</td><td class='' data-sort='binary'>binary</td><td class='num' data-sort='11.44'>11.44</td><td class='num' data-sort='7.04'>7.04</td><td class='num' data-sort='4.77'>4.77</td><td class='num' data-sort='7.19'>7.19</td><td class='num' data-sort='11.04'>11.04</td><td class='num' data-sort='6.03'>6.03</td><td class='num' data-sort='12.89'>12.89</td><td class='num' data-sort='4.18'>4.18</td><td class='num' data-sort='8.75'>8.75</td><td class='num' data-sort='8.81'>8.81</td><td class='num' data-sort='8.25'>8.25</td><td class='num' data-sort='12.19'>12.19</td><td class='num' data-sort='4.7'>4.70</td><td class='num' data-sort='5.11'>5.11</td><td class='num' data-sort='7.61'>7.61</td></tr><tr><td class='method' data-sort='Task-6346831288'>Task-6346831288</td><td class='' data-sort='multiclass'>multiclass</td><td class='num' data-sort='10.72'>10.72</td><td class='num' data-sort='5.16'>5.16</td><td class='num' data-sort='11.43'>11.43</td><td class='num' data-sort='10.59'>10.59</td><td class='num' data-sort='7.19'>7.19</td><td class='num' data-sort='10.28'>10.28</td><td class='num' data-sort='8.29'>8.29</td><td class='num' data-sort='13.01'>13.01</td><td class='num' data-sort='4.27'>4.27</td><td class='num' data-sort='5.92'>5.92</td><td class='num' data-sort='3.51'>3.51</td><td class='num' data-sort='9.27'>9.27</td><td class='num' data-sort='8.29'>8.29</td><td class='num' data-sort='6.09'>6.09</td><td class='num' data-sort='5.99'>5.99</td></tr><tr><td class='method' data-sort='Task-6392717221'>Task-6392717221</td><td class='' data-sort='binary'>binary</td><td class='num' data-sort='7.68'>7.68</td><td class='num' data-sort='3.87'>3.87</td><td class='num' data-sort='5.69'>5.69</td><td class='num' data-sort='5.23'>5.23</td><td class='num' data-sort='9.24'>9.24</td><td class='num' data-sort='11.72'>11.72</td><td class='num' data-sort='15.0'>15.00</td><td class='num' data-sort='5.59'>5.59</td><td class='num' data-sort='3.4'>3.40</td><td class='num' data-sort='5.33'>5.33</td><td class='num' data-sort='2.96'>2.96</td><td class='num' data-sort='12.71'>12.71</td><td class='num' data-sort='12.96'>12.96</td><td class='num' data-sort='8.97'>8.97</td><td class='num' data-sort='9.65'>9.65</td></tr><tr><td class='method' data-sort='Task-6680748470'>Task-6680748470</td><td class='' data-sort='binary'>binary</td><td class='num' data-sort='5.51'>5.51</td><td class='num' data-sort='4.86'>4.86</td><td class='num' data-sort='8.5'>8.50</td><td class='num' data-sort='7.39'>7.39</td><td class='num' data-sort='8.56'>8.56</td><td class='num' data-sort='7.76'>7.76</td><td class='num' data-sort='11.31'>11.31</td><td class='num' data-sort='9.57'>9.57</td><td class='num' data-sort='6.57'>6.57</td><td class='num' data-sort='9.64'>9.64</td><td class='num' data-sort='4.4'>4.40</td><td class='num' data-sort='10.53'>10.53</td><td class='num' data-sort='7.44'>7.44</td><td class='num' data-sort='9.33'>9.33</td><td class='num' data-sort='8.63'>8.63</td></tr><tr><td class='method' data-sort='Task-6789920400'>Task-6789920400</td><td class='' data-sort='binary'>binary</td><td class='num' data-sort='7.23'>7.23</td><td class='num' data-sort='7.45'>7.45</td><td class='num' data-sort='6.23'>6.23</td><td class='num' data-sort='7.31'>7.31</td><td class='num' data-sort='7.11'>7.11</td><td class='num' data-sort='11.11'>11.11</td><td class='num' data-sort='13.89'>13.89</td><td class='num' data-sort='5.27'>5.27</td><td class='num' data-sort='7.49'>7.49</td><td class='num' data-sort='6.31'>6.31</td><td class='num' data-sort='5.86'>5.86</td><td class='num' data-sort='10.75'>10.75</td><td class='num' data-sort='10.34'>10.34</td><td class='num' data-sort='7.73'>7.73</td><td class='num' data-sort='5.91'>5.91</td></tr><tr><td class='method' data-sort='Task-6859627074'>Task-6859627074</td><td class='' data-sort='binary'>binary</td><td class='num' data-sort='13.05'>13.05</td><td class='num' data-sort='7.02'>7.02</td><td class='num' data-sort='11.47'>11.47</td><td class='num' data-sort='10.83'>10.83</td><td class='num' data-sort='3.43'>3.43</td><td class='num' data-sort='7.76'>7.76</td><td class='num' data-sort='9.41'>9.41</td><td class='num' data-sort='11.49'>11.49</td><td class='num' data-sort='3.89'>3.89</td><td class='num' data-sort='6.55'>6.55</td><td class='num' data-sort='7.25'>7.25</td><td class='num' data-sort='9.3'>9.30</td><td class='num' data-sort='6.51'>6.51</td><td class='num' data-sort='5.67'>5.67</td><td class='num' data-sort='6.37'>6.37</td></tr><tr><td class='method' data-sort='Task-7014870989'>Task-7014870989</td><td class='' data-sort='multiclass'>multiclass</td><td class='num' data-sort='9.85'>9.85</td><td class='num' data-sort='5.27'>5.27</td><td class='num' data-sort='6.15'>6.15</td><td class='num' data-sort='6.15'>6.15</td><td class='num' data-sort='8.57'>8.57</td><td class='num' data-sort='10.55'>10.55</td><td class='num' data-sort='13.39'>13.39</td><td class='num' data-sort='6.57'>6.57</td><td class='num' data-sort='6.25'>6.25</td><td class='num' data-sort='4.48'>4.48</td><td class='num' data-sort='5.44'>5.44</td><td class='num' data-sort='11.97'>11.97</td><td class='num' data-sort='11.95'>11.95</td><td class='num' data-sort='8.17'>8.17</td><td class='num' data-sort='5.24'>5.24</td></tr><tr><td class='method' data-sort='Task-7152814512'>Task-7152814512</td><td class='' data-sort='binary'>binary</td><td class='num' data-sort='4.31'>4.31</td><td class='num' data-sort='4.44'>4.44</td><td class='num' data-sort='8.17'>8.17</td><td class='num' data-sort='9.15'>9.15</td><td class='num' data-sort='9.15'>9.15</td><td class='num' data-sort='6.27'>6.27</td><td class='num' data-sort='14.69'>14.69</td><td class='num' data-sort='8.32'>8.32</td><td class='num' data-sort='9.35'>9.35</td><td class='num' data-sort='6.79'>6.79</td><td class='num' data-sort='5.71'>5.71</td><td class='num' data-sort='12.51'>12.51</td><td class='num' data-sort='5.35'>5.35</td><td class='num' data-sort='6.75'>6.75</td><td class='num' data-sort='9.05'>9.05</td></tr><tr><td class='method' data-sort='Task-7219454773'>Task-7219454773</td><td class='' data-sort='binary'>binary</td><td class='num' data-sort='9.23'>9.23</td><td class='num' data-sort='5.95'>5.95</td><td class='num' data-sort='7.67'>7.67</td><td class='num' data-sort='8.89'>8.89</td><td class='num' data-sort='6.29'>6.29</td><td class='num' data-sort='6.85'>6.85</td><td class='num' data-sort='11.43'>11.43</td><td class='num' data-sort='8.02'>8.02</td><td class='num' data-sort='8.01'>8.01</td><td class='num' data-sort='5.67'>5.67</td><td class='num' data-sort='6.16'>6.16</td><td class='num' data-sort='9.36'>9.36</td><td class='num' data-sort='9.93'>9.93</td><td class='num' data-sort='7.61'>7.61</td><td class='num' data-sort='8.92'>8.92</td></tr><tr><td class='method' data-sort='Task-7315435704'>Task-7315435704</td><td class='' data-sort='binary'>binary</td><td class='num' data-sort='5.01'>5.01</td><td class='num' data-sort='7.36'>7.36</td><td class='num' data-sort='10.92'>10.92</td><td class='num' data-sort='5.7'>5.70</td><td class='num' data-sort='9.11'>9.11</td><td class='num' data-sort='7.43'>7.43</td><td class='num' data-sort='8.52'>8.52</td><td class='num' data-sort='10.92'>10.92</td><td class='num' data-sort='4.41'>4.41</td><td class='num' data-sort='10.33'>10.33</td><td class='num' data-sort='8.27'>8.27</td><td class='num' data-sort='8.84'>8.84</td><td class='num' data-sort='8.01'>8.01</td><td class='num' data-sort='9.46'>9.46</td><td class='num' data-sort='5.71'>5.71</td></tr><tr><td class='method' data-sort='Task-7493221688'>Task-7493221688</td><td class='' data-sort='regression'>regression</td><td class='num' data-sort='7.89'>7.89</td><td class='num' data-sort='6.73'>6.73</td><td class='num' data-sort='8.36'>8.36</td><td class='num' data-sort='6.39'>6.39</td><td class='num' data-sort='7.79'>7.79</td><td class='num' data-sort='7.57'>7.57</td><td class='num' data-sort='12.43'>12.43</td><td class='num' data-sort='8.37'>8.37</td><td class='num' data-sort='7.59'>7.59</td><td class='num' data-sort='5.97'>5.97</td><td class='num' data-sort='6.1'>6.10</td><td class='num' data-sort='10.11'>10.11</td><td class='num' data-sort='8.55'>8.55</td><td class='num' data-sort='9.81'>9.81</td><td class='num' data-sort='6.34'>6.34</td></tr><tr><td class='method' data-sort='Task-7603283006'>Task-7603283006</td><td class='' data-sort='regression'>regression</td><td class='num' data-sort='8.08'>8.08</td><td class='num' data-sort='4.27'>4.27</td><td class='num' data-sort='9.77'>9.77</td><td class='num' data-sort='7.47'>7.47</td><td class='num' data-sort='8.78'>8.78</td><td class='num' data-sort='7.23'>7.23</td><td class='num' data-sort='12.84'>12.84</td><td class='num' data-sort='9.82'>9.82</td><td class='num' data-sort='4.83'>4.83</td><td class='num' data-sort='7.68'>7.68</td><td class='num' data-sort='3.89'>3.89</td><td class='num' data-sort='9.64'>9.64</td><td class='num' data-sort='7.35'>7.35</td><td class='num' data-sort='10.03'>10.03</td><td class='num' data-sort='8.33'>8.33</td></tr><tr><td class='method' data-sort='Task-7612659684'>Task-7612659684</td><td class='' data-sort='multiclass'>multiclass</td><td class='num' data-sort='8.97'>8.97</td><td class='num' data-sort='4.24'>4.24</td><td class='num' data-sort='11.21'>11.21</td><td class='num' data-sort='7.73'>7.73</td><td class='num' data-sort='7.23'>7.23</td><td class='num' data-sort='8.79'>8.79</td><td class='num' data-sort='12.57'>12.57</td><td class='num' data-sort='7.87'>7.87</td><td class='num' data-sort='3.25'>3.25</td><td class='num' data-sort='8.29'>8.29</td><td class='num' data-sort='5.17'>5.17</td><td class='num' data-sort='10.12'>10.12</td><td class='num' data-sort='8.07'>8.07</td><td class='num' data-sort='8.45'>8.45</td><td class='num' data-sort='8.03'>8.03</td></tr><tr><td class='method' data-sort='Task-7641908575'>Task-7641908575</td><td class='' data-sort='binary'>binary</td><td class='num' data-sort='9.37'>9.37</td><td class='num' data-sort='8.8'>8.80</td><td class='num' data-sort='6.61'>6.61</td><td class='num' data-sort='7.41'>7.41</td><td class='num' data-sort='6.72'>6.72</td><td class='num' data-sort='12.05'>12.05</td><td class='num' data-sort='13.77'>13.77</td><td class='num' data-sort='6.65'>6.65</td><td class='num' data-sort='6.42'>6.42</td><td class='num' data-sort='5.17'>5.17</td><td class='num' data-sort='9.33'>9.33</td><td class='num' data-sort='10.82'>10.82</td><td class='num' data-sort='4.64'>4.64</td><td class='num' data-sort='4.89'>4.89</td><td class='num' data-sort='7.33'>7.33</td></tr><tr><td class='method' data-sort='Task-7704446119'>Task-7704446119</td><td class='' data-sort='regression'>regression</td><td class='num' data-sort='5.84'>5.84</td><td class='num' data-sort='10.72'>10.72</td><td class='num' data-sort='3.8'>3.80</td><td class='num' data-sort='3.56'>3.56</td><td class='num' data-sort='8.51'>8.51</td><td class='num' data-sort='9.39'>9.39</td><td class='num' data-sort='11.8'>11.80</td><td class='num' data-sort='3.63'>3.63</td><td class='num' data-sort='9.39'>9.39</td><td class='num' data-sort='8.35'>8.35</td><td class='num' data-sort='10.89'>10.89</td><td class='num' data-sort='9.25'>9.25</td><td class='num' data-sort='9.47'>9.47</td><td class='num' data-sort='7.28'>7.28</td><td class='num' data-sort='8.13'>8.13</td></tr><tr><td class='method' data-sort='Task-7911827689'>Task-7911827689</td><td class='' data-sort='binary'>binary</td><td class='num' data-sort='10.55'>10.55</td><td class='num' data-sort='6.23'>6.23</td><td class='num' data-sort='3.39'>3.39</td><td class='num' data-sort='4.99'>4.99</td><td class='num' data-sort='6.16'>6.16</td><td class='num' data-sort='8.96'>8.96</td><td class='num' data-sort='13.63'>13.63</td><td class='num' data-sort='7.63'>7.63</td><td class='num' data-sort='7.11'>7.11</td><td class='num' data-sort='5.6'>5.60</td><td class='num' data-sort='5.49'>5.49</td><td class='num' data-sort='10.07'>10.07</td><td class='num' data-sort='9.39'>9.39</td><td class='num' data-sort='11.64'>11.64</td><td class='num' data-sort='9.17'>9.17</td></tr><tr><td class='method' data-sort='Task-8002938196'>Task-8002938196</td><td class='' data-sort='binary'>binary</td><td class='num' data-sort='7.35'>7.35</td><td class='num' data-sort='5.9'>5.90</td><td class='num' data-sort='6.93'>6.93</td><td class='num' data-sort='6.34'>6.34</td><td class='num' data-sort='8.11'>8.11</td><td class='num' data-sort='9.09'>9.09</td><td class='num' data-sort='10.71'>10.71</td><td class='num' data-sort='6.51'>6.51</td><td class='num' data-sort='7.6'>7.60</td><td class='num' data-sort='7.17'>7.17</td><td class='num' data-sort='6.85'>6.85</td><td class='num' data-sort='10.56'>10.56</td><td class='num' data-sort='10.59'>10.59</td><td class='num' data-sort='9.51'>9.51</td><td class='num' data-sort='6.78'>6.78</td></tr><tr><td class='method' data-sort='Task-8017089756'>Task-8017089756</td><td class='' data-sort='binary'>binary</td><td class='num' data-sort='8.97'>8.97</td><td class='num' data-sort='8.13'>8.13</td><td class='num' data-sort='7.69'>7.69</td><td class='num' data-sort='7.54'>7.54</td><td class='num' data-sort='7.3'>7.30</td><td class='num' data-sort='9.24'>9.24</td><td class='num' data-sort='11.63'>11.63</td><td class='num' data-sort='8.11'>8.11</td><td class='num' data-sort='6.63'>6.63</td><td class='num' data-sort='6.28'>6.28</td><td class='num' data-sort='6.81'>6.81</td><td class='num' data-sort='8.33'>8.33</td><td class='num' data-sort='8.49'>8.49</td><td class='num' data-sort='8.31'>8.31</td><td class='num' data-sort='6.55'>6.55</td></tr><tr><td class='method' data-sort='Task-8047705376'>Task-8047705376</td><td class='' data-sort='binary'>binary</td><td class='num' data-sort='6.29'>6.29</td><td class='num' data-sort='6.95'>6.95</td><td class='num' data-sort='4.65'>4.65</td><td class='num' data-sort='8.67'>8.67</td><td class='num' data-sort='7.86'>7.86</td><td class='num' data-sort='9.66'>9.66</td><td class='num' data-sort='8.01'>8.01</td><td class='num' data-sort='5.29'>5.29</td><td class='num' data-sort='8.52'>8.52</td><td class='num' data-sort='8.81'>8.81</td><td class='num' data-sort='6.09'>6.09</td><td class='num' data-sort='11.29'>11.29</td><td class='num' data-sort='10.75'>10.75</td><td class='num' data-sort='9.61'>9.61</td><td class='num' data-sort='7.55'>7.55</td></tr><tr><td class='method' data-sort='Task-8120463491'>Task-8120463491</td><td class='' data-sort='multiclass'>multiclass</td><td class='num' data-sort='7.47'>7.47</td><td class='num' data-sort='5.95'>5.95</td><td class='num' data-sort='6.8'>6.80</td><td class='num' data-sort='8.84'>8.84</td><td class='num' data-sort='8.53'>8.53</td><td class='num' data-sort='7.61'>7.61</td><td class='num' data-sort='12.8'>12.80</td><td class='num' data-sort='5.31'>5.31</td><td class='num' data-sort='8.65'>8.65</td><td class='num' data-sort='7.79'>7.79</td><td class='num' data-sort='4.68'>4.68</td><td class='num' data-sort='12.75'>12.75</td><td class='num' data-sort='6.44'>6.44</td><td class='num' data-sort='7.62'>7.62</td><td class='num' data-sort='8.77'>8.77</td></tr><tr><td class='method' data-sort='Task-8167107051'>Task-8167107051</td><td class='' data-sort='binary'>binary</td><td class='num' data-sort='6.6'>6.60</td><td class='num' data-sort='7.54'>7.54</td><td class='num' data-sort='2.29'>2.29</td><td class='num' data-sort='6.41'>6.41</td><td class='num' data-sort='9.41'>9.41</td><td class='num' data-sort='9.27'>9.27</td><td class='num' data-sort='12.54'>12.54</td><td class='num' data-sort='4.17'>4.17</td><td class='num' data-sort='11.0'>11.00</td><td class='num' data-sort='9.93'>9.93</td><td class='num' data-sort='4.86'>4.86</td><td class='num' data-sort='10.37'>10.37</td><td class='num' data-sort='9.95'>9.95</td><td class='num' data-sort='9.91'>9.91</td><td class='num' data-sort='5.77'>5.77</td></tr><tr><td class='method' data-sort='Task-8256787811'>Task-8256787811</td><td class='' data-sort='multiclass'>multiclass</td><td class='num' data-sort='6.79'>6.79</td><td class='num' data-sort='6.82'>6.82</td><td class='num' data-sort='8.39'>8.39</td><td class='num' data-sort='6.31'>6.31</td><td class='num' data-sort='7.82'>7.82</td><td class='num' data-sort='13.25'>13.25</td><td class='num' data-sort='8.63'>8.63</td><td class='num' data-sort='9.63'>9.63</td><td class='num' data-sort='5.95'>5.95</td><td class='num' data-sort='7.13'>7.13</td><td class='num' data-sort='4.25'>4.25</td><td class='num' data-sort='9.03'>9.03</td><td class='num' data-sort='9.45'>9.45</td><td class='num' data-sort='7.83'>7.83</td><td class='num' data-sort='8.71'>8.71</td></tr><tr><td class='method' data-sort='Task-8467301507'>Task-8467301507</td><td class='' data-sort='binary'>binary</td><td class='num' data-sort='6.19'>6.19</td><td class='num' data-sort='2.55'>2.55</td><td class='num' data-sort='3.52'>3.52</td><td class='num' data-sort='7.16'>7.16</td><td class='num' data-sort='10.67'>10.67</td><td class='num' data-sort='10.48'>10.48</td><td class='num' data-sort='14.96'>14.96</td><td class='num' data-sort='3.97'>3.97</td><td class='num' data-sort='5.75'>5.75</td><td class='num' data-sort='7.62'>7.62</td><td class='num' data-sort='3.75'>3.75</td><td class='num' data-sort='12.91'>12.91</td><td class='num' data-sort='13.35'>13.35</td><td class='num' data-sort='10.61'>10.61</td><td class='num' data-sort='6.53'>6.53</td></tr><tr><td class='method' data-sort='Task-8467603651'>Task-8467603651</td><td class='' data-sort='regression'>regression</td><td class='num' data-sort='7.23'>7.23</td><td class='num' data-sort='6.76'>6.76</td><td class='num' data-sort='6.39'>6.39</td><td class='num' data-sort='10.43'>10.43</td><td class='num' data-sort='8.59'>8.59</td><td class='num' data-sort='6.72'>6.72</td><td class='num' data-sort='11.07'>11.07</td><td class='num' data-sort='6.65'>6.65</td><td class='num' data-sort='9.64'>9.64</td><td class='num' data-sort='7.99'>7.99</td><td class='num' data-sort='5.44'>5.44</td><td class='num' data-sort='9.24'>9.24</td><td class='num' data-sort='9.31'>9.31</td><td class='num' data-sort='8.08'>8.08</td><td class='num' data-sort='6.48'>6.48</td></tr><tr><td class='method' data-sort='Task-8480597872'>Task-8480597872</td><td class='' data-sort='binary'>binary</td><td class='num' data-sort='8.51'>8.51</td><td class='num' data-sort='8.63'>8.63</td><td class='num' data-sort='10.15'>10.15</td><td class='num' data-sort='6.6'>6.60</td><td class='num' data-sort='7.14'>7.14</td><td class='num' data-sort='7.86'>7.86</td><td class='num' data-sort='6.73'>6.73</td><td class='num' data-sort='10.07'>10.07</td><td class='num' data-sort='7.06'>7.06</td><td class='num' data-sort='6.93'>6.93</td><td class='num' data-sort='8.32'>8.32</td><td class='num' data-sort='7.91'>7.91</td><td class='num' data-sort='5.43'>5.43</td><td class='num' data-sort='8.55'>8.55</td><td class='num' data-sort='10.12'>10.12</td></tr><tr><td class='method' data-sort='Task-8600470218'>Task-8600470218</td><td class='' data-sort='binary'>binary</td><td class='num' data-sort='6.79'>6.79</td><td class='num' data-sort='7.79'>7.79</td><td class='num' data-sort='5.83'>5.83</td><td class='num' data-sort='8.21'>8.21</td><td class='num' data-sort='8.6'>8.60</td><td class='num' data-sort='8.57'>8.57</td><td class='num' data-sort='13.07'>13.07</td><td class='num' data-sort='8.66'>8.66</td><td class='num' data-sort='8.04'>8.04</td><td class='num' data-sort='5.81'>5.81</td><td class='num' data-sort='8.43'>8.43</td><td class='num' data-sort='10.57'>10.57</td><td class='num' data-sort='6.37'>6.37</td><td class='num' data-sort='6.27'>6.27</td><td class='num' data-sort='6.99'>6.99</td></tr><tr><td class='method' data-sort='Task-8702448794'>Task-8702448794</td><td class='' data-sort='binary'>binary</td><td class='num' data-sort='7.0'>7.00</td><td class='num' data-sort='3.85'>3.85</td><td class='num' data-sort='2.95'>2.95</td><td class='num' data-sort='5.03'>5.03</td><td class='num' data-sort='10.7'>10.70</td><td class='num' data-sort='8.01'>8.01</td><td class='num' data-sort='14.99'>14.99</td><td class='num' data-sort='4.61'>4.61</td><td class='num' data-sort='5.19'>5.19</td><td class='num' data-sort='12.59'>12.59</td><td class='num' data-sort='2.67'>2.67</td><td class='num' data-sort='12.4'>12.40</td><td class='num' data-sort='11.61'>11.61</td><td class='num' data-sort='7.71'>7.71</td><td class='num' data-sort='10.7'>10.70</td></tr><tr><td class='method' data-sort='Task-904079219'>Task-904079219</td><td class='' data-sort='regression'>regression</td><td class='num' data-sort='5.91'>5.91</td><td class='num' data-sort='3.03'>3.03</td><td class='num' data-sort='5.28'>5.28</td><td class='num' data-sort='5.72'>5.72</td><td class='num' data-sort='13.05'>13.05</td><td class='num' data-sort='10.2'>10.20</td><td class='num' data-sort='7.71'>7.71</td><td class='num' data-sort='4.92'>4.92</td><td class='num' data-sort='4.35'>4.35</td><td class='num' data-sort='12.67'>12.67</td><td class='num' data-sort='1.52'>1.52</td><td class='num' data-sort='11.99'>11.99</td><td class='num' data-sort='11.8'>11.80</td><td class='num' data-sort='8.8'>8.80</td><td class='num' data-sort='13.07'>13.07</td></tr><tr><td class='method' data-sort='Task-9955825721'>Task-9955825721</td><td class='' data-sort='binary'>binary</td><td class='num' data-sort='8.86'>8.86</td><td class='num' data-sort='7.81'>7.81</td><td class='num' data-sort='5.81'>5.81</td><td class='num' data-sort='6.46'>6.46</td><td class='num' data-sort='7.86'>7.86</td><td class='num' data-sort='5.33'>5.33</td><td class='num' data-sort='12.43'>12.43</td><td class='num' data-sort='6.25'>6.25</td><td class='num' data-sort='10.51'>10.51</td><td class='num' data-sort='9.01'>9.01</td><td class='num' data-sort='7.34'>7.34</td><td class='num' data-sort='10.87'>10.87</td><td class='num' data-sort='5.24'>5.24</td><td class='num' data-sort='8.27'>8.27</td><td class='num' data-sort='7.96'>7.96</td></tr></tbody></table></div><script type='text/plain' id='sa-csv'>dataset,problem_type,(R)ReliefF,CART,ElasticNet,F-Test,GainRatio,LOCO,LaplacianScore,Lasso,MI,MarkovBlanket,RFImportance,Random,SBE,SFS,mRMR
|
| 31 |
Task-1087039797,binary,8.17,3.48,11.93,11.27,9.17,5.76,13.39,11.76,8.33,5.29,3.69,10.23,6.81,6.16,4.55
|
| 32 |
Task-1119718077,regression,8.04,7.27,8.83,10.8,6.71,7.96,6.51,8.83,7.2,7.94,7.65,8.23,8.42,7.88,7.74
|
|
|
|
| 26 |
td .sub { color:var(--muted); font-size:12px; font-weight:400; margin-left:6px; }
|
| 27 |
td.na { color:var(--muted); }
|
| 28 |
.pill { display:inline-block; padding:0 8px; border:1px solid var(--line); border-radius:999px;
|
| 29 |
+
font-size:12px; color:var(--fg); }
|
| 30 |
+
.pill.cat-search { border-color:var(--accent); }
|
| 31 |
+
.pill.cat-baseline { color:var(--muted); }
|
| 32 |
</style></head><body><div class='toolbar'><span class='hint'>Mean rank of each method per dataset (lower is better). Click a column to sort.</span><button onclick="saCsv('selectarena_per_dataset_ranks.csv')">CSV</button></div><div class='wrap'><table id='sa-table'><thead><tr><th class=''>Dataset</th><th class=''>Problem type</th><th class='num'>(R)ReliefF</th><th class='num'>CART</th><th class='num'>ElasticNet</th><th class='num'>F-Test</th><th class='num'>GainRatio</th><th class='num'>LOCO</th><th class='num'>LaplacianScore</th><th class='num'>Lasso</th><th class='num'>MI</th><th class='num'>MarkovBlanket</th><th class='num'>RFImportance</th><th class='num'>Random</th><th class='num'>SBE</th><th class='num'>SFS</th><th class='num'>mRMR</th></tr></thead><tbody><tr><td class='method' data-sort='Task-1087039797'>Task-1087039797</td><td class='' data-sort='binary'>binary</td><td class='num' data-sort='8.17'>8.17</td><td class='num' data-sort='3.48'>3.48</td><td class='num' data-sort='11.93'>11.93</td><td class='num' data-sort='11.27'>11.27</td><td class='num' data-sort='9.17'>9.17</td><td class='num' data-sort='5.76'>5.76</td><td class='num' data-sort='13.39'>13.39</td><td class='num' data-sort='11.76'>11.76</td><td class='num' data-sort='8.33'>8.33</td><td class='num' data-sort='5.29'>5.29</td><td class='num' data-sort='3.69'>3.69</td><td class='num' data-sort='10.23'>10.23</td><td class='num' data-sort='6.81'>6.81</td><td class='num' data-sort='6.16'>6.16</td><td class='num' data-sort='4.55'>4.55</td></tr><tr><td class='method' data-sort='Task-1119718077'>Task-1119718077</td><td class='' data-sort='regression'>regression</td><td class='num' data-sort='8.04'>8.04</td><td class='num' data-sort='7.27'>7.27</td><td class='num' data-sort='8.83'>8.83</td><td class='num' data-sort='10.8'>10.80</td><td class='num' data-sort='6.71'>6.71</td><td class='num' data-sort='7.96'>7.96</td><td class='num' data-sort='6.51'>6.51</td><td class='num' data-sort='8.83'>8.83</td><td class='num' data-sort='7.2'>7.20</td><td class='num' data-sort='7.94'>7.94</td><td class='num' data-sort='7.65'>7.65</td><td class='num' data-sort='8.23'>8.23</td><td class='num' data-sort='8.42'>8.42</td><td class='num' data-sort='7.88'>7.88</td><td class='num' data-sort='7.74'>7.74</td></tr><tr><td class='method' data-sort='Task-1190597780'>Task-1190597780</td><td class='' data-sort='binary'>binary</td><td class='num' data-sort='6.51'>6.51</td><td class='num' data-sort='9.95'>9.95</td><td class='num' data-sort='7.53'>7.53</td><td class='num' data-sort='6.69'>6.69</td><td class='num' data-sort='7.26'>7.26</td><td class='num' data-sort='8.39'>8.39</td><td class='num' data-sort='7.87'>7.87</td><td class='num' data-sort='7.0'>7.00</td><td class='num' data-sort='6.91'>6.91</td><td class='num' data-sort='8.5'>8.50</td><td class='num' data-sort='7.94'>7.94</td><td class='num' data-sort='9.58'>9.58</td><td class='num' data-sort='9.13'>9.13</td><td class='num' data-sort='9.08'>9.08</td><td class='num' data-sort='7.66'>7.66</td></tr><tr><td class='method' data-sort='Task-1429652751'>Task-1429652751</td><td class='' data-sort='binary'>binary</td><td class='num' data-sort='8.35'>8.35</td><td class='num' data-sort='7.29'>7.29</td><td class='num' data-sort='6.4'>6.40</td><td class='num' data-sort='8.08'>8.08</td><td class='num' data-sort='7.17'>7.17</td><td class='num' data-sort='10.55'>10.55</td><td class='num' data-sort='8.47'>8.47</td><td class='num' data-sort='7.01'>7.01</td><td class='num' data-sort='7.88'>7.88</td><td class='num' data-sort='7.71'>7.71</td><td class='num' data-sort='7.32'>7.32</td><td class='num' data-sort='9.98'>9.98</td><td class='num' data-sort='9.11'>9.11</td><td class='num' data-sort='7.92'>7.92</td><td class='num' data-sort='6.76'>6.76</td></tr><tr><td class='method' data-sort='Task-1458608432'>Task-1458608432</td><td class='' data-sort='binary'>binary</td><td class='num' data-sort='14.12'>14.12</td><td class='num' data-sort='6.07'>6.07</td><td class='num' data-sort='4.05'>4.05</td><td class='num' data-sort='2.73'>2.73</td><td class='num' data-sort='4.9'>4.90</td><td class='num' data-sort='9.72'>9.72</td><td class='num' data-sort='14.44'>14.44</td><td class='num' data-sort='4.06'>4.06</td><td class='num' data-sort='13.23'>13.23</td><td class='num' data-sort='5.53'>5.53</td><td class='num' data-sort='4.03'>4.03</td><td class='num' data-sort='11.31'>11.31</td><td class='num' data-sort='11.19'>11.19</td><td class='num' data-sort='9.89'>9.89</td><td class='num' data-sort='4.64'>4.64</td></tr><tr><td class='method' data-sort='Task-1470619609'>Task-1470619609</td><td class='' data-sort='binary'>binary</td><td class='num' data-sort='6.89'>6.89</td><td class='num' data-sort='6.53'>6.53</td><td class='num' data-sort='5.31'>5.31</td><td class='num' data-sort='7.21'>7.21</td><td class='num' data-sort='6.27'>6.27</td><td class='num' data-sort='11.73'>11.73</td><td class='num' data-sort='14.87'>14.87</td><td class='num' data-sort='4.36'>4.36</td><td class='num' data-sort='5.2'>5.20</td><td class='num' data-sort='6.28'>6.28</td><td class='num' data-sort='4.69'>4.69</td><td class='num' data-sort='11.83'>11.83</td><td class='num' data-sort='11.76'>11.76</td><td class='num' data-sort='11.1'>11.10</td><td class='num' data-sort='5.97'>5.97</td></tr><tr><td class='method' data-sort='Task-1537910476'>Task-1537910476</td><td class='' data-sort='multiclass'>multiclass</td><td class='num' data-sort='8.61'>8.61</td><td class='num' data-sort='8.28'>8.28</td><td class='num' data-sort='6.59'>6.59</td><td class='num' data-sort='6.53'>6.53</td><td class='num' data-sort='7.3'>7.30</td><td class='num' data-sort='8.38'>8.38</td><td class='num' data-sort='10.38'>10.38</td><td class='num' data-sort='6.51'>6.51</td><td class='num' data-sort='7.05'>7.05</td><td class='num' data-sort='6.5'>6.50</td><td class='num' data-sort='6.45'>6.45</td><td class='num' data-sort='9.79'>9.79</td><td class='num' data-sort='7.76'>7.76</td><td class='num' data-sort='12.46'>12.46</td><td class='num' data-sort='7.4'>7.40</td></tr><tr><td class='method' data-sort='Task-1548247551'>Task-1548247551</td><td class='' data-sort='multiclass'>multiclass</td><td class='num' data-sort='4.36'>4.36</td><td class='num' data-sort='10.63'>10.63</td><td class='num' data-sort='4.33'>4.33</td><td class='num' data-sort='5.45'>5.45</td><td class='num' data-sort='5.99'>5.99</td><td class='num' data-sort='11.75'>11.75</td><td class='num' data-sort='13.51'>13.51</td><td class='num' data-sort='5.03'>5.03</td><td class='num' data-sort='5.97'>5.97</td><td class='num' data-sort='6.49'>6.49</td><td class='num' data-sort='4.35'>4.35</td><td class='num' data-sort='11.96'>11.96</td><td class='num' data-sort='12.08'>12.08</td><td class='num' data-sort='12.08'>12.08</td><td class='num' data-sort='6.03'>6.03</td></tr><tr><td class='method' data-sort='Task-156413859'>Task-156413859</td><td class='' data-sort='binary'>binary</td><td class='num' data-sort='6.73'>6.73</td><td class='num' data-sort='5.13'>5.13</td><td class='num' data-sort='9.99'>9.99</td><td class='num' data-sort='10.83'>10.83</td><td class='num' data-sort='7.89'>7.89</td><td class='num' data-sort='4.34'>4.34</td><td class='num' data-sort='14.51'>14.51</td><td class='num' data-sort='9.67'>9.67</td><td class='num' data-sort='6.69'>6.69</td><td class='num' data-sort='8.0'>8.00</td><td class='num' data-sort='4.67'>4.67</td><td class='num' data-sort='11.84'>11.84</td><td class='num' data-sort='3.59'>3.59</td><td class='num' data-sort='5.82'>5.82</td><td class='num' data-sort='10.32'>10.32</td></tr><tr><td class='method' data-sort='Task-1713586811'>Task-1713586811</td><td class='' data-sort='binary'>binary</td><td class='num' data-sort='10.27'>10.27</td><td class='num' data-sort='9.29'>9.29</td><td class='num' data-sort='8.33'>8.33</td><td class='num' data-sort='6.31'>6.31</td><td class='num' data-sort='6.89'>6.89</td><td class='num' data-sort='4.46'>4.46</td><td class='num' data-sort='13.69'>13.69</td><td class='num' data-sort='8.33'>8.33</td><td class='num' data-sort='4.35'>4.35</td><td class='num' data-sort='6.85'>6.85</td><td class='num' data-sort='9.99'>9.99</td><td class='num' data-sort='13.23'>13.23</td><td class='num' data-sort='5.09'>5.09</td><td class='num' data-sort='4.86'>4.86</td><td class='num' data-sort='8.05'>8.05</td></tr><tr><td class='method' data-sort='Task-1737676527'>Task-1737676527</td><td class='' data-sort='regression'>regression</td><td class='num' data-sort='8.84'>8.84</td><td class='num' data-sort='3.17'>3.17</td><td class='num' data-sort='5.44'>5.44</td><td class='num' data-sort='8.93'>8.93</td><td class='num' data-sort='7.55'>7.55</td><td class='num' data-sort='3.75'>3.75</td><td class='num' data-sort='14.93'>14.93</td><td class='num' data-sort='5.43'>5.43</td><td class='num' data-sort='5.27'>5.27</td><td class='num' data-sort='7.62'>7.62</td><td class='num' data-sort='3.55'>3.55</td><td class='num' data-sort='13.31'>13.31</td><td class='num' data-sort='13.69'>13.69</td><td class='num' data-sort='8.0'>8.00</td><td class='num' data-sort='10.52'>10.52</td></tr><tr><td class='method' data-sort='Task-1747559729'>Task-1747559729</td><td class='' data-sort='multiclass'>multiclass</td><td class='num' data-sort='9.89'>9.89</td><td class='num' data-sort='5.05'>5.05</td><td class='num' data-sort='6.16'>6.16</td><td class='num' data-sort='9.97'>9.97</td><td class='num' data-sort='7.06'>7.06</td><td class='num' data-sort='4.71'>4.71</td><td class='num' data-sort='13.64'>13.64</td><td class='num' data-sort='6.14'>6.14</td><td class='num' data-sort='7.12'>7.12</td><td class='num' data-sort='5.66'>5.66</td><td class='num' data-sort='6.83'>6.83</td><td class='num' data-sort='12.35'>12.35</td><td class='num' data-sort='4.95'>4.95</td><td class='num' data-sort='7.87'>7.87</td><td class='num' data-sort='12.6'>12.60</td></tr><tr><td class='method' data-sort='Task-1767785790'>Task-1767785790</td><td class='' data-sort='binary'>binary</td><td class='num' data-sort='8.27'>8.27</td><td class='num' data-sort='8.17'>8.17</td><td class='num' data-sort='8.43'>8.43</td><td class='num' data-sort='8.26'>8.26</td><td class='num' data-sort='7.73'>7.73</td><td class='num' data-sort='7.31'>7.31</td><td class='num' data-sort='8.89'>8.89</td><td class='num' data-sort='7.9'>7.90</td><td class='num' data-sort='7.27'>7.27</td><td class='num' data-sort='6.38'>6.38</td><td class='num' data-sort='9.08'>9.08</td><td class='num' data-sort='7.97'>7.97</td><td class='num' data-sort='6.63'>6.63</td><td class='num' data-sort='10.03'>10.03</td><td class='num' data-sort='7.69'>7.69</td></tr><tr><td class='method' data-sort='Task-2209824369'>Task-2209824369</td><td class='' data-sort='binary'>binary</td><td class='num' data-sort='6.43'>6.43</td><td class='num' data-sort='5.28'>5.28</td><td class='num' data-sort='8.42'>8.42</td><td class='num' data-sort='12.05'>12.05</td><td class='num' data-sort='11.93'>11.93</td><td class='num' data-sort='5.47'>5.47</td><td class='num' data-sort='7.54'>7.54</td><td class='num' data-sort='8.45'>8.45</td><td class='num' data-sort='8.06'>8.06</td><td class='num' data-sort='9.03'>9.03</td><td class='num' data-sort='7.69'>7.69</td><td class='num' data-sort='12.4'>12.40</td><td class='num' data-sort='2.49'>2.49</td><td class='num' data-sort='2.61'>2.61</td><td class='num' data-sort='12.15'>12.15</td></tr><tr><td class='method' data-sort='Task-2263383072'>Task-2263383072</td><td class='' data-sort='binary'>binary</td><td class='num' data-sort='7.45'>7.45</td><td class='num' data-sort='7.36'>7.36</td><td class='num' data-sort='8.25'>8.25</td><td class='num' data-sort='7.35'>7.35</td><td class='num' data-sort='7.37'>7.37</td><td class='num' data-sort='8.13'>8.13</td><td class='num' data-sort='13.37'>13.37</td><td class='num' data-sort='8.03'>8.03</td><td class='num' data-sort='6.67'>6.67</td><td class='num' data-sort='4.32'>4.32</td><td class='num' data-sort='7.36'>7.36</td><td class='num' data-sort='13.23'>13.23</td><td class='num' data-sort='6.65'>6.65</td><td class='num' data-sort='8.27'>8.27</td><td class='num' data-sort='6.19'>6.19</td></tr><tr><td class='method' data-sort='Task-2269177443'>Task-2269177443</td><td class='' data-sort='multiclass'>multiclass</td><td class='num' data-sort='5.35'>5.35</td><td class='num' data-sort='5.23'>5.23</td><td class='num' data-sort='3.94'>3.94</td><td class='num' data-sort='5.99'>5.99</td><td class='num' data-sort='10.97'>10.97</td><td class='num' data-sort='6.72'>6.72</td><td class='num' data-sort='12.91'>12.91</td><td class='num' data-sort='4.66'>4.66</td><td class='num' data-sort='5.0'>5.00</td><td class='num' data-sort='9.85'>9.85</td><td class='num' data-sort='5.71'>5.71</td><td class='num' data-sort='12.24'>12.24</td><td class='num' data-sort='8.88'>8.88</td><td class='num' data-sort='9.88'>9.88</td><td class='num' data-sort='12.68'>12.68</td></tr><tr><td class='method' data-sort='Task-2582981881'>Task-2582981881</td><td class='' data-sort='binary'>binary</td><td class='num' data-sort='6.4'>6.40</td><td class='num' data-sort='8.39'>8.39</td><td class='num' data-sort='4.23'>4.23</td><td class='num' data-sort='8.16'>8.16</td><td class='num' data-sort='10.09'>10.09</td><td class='num' data-sort='10.05'>10.05</td><td class='num' data-sort='12.67'>12.67</td><td class='num' data-sort='5.93'>5.93</td><td class='num' data-sort='8.11'>8.11</td><td class='num' data-sort='8.95'>8.95</td><td class='num' data-sort='4.67'>4.67</td><td class='num' data-sort='7.91'>7.91</td><td class='num' data-sort='8.87'>8.87</td><td class='num' data-sort='5.49'>5.49</td><td class='num' data-sort='10.09'>10.09</td></tr><tr><td class='method' data-sort='Task-2807200459'>Task-2807200459</td><td class='' data-sort='binary'>binary</td><td class='num' data-sort='11.09'>11.09</td><td class='num' data-sort='9.55'>9.55</td><td class='num' data-sort='5.79'>5.79</td><td class='num' data-sort='6.59'>6.59</td><td class='num' data-sort='10.19'>10.19</td><td class='num' data-sort='5.12'>5.12</td><td class='num' data-sort='10.9'>10.90</td><td class='num' data-sort='5.77'>5.77</td><td class='num' data-sort='6.11'>6.11</td><td class='num' data-sort='7.91'>7.91</td><td class='num' data-sort='10.75'>10.75</td><td class='num' data-sort='10.58'>10.58</td><td class='num' data-sort='4.23'>4.23</td><td class='num' data-sort='5.73'>5.73</td><td class='num' data-sort='9.71'>9.71</td></tr><tr><td class='method' data-sort='Task-2824973323'>Task-2824973323</td><td class='' data-sort='multiclass'>multiclass</td><td class='num' data-sort='6.41'>6.41</td><td class='num' data-sort='4.27'>4.27</td><td class='num' data-sort='7.01'>7.01</td><td class='num' data-sort='5.09'>5.09</td><td class='num' data-sort='9.12'>9.12</td><td class='num' data-sort='7.48'>7.48</td><td class='num' data-sort='11.53'>11.53</td><td class='num' data-sort='7.3'>7.30</td><td class='num' data-sort='3.65'>3.65</td><td class='num' data-sort='7.95'>7.95</td><td class='num' data-sort='6.01'>6.01</td><td class='num' data-sort='13.27'>13.27</td><td class='num' data-sort='12.37'>12.37</td><td class='num' data-sort='10.87'>10.87</td><td class='num' data-sort='7.65'>7.65</td></tr><tr><td class='method' data-sort='Task-2922517334'>Task-2922517334</td><td class='' data-sort='regression'>regression</td><td class='num' data-sort='9.56'>9.56</td><td class='num' data-sort='5.04'>5.04</td><td class='num' data-sort='7.77'>7.77</td><td class='num' data-sort='8.19'>8.19</td><td class='num' data-sort='7.87'>7.87</td><td class='num' data-sort='7.5'>7.50</td><td class='num' data-sort='13.72'>13.72</td><td class='num' data-sort='7.89'>7.89</td><td class='num' data-sort='7.78'>7.78</td><td class='num' data-sort='8.11'>8.11</td><td class='num' data-sort='5.46'>5.46</td><td class='num' data-sort='13.79'>13.79</td><td class='num' data-sort='3.91'>3.91</td><td class='num' data-sort='4.59'>4.59</td><td class='num' data-sort='8.84'>8.84</td></tr><tr><td class='method' data-sort='Task-3274731682'>Task-3274731682</td><td class='' data-sort='regression'>regression</td><td class='num' data-sort='5.67'>5.67</td><td class='num' data-sort='2.97'>2.97</td><td class='num' data-sort='8.49'>8.49</td><td class='num' data-sort='10.79'>10.79</td><td class='num' data-sort='7.39'>7.39</td><td class='num' data-sort='10.15'>10.15</td><td class='num' data-sort='11.87'>11.87</td><td class='num' data-sort='8.45'>8.45</td><td class='num' data-sort='8.65'>8.65</td><td class='num' data-sort='6.31'>6.31</td><td class='num' data-sort='2.78'>2.78</td><td class='num' data-sort='11.85'>11.85</td><td class='num' data-sort='5.47'>5.47</td><td class='num' data-sort='8.63'>8.63</td><td class='num' data-sort='10.53'>10.53</td></tr><tr><td class='method' data-sort='Task-3294278573'>Task-3294278573</td><td class='' data-sort='binary'>binary</td><td class='num' data-sort='5.53'>5.53</td><td class='num' data-sort='10.19'>10.19</td><td class='num' data-sort='5.85'>5.85</td><td class='num' data-sort='5.77'>5.77</td><td class='num' data-sort='5.39'>5.39</td><td class='num' data-sort='11.27'>11.27</td><td class='num' data-sort='11.37'>11.37</td><td class='num' data-sort='6.59'>6.59</td><td class='num' data-sort='5.54'>5.54</td><td class='num' data-sort='6.97'>6.97</td><td class='num' data-sort='5.33'>5.33</td><td class='num' data-sort='12.23'>12.23</td><td class='num' data-sort='11.11'>11.11</td><td class='num' data-sort='11.19'>11.19</td><td class='num' data-sort='5.65'>5.65</td></tr><tr><td class='method' data-sort='Task-3421250263'>Task-3421250263</td><td class='' data-sort='binary'>binary</td><td class='num' data-sort='6.83'>6.83</td><td class='num' data-sort='9.0'>9.00</td><td class='num' data-sort='6.89'>6.89</td><td class='num' data-sort='7.27'>7.27</td><td class='num' data-sort='5.91'>5.91</td><td class='num' data-sort='9.03'>9.03</td><td class='num' data-sort='12.43'>12.43</td><td class='num' data-sort='7.73'>7.73</td><td class='num' data-sort='5.83'>5.83</td><td class='num' data-sort='8.26'>8.26</td><td class='num' data-sort='6.93'>6.93</td><td class='num' data-sort='8.74'>8.74</td><td class='num' data-sort='8.76'>8.76</td><td class='num' data-sort='8.15'>8.15</td><td class='num' data-sort='8.23'>8.23</td></tr><tr><td class='method' data-sort='Task-3562356624'>Task-3562356624</td><td class='' data-sort='binary'>binary</td><td class='num' data-sort='9.99'>9.99</td><td class='num' data-sort='8.96'>8.96</td><td class='num' data-sort='6.92'>6.92</td><td class='num' data-sort='6.59'>6.59</td><td class='num' data-sort='9.79'>9.79</td><td class='num' data-sort='8.55'>8.55</td><td class='num' data-sort='5.27'>5.27</td><td class='num' data-sort='6.92'>6.92</td><td class='num' data-sort='9.33'>9.33</td><td class='num' data-sort='8.43'>8.43</td><td class='num' data-sort='8.45'>8.45</td><td class='num' data-sort='6.43'>6.43</td><td class='num' data-sort='6.87'>6.87</td><td class='num' data-sort='6.39'>6.39</td><td class='num' data-sort='11.11'>11.11</td></tr><tr><td class='method' data-sort='Task-4023552040'>Task-4023552040</td><td class='' data-sort='binary'>binary</td><td class='num' data-sort='10.35'>10.35</td><td class='num' data-sort='5.77'>5.77</td><td class='num' data-sort='5.45'>5.45</td><td class='num' data-sort='5.37'>5.37</td><td class='num' data-sort='11.92'>11.92</td><td class='num' data-sort='9.79'>9.79</td><td class='num' data-sort='12.47'>12.47</td><td class='num' data-sort='5.27'>5.27</td><td class='num' data-sort='8.37'>8.37</td><td class='num' data-sort='12.52'>12.52</td><td class='num' data-sort='6.29'>6.29</td><td class='num' data-sort='12.76'>12.76</td><td class='num' data-sort='3.79'>3.79</td><td class='num' data-sort='4.36'>4.36</td><td class='num' data-sort='5.55'>5.55</td></tr><tr><td class='method' data-sort='Task-4265382138'>Task-4265382138</td><td class='' data-sort='binary'>binary</td><td class='num' data-sort='12.05'>12.05</td><td class='num' data-sort='4.87'>4.87</td><td class='num' data-sort='5.08'>5.08</td><td class='num' data-sort='8.73'>8.73</td><td class='num' data-sort='10.39'>10.39</td><td class='num' data-sort='9.77'>9.77</td><td class='num' data-sort='13.05'>13.05</td><td class='num' data-sort='5.99'>5.99</td><td class='num' data-sort='7.51'>7.51</td><td class='num' data-sort='4.93'>4.93</td><td class='num' data-sort='3.89'>3.89</td><td class='num' data-sort='8.03'>8.03</td><td class='num' data-sort='10.21'>10.21</td><td class='num' data-sort='7.19'>7.19</td><td class='num' data-sort='8.31'>8.31</td></tr><tr><td class='method' data-sort='Task-4507720433'>Task-4507720433</td><td class='' data-sort='multiclass'>multiclass</td><td class='num' data-sort='8.33'>8.33</td><td class='num' data-sort='1.79'>1.79</td><td class='num' data-sort='3.67'>3.67</td><td class='num' data-sort='8.2'>8.20</td><td class='num' data-sort='12.01'>12.01</td><td class='num' data-sort='12.4'>12.40</td><td class='num' data-sort='7.44'>7.44</td><td class='num' data-sort='4.68'>4.68</td><td class='num' data-sort='8.6'>8.60</td><td class='num' data-sort='3.29'>3.29</td><td class='num' data-sort='2.47'>2.47</td><td class='num' data-sort='13.33'>13.33</td><td class='num' data-sort='13.23'>13.23</td><td class='num' data-sort='13.41'>13.41</td><td class='num' data-sort='7.15'>7.15</td></tr><tr><td class='method' data-sort='Task-4532501802'>Task-4532501802</td><td class='' data-sort='binary'>binary</td><td class='num' data-sort='7.61'>7.61</td><td class='num' data-sort='9.05'>9.05</td><td class='num' data-sort='7.58'>7.58</td><td class='num' data-sort='7.8'>7.80</td><td class='num' data-sort='7.75'>7.75</td><td class='num' data-sort='8.52'>8.52</td><td class='num' data-sort='6.9'>6.90</td><td class='num' data-sort='8.49'>8.49</td><td class='num' data-sort='7.63'>7.63</td><td class='num' data-sort='6.77'>6.77</td><td class='num' data-sort='6.2'>6.20</td><td class='num' data-sort='9.53'>9.53</td><td class='num' data-sort='7.86'>7.86</td><td class='num' data-sort='9.63'>9.63</td><td class='num' data-sort='8.67'>8.67</td></tr><tr><td class='method' data-sort='Task-4548273248'>Task-4548273248</td><td class='' data-sort='multiclass'>multiclass</td><td class='num' data-sort='6.55'>6.55</td><td class='num' data-sort='7.97'>7.97</td><td class='num' data-sort='5.87'>5.87</td><td class='num' data-sort='8.57'>8.57</td><td class='num' data-sort='5.17'>5.17</td><td class='num' data-sort='9.76'>9.76</td><td class='num' data-sort='14.2'>14.20</td><td class='num' data-sort='8.55'>8.55</td><td class='num' data-sort='5.68'>5.68</td><td class='num' data-sort='6.17'>6.17</td><td class='num' data-sort='5.93'>5.93</td><td class='num' data-sort='9.96'>9.96</td><td class='num' data-sort='10.17'>10.17</td><td class='num' data-sort='9.46'>9.46</td><td class='num' data-sort='5.99'>5.99</td></tr><tr><td class='method' data-sort='Task-4591045950'>Task-4591045950</td><td class='' data-sort='binary'>binary</td><td class='num' data-sort='5.92'>5.92</td><td class='num' data-sort='6.0'>6.00</td><td class='num' data-sort='12.99'>12.99</td><td class='num' data-sort='4.73'>4.73</td><td class='num' data-sort='5.29'>5.29</td><td class='num' data-sort='3.57'>3.57</td><td class='num' data-sort='13.25'>13.25</td><td class='num' data-sort='13.05'>13.05</td><td class='num' data-sort='9.47'>9.47</td><td class='num' data-sort='4.41'>4.41</td><td class='num' data-sort='6.13'>6.13</td><td class='num' data-sort='12.49'>12.49</td><td class='num' data-sort='12.91'>12.91</td><td class='num' data-sort='4.01'>4.01</td><td class='num' data-sort='5.76'>5.76</td></tr><tr><td class='method' data-sort='Task-4613115627'>Task-4613115627</td><td class='' data-sort='binary'>binary</td><td class='num' data-sort='6.23'>6.23</td><td class='num' data-sort='3.36'>3.36</td><td class='num' data-sort='5.25'>5.25</td><td class='num' data-sort='5.96'>5.96</td><td class='num' data-sort='11.37'>11.37</td><td class='num' data-sort='11.37'>11.37</td><td class='num' data-sort='12.72'>12.72</td><td class='num' data-sort='4.23'>4.23</td><td class='num' data-sort='5.51'>5.51</td><td class='num' data-sort='10.48'>10.48</td><td class='num' data-sort='4.17'>4.17</td><td class='num' data-sort='11.24'>11.24</td><td class='num' data-sort='11.08'>11.08</td><td class='num' data-sort='5.67'>5.67</td><td class='num' data-sort='11.37'>11.37</td></tr><tr><td class='method' data-sort='Task-4673108304'>Task-4673108304</td><td class='' data-sort='binary'>binary</td><td class='num' data-sort='10.35'>10.35</td><td class='num' data-sort='8.08'>8.08</td><td class='num' data-sort='5.9'>5.90</td><td class='num' data-sort='10.15'>10.15</td><td class='num' data-sort='7.87'>7.87</td><td class='num' data-sort='8.28'>8.28</td><td class='num' data-sort='8.21'>8.21</td><td class='num' data-sort='6.31'>6.31</td><td class='num' data-sort='8.05'>8.05</td><td class='num' data-sort='7.87'>7.87</td><td class='num' data-sort='7.37'>7.37</td><td class='num' data-sort='8.09'>8.09</td><td class='num' data-sort='8.85'>8.85</td><td class='num' data-sort='6.71'>6.71</td><td class='num' data-sort='7.91'>7.91</td></tr><tr><td class='method' data-sort='Task-4699939988'>Task-4699939988</td><td class='' data-sort='binary'>binary</td><td class='num' data-sort='7.9'>7.90</td><td class='num' data-sort='5.36'>5.36</td><td class='num' data-sort='9.07'>9.07</td><td class='num' data-sort='7.43'>7.43</td><td class='num' data-sort='5.98'>5.98</td><td class='num' data-sort='7.15'>7.15</td><td class='num' data-sort='14.27'>14.27</td><td class='num' data-sort='9.07'>9.07</td><td class='num' data-sort='8.92'>8.92</td><td class='num' data-sort='7.61'>7.61</td><td class='num' data-sort='5.45'>5.45</td><td class='num' data-sort='9.41'>9.41</td><td class='num' data-sort='4.09'>4.09</td><td class='num' data-sort='9.05'>9.05</td><td class='num' data-sort='9.24'>9.24</td></tr><tr><td class='method' data-sort='Task-4867148065'>Task-4867148065</td><td class='' data-sort='binary'>binary</td><td class='num' data-sort='13.15'>13.15</td><td class='num' data-sort='4.73'>4.73</td><td class='num' data-sort='8.38'>8.38</td><td class='num' data-sort='4.19'>4.19</td><td class='num' data-sort='6.85'>6.85</td><td class='num' data-sort='7.36'>7.36</td><td class='num' data-sort='14.97'>14.97</td><td class='num' data-sort='10.65'>10.65</td><td class='num' data-sort='4.97'>4.97</td><td class='num' data-sort='7.4'>7.40</td><td class='num' data-sort='6.19'>6.19</td><td class='num' data-sort='8.01'>8.01</td><td class='num' data-sort='10.65'>10.65</td><td class='num' data-sort='4.64'>4.64</td><td class='num' data-sort='7.85'>7.85</td></tr><tr><td class='method' data-sort='Task-5047778267'>Task-5047778267</td><td class='' data-sort='binary'>binary</td><td class='num' data-sort='4.6'>4.60</td><td class='num' data-sort='8.95'>8.95</td><td class='num' data-sort='5.03'>5.03</td><td class='num' data-sort='5.43'>5.43</td><td class='num' data-sort='5.29'>5.29</td><td class='num' data-sort='12.49'>12.49</td><td class='num' data-sort='13.63'>13.63</td><td class='num' data-sort='5.43'>5.43</td><td class='num' data-sort='5.97'>5.97</td><td class='num' data-sort='7.02'>7.02</td><td class='num' data-sort='6.21'>6.21</td><td class='num' data-sort='12.15'>12.15</td><td class='num' data-sort='12.19'>12.19</td><td class='num' data-sort='10.11'>10.11</td><td class='num' data-sort='5.49'>5.49</td></tr><tr><td class='method' data-sort='Task-5061570515'>Task-5061570515</td><td class='' data-sort='binary'>binary</td><td class='num' data-sort='7.71'>7.71</td><td class='num' data-sort='7.22'>7.22</td><td class='num' data-sort='5.86'>5.86</td><td class='num' data-sort='4.45'>4.45</td><td class='num' data-sort='11.55'>11.55</td><td class='num' data-sort='7.25'>7.25</td><td class='num' data-sort='12.81'>12.81</td><td class='num' data-sort='5.55'>5.55</td><td class='num' data-sort='6.09'>6.09</td><td class='num' data-sort='12.29'>12.29</td><td class='num' data-sort='5.65'>5.65</td><td class='num' data-sort='11.33'>11.33</td><td class='num' data-sort='6.2'>6.20</td><td class='num' data-sort='7.33'>7.33</td><td class='num' data-sort='8.69'>8.69</td></tr><tr><td class='method' data-sort='Task-5079031577'>Task-5079031577</td><td class='' data-sort='regression'>regression</td><td class='num' data-sort='6.12'>6.12</td><td class='num' data-sort='6.69'>6.69</td><td class='num' data-sort='5.97'>5.97</td><td class='num' data-sort='8.65'>8.65</td><td class='num' data-sort='9.19'>9.19</td><td class='num' data-sort='8.87'>8.87</td><td class='num' data-sort='9.03'>9.03</td><td class='num' data-sort='5.97'>5.97</td><td class='num' data-sort='9.27'>9.27</td><td class='num' data-sort='9.22'>9.22</td><td class='num' data-sort='6.08'>6.08</td><td class='num' data-sort='12.32'>12.32</td><td class='num' data-sort='7.08'>7.08</td><td class='num' data-sort='6.63'>6.63</td><td class='num' data-sort='8.92'>8.92</td></tr><tr><td class='method' data-sort='Task-5168164060'>Task-5168164060</td><td class='' data-sort='binary'>binary</td><td class='num' data-sort='6.19'>6.19</td><td class='num' data-sort='11.7'>11.70</td><td class='num' data-sort='4.99'>4.99</td><td class='num' data-sort='6.31'>6.31</td><td class='num' data-sort='9.88'>9.88</td><td class='num' data-sort='9.07'>9.07</td><td class='num' data-sort='7.24'>7.24</td><td class='num' data-sort='6.73'>6.73</td><td class='num' data-sort='6.1'>6.10</td><td class='num' data-sort='10.4'>10.40</td><td class='num' data-sort='9.46'>9.46</td><td class='num' data-sort='11.45'>11.45</td><td class='num' data-sort='7.24'>7.24</td><td class='num' data-sort='6.28'>6.28</td><td class='num' data-sort='6.96'>6.96</td></tr><tr><td class='method' data-sort='Task-5416034620'>Task-5416034620</td><td class='' data-sort='binary'>binary</td><td class='num' data-sort='10.92'>10.92</td><td class='num' data-sort='10.48'>10.48</td><td class='num' data-sort='6.95'>6.95</td><td class='num' data-sort='5.63'>5.63</td><td class='num' data-sort='11.11'>11.11</td><td class='num' data-sort='4.23'>4.23</td><td class='num' data-sort='12.25'>12.25</td><td class='num' data-sort='7.09'>7.09</td><td class='num' data-sort='3.98'>3.98</td><td class='num' data-sort='10.55'>10.55</td><td class='num' data-sort='10.36'>10.36</td><td class='num' data-sort='8.87'>8.87</td><td class='num' data-sort='5.07'>5.07</td><td class='num' data-sort='6.12'>6.12</td><td class='num' data-sort='6.4'>6.40</td></tr><tr><td class='method' data-sort='Task-5622217616'>Task-5622217616</td><td class='' data-sort='binary'>binary</td><td class='num' data-sort='9.35'>9.35</td><td class='num' data-sort='8.57'>8.57</td><td class='num' data-sort='7.77'>7.77</td><td class='num' data-sort='8.96'>8.96</td><td class='num' data-sort='6.07'>6.07</td><td class='num' data-sort='9.43'>9.43</td><td class='num' data-sort='7.92'>7.92</td><td class='num' data-sort='7.77'>7.77</td><td class='num' data-sort='7.67'>7.67</td><td class='num' data-sort='8.33'>8.33</td><td class='num' data-sort='7.27'>7.27</td><td class='num' data-sort='8.91'>8.91</td><td class='num' data-sort='7.6'>7.60</td><td class='num' data-sort='6.87'>6.87</td><td class='num' data-sort='7.5'>7.50</td></tr><tr><td class='method' data-sort='Task-5745371303'>Task-5745371303</td><td class='' data-sort='binary'>binary</td><td class='num' data-sort='8.71'>8.71</td><td class='num' data-sort='9.07'>9.07</td><td class='num' data-sort='6.98'>6.98</td><td class='num' data-sort='7.93'>7.93</td><td class='num' data-sort='8.47'>8.47</td><td class='num' data-sort='7.07'>7.07</td><td class='num' data-sort='13.15'>13.15</td><td class='num' data-sort='6.95'>6.95</td><td class='num' data-sort='6.63'>6.63</td><td class='num' data-sort='5.76'>5.76</td><td class='num' data-sort='8.18'>8.18</td><td class='num' data-sort='12.63'>12.63</td><td class='num' data-sort='4.8'>4.80</td><td class='num' data-sort='7.75'>7.75</td><td class='num' data-sort='5.92'>5.92</td></tr><tr><td class='method' data-sort='Task-5763189736'>Task-5763189736</td><td class='' data-sort='binary'>binary</td><td class='num' data-sort='7.33'>7.33</td><td class='num' data-sort='8.01'>8.01</td><td class='num' data-sort='6.99'>6.99</td><td class='num' data-sort='6.82'>6.82</td><td class='num' data-sort='7.07'>7.07</td><td class='num' data-sort='6.27'>6.27</td><td class='num' data-sort='13.27'>13.27</td><td class='num' data-sort='8.77'>8.77</td><td class='num' data-sort='8.95'>8.95</td><td class='num' data-sort='5.37'>5.37</td><td class='num' data-sort='8.12'>8.12</td><td class='num' data-sort='10.56'>10.56</td><td class='num' data-sort='5.67'>5.67</td><td class='num' data-sort='8.82'>8.82</td><td class='num' data-sort='7.98'>7.98</td></tr><tr><td class='method' data-sort='Task-616857086'>Task-616857086</td><td class='' data-sort='binary'>binary</td><td class='num' data-sort='11.44'>11.44</td><td class='num' data-sort='7.04'>7.04</td><td class='num' data-sort='4.77'>4.77</td><td class='num' data-sort='7.19'>7.19</td><td class='num' data-sort='11.04'>11.04</td><td class='num' data-sort='6.03'>6.03</td><td class='num' data-sort='12.89'>12.89</td><td class='num' data-sort='4.18'>4.18</td><td class='num' data-sort='8.75'>8.75</td><td class='num' data-sort='8.81'>8.81</td><td class='num' data-sort='8.25'>8.25</td><td class='num' data-sort='12.19'>12.19</td><td class='num' data-sort='4.7'>4.70</td><td class='num' data-sort='5.11'>5.11</td><td class='num' data-sort='7.61'>7.61</td></tr><tr><td class='method' data-sort='Task-6346831288'>Task-6346831288</td><td class='' data-sort='multiclass'>multiclass</td><td class='num' data-sort='10.72'>10.72</td><td class='num' data-sort='5.16'>5.16</td><td class='num' data-sort='11.43'>11.43</td><td class='num' data-sort='10.59'>10.59</td><td class='num' data-sort='7.19'>7.19</td><td class='num' data-sort='10.28'>10.28</td><td class='num' data-sort='8.29'>8.29</td><td class='num' data-sort='13.01'>13.01</td><td class='num' data-sort='4.27'>4.27</td><td class='num' data-sort='5.92'>5.92</td><td class='num' data-sort='3.51'>3.51</td><td class='num' data-sort='9.27'>9.27</td><td class='num' data-sort='8.29'>8.29</td><td class='num' data-sort='6.09'>6.09</td><td class='num' data-sort='5.99'>5.99</td></tr><tr><td class='method' data-sort='Task-6392717221'>Task-6392717221</td><td class='' data-sort='binary'>binary</td><td class='num' data-sort='7.68'>7.68</td><td class='num' data-sort='3.87'>3.87</td><td class='num' data-sort='5.69'>5.69</td><td class='num' data-sort='5.23'>5.23</td><td class='num' data-sort='9.24'>9.24</td><td class='num' data-sort='11.72'>11.72</td><td class='num' data-sort='15.0'>15.00</td><td class='num' data-sort='5.59'>5.59</td><td class='num' data-sort='3.4'>3.40</td><td class='num' data-sort='5.33'>5.33</td><td class='num' data-sort='2.96'>2.96</td><td class='num' data-sort='12.71'>12.71</td><td class='num' data-sort='12.96'>12.96</td><td class='num' data-sort='8.97'>8.97</td><td class='num' data-sort='9.65'>9.65</td></tr><tr><td class='method' data-sort='Task-6680748470'>Task-6680748470</td><td class='' data-sort='binary'>binary</td><td class='num' data-sort='5.51'>5.51</td><td class='num' data-sort='4.86'>4.86</td><td class='num' data-sort='8.5'>8.50</td><td class='num' data-sort='7.39'>7.39</td><td class='num' data-sort='8.56'>8.56</td><td class='num' data-sort='7.76'>7.76</td><td class='num' data-sort='11.31'>11.31</td><td class='num' data-sort='9.57'>9.57</td><td class='num' data-sort='6.57'>6.57</td><td class='num' data-sort='9.64'>9.64</td><td class='num' data-sort='4.4'>4.40</td><td class='num' data-sort='10.53'>10.53</td><td class='num' data-sort='7.44'>7.44</td><td class='num' data-sort='9.33'>9.33</td><td class='num' data-sort='8.63'>8.63</td></tr><tr><td class='method' data-sort='Task-6789920400'>Task-6789920400</td><td class='' data-sort='binary'>binary</td><td class='num' data-sort='7.23'>7.23</td><td class='num' data-sort='7.45'>7.45</td><td class='num' data-sort='6.23'>6.23</td><td class='num' data-sort='7.31'>7.31</td><td class='num' data-sort='7.11'>7.11</td><td class='num' data-sort='11.11'>11.11</td><td class='num' data-sort='13.89'>13.89</td><td class='num' data-sort='5.27'>5.27</td><td class='num' data-sort='7.49'>7.49</td><td class='num' data-sort='6.31'>6.31</td><td class='num' data-sort='5.86'>5.86</td><td class='num' data-sort='10.75'>10.75</td><td class='num' data-sort='10.34'>10.34</td><td class='num' data-sort='7.73'>7.73</td><td class='num' data-sort='5.91'>5.91</td></tr><tr><td class='method' data-sort='Task-6859627074'>Task-6859627074</td><td class='' data-sort='binary'>binary</td><td class='num' data-sort='13.05'>13.05</td><td class='num' data-sort='7.02'>7.02</td><td class='num' data-sort='11.47'>11.47</td><td class='num' data-sort='10.83'>10.83</td><td class='num' data-sort='3.43'>3.43</td><td class='num' data-sort='7.76'>7.76</td><td class='num' data-sort='9.41'>9.41</td><td class='num' data-sort='11.49'>11.49</td><td class='num' data-sort='3.89'>3.89</td><td class='num' data-sort='6.55'>6.55</td><td class='num' data-sort='7.25'>7.25</td><td class='num' data-sort='9.3'>9.30</td><td class='num' data-sort='6.51'>6.51</td><td class='num' data-sort='5.67'>5.67</td><td class='num' data-sort='6.37'>6.37</td></tr><tr><td class='method' data-sort='Task-7014870989'>Task-7014870989</td><td class='' data-sort='multiclass'>multiclass</td><td class='num' data-sort='9.85'>9.85</td><td class='num' data-sort='5.27'>5.27</td><td class='num' data-sort='6.15'>6.15</td><td class='num' data-sort='6.15'>6.15</td><td class='num' data-sort='8.57'>8.57</td><td class='num' data-sort='10.55'>10.55</td><td class='num' data-sort='13.39'>13.39</td><td class='num' data-sort='6.57'>6.57</td><td class='num' data-sort='6.25'>6.25</td><td class='num' data-sort='4.48'>4.48</td><td class='num' data-sort='5.44'>5.44</td><td class='num' data-sort='11.97'>11.97</td><td class='num' data-sort='11.95'>11.95</td><td class='num' data-sort='8.17'>8.17</td><td class='num' data-sort='5.24'>5.24</td></tr><tr><td class='method' data-sort='Task-7152814512'>Task-7152814512</td><td class='' data-sort='binary'>binary</td><td class='num' data-sort='4.31'>4.31</td><td class='num' data-sort='4.44'>4.44</td><td class='num' data-sort='8.17'>8.17</td><td class='num' data-sort='9.15'>9.15</td><td class='num' data-sort='9.15'>9.15</td><td class='num' data-sort='6.27'>6.27</td><td class='num' data-sort='14.69'>14.69</td><td class='num' data-sort='8.32'>8.32</td><td class='num' data-sort='9.35'>9.35</td><td class='num' data-sort='6.79'>6.79</td><td class='num' data-sort='5.71'>5.71</td><td class='num' data-sort='12.51'>12.51</td><td class='num' data-sort='5.35'>5.35</td><td class='num' data-sort='6.75'>6.75</td><td class='num' data-sort='9.05'>9.05</td></tr><tr><td class='method' data-sort='Task-7219454773'>Task-7219454773</td><td class='' data-sort='binary'>binary</td><td class='num' data-sort='9.23'>9.23</td><td class='num' data-sort='5.95'>5.95</td><td class='num' data-sort='7.67'>7.67</td><td class='num' data-sort='8.89'>8.89</td><td class='num' data-sort='6.29'>6.29</td><td class='num' data-sort='6.85'>6.85</td><td class='num' data-sort='11.43'>11.43</td><td class='num' data-sort='8.02'>8.02</td><td class='num' data-sort='8.01'>8.01</td><td class='num' data-sort='5.67'>5.67</td><td class='num' data-sort='6.16'>6.16</td><td class='num' data-sort='9.36'>9.36</td><td class='num' data-sort='9.93'>9.93</td><td class='num' data-sort='7.61'>7.61</td><td class='num' data-sort='8.92'>8.92</td></tr><tr><td class='method' data-sort='Task-7315435704'>Task-7315435704</td><td class='' data-sort='binary'>binary</td><td class='num' data-sort='5.01'>5.01</td><td class='num' data-sort='7.36'>7.36</td><td class='num' data-sort='10.92'>10.92</td><td class='num' data-sort='5.7'>5.70</td><td class='num' data-sort='9.11'>9.11</td><td class='num' data-sort='7.43'>7.43</td><td class='num' data-sort='8.52'>8.52</td><td class='num' data-sort='10.92'>10.92</td><td class='num' data-sort='4.41'>4.41</td><td class='num' data-sort='10.33'>10.33</td><td class='num' data-sort='8.27'>8.27</td><td class='num' data-sort='8.84'>8.84</td><td class='num' data-sort='8.01'>8.01</td><td class='num' data-sort='9.46'>9.46</td><td class='num' data-sort='5.71'>5.71</td></tr><tr><td class='method' data-sort='Task-7493221688'>Task-7493221688</td><td class='' data-sort='regression'>regression</td><td class='num' data-sort='7.89'>7.89</td><td class='num' data-sort='6.73'>6.73</td><td class='num' data-sort='8.36'>8.36</td><td class='num' data-sort='6.39'>6.39</td><td class='num' data-sort='7.79'>7.79</td><td class='num' data-sort='7.57'>7.57</td><td class='num' data-sort='12.43'>12.43</td><td class='num' data-sort='8.37'>8.37</td><td class='num' data-sort='7.59'>7.59</td><td class='num' data-sort='5.97'>5.97</td><td class='num' data-sort='6.1'>6.10</td><td class='num' data-sort='10.11'>10.11</td><td class='num' data-sort='8.55'>8.55</td><td class='num' data-sort='9.81'>9.81</td><td class='num' data-sort='6.34'>6.34</td></tr><tr><td class='method' data-sort='Task-7603283006'>Task-7603283006</td><td class='' data-sort='regression'>regression</td><td class='num' data-sort='8.08'>8.08</td><td class='num' data-sort='4.27'>4.27</td><td class='num' data-sort='9.77'>9.77</td><td class='num' data-sort='7.47'>7.47</td><td class='num' data-sort='8.78'>8.78</td><td class='num' data-sort='7.23'>7.23</td><td class='num' data-sort='12.84'>12.84</td><td class='num' data-sort='9.82'>9.82</td><td class='num' data-sort='4.83'>4.83</td><td class='num' data-sort='7.68'>7.68</td><td class='num' data-sort='3.89'>3.89</td><td class='num' data-sort='9.64'>9.64</td><td class='num' data-sort='7.35'>7.35</td><td class='num' data-sort='10.03'>10.03</td><td class='num' data-sort='8.33'>8.33</td></tr><tr><td class='method' data-sort='Task-7612659684'>Task-7612659684</td><td class='' data-sort='multiclass'>multiclass</td><td class='num' data-sort='8.97'>8.97</td><td class='num' data-sort='4.24'>4.24</td><td class='num' data-sort='11.21'>11.21</td><td class='num' data-sort='7.73'>7.73</td><td class='num' data-sort='7.23'>7.23</td><td class='num' data-sort='8.79'>8.79</td><td class='num' data-sort='12.57'>12.57</td><td class='num' data-sort='7.87'>7.87</td><td class='num' data-sort='3.25'>3.25</td><td class='num' data-sort='8.29'>8.29</td><td class='num' data-sort='5.17'>5.17</td><td class='num' data-sort='10.12'>10.12</td><td class='num' data-sort='8.07'>8.07</td><td class='num' data-sort='8.45'>8.45</td><td class='num' data-sort='8.03'>8.03</td></tr><tr><td class='method' data-sort='Task-7641908575'>Task-7641908575</td><td class='' data-sort='binary'>binary</td><td class='num' data-sort='9.37'>9.37</td><td class='num' data-sort='8.8'>8.80</td><td class='num' data-sort='6.61'>6.61</td><td class='num' data-sort='7.41'>7.41</td><td class='num' data-sort='6.72'>6.72</td><td class='num' data-sort='12.05'>12.05</td><td class='num' data-sort='13.77'>13.77</td><td class='num' data-sort='6.65'>6.65</td><td class='num' data-sort='6.42'>6.42</td><td class='num' data-sort='5.17'>5.17</td><td class='num' data-sort='9.33'>9.33</td><td class='num' data-sort='10.82'>10.82</td><td class='num' data-sort='4.64'>4.64</td><td class='num' data-sort='4.89'>4.89</td><td class='num' data-sort='7.33'>7.33</td></tr><tr><td class='method' data-sort='Task-7704446119'>Task-7704446119</td><td class='' data-sort='regression'>regression</td><td class='num' data-sort='5.84'>5.84</td><td class='num' data-sort='10.72'>10.72</td><td class='num' data-sort='3.8'>3.80</td><td class='num' data-sort='3.56'>3.56</td><td class='num' data-sort='8.51'>8.51</td><td class='num' data-sort='9.39'>9.39</td><td class='num' data-sort='11.8'>11.80</td><td class='num' data-sort='3.63'>3.63</td><td class='num' data-sort='9.39'>9.39</td><td class='num' data-sort='8.35'>8.35</td><td class='num' data-sort='10.89'>10.89</td><td class='num' data-sort='9.25'>9.25</td><td class='num' data-sort='9.47'>9.47</td><td class='num' data-sort='7.28'>7.28</td><td class='num' data-sort='8.13'>8.13</td></tr><tr><td class='method' data-sort='Task-7911827689'>Task-7911827689</td><td class='' data-sort='binary'>binary</td><td class='num' data-sort='10.55'>10.55</td><td class='num' data-sort='6.23'>6.23</td><td class='num' data-sort='3.39'>3.39</td><td class='num' data-sort='4.99'>4.99</td><td class='num' data-sort='6.16'>6.16</td><td class='num' data-sort='8.96'>8.96</td><td class='num' data-sort='13.63'>13.63</td><td class='num' data-sort='7.63'>7.63</td><td class='num' data-sort='7.11'>7.11</td><td class='num' data-sort='5.6'>5.60</td><td class='num' data-sort='5.49'>5.49</td><td class='num' data-sort='10.07'>10.07</td><td class='num' data-sort='9.39'>9.39</td><td class='num' data-sort='11.64'>11.64</td><td class='num' data-sort='9.17'>9.17</td></tr><tr><td class='method' data-sort='Task-8002938196'>Task-8002938196</td><td class='' data-sort='binary'>binary</td><td class='num' data-sort='7.35'>7.35</td><td class='num' data-sort='5.9'>5.90</td><td class='num' data-sort='6.93'>6.93</td><td class='num' data-sort='6.34'>6.34</td><td class='num' data-sort='8.11'>8.11</td><td class='num' data-sort='9.09'>9.09</td><td class='num' data-sort='10.71'>10.71</td><td class='num' data-sort='6.51'>6.51</td><td class='num' data-sort='7.6'>7.60</td><td class='num' data-sort='7.17'>7.17</td><td class='num' data-sort='6.85'>6.85</td><td class='num' data-sort='10.56'>10.56</td><td class='num' data-sort='10.59'>10.59</td><td class='num' data-sort='9.51'>9.51</td><td class='num' data-sort='6.78'>6.78</td></tr><tr><td class='method' data-sort='Task-8017089756'>Task-8017089756</td><td class='' data-sort='binary'>binary</td><td class='num' data-sort='8.97'>8.97</td><td class='num' data-sort='8.13'>8.13</td><td class='num' data-sort='7.69'>7.69</td><td class='num' data-sort='7.54'>7.54</td><td class='num' data-sort='7.3'>7.30</td><td class='num' data-sort='9.24'>9.24</td><td class='num' data-sort='11.63'>11.63</td><td class='num' data-sort='8.11'>8.11</td><td class='num' data-sort='6.63'>6.63</td><td class='num' data-sort='6.28'>6.28</td><td class='num' data-sort='6.81'>6.81</td><td class='num' data-sort='8.33'>8.33</td><td class='num' data-sort='8.49'>8.49</td><td class='num' data-sort='8.31'>8.31</td><td class='num' data-sort='6.55'>6.55</td></tr><tr><td class='method' data-sort='Task-8047705376'>Task-8047705376</td><td class='' data-sort='binary'>binary</td><td class='num' data-sort='6.29'>6.29</td><td class='num' data-sort='6.95'>6.95</td><td class='num' data-sort='4.65'>4.65</td><td class='num' data-sort='8.67'>8.67</td><td class='num' data-sort='7.86'>7.86</td><td class='num' data-sort='9.66'>9.66</td><td class='num' data-sort='8.01'>8.01</td><td class='num' data-sort='5.29'>5.29</td><td class='num' data-sort='8.52'>8.52</td><td class='num' data-sort='8.81'>8.81</td><td class='num' data-sort='6.09'>6.09</td><td class='num' data-sort='11.29'>11.29</td><td class='num' data-sort='10.75'>10.75</td><td class='num' data-sort='9.61'>9.61</td><td class='num' data-sort='7.55'>7.55</td></tr><tr><td class='method' data-sort='Task-8120463491'>Task-8120463491</td><td class='' data-sort='multiclass'>multiclass</td><td class='num' data-sort='7.47'>7.47</td><td class='num' data-sort='5.95'>5.95</td><td class='num' data-sort='6.8'>6.80</td><td class='num' data-sort='8.84'>8.84</td><td class='num' data-sort='8.53'>8.53</td><td class='num' data-sort='7.61'>7.61</td><td class='num' data-sort='12.8'>12.80</td><td class='num' data-sort='5.31'>5.31</td><td class='num' data-sort='8.65'>8.65</td><td class='num' data-sort='7.79'>7.79</td><td class='num' data-sort='4.68'>4.68</td><td class='num' data-sort='12.75'>12.75</td><td class='num' data-sort='6.44'>6.44</td><td class='num' data-sort='7.62'>7.62</td><td class='num' data-sort='8.77'>8.77</td></tr><tr><td class='method' data-sort='Task-8167107051'>Task-8167107051</td><td class='' data-sort='binary'>binary</td><td class='num' data-sort='6.6'>6.60</td><td class='num' data-sort='7.54'>7.54</td><td class='num' data-sort='2.29'>2.29</td><td class='num' data-sort='6.41'>6.41</td><td class='num' data-sort='9.41'>9.41</td><td class='num' data-sort='9.27'>9.27</td><td class='num' data-sort='12.54'>12.54</td><td class='num' data-sort='4.17'>4.17</td><td class='num' data-sort='11.0'>11.00</td><td class='num' data-sort='9.93'>9.93</td><td class='num' data-sort='4.86'>4.86</td><td class='num' data-sort='10.37'>10.37</td><td class='num' data-sort='9.95'>9.95</td><td class='num' data-sort='9.91'>9.91</td><td class='num' data-sort='5.77'>5.77</td></tr><tr><td class='method' data-sort='Task-8256787811'>Task-8256787811</td><td class='' data-sort='multiclass'>multiclass</td><td class='num' data-sort='6.79'>6.79</td><td class='num' data-sort='6.82'>6.82</td><td class='num' data-sort='8.39'>8.39</td><td class='num' data-sort='6.31'>6.31</td><td class='num' data-sort='7.82'>7.82</td><td class='num' data-sort='13.25'>13.25</td><td class='num' data-sort='8.63'>8.63</td><td class='num' data-sort='9.63'>9.63</td><td class='num' data-sort='5.95'>5.95</td><td class='num' data-sort='7.13'>7.13</td><td class='num' data-sort='4.25'>4.25</td><td class='num' data-sort='9.03'>9.03</td><td class='num' data-sort='9.45'>9.45</td><td class='num' data-sort='7.83'>7.83</td><td class='num' data-sort='8.71'>8.71</td></tr><tr><td class='method' data-sort='Task-8467301507'>Task-8467301507</td><td class='' data-sort='binary'>binary</td><td class='num' data-sort='6.19'>6.19</td><td class='num' data-sort='2.55'>2.55</td><td class='num' data-sort='3.52'>3.52</td><td class='num' data-sort='7.16'>7.16</td><td class='num' data-sort='10.67'>10.67</td><td class='num' data-sort='10.48'>10.48</td><td class='num' data-sort='14.96'>14.96</td><td class='num' data-sort='3.97'>3.97</td><td class='num' data-sort='5.75'>5.75</td><td class='num' data-sort='7.62'>7.62</td><td class='num' data-sort='3.75'>3.75</td><td class='num' data-sort='12.91'>12.91</td><td class='num' data-sort='13.35'>13.35</td><td class='num' data-sort='10.61'>10.61</td><td class='num' data-sort='6.53'>6.53</td></tr><tr><td class='method' data-sort='Task-8467603651'>Task-8467603651</td><td class='' data-sort='regression'>regression</td><td class='num' data-sort='7.23'>7.23</td><td class='num' data-sort='6.76'>6.76</td><td class='num' data-sort='6.39'>6.39</td><td class='num' data-sort='10.43'>10.43</td><td class='num' data-sort='8.59'>8.59</td><td class='num' data-sort='6.72'>6.72</td><td class='num' data-sort='11.07'>11.07</td><td class='num' data-sort='6.65'>6.65</td><td class='num' data-sort='9.64'>9.64</td><td class='num' data-sort='7.99'>7.99</td><td class='num' data-sort='5.44'>5.44</td><td class='num' data-sort='9.24'>9.24</td><td class='num' data-sort='9.31'>9.31</td><td class='num' data-sort='8.08'>8.08</td><td class='num' data-sort='6.48'>6.48</td></tr><tr><td class='method' data-sort='Task-8480597872'>Task-8480597872</td><td class='' data-sort='binary'>binary</td><td class='num' data-sort='8.51'>8.51</td><td class='num' data-sort='8.63'>8.63</td><td class='num' data-sort='10.15'>10.15</td><td class='num' data-sort='6.6'>6.60</td><td class='num' data-sort='7.14'>7.14</td><td class='num' data-sort='7.86'>7.86</td><td class='num' data-sort='6.73'>6.73</td><td class='num' data-sort='10.07'>10.07</td><td class='num' data-sort='7.06'>7.06</td><td class='num' data-sort='6.93'>6.93</td><td class='num' data-sort='8.32'>8.32</td><td class='num' data-sort='7.91'>7.91</td><td class='num' data-sort='5.43'>5.43</td><td class='num' data-sort='8.55'>8.55</td><td class='num' data-sort='10.12'>10.12</td></tr><tr><td class='method' data-sort='Task-8600470218'>Task-8600470218</td><td class='' data-sort='binary'>binary</td><td class='num' data-sort='6.79'>6.79</td><td class='num' data-sort='7.79'>7.79</td><td class='num' data-sort='5.83'>5.83</td><td class='num' data-sort='8.21'>8.21</td><td class='num' data-sort='8.6'>8.60</td><td class='num' data-sort='8.57'>8.57</td><td class='num' data-sort='13.07'>13.07</td><td class='num' data-sort='8.66'>8.66</td><td class='num' data-sort='8.04'>8.04</td><td class='num' data-sort='5.81'>5.81</td><td class='num' data-sort='8.43'>8.43</td><td class='num' data-sort='10.57'>10.57</td><td class='num' data-sort='6.37'>6.37</td><td class='num' data-sort='6.27'>6.27</td><td class='num' data-sort='6.99'>6.99</td></tr><tr><td class='method' data-sort='Task-8702448794'>Task-8702448794</td><td class='' data-sort='binary'>binary</td><td class='num' data-sort='7.0'>7.00</td><td class='num' data-sort='3.85'>3.85</td><td class='num' data-sort='2.95'>2.95</td><td class='num' data-sort='5.03'>5.03</td><td class='num' data-sort='10.7'>10.70</td><td class='num' data-sort='8.01'>8.01</td><td class='num' data-sort='14.99'>14.99</td><td class='num' data-sort='4.61'>4.61</td><td class='num' data-sort='5.19'>5.19</td><td class='num' data-sort='12.59'>12.59</td><td class='num' data-sort='2.67'>2.67</td><td class='num' data-sort='12.4'>12.40</td><td class='num' data-sort='11.61'>11.61</td><td class='num' data-sort='7.71'>7.71</td><td class='num' data-sort='10.7'>10.70</td></tr><tr><td class='method' data-sort='Task-904079219'>Task-904079219</td><td class='' data-sort='regression'>regression</td><td class='num' data-sort='5.91'>5.91</td><td class='num' data-sort='3.03'>3.03</td><td class='num' data-sort='5.28'>5.28</td><td class='num' data-sort='5.72'>5.72</td><td class='num' data-sort='13.05'>13.05</td><td class='num' data-sort='10.2'>10.20</td><td class='num' data-sort='7.71'>7.71</td><td class='num' data-sort='4.92'>4.92</td><td class='num' data-sort='4.35'>4.35</td><td class='num' data-sort='12.67'>12.67</td><td class='num' data-sort='1.52'>1.52</td><td class='num' data-sort='11.99'>11.99</td><td class='num' data-sort='11.8'>11.80</td><td class='num' data-sort='8.8'>8.80</td><td class='num' data-sort='13.07'>13.07</td></tr><tr><td class='method' data-sort='Task-9955825721'>Task-9955825721</td><td class='' data-sort='binary'>binary</td><td class='num' data-sort='8.86'>8.86</td><td class='num' data-sort='7.81'>7.81</td><td class='num' data-sort='5.81'>5.81</td><td class='num' data-sort='6.46'>6.46</td><td class='num' data-sort='7.86'>7.86</td><td class='num' data-sort='5.33'>5.33</td><td class='num' data-sort='12.43'>12.43</td><td class='num' data-sort='6.25'>6.25</td><td class='num' data-sort='10.51'>10.51</td><td class='num' data-sort='9.01'>9.01</td><td class='num' data-sort='7.34'>7.34</td><td class='num' data-sort='10.87'>10.87</td><td class='num' data-sort='5.24'>5.24</td><td class='num' data-sort='8.27'>8.27</td><td class='num' data-sort='7.96'>7.96</td></tr></tbody></table></div><script type='text/plain' id='sa-csv'>dataset,problem_type,(R)ReliefF,CART,ElasticNet,F-Test,GainRatio,LOCO,LaplacianScore,Lasso,MI,MarkovBlanket,RFImportance,Random,SBE,SFS,mRMR
|
| 33 |
Task-1087039797,binary,8.17,3.48,11.93,11.27,9.17,5.76,13.39,11.76,8.33,5.29,3.69,10.23,6.81,6.16,4.55
|
| 34 |
Task-1119718077,regression,8.04,7.27,8.83,10.8,6.71,7.96,6.51,8.83,7.2,7.94,7.65,8.23,8.42,7.88,7.74
|
data/stability_table.html
CHANGED
|
@@ -26,7 +26,9 @@ td.method { font-weight:600; }
|
|
| 26 |
td .sub { color:var(--muted); font-size:12px; font-weight:400; margin-left:6px; }
|
| 27 |
td.na { color:var(--muted); }
|
| 28 |
.pill { display:inline-block; padding:0 8px; border:1px solid var(--line); border-radius:999px;
|
| 29 |
-
font-size:12px; color:var(--
|
|
|
|
|
|
|
| 30 |
</style></head><body><div class='toolbar'><span class='hint'>Click a column to sort.</span><button onclick="saCsv('selectarena_stability.csv')">CSV</button></div><div class='wrap'><table id='sa-table'><thead><tr><th class=''>Method</th><th class='num' title='Mean over datasets and budgets.'>Stability β</th><th class='num'>Budget 1 (smallest)</th><th class='num'>Budget 2</th><th class='num'>Budget 3</th><th class='num'>Budget 4</th><th class='num'>Budget 5 (largest)</th></tr></thead><tbody><tr><td class='method' data-sort='F-Test'>F-Test</td><td class='num' data-sort='0.679'>0.679</td><td class='num' data-sort='0.7253'>0.725</td><td class='num' data-sort='0.734'>0.734</td><td class='num' data-sort='0.7056'>0.706</td><td class='num' data-sort='0.6678'>0.668</td><td class='num' data-sort='0.5622'>0.562</td></tr><tr><td class='method' data-sort='LaplacianScore'>LaplacianScore</td><td class='num' data-sort='0.6518'>0.652</td><td class='num' data-sort='0.5883'>0.588</td><td class='num' data-sort='0.6445'>0.644</td><td class='num' data-sort='0.6596'>0.660</td><td class='num' data-sort='0.6889'>0.689</td><td class='num' data-sort='0.6777'>0.678</td></tr><tr><td class='method' data-sort='MI'>MI</td><td class='num' data-sort='0.6474'>0.647</td><td class='num' data-sort='0.6455'>0.645</td><td class='num' data-sort='0.6726'>0.673</td><td class='num' data-sort='0.6829'>0.683</td><td class='num' data-sort='0.6494'>0.649</td><td class='num' data-sort='0.5868'>0.587</td></tr><tr><td class='method' data-sort='(R)ReliefF'>(R)ReliefF</td><td class='num' data-sort='0.646'>0.646</td><td class='num' data-sort='0.6415'>0.641</td><td class='num' data-sort='0.647'>0.647</td><td class='num' data-sort='0.6725'>0.672</td><td class='num' data-sort='0.6563'>0.656</td><td class='num' data-sort='0.6124'>0.612</td></tr><tr><td class='method' data-sort='GainRatio'>GainRatio</td><td class='num' data-sort='0.6441'>0.644</td><td class='num' data-sort='0.6788'>0.679</td><td class='num' data-sort='0.6807'>0.681</td><td class='num' data-sort='0.6703'>0.670</td><td class='num' data-sort='0.641'>0.641</td><td class='num' data-sort='0.5498'>0.550</td></tr><tr><td class='method' data-sort='RFImportance'>RFImportance</td><td class='num' data-sort='0.6432'>0.643</td><td class='num' data-sort='0.6282'>0.628</td><td class='num' data-sort='0.6529'>0.653</td><td class='num' data-sort='0.6762'>0.676</td><td class='num' data-sort='0.6613'>0.661</td><td class='num' data-sort='0.5974'>0.597</td></tr><tr><td class='method' data-sort='mRMR'>mRMR</td><td class='num' data-sort='0.6036'>0.604</td><td class='num' data-sort='0.5276'>0.528</td><td class='num' data-sort='0.5784'>0.578</td><td class='num' data-sort='0.6139'>0.614</td><td class='num' data-sort='0.656'>0.656</td><td class='num' data-sort='0.651'>0.651</td></tr><tr><td class='method' data-sort='ElasticNet'>ElasticNet</td><td class='num' data-sort='0.4472'>0.447</td><td class='num' data-sort='0.5094'>0.509</td><td class='num' data-sort='0.5187'>0.519</td><td class='num' data-sort='0.4808'>0.481</td><td class='num' data-sort='0.4188'>0.419</td><td class='num' data-sort='0.3083'>0.308</td></tr><tr><td class='method' data-sort='CART'>CART</td><td class='num' data-sort='0.422'>0.422</td><td class='num' data-sort='0.4838'>0.484</td><td class='num' data-sort='0.4556'>0.456</td><td class='num' data-sort='0.4444'>0.444</td><td class='num' data-sort='0.4018'>0.402</td><td class='num' data-sort='0.3241'>0.324</td></tr><tr><td class='method' data-sort='MarkovBlanket'>MarkovBlanket</td><td class='num' data-sort='0.413'>0.413</td><td class='num' data-sort='0.4177'>0.418</td><td class='num' data-sort='0.3958'>0.396</td><td class='num' data-sort='0.4132'>0.413</td><td class='num' data-sort='0.4337'>0.434</td><td class='num' data-sort='0.4045'>0.405</td></tr><tr><td class='method' data-sort='Lasso'>Lasso</td><td class='num' data-sort='0.3623'>0.362</td><td class='num' data-sort='0.4208'>0.421</td><td class='num' data-sort='0.4216'>0.422</td><td class='num' data-sort='0.3889'>0.389</td><td class='num' data-sort='0.3343'>0.334</td><td class='num' data-sort='0.2457'>0.246</td></tr><tr><td class='method' data-sort='SFS'>SFS</td><td class='num' data-sort='0.2668'>0.267</td><td class='num' data-sort='0.3677'>0.368</td><td class='num' data-sort='0.2781'>0.278</td><td class='num' data-sort='0.2392'>0.239</td><td class='num' data-sort='0.2248'>0.225</td><td class='num' data-sort='0.1758'>0.176</td></tr><tr><td class='method' data-sort='LOCO'>LOCO</td><td class='num' data-sort='0.1727'>0.173</td><td class='num' data-sort='0.2993'>0.299</td><td class='num' data-sort='0.2199'>0.220</td><td class='num' data-sort='0.1514'>0.151</td><td class='num' data-sort='0.1136'>0.114</td><td class='num' data-sort='0.0795'>0.080</td></tr><tr><td class='method' data-sort='SBE'>SBE</td><td class='num' data-sort='0.1323'>0.132</td><td class='num' data-sort='0.1945'>0.195</td><td class='num' data-sort='0.1753'>0.175</td><td class='num' data-sort='0.1437'>0.144</td><td class='num' data-sort='0.0949'>0.095</td><td class='num' data-sort='0.0594'>0.059</td></tr><tr><td class='method' data-sort='Random'>Random</td><td class='num' data-sort='0.0006'>0.001</td><td class='num' data-sort='-0.0013'>-0.001</td><td class='num' data-sort='0.001'>0.001</td><td class='num' data-sort='-0.0007'>-0.001</td><td class='num' data-sort='0.002'>0.002</td><td class='num' data-sort='0.002'>0.002</td></tr></tbody></table></div><script type='text/plain' id='sa-csv'>method,stability,stability_budget_0,stability_budget_1,stability_budget_2,stability_budget_3,stability_budget_4
|
| 31 |
F-Test,0.679,0.7253,0.734,0.7056,0.6678,0.5622
|
| 32 |
LaplacianScore,0.6518,0.5883,0.6445,0.6596,0.6889,0.6777
|
|
|
|
| 26 |
td .sub { color:var(--muted); font-size:12px; font-weight:400; margin-left:6px; }
|
| 27 |
td.na { color:var(--muted); }
|
| 28 |
.pill { display:inline-block; padding:0 8px; border:1px solid var(--line); border-radius:999px;
|
| 29 |
+
font-size:12px; color:var(--fg); }
|
| 30 |
+
.pill.cat-search { border-color:var(--accent); }
|
| 31 |
+
.pill.cat-baseline { color:var(--muted); }
|
| 32 |
</style></head><body><div class='toolbar'><span class='hint'>Click a column to sort.</span><button onclick="saCsv('selectarena_stability.csv')">CSV</button></div><div class='wrap'><table id='sa-table'><thead><tr><th class=''>Method</th><th class='num' title='Mean over datasets and budgets.'>Stability β</th><th class='num'>Budget 1 (smallest)</th><th class='num'>Budget 2</th><th class='num'>Budget 3</th><th class='num'>Budget 4</th><th class='num'>Budget 5 (largest)</th></tr></thead><tbody><tr><td class='method' data-sort='F-Test'>F-Test</td><td class='num' data-sort='0.679'>0.679</td><td class='num' data-sort='0.7253'>0.725</td><td class='num' data-sort='0.734'>0.734</td><td class='num' data-sort='0.7056'>0.706</td><td class='num' data-sort='0.6678'>0.668</td><td class='num' data-sort='0.5622'>0.562</td></tr><tr><td class='method' data-sort='LaplacianScore'>LaplacianScore</td><td class='num' data-sort='0.6518'>0.652</td><td class='num' data-sort='0.5883'>0.588</td><td class='num' data-sort='0.6445'>0.644</td><td class='num' data-sort='0.6596'>0.660</td><td class='num' data-sort='0.6889'>0.689</td><td class='num' data-sort='0.6777'>0.678</td></tr><tr><td class='method' data-sort='MI'>MI</td><td class='num' data-sort='0.6474'>0.647</td><td class='num' data-sort='0.6455'>0.645</td><td class='num' data-sort='0.6726'>0.673</td><td class='num' data-sort='0.6829'>0.683</td><td class='num' data-sort='0.6494'>0.649</td><td class='num' data-sort='0.5868'>0.587</td></tr><tr><td class='method' data-sort='(R)ReliefF'>(R)ReliefF</td><td class='num' data-sort='0.646'>0.646</td><td class='num' data-sort='0.6415'>0.641</td><td class='num' data-sort='0.647'>0.647</td><td class='num' data-sort='0.6725'>0.672</td><td class='num' data-sort='0.6563'>0.656</td><td class='num' data-sort='0.6124'>0.612</td></tr><tr><td class='method' data-sort='GainRatio'>GainRatio</td><td class='num' data-sort='0.6441'>0.644</td><td class='num' data-sort='0.6788'>0.679</td><td class='num' data-sort='0.6807'>0.681</td><td class='num' data-sort='0.6703'>0.670</td><td class='num' data-sort='0.641'>0.641</td><td class='num' data-sort='0.5498'>0.550</td></tr><tr><td class='method' data-sort='RFImportance'>RFImportance</td><td class='num' data-sort='0.6432'>0.643</td><td class='num' data-sort='0.6282'>0.628</td><td class='num' data-sort='0.6529'>0.653</td><td class='num' data-sort='0.6762'>0.676</td><td class='num' data-sort='0.6613'>0.661</td><td class='num' data-sort='0.5974'>0.597</td></tr><tr><td class='method' data-sort='mRMR'>mRMR</td><td class='num' data-sort='0.6036'>0.604</td><td class='num' data-sort='0.5276'>0.528</td><td class='num' data-sort='0.5784'>0.578</td><td class='num' data-sort='0.6139'>0.614</td><td class='num' data-sort='0.656'>0.656</td><td class='num' data-sort='0.651'>0.651</td></tr><tr><td class='method' data-sort='ElasticNet'>ElasticNet</td><td class='num' data-sort='0.4472'>0.447</td><td class='num' data-sort='0.5094'>0.509</td><td class='num' data-sort='0.5187'>0.519</td><td class='num' data-sort='0.4808'>0.481</td><td class='num' data-sort='0.4188'>0.419</td><td class='num' data-sort='0.3083'>0.308</td></tr><tr><td class='method' data-sort='CART'>CART</td><td class='num' data-sort='0.422'>0.422</td><td class='num' data-sort='0.4838'>0.484</td><td class='num' data-sort='0.4556'>0.456</td><td class='num' data-sort='0.4444'>0.444</td><td class='num' data-sort='0.4018'>0.402</td><td class='num' data-sort='0.3241'>0.324</td></tr><tr><td class='method' data-sort='MarkovBlanket'>MarkovBlanket</td><td class='num' data-sort='0.413'>0.413</td><td class='num' data-sort='0.4177'>0.418</td><td class='num' data-sort='0.3958'>0.396</td><td class='num' data-sort='0.4132'>0.413</td><td class='num' data-sort='0.4337'>0.434</td><td class='num' data-sort='0.4045'>0.405</td></tr><tr><td class='method' data-sort='Lasso'>Lasso</td><td class='num' data-sort='0.3623'>0.362</td><td class='num' data-sort='0.4208'>0.421</td><td class='num' data-sort='0.4216'>0.422</td><td class='num' data-sort='0.3889'>0.389</td><td class='num' data-sort='0.3343'>0.334</td><td class='num' data-sort='0.2457'>0.246</td></tr><tr><td class='method' data-sort='SFS'>SFS</td><td class='num' data-sort='0.2668'>0.267</td><td class='num' data-sort='0.3677'>0.368</td><td class='num' data-sort='0.2781'>0.278</td><td class='num' data-sort='0.2392'>0.239</td><td class='num' data-sort='0.2248'>0.225</td><td class='num' data-sort='0.1758'>0.176</td></tr><tr><td class='method' data-sort='LOCO'>LOCO</td><td class='num' data-sort='0.1727'>0.173</td><td class='num' data-sort='0.2993'>0.299</td><td class='num' data-sort='0.2199'>0.220</td><td class='num' data-sort='0.1514'>0.151</td><td class='num' data-sort='0.1136'>0.114</td><td class='num' data-sort='0.0795'>0.080</td></tr><tr><td class='method' data-sort='SBE'>SBE</td><td class='num' data-sort='0.1323'>0.132</td><td class='num' data-sort='0.1945'>0.195</td><td class='num' data-sort='0.1753'>0.175</td><td class='num' data-sort='0.1437'>0.144</td><td class='num' data-sort='0.0949'>0.095</td><td class='num' data-sort='0.0594'>0.059</td></tr><tr><td class='method' data-sort='Random'>Random</td><td class='num' data-sort='0.0006'>0.001</td><td class='num' data-sort='-0.0013'>-0.001</td><td class='num' data-sort='0.001'>0.001</td><td class='num' data-sort='-0.0007'>-0.001</td><td class='num' data-sort='0.002'>0.002</td><td class='num' data-sort='0.002'>0.002</td></tr></tbody></table></div><script type='text/plain' id='sa-csv'>method,stability,stability_budget_0,stability_budget_1,stability_budget_2,stability_budget_3,stability_budget_4
|
| 33 |
F-Test,0.679,0.7253,0.734,0.7056,0.6678,0.5622
|
| 34 |
LaplacianScore,0.6518,0.5883,0.6445,0.6596,0.6889,0.6777
|
data/subsets/tasks_all/models_all/budgets_0/elo_explorer.html
CHANGED
|
@@ -19,67 +19,67 @@ svg { display:block; width:100%; height:auto; }
|
|
| 19 |
.sa-ink { stroke:var(--fg); }
|
| 20 |
.sa-hit { fill:transparent; }
|
| 21 |
.sa-hit:hover { fill:var(--fg); fill-opacity:0.06; }
|
| 22 |
-
</style><defs><pattern id='sa-na' width='6' height='6' patternUnits='userSpaceOnUse' patternTransform='rotate(45)'><rect width='6' height='6' fill='#e5e7eb'/><line x1='0' y1='0' x2='0' y2='6' stroke='#9ca3af' stroke-width='2'/></pattern><linearGradient id='sa-g-stab' x1='0' y1='1' x2='0' y2='0'><stop offset='0' stop-color='#ffffff'/><stop offset='1' stop-color='#0072B2'/></linearGradient><linearGradient id='sa-g-val' x1='0' y1='1' x2='0' y2='0'><stop offset='0' stop-color='#ffffff'/><stop offset='1' stop-color='#D55E00'/></linearGradient></defs><line class='sa-grid' x1='64' x2='1024' y1='287.6' y2='287.6' stroke-width='1'/><text class='sa-muted' x='56' y='291.6' text-anchor='end' font-size='12'>900</text><line class='sa-grid' x1='64' x2='1024' y1='233.7' y2='233.7' stroke-width='1'/><text class='sa-muted' x='56' y='237.7' text-anchor='end' font-size='12'>1000</text><line class='sa-grid' x1='64' x2='1024' y1='179.7' y2='179.7' stroke-width='1'/><text class='sa-muted' x='56' y='183.7' text-anchor='end' font-size='12'>1100</text><line class='sa-grid' x1='64' x2='1024' y1='125.8' y2='125.8' stroke-width='1'/><text class='sa-muted' x='56' y='129.8' text-anchor='end' font-size='12'>1200</text><line class='sa-grid' x1='64' x2='1024' y1='71.9' y2='71.9' stroke-width='1'/><text class='sa-muted' x='56' y='75.9' text-anchor='end' font-size='12'>1300</text><line class='sa-grid' x1='64' x2='1024' y1='18.0' y2='18.0' stroke-width='1'/><text class='sa-muted' x='56' y='22.0' text-anchor='end' font-size='12'>1400</text><text class='sa-text' transform='translate(16 173) rotate(-90)' text-anchor='middle' font-size='13'>Elo</text><line class='sa-ink' x1='64' x2='1024' y1='233.7' y2='233.7' stroke-width='1' stroke-dasharray='4 4' opacity='0.5'/><rect x='73.6' y='253.6' width='22.4' height='74.4' fill='#69acd2' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='96.0' y='253.6' width='22.4' height='74.4' fill='#db7727' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='96.0' x2='96.0' y1='221.3' y2='294.0'/><line x1='91.0' x2='101.0' y1='221.3' y2='221.3'/><line x1='91.0' x2='101.0' y1='294.0' y2='294.0'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(100.0 342.0) rotate(-35)'>LaplacianScore</text><rect class='sa-hit' x='64.0' y='18' width='64' height='414'><title>LaplacianScore (
|
| 23 |
Elo 963 [95% CI 888β1023]
|
| 24 |
Stability 0.588
|
| 25 |
-
Validity 0.847</title></rect><rect x='137.6' y='233.7' width='22.4' height='94.3' fill='#ffffff' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='160.0' y='233.7' width='22.4' height='94.3' fill='#eab081' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='160.0' x2='160.0' y1='207.8' y2='260.6'/><line x1='155.0' x2='165.0' y1='207.8' y2='207.8'/><line x1='155.0' x2='165.0' y1='260.6' y2='260.6'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(164.0 342.0) rotate(-35)'>Random</text><rect class='sa-hit' x='128.0' y='18' width='64' height='414'><title>Random (Baseline)
|
| 26 |
Elo 1000 [95% CI 950β1048]
|
| 27 |
Stability -0.001
|
| 28 |
-
Validity 0.493</title></rect><rect x='201.6' y='138.8' width='22.4' height='189.2' fill='#b3d5e8' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='224.0' y='138.8' width='22.4' height='189.2' fill='#e39455' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='224.0' x2='224.0' y1='116.1' y2='161.4'/><line x1='219.0' x2='229.0' y1='116.1' y2='116.1'/><line x1='219.0' x2='229.0' y1='161.4' y2='161.4'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(228.0 342.0) rotate(-35)'>LOCO</text><rect class='sa-hit' x='192.0' y='18' width='64' height='414'><title>LOCO (
|
| 29 |
Elo 1176 [95% CI 1134β1218]
|
| 30 |
Stability 0.299
|
| 31 |
-
Validity 0.665</title></rect><rect x='265.6' y='133.4' width='22.4' height='194.6' fill='#cde4f0' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='288.0' y='133.4' width='22.4' height='194.6' fill='#e59b61' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='288.0' x2='288.0' y1='107.0' y2='157.6'/><line x1='283.0' x2='293.0' y1='107.0' y2='107.0'/><line x1='283.0' x2='293.0' y1='157.6' y2='157.6'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(292.0 342.0) rotate(-35)'>SBE</text><rect class='sa-hit' x='256.0' y='18' width='64' height='414'><title>SBE (
|
| 32 |
Elo 1186 [95% CI 1141β1235]
|
| 33 |
Stability 0.195
|
| 34 |
-
Validity 0.621</title></rect><rect x='329.6' y='122.6' width='22.4' height='205.4' fill='#5ba4ce' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='352.0' y='122.6' width='22.4' height='205.4' fill='#da7220' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='352.0' x2='352.0' y1='99.4' y2='145.8'/><line x1='347.0' x2='357.0' y1='99.4' y2='99.4'/><line x1='347.0' x2='357.0' y1='145.8' y2='145.8'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(356.0 342.0) rotate(-35)'>(R)ReliefF</text><rect class='sa-hit' x='320.0' y='18' width='64' height='414'><title>(R)ReliefF (
|
| 35 |
Elo 1206 [95% CI 1163β1249]
|
| 36 |
Stability 0.642
|
| 37 |
-
Validity 0.873</title></rect><rect x='393.6' y='103.2' width='22.4' height='224.8' fill='#529fcb' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='416.0' y='103.2' width='22.4' height='224.8' fill='#d7670f' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='416.0' x2='416.0' y1='81.6' y2='125.3'/><line x1='411.0' x2='421.0' y1='81.6' y2='81.6'/><line x1='411.0' x2='421.0' y1='125.3' y2='125.3'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(420.0 342.0) rotate(-35)'>GainRatio</text><rect class='sa-hit' x='384.0' y='18' width='64' height='414'><title>GainRatio (
|
| 38 |
Elo 1242 [95% CI 1201β1282]
|
| 39 |
Stability 0.679
|
| 40 |
-
Validity 0.942</title></rect><rect x='457.6' y='98.9' width='22.4' height='229.1' fill='#94c4df' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='480.0' y='98.9' width='22.4' height='229.1' fill='#da701c' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='480.0' x2='480.0' y1='80.0' y2='118.8'/><line x1='475.0' x2='485.0' y1='80.0' y2='80.0'/><line x1='475.0' x2='485.0' y1='118.8' y2='118.8'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(484.0 342.0) rotate(-35)'>Lasso</text><rect class='sa-hit' x='448.0' y='18' width='64' height='414'><title>Lasso (
|
| 41 |
Elo 1250 [95% CI 1213β1285]
|
| 42 |
Stability 0.421
|
| 43 |
-
Validity 0.890</title></rect><rect x='521.6' y='96.2' width='22.4' height='231.8' fill='#78b5d6' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='544.0' y='96.2' width='22.4' height='231.8' fill='#de8238' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='544.0' x2='544.0' y1='75.7' y2='117.2'/><line x1='539.0' x2='549.0' y1='75.7' y2='75.7'/><line x1='539.0' x2='549.0' y1='117.2' y2='117.2'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(548.0 342.0) rotate(-35)'>mRMR</text><rect class='sa-hit' x='512.0' y='18' width='64' height='414'><title>mRMR (
|
| 44 |
Elo 1255 [95% CI 1216β1293]
|
| 45 |
Stability 0.528
|
| 46 |
-
Validity 0.779</title></rect><rect x='585.6' y='88.1' width='22.4' height='239.9' fill='#a1cbe3' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='608.0' y='88.1' width='22.4' height='239.9' fill='#e08640' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='608.0' x2='608.0' y1='64.4' y2='110.2'/><line x1='603.0' x2='613.0' y1='64.4' y2='64.4'/><line x1='603.0' x2='613.0' y1='110.2' y2='110.2'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(612.0 342.0) rotate(-35)'>SFS</text><rect class='sa-hit' x='576.0' y='18' width='64' height='414'><title>SFS (
|
| 47 |
Elo 1270 [95% CI 1229β1314]
|
| 48 |
Stability 0.368
|
| 49 |
-
Validity 0.750</title></rect><rect x='649.6' y='80.5' width='22.4' height='247.5' fill='#7db7d8' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='672.0' y='80.5' width='22.4' height='247.5' fill='#d96c16' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='672.0' x2='672.0' y1='58.4' y2='101.0'/><line x1='667.0' x2='677.0' y1='58.4' y2='58.4'/><line x1='667.0' x2='677.0' y1='101.0' y2='101.0'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(676.0 342.0) rotate(-35)'>ElasticNet</text><rect class='sa-hit' x='640.0' y='18' width='64' height='414'><title>ElasticNet (
|
| 50 |
Elo 1284 [95% CI 1246β1325]
|
| 51 |
Stability 0.509
|
| 52 |
-
Validity 0.915</title></rect><rect x='713.6' y='77.3' width='22.4' height='250.7' fill='#4699c7' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='736.0' y='77.3' width='22.4' height='250.7' fill='#d7660c' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='736.0' x2='736.0' y1='55.7' y2='98.3'/><line x1='731.0' x2='741.0' y1='55.7' y2='55.7'/><line x1='731.0' x2='741.0' y1='98.3' y2='98.3'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(740.0 342.0) rotate(-35)'>F-Test</text><rect class='sa-hit' x='704.0' y='18' width='64' height='414'><title>F-Test (
|
| 53 |
Elo 1290 [95% CI 1251β1330]
|
| 54 |
Stability 0.725
|
| 55 |
-
Validity 0.952</title></rect><rect x='777.6' y='77.3' width='22.4' height='250.7' fill='#5ba4cd' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='800.0' y='77.3' width='22.4' height='250.7' fill='#d96e1a' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='800.0' x2='800.0' y1='58.4' y2='97.3'/><line x1='795.0' x2='805.0' y1='58.4' y2='58.4'/><line x1='795.0' x2='805.0' y1='97.3' y2='97.3'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(804.0 342.0) rotate(-35)'>MI</text><rect class='sa-hit' x='768.0' y='18' width='64' height='414'><title>MI (
|
| 56 |
Elo 1290 [95% CI 1253β1325]
|
| 57 |
Stability 0.645
|
| 58 |
-
Validity 0.898</title></rect><rect x='841.6' y='67.6' width='22.4' height='260.4' fill='#94c4df' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='864.0' y='67.6' width='22.4' height='260.4' fill='#ebb488' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='864.0' x2='864.0' y1='46.0' y2='88.6'/><line x1='859.0' x2='869.0' y1='46.0' y2='46.0'/><line x1='859.0' x2='869.0' y1='88.6' y2='88.6'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(868.0 342.0) rotate(-35)'>MarkovBlanket</text><rect class='sa-hit' x='832.0' y='18' width='64' height='414'><title>MarkovBlanket (
|
| 59 |
Elo 1308 [95% CI 1269β1348]
|
| 60 |
Stability 0.418
|
| 61 |
-
Validity 0.465</title></rect><rect x='905.6' y='63.8' width='22.4' height='264.2' fill='#84bbda' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='928.0' y='63.8' width='22.4' height='264.2' fill='#e08945' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='928.0' x2='928.0' y1='39.6' y2='87.5'/><line x1='923.0' x2='933.0' y1='39.6' y2='39.6'/><line x1='923.0' x2='933.0' y1='87.5' y2='87.5'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(932.0 342.0) rotate(-35)'>CART</text><rect class='sa-hit' x='896.0' y='18' width='64' height='414'><title>CART (
|
| 62 |
Elo 1315 [95% CI 1271β1360]
|
| 63 |
Stability 0.484
|
| 64 |
-
Validity 0.731</title></rect><rect x='969.6' y='48.7' width='22.4' height='279.3' fill='#5fa6cf' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='992.0' y='48.7' width='22.4' height='279.3' fill='#dd7d32' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='992.0' x2='992.0' y1='21.8' y2='74.1'/><line x1='987.0' x2='997.0' y1='21.8' y2='21.8'/><line x1='987.0' x2='997.0' y1='74.1' y2='74.1'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(996.0 342.0) rotate(-35)'>RFImportance</text><rect class='sa-hit' x='960.0' y='18' width='64' height='414'><title>RFImportance (
|
| 65 |
Elo 1343 [95% CI 1296β1393]
|
| 66 |
Stability 0.628
|
| 67 |
-
Validity 0.805</title></rect><line class='sa-ink' x1='64' x2='1024' y1='328.0' y2='328.0' stroke-width='1' opacity='0.4'/><rect x='1070' y='40.0' width='12' height='107.0' fill='url(#sa-g-stab)' stroke='#000' stroke-width='0.6'/><text class='sa-text' x='1076' y='30.0' text-anchor='middle' font-size='12'>stable</text><text class='sa-text' x='1076' y='165.0' text-anchor='middle' font-size='12'>unstable</text><text class='sa-muted' x='1088' y='151.0' font-size='11'>0.0</text><text class='sa-muted' x='1088' y='97.5' font-size='11'>0.5</text><text class='sa-muted' x='1088' y='44.0' font-size='11'>1.0</text><rect x='1070' y='207.0' width='12' height='107.0' fill='url(#sa-g-val)' stroke='#000' stroke-width='0.6'/><text class='sa-text' x='1076' y='197.0' text-anchor='middle' font-size='12'>valid</text><text class='sa-text' x='1076' y='332.0' text-anchor='middle' font-size='12'>invalid</text><text class='sa-muted' x='1088' y='318.0' font-size='11'>0.0</text><text class='sa-muted' x='1088' y='264.5' font-size='11'>0.5</text><text class='sa-muted' x='1088' y='211.0' font-size='11'>1.0</text><rect x='1070' y='352.0' width='12' height='12' fill='url(#sa-na)' stroke='#000' stroke-width='0.6'/><text class='sa-muted' x='1088' y='362.0' font-size='11'>no result</text></svg><script type='text/plain' id='sa-csv'>Method,
|
| 68 |
-
RFImportance,
|
| 69 |
-
CART,
|
| 70 |
-
MarkovBlanket,
|
| 71 |
-
F-Test,
|
| 72 |
-
MI,
|
| 73 |
-
ElasticNet,
|
| 74 |
-
SFS,
|
| 75 |
-
mRMR,
|
| 76 |
-
Lasso,
|
| 77 |
-
GainRatio,
|
| 78 |
-
(R)ReliefF,
|
| 79 |
-
SBE,
|
| 80 |
-
LOCO,
|
| 81 |
-
Random,Baseline,1000.0,950.0,1048.0,-0.001,0.493
|
| 82 |
-
LaplacianScore,
|
| 83 |
</script><script>
|
| 84 |
function saDownload(name, blob) {
|
| 85 |
var a = document.createElement('a');
|
|
|
|
| 19 |
.sa-ink { stroke:var(--fg); }
|
| 20 |
.sa-hit { fill:transparent; }
|
| 21 |
.sa-hit:hover { fill:var(--fg); fill-opacity:0.06; }
|
| 22 |
+
</style><defs><pattern id='sa-na' width='6' height='6' patternUnits='userSpaceOnUse' patternTransform='rotate(45)'><rect width='6' height='6' fill='#e5e7eb'/><line x1='0' y1='0' x2='0' y2='6' stroke='#9ca3af' stroke-width='2'/></pattern><linearGradient id='sa-g-stab' x1='0' y1='1' x2='0' y2='0'><stop offset='0' stop-color='#ffffff'/><stop offset='1' stop-color='#0072B2'/></linearGradient><linearGradient id='sa-g-val' x1='0' y1='1' x2='0' y2='0'><stop offset='0' stop-color='#ffffff'/><stop offset='1' stop-color='#D55E00'/></linearGradient></defs><line class='sa-grid' x1='64' x2='1024' y1='287.6' y2='287.6' stroke-width='1'/><text class='sa-muted' x='56' y='291.6' text-anchor='end' font-size='12'>900</text><line class='sa-grid' x1='64' x2='1024' y1='233.7' y2='233.7' stroke-width='1'/><text class='sa-muted' x='56' y='237.7' text-anchor='end' font-size='12'>1000</text><line class='sa-grid' x1='64' x2='1024' y1='179.7' y2='179.7' stroke-width='1'/><text class='sa-muted' x='56' y='183.7' text-anchor='end' font-size='12'>1100</text><line class='sa-grid' x1='64' x2='1024' y1='125.8' y2='125.8' stroke-width='1'/><text class='sa-muted' x='56' y='129.8' text-anchor='end' font-size='12'>1200</text><line class='sa-grid' x1='64' x2='1024' y1='71.9' y2='71.9' stroke-width='1'/><text class='sa-muted' x='56' y='75.9' text-anchor='end' font-size='12'>1300</text><line class='sa-grid' x1='64' x2='1024' y1='18.0' y2='18.0' stroke-width='1'/><text class='sa-muted' x='56' y='22.0' text-anchor='end' font-size='12'>1400</text><text class='sa-text' transform='translate(16 173) rotate(-90)' text-anchor='middle' font-size='13'>Elo</text><line class='sa-ink' x1='64' x2='1024' y1='233.7' y2='233.7' stroke-width='1' stroke-dasharray='4 4' opacity='0.5'/><rect x='73.6' y='253.6' width='22.4' height='74.4' fill='#69acd2' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='96.0' y='253.6' width='22.4' height='74.4' fill='#db7727' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='96.0' x2='96.0' y1='221.3' y2='294.0'/><line x1='91.0' x2='101.0' y1='221.3' y2='221.3'/><line x1='91.0' x2='101.0' y1='294.0' y2='294.0'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(100.0 342.0) rotate(-35)'>LaplacianScore</text><rect class='sa-hit' x='64.0' y='18' width='64' height='414'><title>LaplacianScore (Distance, non-search)
|
| 23 |
Elo 963 [95% CI 888β1023]
|
| 24 |
Stability 0.588
|
| 25 |
+
Validity 0.847</title></rect><rect x='137.6' y='233.7' width='22.4' height='94.3' fill='#ffffff' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='160.0' y='233.7' width='22.4' height='94.3' fill='#eab081' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='160.0' x2='160.0' y1='207.8' y2='260.6'/><line x1='155.0' x2='165.0' y1='207.8' y2='207.8'/><line x1='155.0' x2='165.0' y1='260.6' y2='260.6'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(164.0 342.0) rotate(-35)'>Random</text><rect class='sa-hit' x='128.0' y='18' width='64' height='414'><title>Random (Baseline, baseline)
|
| 26 |
Elo 1000 [95% CI 950β1048]
|
| 27 |
Stability -0.001
|
| 28 |
+
Validity 0.493</title></rect><rect x='201.6' y='138.8' width='22.4' height='189.2' fill='#b3d5e8' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='224.0' y='138.8' width='22.4' height='189.2' fill='#e39455' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='224.0' x2='224.0' y1='116.1' y2='161.4'/><line x1='219.0' x2='229.0' y1='116.1' y2='116.1'/><line x1='219.0' x2='229.0' y1='161.4' y2='161.4'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(228.0 342.0) rotate(-35)'>LOCO</text><rect class='sa-hit' x='192.0' y='18' width='64' height='414'><title>LOCO (Leave-one-out, search)
|
| 29 |
Elo 1176 [95% CI 1134β1218]
|
| 30 |
Stability 0.299
|
| 31 |
+
Validity 0.665</title></rect><rect x='265.6' y='133.4' width='22.4' height='194.6' fill='#cde4f0' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='288.0' y='133.4' width='22.4' height='194.6' fill='#e59b61' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='288.0' x2='288.0' y1='107.0' y2='157.6'/><line x1='283.0' x2='293.0' y1='107.0' y2='107.0'/><line x1='283.0' x2='293.0' y1='157.6' y2='157.6'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(292.0 342.0) rotate(-35)'>SBE</text><rect class='sa-hit' x='256.0' y='18' width='64' height='414'><title>SBE (Backward search, search)
|
| 32 |
Elo 1186 [95% CI 1141β1235]
|
| 33 |
Stability 0.195
|
| 34 |
+
Validity 0.621</title></rect><rect x='329.6' y='122.6' width='22.4' height='205.4' fill='#5ba4ce' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='352.0' y='122.6' width='22.4' height='205.4' fill='#da7220' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='352.0' x2='352.0' y1='99.4' y2='145.8'/><line x1='347.0' x2='357.0' y1='99.4' y2='99.4'/><line x1='347.0' x2='357.0' y1='145.8' y2='145.8'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(356.0 342.0) rotate(-35)'>(R)ReliefF</text><rect class='sa-hit' x='320.0' y='18' width='64' height='414'><title>(R)ReliefF (Distance, non-search)
|
| 35 |
Elo 1206 [95% CI 1163β1249]
|
| 36 |
Stability 0.642
|
| 37 |
+
Validity 0.873</title></rect><rect x='393.6' y='103.2' width='22.4' height='224.8' fill='#529fcb' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='416.0' y='103.2' width='22.4' height='224.8' fill='#d7670f' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='416.0' x2='416.0' y1='81.6' y2='125.3'/><line x1='411.0' x2='421.0' y1='81.6' y2='81.6'/><line x1='411.0' x2='421.0' y1='125.3' y2='125.3'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(420.0 342.0) rotate(-35)'>GainRatio</text><rect class='sa-hit' x='384.0' y='18' width='64' height='414'><title>GainRatio (Information theory, non-search)
|
| 38 |
Elo 1242 [95% CI 1201β1282]
|
| 39 |
Stability 0.679
|
| 40 |
+
Validity 0.942</title></rect><rect x='457.6' y='98.9' width='22.4' height='229.1' fill='#94c4df' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='480.0' y='98.9' width='22.4' height='229.1' fill='#da701c' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='480.0' x2='480.0' y1='80.0' y2='118.8'/><line x1='475.0' x2='485.0' y1='80.0' y2='80.0'/><line x1='475.0' x2='485.0' y1='118.8' y2='118.8'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(484.0 342.0) rotate(-35)'>Lasso</text><rect class='sa-hit' x='448.0' y='18' width='64' height='414'><title>Lasso (Regularization, non-search)
|
| 41 |
Elo 1250 [95% CI 1213β1285]
|
| 42 |
Stability 0.421
|
| 43 |
+
Validity 0.890</title></rect><rect x='521.6' y='96.2' width='22.4' height='231.8' fill='#78b5d6' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='544.0' y='96.2' width='22.4' height='231.8' fill='#de8238' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='544.0' x2='544.0' y1='75.7' y2='117.2'/><line x1='539.0' x2='549.0' y1='75.7' y2='75.7'/><line x1='539.0' x2='549.0' y1='117.2' y2='117.2'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(548.0 342.0) rotate(-35)'>mRMR</text><rect class='sa-hit' x='512.0' y='18' width='64' height='414'><title>mRMR (Information theory, non-search)
|
| 44 |
Elo 1255 [95% CI 1216β1293]
|
| 45 |
Stability 0.528
|
| 46 |
+
Validity 0.779</title></rect><rect x='585.6' y='88.1' width='22.4' height='239.9' fill='#a1cbe3' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='608.0' y='88.1' width='22.4' height='239.9' fill='#e08640' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='608.0' x2='608.0' y1='64.4' y2='110.2'/><line x1='603.0' x2='613.0' y1='64.4' y2='64.4'/><line x1='603.0' x2='613.0' y1='110.2' y2='110.2'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(612.0 342.0) rotate(-35)'>SFS</text><rect class='sa-hit' x='576.0' y='18' width='64' height='414'><title>SFS (Forward search, search)
|
| 47 |
Elo 1270 [95% CI 1229β1314]
|
| 48 |
Stability 0.368
|
| 49 |
+
Validity 0.750</title></rect><rect x='649.6' y='80.5' width='22.4' height='247.5' fill='#7db7d8' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='672.0' y='80.5' width='22.4' height='247.5' fill='#d96c16' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='672.0' x2='672.0' y1='58.4' y2='101.0'/><line x1='667.0' x2='677.0' y1='58.4' y2='58.4'/><line x1='667.0' x2='677.0' y1='101.0' y2='101.0'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(676.0 342.0) rotate(-35)'>ElasticNet</text><rect class='sa-hit' x='640.0' y='18' width='64' height='414'><title>ElasticNet (Regularization, non-search)
|
| 50 |
Elo 1284 [95% CI 1246β1325]
|
| 51 |
Stability 0.509
|
| 52 |
+
Validity 0.915</title></rect><rect x='713.6' y='77.3' width='22.4' height='250.7' fill='#4699c7' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='736.0' y='77.3' width='22.4' height='250.7' fill='#d7660c' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='736.0' x2='736.0' y1='55.7' y2='98.3'/><line x1='731.0' x2='741.0' y1='55.7' y2='55.7'/><line x1='731.0' x2='741.0' y1='98.3' y2='98.3'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(740.0 342.0) rotate(-35)'>F-Test</text><rect class='sa-hit' x='704.0' y='18' width='64' height='414'><title>F-Test (Correlation, non-search)
|
| 53 |
Elo 1290 [95% CI 1251β1330]
|
| 54 |
Stability 0.725
|
| 55 |
+
Validity 0.952</title></rect><rect x='777.6' y='77.3' width='22.4' height='250.7' fill='#5ba4cd' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='800.0' y='77.3' width='22.4' height='250.7' fill='#d96e1a' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='800.0' x2='800.0' y1='58.4' y2='97.3'/><line x1='795.0' x2='805.0' y1='58.4' y2='58.4'/><line x1='795.0' x2='805.0' y1='97.3' y2='97.3'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(804.0 342.0) rotate(-35)'>MI</text><rect class='sa-hit' x='768.0' y='18' width='64' height='414'><title>MI (Information theory, non-search)
|
| 56 |
Elo 1290 [95% CI 1253β1325]
|
| 57 |
Stability 0.645
|
| 58 |
+
Validity 0.898</title></rect><rect x='841.6' y='67.6' width='22.4' height='260.4' fill='#94c4df' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='864.0' y='67.6' width='22.4' height='260.4' fill='#ebb488' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='864.0' x2='864.0' y1='46.0' y2='88.6'/><line x1='859.0' x2='869.0' y1='46.0' y2='46.0'/><line x1='859.0' x2='869.0' y1='88.6' y2='88.6'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(868.0 342.0) rotate(-35)'>MarkovBlanket</text><rect class='sa-hit' x='832.0' y='18' width='64' height='414'><title>MarkovBlanket (Backward search, search)
|
| 59 |
Elo 1308 [95% CI 1269β1348]
|
| 60 |
Stability 0.418
|
| 61 |
+
Validity 0.465</title></rect><rect x='905.6' y='63.8' width='22.4' height='264.2' fill='#84bbda' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='928.0' y='63.8' width='22.4' height='264.2' fill='#e08945' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='928.0' x2='928.0' y1='39.6' y2='87.5'/><line x1='923.0' x2='933.0' y1='39.6' y2='39.6'/><line x1='923.0' x2='933.0' y1='87.5' y2='87.5'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(932.0 342.0) rotate(-35)'>CART</text><rect class='sa-hit' x='896.0' y='18' width='64' height='414'><title>CART (Tree, non-search)
|
| 62 |
Elo 1315 [95% CI 1271β1360]
|
| 63 |
Stability 0.484
|
| 64 |
+
Validity 0.731</title></rect><rect x='969.6' y='48.7' width='22.4' height='279.3' fill='#5fa6cf' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='992.0' y='48.7' width='22.4' height='279.3' fill='#dd7d32' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='992.0' x2='992.0' y1='21.8' y2='74.1'/><line x1='987.0' x2='997.0' y1='21.8' y2='21.8'/><line x1='987.0' x2='997.0' y1='74.1' y2='74.1'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(996.0 342.0) rotate(-35)'>RFImportance</text><rect class='sa-hit' x='960.0' y='18' width='64' height='414'><title>RFImportance (Tree, non-search)
|
| 65 |
Elo 1343 [95% CI 1296β1393]
|
| 66 |
Stability 0.628
|
| 67 |
+
Validity 0.805</title></rect><line class='sa-ink' x1='64' x2='1024' y1='328.0' y2='328.0' stroke-width='1' opacity='0.4'/><rect x='1070' y='40.0' width='12' height='107.0' fill='url(#sa-g-stab)' stroke='#000' stroke-width='0.6'/><text class='sa-text' x='1076' y='30.0' text-anchor='middle' font-size='12'>stable</text><text class='sa-text' x='1076' y='165.0' text-anchor='middle' font-size='12'>unstable</text><text class='sa-muted' x='1088' y='151.0' font-size='11'>0.0</text><text class='sa-muted' x='1088' y='97.5' font-size='11'>0.5</text><text class='sa-muted' x='1088' y='44.0' font-size='11'>1.0</text><rect x='1070' y='207.0' width='12' height='107.0' fill='url(#sa-g-val)' stroke='#000' stroke-width='0.6'/><text class='sa-text' x='1076' y='197.0' text-anchor='middle' font-size='12'>valid</text><text class='sa-text' x='1076' y='332.0' text-anchor='middle' font-size='12'>invalid</text><text class='sa-muted' x='1088' y='318.0' font-size='11'>0.0</text><text class='sa-muted' x='1088' y='264.5' font-size='11'>0.5</text><text class='sa-muted' x='1088' y='211.0' font-size='11'>1.0</text><rect x='1070' y='352.0' width='12' height='12' fill='url(#sa-na)' stroke='#000' stroke-width='0.6'/><text class='sa-muted' x='1088' y='362.0' font-size='11'>no result</text></svg><script type='text/plain' id='sa-csv'>Method,Category,Subgroup,Elo,Elo CI low,Elo CI high,Stability,Validity
|
| 68 |
+
RFImportance,Non-search,Tree,1343.0,1296.0,1393.0,0.628,0.805
|
| 69 |
+
CART,Non-search,Tree,1315.0,1271.0,1360.0,0.484,0.731
|
| 70 |
+
MarkovBlanket,Search,Backward search,1308.0,1269.0,1348.0,0.418,0.465
|
| 71 |
+
F-Test,Non-search,Correlation,1290.0,1251.0,1330.0,0.725,0.952
|
| 72 |
+
MI,Non-search,Information theory,1290.0,1253.0,1325.0,0.645,0.898
|
| 73 |
+
ElasticNet,Non-search,Regularization,1284.0,1246.0,1325.0,0.509,0.915
|
| 74 |
+
SFS,Search,Forward search,1270.0,1229.0,1314.0,0.368,0.75
|
| 75 |
+
mRMR,Non-search,Information theory,1255.0,1216.0,1293.0,0.528,0.779
|
| 76 |
+
Lasso,Non-search,Regularization,1250.0,1213.0,1285.0,0.421,0.89
|
| 77 |
+
GainRatio,Non-search,Information theory,1242.0,1201.0,1282.0,0.679,0.942
|
| 78 |
+
(R)ReliefF,Non-search,Distance,1206.0,1163.0,1249.0,0.642,0.873
|
| 79 |
+
SBE,Search,Backward search,1186.0,1141.0,1235.0,0.195,0.621
|
| 80 |
+
LOCO,Search,Leave-one-out,1176.0,1134.0,1218.0,0.299,0.665
|
| 81 |
+
Random,Baseline,Baseline,1000.0,950.0,1048.0,-0.001,0.493
|
| 82 |
+
LaplacianScore,Non-search,Distance,963.0,888.0,1023.0,0.588,0.847
|
| 83 |
</script><script>
|
| 84 |
function saDownload(name, blob) {
|
| 85 |
var a = document.createElement('a');
|
data/subsets/tasks_all/models_all/budgets_0/leaderboard_table.html
CHANGED
|
@@ -26,23 +26,25 @@ td.method { font-weight:600; }
|
|
| 26 |
td .sub { color:var(--muted); font-size:12px; font-weight:400; margin-left:6px; }
|
| 27 |
td.na { color:var(--muted); }
|
| 28 |
.pill { display:inline-block; padding:0 8px; border:1px solid var(--line); border-radius:999px;
|
| 29 |
-
font-size:12px; color:var(--
|
| 30 |
-
</style></head><body><div class='toolbar'><span class='hint'>Click a column to sort. Hover a header for its definition.</span><button onclick="saCsv('selectarena_leaderboard.csv')">CSV</button></div><div class='wrap'><table id='sa-table'><thead><tr><th class='num'>#</th><th class=''>Method</th><th class='' title='Filter: scores features from the data alone. Wrapper: searches feature subsets by refitting a proxy model. Embedded: reads the selection off a fitted model.'>Type</th><th class='num' title='Bradley-Terry Elo from pairwise matches within each dataset, budget and downstream model. Random = 1000. +/β give the 95% bootstrap interval.'>Elo β</th><th class='num' title='Mean rank of the downstream error.'>Avg. rank β</th><th class='num' title='Share of pairwise comparisons won, in %. Ties count half.'>Win rate β</th><th class='num' title='Mean relative gap (error β best error) / error to the best method, in %.'>Improvability β</th><th class='num' title='Mean ratio of the error to the same model trained on all features. Below 1: the selection helped.'>Error vs. all β</th><th class='num' title='Agreement of the selected features across bootstrap resamples (Nogueira et al., 2018).'>Stability β</th><th class='num' title='Share of the budget filled with original features when as many noise features are added.'>Validity β</th><th class='num' title='Median wall-clock time of the selection.'>Time (s) β</th><th class='num' title='Share of runs where the method ran out of time and random features filled the budget.'>Fallback (%)</th><th class='num'>#Datasets</th></tr></thead><tbody><tr><td class='num' data-sort='1'>1</td><td class='method' data-sort='RFImportance'>RFImportance</td><td class='' data-sort='Embedded'><span class='pill'>π³ Embedded</span></td><td class='num' data-sort='1343.0'>1343<span class='sub'>+50 / β47</span></td><td class='num' data-sort='6.29'>6.29</td><td class='num' data-sort='62.2'>62.2</td><td class='num' data-sort='20.0'>20.0</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.628'>0.628</td><td class='num' data-sort='0.805'>0.805</td><td class='num' data-sort='0.2'>0.2</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='2'>2</td><td class='method' data-sort='CART'>CART</td><td class='' data-sort='Embedded'><span class='pill'>π³ Embedded</span></td><td class='num' data-sort='1315.0'>1315<span class='sub'>+45 / β44</span></td><td class='num' data-sort='6.56'>6.56</td><td class='num' data-sort='60.3'>60.3</td><td class='num' data-sort='22.6'>22.6</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.484'>0.484</td><td class='num' data-sort='0.731'>0.731</td><td class='num' data-sort='0.1'>0.1</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='3'>3</td><td class='method' data-sort='MarkovBlanket'>MarkovBlanket</td><td class='' data-sort='Filter'><span class='pill'>π Filter</span></td><td class='num' data-sort='1308.0'>1308<span class='sub'>+40 / β39</span></td><td class='num' data-sort='6.67'>6.67</td><td class='num' data-sort='59.5'>59.5</td><td class='num' data-sort='21.8'>21.8</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.418'>0.418</td><td class='num' data-sort='0.465'>0.465</td><td class='num' data-sort='0.2'>0.2</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='4'>4</td><td class='method' data-sort='F-Test'>F-Test</td><td class='' data-sort='Filter'><span class='pill'>π Filter</span></td><td class='num' data-sort='1290.0'>1290<span class='sub'>+40 / β39</span></td><td class='num' data-sort='7.11'>7.11</td><td class='num' data-sort='56.4'>56.4</td><td class='num' data-sort='23.8'>23.8</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.725'>0.725</td><td class='num' data-sort='0.952'>0.952</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='5'>5</td><td class='method' data-sort='MI'>MI</td><td class='' data-sort='Filter'><span class='pill'>π Filter</span></td><td class='num' data-sort='1290.0'>1290<span class='sub'>+35 / β37</span></td><td class='num' data-sort='6.95'>6.95</td><td class='num' data-sort='57.5'>57.5</td><td class='num' data-sort='23.0'>23.0</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.645'>0.645</td><td class='num' data-sort='0.898'>0.898</td><td class='num' data-sort='0.4'>0.4</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='6'>6</td><td class='method' data-sort='ElasticNet'>ElasticNet</td><td class='' data-sort='Embedded'><span class='pill'>π³ Embedded</span></td><td class='num' data-sort='1284.0'>1284<span class='sub'>+41 / β38</span></td><td class='num' data-sort='6.95'>6.95</td><td class='num' data-sort='57.5'>57.5</td><td class='num' data-sort='24.4'>24.4</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.509'>0.509</td><td class='num' data-sort='0.915'>0.915</td><td class='num' data-sort='7.5'>7.5</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='7'>7</td><td class='method' data-sort='SFS'>SFS</td><td class='' data-sort='Wrapper'><span class='pill'>π Wrapper</span></td><td class='num' data-sort='1270.0'>1270<span class='sub'>+44 / β41</span></td><td class='num' data-sort='7.13'>7.13</td><td class='num' data-sort='56.2'>56.2</td><td class='num' data-sort='24.5'>24.5</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.368'>0.368</td><td class='num' data-sort='0.75'>0.750</td><td class='num' data-sort='941.9'>941.9</td><td class='num' data-sort='29.1'>29.1</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='8'>8</td><td class='method' data-sort='mRMR'>mRMR</td><td class='' data-sort='Filter'><span class='pill'>π Filter</span></td><td class='num' data-sort='1255.0'>1255<span class='sub'>+38 / β39</span></td><td class='num' data-sort='7.68'>7.68</td><td class='num' data-sort='52.3'>52.3</td><td class='num' data-sort='24.7'>24.7</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.528'>0.528</td><td class='num' data-sort='0.779'>0.779</td><td class='num' data-sort='0.3'>0.3</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='9'>9</td><td class='method' data-sort='Lasso'>Lasso</td><td class='' data-sort='Embedded'><span class='pill'>π³ Embedded</span></td><td class='num' data-sort='1250.0'>1250<span class='sub'>+35 / β37</span></td><td class='num' data-sort='7.42'>7.42</td><td class='num' data-sort='54.1'>54.1</td><td class='num' data-sort='25.7'>25.7</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.421'>0.421</td><td class='num' data-sort='0.89'>0.890</td><td class='num' data-sort='2.2'>2.2</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='10'>10</td><td class='method' data-sort='GainRatio'>GainRatio</td><td class='' data-sort='Filter'><span class='pill'>π Filter</span></td><td class='num' data-sort='1242.0'>1242<span class='sub'>+40 / β41</span></td><td class='num' data-sort='7.98'>7.98</td><td class='num' data-sort='50.1'>50.1</td><td class='num' data-sort='26.5'>26.5</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.679'>0.679</td><td class='num' data-sort='0.942'>0.942</td><td class='num' data-sort='0.2'>0.2</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='11'>11</td><td class='method' data-sort='(R)ReliefF'>(R)ReliefF</td><td class='' data-sort='Filter'><span class='pill'>π Filter</span></td><td class='num' data-sort='1206.0'>1206<span class='sub'>+43 / β43</span></td><td class='num' data-sort='8.47'>8.47</td><td class='num' data-sort='46.6'>46.6</td><td class='num' data-sort='28.4'>28.4</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.642'>0.642</td><td class='num' data-sort='0.873'>0.873</td><td class='num' data-sort='14.2'>14.2</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='12'>12</td><td class='method' data-sort='SBE'>SBE</td><td class='' data-sort='Wrapper'><span class='pill'>π Wrapper</span></td><td class='num' data-sort='1186.0'>1186<span class='sub'>+49 / β45</span></td><td class='num' data-sort='8.7'>8.70</td><td class='num' data-sort='45.0'>45.0</td><td class='num' data-sort='30.3'>30.3</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.195'>0.195</td><td class='num' data-sort='0.621'>0.621</td><td class='num' data-sort='3579.1'>3,579.1</td><td class='num' data-sort='49.6'>49.6</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='13'>13</td><td class='method' data-sort='LOCO'>LOCO</td><td class='' data-sort='Wrapper'><span class='pill'>π Wrapper</span></td><td class='num' data-sort='1176.0'>1176<span class='sub'>+42 / β42</span></td><td class='num' data-sort='8.54'>8.54</td><td class='num' data-sort='46.1'>46.1</td><td class='num' data-sort='29.9'>29.9</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.299'>0.299</td><td class='num' data-sort='0.665'>0.665</td><td class='num' data-sort='242.0'>242.0</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='14'>14</td><td class='method' data-sort='Random'>Random</td><td class='' data-sort='Baseline'><span class='pill'>βͺ Baseline</span></td><td class='num' data-sort='1000.0'>1000<span class='sub'>+48 / β50</span></td><td class='num' data-sort='11.38'>11.38</td><td class='num' data-sort='25.9'>25.9</td><td class='num' data-sort='42.0'>42.0</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='-0.001'>-0.001</td><td class='num' data-sort='0.493'>0.493</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='100.0'>100.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='15'>15</td><td class='method' data-sort='LaplacianScore'>LaplacianScore</td><td class='' data-sort='Filter'><span class='pill'>π Filter</span></td><td class='num' data-sort='963.0'>963<span class='sub'>+60 / β75</span></td><td class='num' data-sort='12.16'>12.16</td><td class='num' data-sort='20.3'>20.3</td><td class='num' data-sort='45.3'>45.3</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.588'>0.588</td><td class='num' data-sort='0.847'>0.847</td><td class='num' data-sort='0.2'>0.2</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr></tbody></table></div><script type='text/plain' id='sa-csv'>Rank,Type,TypeName,Method,Elo,Elo CI low,Elo CI high,Avg. rank,Win rate (%),Improvability (%),Error vs. all features,Stability,Validity,Selection time (s),Fallback rate (%),#Datasets,#Comparisons
|
| 31 |
-
|
| 32 |
-
2,π³,Embedded,CART,1315.0,1271.0,1360.0,6.56,60.3,22.6,,0.484,0.731,0.1,0.0,71,1065
|
| 33 |
-
|
| 34 |
-
|
| 35 |
-
|
| 36 |
-
|
| 37 |
-
|
| 38 |
-
|
| 39 |
-
|
| 40 |
-
|
| 41 |
-
|
| 42 |
-
|
| 43 |
-
|
| 44 |
-
|
| 45 |
-
|
|
|
|
|
|
|
| 46 |
</script><script>
|
| 47 |
function saDownload(name, blob) {
|
| 48 |
var a = document.createElement('a');
|
|
|
|
| 26 |
td .sub { color:var(--muted); font-size:12px; font-weight:400; margin-left:6px; }
|
| 27 |
td.na { color:var(--muted); }
|
| 28 |
.pill { display:inline-block; padding:0 8px; border:1px solid var(--line); border-radius:999px;
|
| 29 |
+
font-size:12px; color:var(--fg); }
|
| 30 |
+
.pill.cat-search { border-color:var(--accent); }
|
| 31 |
+
.pill.cat-baseline { color:var(--muted); }
|
| 32 |
+
</style></head><body><div class='toolbar'><span class='hint'>Click a column to sort. Hover a header for its definition.</span><button onclick="saCsv('selectarena_leaderboard.csv')">CSV</button></div><div class='wrap'><table id='sa-table'><thead><tr><th class='num'>#</th><th class=''>Method</th><th class='' title='Taxonomy of the paper (Appendix A). Search methods evaluate many feature subsets with the model (forward, backward, leave-one-out); non-search methods use a proxy instead (correlation, distance, information theory, tree, regularization).'>Type</th><th class='num' title='Bradley-Terry Elo from pairwise matches within each dataset, budget and downstream model. Random = 1000. +/β give the 95% bootstrap interval.'>Elo β</th><th class='num' title='Mean rank of the downstream error.'>Avg. rank β</th><th class='num' title='Share of pairwise comparisons won, in %. Ties count half.'>Win rate β</th><th class='num' title='Mean relative gap (error β best error) / error to the best method, in %.'>Improvability β</th><th class='num' title='Mean ratio of the error to the same model trained on all features. Below 1: the selection helped.'>Error vs. all β</th><th class='num' title='Agreement of the selected features across bootstrap resamples (Nogueira et al., 2018).'>Stability β</th><th class='num' title='Share of the budget filled with original features when as many noise features are added.'>Validity β</th><th class='num' title='Median wall-clock time of the selection.'>Time (s) β</th><th class='num' title='Share of runs where the method ran out of time and random features filled the budget.'>Fallback (%)</th><th class='num'>#Datasets</th></tr></thead><tbody><tr><td class='num' data-sort='1'>1</td><td class='method' data-sort='RFImportance'>RFImportance</td><td class='' data-sort='Tree'><span class='pill cat-non-search'>Tree</span><span class='sub'>non-search</span></td><td class='num' data-sort='1343.0'>1343<span class='sub'>+50 / β47</span></td><td class='num' data-sort='6.29'>6.29</td><td class='num' data-sort='62.2'>62.2</td><td class='num' data-sort='20.0'>20.0</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.628'>0.628</td><td class='num' data-sort='0.805'>0.805</td><td class='num' data-sort='0.2'>0.2</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='2'>2</td><td class='method' data-sort='CART'>CART</td><td class='' data-sort='Tree'><span class='pill cat-non-search'>Tree</span><span class='sub'>non-search</span></td><td class='num' data-sort='1315.0'>1315<span class='sub'>+45 / β44</span></td><td class='num' data-sort='6.56'>6.56</td><td class='num' data-sort='60.3'>60.3</td><td class='num' data-sort='22.6'>22.6</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.484'>0.484</td><td class='num' data-sort='0.731'>0.731</td><td class='num' data-sort='0.1'>0.1</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='3'>3</td><td class='method' data-sort='MarkovBlanket'>MarkovBlanket</td><td class='' data-sort='Backward search'><span class='pill cat-search'>Backward search</span><span class='sub'>search</span></td><td class='num' data-sort='1308.0'>1308<span class='sub'>+40 / β39</span></td><td class='num' data-sort='6.67'>6.67</td><td class='num' data-sort='59.5'>59.5</td><td class='num' data-sort='21.8'>21.8</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.418'>0.418</td><td class='num' data-sort='0.465'>0.465</td><td class='num' data-sort='0.2'>0.2</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='4'>4</td><td class='method' data-sort='F-Test'>F-Test</td><td class='' data-sort='Correlation'><span class='pill cat-non-search'>Correlation</span><span class='sub'>non-search</span></td><td class='num' data-sort='1290.0'>1290<span class='sub'>+40 / β39</span></td><td class='num' data-sort='7.11'>7.11</td><td class='num' data-sort='56.4'>56.4</td><td class='num' data-sort='23.8'>23.8</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.725'>0.725</td><td class='num' data-sort='0.952'>0.952</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='5'>5</td><td class='method' data-sort='MI'>MI</td><td class='' data-sort='Information theory'><span class='pill cat-non-search'>Information theory</span><span class='sub'>non-search</span></td><td class='num' data-sort='1290.0'>1290<span class='sub'>+35 / β37</span></td><td class='num' data-sort='6.95'>6.95</td><td class='num' data-sort='57.5'>57.5</td><td class='num' data-sort='23.0'>23.0</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.645'>0.645</td><td class='num' data-sort='0.898'>0.898</td><td class='num' data-sort='0.4'>0.4</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='6'>6</td><td class='method' data-sort='ElasticNet'>ElasticNet</td><td class='' data-sort='Regularization'><span class='pill cat-non-search'>Regularization</span><span class='sub'>non-search</span></td><td class='num' data-sort='1284.0'>1284<span class='sub'>+41 / β38</span></td><td class='num' data-sort='6.95'>6.95</td><td class='num' data-sort='57.5'>57.5</td><td class='num' data-sort='24.4'>24.4</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.509'>0.509</td><td class='num' data-sort='0.915'>0.915</td><td class='num' data-sort='7.5'>7.5</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='7'>7</td><td class='method' data-sort='SFS'>SFS</td><td class='' data-sort='Forward search'><span class='pill cat-search'>Forward search</span><span class='sub'>search</span></td><td class='num' data-sort='1270.0'>1270<span class='sub'>+44 / β41</span></td><td class='num' data-sort='7.13'>7.13</td><td class='num' data-sort='56.2'>56.2</td><td class='num' data-sort='24.5'>24.5</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.368'>0.368</td><td class='num' data-sort='0.75'>0.750</td><td class='num' data-sort='941.9'>941.9</td><td class='num' data-sort='29.1'>29.1</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='8'>8</td><td class='method' data-sort='mRMR'>mRMR</td><td class='' data-sort='Information theory'><span class='pill cat-non-search'>Information theory</span><span class='sub'>non-search</span></td><td class='num' data-sort='1255.0'>1255<span class='sub'>+38 / β39</span></td><td class='num' data-sort='7.68'>7.68</td><td class='num' data-sort='52.3'>52.3</td><td class='num' data-sort='24.7'>24.7</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.528'>0.528</td><td class='num' data-sort='0.779'>0.779</td><td class='num' data-sort='0.3'>0.3</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='9'>9</td><td class='method' data-sort='Lasso'>Lasso</td><td class='' data-sort='Regularization'><span class='pill cat-non-search'>Regularization</span><span class='sub'>non-search</span></td><td class='num' data-sort='1250.0'>1250<span class='sub'>+35 / β37</span></td><td class='num' data-sort='7.42'>7.42</td><td class='num' data-sort='54.1'>54.1</td><td class='num' data-sort='25.7'>25.7</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.421'>0.421</td><td class='num' data-sort='0.89'>0.890</td><td class='num' data-sort='2.2'>2.2</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='10'>10</td><td class='method' data-sort='GainRatio'>GainRatio</td><td class='' data-sort='Information theory'><span class='pill cat-non-search'>Information theory</span><span class='sub'>non-search</span></td><td class='num' data-sort='1242.0'>1242<span class='sub'>+40 / β41</span></td><td class='num' data-sort='7.98'>7.98</td><td class='num' data-sort='50.1'>50.1</td><td class='num' data-sort='26.5'>26.5</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.679'>0.679</td><td class='num' data-sort='0.942'>0.942</td><td class='num' data-sort='0.2'>0.2</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='11'>11</td><td class='method' data-sort='(R)ReliefF'>(R)ReliefF</td><td class='' data-sort='Distance'><span class='pill cat-non-search'>Distance</span><span class='sub'>non-search</span></td><td class='num' data-sort='1206.0'>1206<span class='sub'>+43 / β43</span></td><td class='num' data-sort='8.47'>8.47</td><td class='num' data-sort='46.6'>46.6</td><td class='num' data-sort='28.4'>28.4</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.642'>0.642</td><td class='num' data-sort='0.873'>0.873</td><td class='num' data-sort='14.2'>14.2</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='12'>12</td><td class='method' data-sort='SBE'>SBE</td><td class='' data-sort='Backward search'><span class='pill cat-search'>Backward search</span><span class='sub'>search</span></td><td class='num' data-sort='1186.0'>1186<span class='sub'>+49 / β45</span></td><td class='num' data-sort='8.7'>8.70</td><td class='num' data-sort='45.0'>45.0</td><td class='num' data-sort='30.3'>30.3</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.195'>0.195</td><td class='num' data-sort='0.621'>0.621</td><td class='num' data-sort='3579.1'>3,579.1</td><td class='num' data-sort='49.6'>49.6</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='13'>13</td><td class='method' data-sort='LOCO'>LOCO</td><td class='' data-sort='Leave-one-out'><span class='pill cat-search'>Leave-one-out</span><span class='sub'>search</span></td><td class='num' data-sort='1176.0'>1176<span class='sub'>+42 / β42</span></td><td class='num' data-sort='8.54'>8.54</td><td class='num' data-sort='46.1'>46.1</td><td class='num' data-sort='29.9'>29.9</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.299'>0.299</td><td class='num' data-sort='0.665'>0.665</td><td class='num' data-sort='242.0'>242.0</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='14'>14</td><td class='method' data-sort='Random'>Random</td><td class='' data-sort='Baseline'><span class='pill cat-baseline'>Baseline</span></td><td class='num' data-sort='1000.0'>1000<span class='sub'>+48 / β50</span></td><td class='num' data-sort='11.38'>11.38</td><td class='num' data-sort='25.9'>25.9</td><td class='num' data-sort='42.0'>42.0</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='-0.001'>-0.001</td><td class='num' data-sort='0.493'>0.493</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='100.0'>100.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='15'>15</td><td class='method' data-sort='LaplacianScore'>LaplacianScore</td><td class='' data-sort='Distance'><span class='pill cat-non-search'>Distance</span><span class='sub'>non-search</span></td><td class='num' data-sort='963.0'>963<span class='sub'>+60 / β75</span></td><td class='num' data-sort='12.16'>12.16</td><td class='num' data-sort='20.3'>20.3</td><td class='num' data-sort='45.3'>45.3</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.588'>0.588</td><td class='num' data-sort='0.847'>0.847</td><td class='num' data-sort='0.2'>0.2</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr></tbody></table></div><script type='text/plain' id='sa-csv'>Rank,Method,Category,Subgroup,Elo,Elo CI low,Elo CI high,Avg. rank,Win rate (%),Improvability (%),Error vs. all features,Stability,Validity,Selection time (s),Fallback rate (%),#Datasets,#Comparisons
|
| 33 |
+
1,RFImportance,Non-search,Tree,1343.0,1296.0,1393.0,6.29,62.2,20.0,,0.628,0.805,0.2,0.0,71,1065
|
| 34 |
+
2,CART,Non-search,Tree,1315.0,1271.0,1360.0,6.56,60.3,22.6,,0.484,0.731,0.1,0.0,71,1065
|
| 35 |
+
3,MarkovBlanket,Search,Backward search,1308.0,1269.0,1348.0,6.67,59.5,21.8,,0.418,0.465,0.2,0.0,71,1065
|
| 36 |
+
4,F-Test,Non-search,Correlation,1290.0,1251.0,1330.0,7.11,56.4,23.8,,0.725,0.952,0.0,0.0,71,1065
|
| 37 |
+
5,MI,Non-search,Information theory,1290.0,1253.0,1325.0,6.95,57.5,23.0,,0.645,0.898,0.4,0.0,71,1065
|
| 38 |
+
6,ElasticNet,Non-search,Regularization,1284.0,1246.0,1325.0,6.95,57.5,24.4,,0.509,0.915,7.5,0.0,71,1065
|
| 39 |
+
7,SFS,Search,Forward search,1270.0,1229.0,1314.0,7.13,56.2,24.5,,0.368,0.75,941.9,29.1,71,1065
|
| 40 |
+
8,mRMR,Non-search,Information theory,1255.0,1216.0,1293.0,7.68,52.3,24.7,,0.528,0.779,0.3,0.0,71,1065
|
| 41 |
+
9,Lasso,Non-search,Regularization,1250.0,1213.0,1285.0,7.42,54.1,25.7,,0.421,0.89,2.2,0.0,71,1065
|
| 42 |
+
10,GainRatio,Non-search,Information theory,1242.0,1201.0,1282.0,7.98,50.1,26.5,,0.679,0.942,0.2,0.0,71,1065
|
| 43 |
+
11,(R)ReliefF,Non-search,Distance,1206.0,1163.0,1249.0,8.47,46.6,28.4,,0.642,0.873,14.2,0.0,71,1065
|
| 44 |
+
12,SBE,Search,Backward search,1186.0,1141.0,1235.0,8.7,45.0,30.3,,0.195,0.621,3579.1,49.6,71,1065
|
| 45 |
+
13,LOCO,Search,Leave-one-out,1176.0,1134.0,1218.0,8.54,46.1,29.9,,0.299,0.665,242.0,0.0,71,1065
|
| 46 |
+
14,Random,Baseline,Baseline,1000.0,950.0,1048.0,11.38,25.9,42.0,,-0.001,0.493,0.0,100.0,71,1065
|
| 47 |
+
15,LaplacianScore,Non-search,Distance,963.0,888.0,1023.0,12.16,20.3,45.3,,0.588,0.847,0.2,0.0,71,1065
|
| 48 |
</script><script>
|
| 49 |
function saDownload(name, blob) {
|
| 50 |
var a = document.createElement('a');
|
data/subsets/tasks_all/models_all/budgets_0/website_leaderboard.csv
CHANGED
|
@@ -1,16 +1,16 @@
|
|
| 1 |
-
Rank,
|
| 2 |
-
1,
|
| 3 |
-
2,
|
| 4 |
-
3,
|
| 5 |
-
4,
|
| 6 |
-
5,
|
| 7 |
-
6,
|
| 8 |
-
7,
|
| 9 |
-
8,
|
| 10 |
-
9,
|
| 11 |
-
10,
|
| 12 |
-
11,
|
| 13 |
-
12,
|
| 14 |
-
13,
|
| 15 |
-
14,
|
| 16 |
-
15,
|
|
|
|
| 1 |
+
Rank,Method,Category,Subgroup,Elo,Elo CI low,Elo CI high,Avg. rank,Win rate (%),Improvability (%),Error vs. all features,Stability,Validity,Selection time (s),Fallback rate (%),#Datasets,#Comparisons
|
| 2 |
+
1,RFImportance,Non-search,Tree,1343.0,1296.0,1393.0,6.29,62.2,20.0,,0.628,0.805,0.2,0.0,71,1065
|
| 3 |
+
2,CART,Non-search,Tree,1315.0,1271.0,1360.0,6.56,60.3,22.6,,0.484,0.731,0.1,0.0,71,1065
|
| 4 |
+
3,MarkovBlanket,Search,Backward search,1308.0,1269.0,1348.0,6.67,59.5,21.8,,0.418,0.465,0.2,0.0,71,1065
|
| 5 |
+
4,F-Test,Non-search,Correlation,1290.0,1251.0,1330.0,7.11,56.4,23.8,,0.725,0.952,0.0,0.0,71,1065
|
| 6 |
+
5,MI,Non-search,Information theory,1290.0,1253.0,1325.0,6.95,57.5,23.0,,0.645,0.898,0.4,0.0,71,1065
|
| 7 |
+
6,ElasticNet,Non-search,Regularization,1284.0,1246.0,1325.0,6.95,57.5,24.4,,0.509,0.915,7.5,0.0,71,1065
|
| 8 |
+
7,SFS,Search,Forward search,1270.0,1229.0,1314.0,7.13,56.2,24.5,,0.368,0.75,941.9,29.1,71,1065
|
| 9 |
+
8,mRMR,Non-search,Information theory,1255.0,1216.0,1293.0,7.68,52.3,24.7,,0.528,0.779,0.3,0.0,71,1065
|
| 10 |
+
9,Lasso,Non-search,Regularization,1250.0,1213.0,1285.0,7.42,54.1,25.7,,0.421,0.89,2.2,0.0,71,1065
|
| 11 |
+
10,GainRatio,Non-search,Information theory,1242.0,1201.0,1282.0,7.98,50.1,26.5,,0.679,0.942,0.2,0.0,71,1065
|
| 12 |
+
11,(R)ReliefF,Non-search,Distance,1206.0,1163.0,1249.0,8.47,46.6,28.4,,0.642,0.873,14.2,0.0,71,1065
|
| 13 |
+
12,SBE,Search,Backward search,1186.0,1141.0,1235.0,8.7,45.0,30.3,,0.195,0.621,3579.1,49.6,71,1065
|
| 14 |
+
13,LOCO,Search,Leave-one-out,1176.0,1134.0,1218.0,8.54,46.1,29.9,,0.299,0.665,242.0,0.0,71,1065
|
| 15 |
+
14,Random,Baseline,Baseline,1000.0,950.0,1048.0,11.38,25.9,42.0,,-0.001,0.493,0.0,100.0,71,1065
|
| 16 |
+
15,LaplacianScore,Non-search,Distance,963.0,888.0,1023.0,12.16,20.3,45.3,,0.588,0.847,0.2,0.0,71,1065
|
data/subsets/tasks_all/models_all/budgets_1/elo_explorer.html
CHANGED
|
@@ -19,67 +19,67 @@ svg { display:block; width:100%; height:auto; }
|
|
| 19 |
.sa-ink { stroke:var(--fg); }
|
| 20 |
.sa-hit { fill:transparent; }
|
| 21 |
.sa-hit:hover { fill:var(--fg); fill-opacity:0.06; }
|
| 22 |
-
</style><defs><pattern id='sa-na' width='6' height='6' patternUnits='userSpaceOnUse' patternTransform='rotate(45)'><rect width='6' height='6' fill='#e5e7eb'/><line x1='0' y1='0' x2='0' y2='6' stroke='#9ca3af' stroke-width='2'/></pattern><linearGradient id='sa-g-stab' x1='0' y1='1' x2='0' y2='0'><stop offset='0' stop-color='#ffffff'/><stop offset='1' stop-color='#0072B2'/></linearGradient><linearGradient id='sa-g-val' x1='0' y1='1' x2='0' y2='0'><stop offset='0' stop-color='#ffffff'/><stop offset='1' stop-color='#D55E00'/></linearGradient></defs><line class='sa-grid' x1='64' x2='1024' y1='287.6' y2='287.6' stroke-width='1'/><text class='sa-muted' x='56' y='291.6' text-anchor='end' font-size='12'>1000</text><line class='sa-grid' x1='64' x2='1024' y1='233.7' y2='233.7' stroke-width='1'/><text class='sa-muted' x='56' y='237.7' text-anchor='end' font-size='12'>1100</text><line class='sa-grid' x1='64' x2='1024' y1='179.7' y2='179.7' stroke-width='1'/><text class='sa-muted' x='56' y='183.7' text-anchor='end' font-size='12'>1200</text><line class='sa-grid' x1='64' x2='1024' y1='125.8' y2='125.8' stroke-width='1'/><text class='sa-muted' x='56' y='129.8' text-anchor='end' font-size='12'>1300</text><line class='sa-grid' x1='64' x2='1024' y1='71.9' y2='71.9' stroke-width='1'/><text class='sa-muted' x='56' y='75.9' text-anchor='end' font-size='12'>1400</text><line class='sa-grid' x1='64' x2='1024' y1='18.0' y2='18.0' stroke-width='1'/><text class='sa-muted' x='56' y='22.0' text-anchor='end' font-size='12'>1500</text><text class='sa-text' transform='translate(16 173) rotate(-90)' text-anchor='middle' font-size='13'>Elo</text><line class='sa-ink' x1='64' x2='1024' y1='287.6' y2='287.6' stroke-width='1' stroke-dasharray='4 4' opacity='0.5'/><rect x='73.6' y='287.6' width='22.4' height='40.4' fill='#ffffff' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='96.0' y='287.6' width='22.4' height='40.4' fill='#eaaf80' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='96.0' x2='96.0' y1='264.9' y2='312.9'/><line x1='91.0' x2='101.0' y1='264.9' y2='264.9'/><line x1='91.0' x2='101.0' y1='312.9' y2='312.9'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(100.0 342.0) rotate(-35)'>Random</text><rect class='sa-hit' x='64.0' y='18' width='64' height='414'><title>Random (Baseline)
|
| 23 |
Elo 1000 [95% CI 953β1042]
|
| 24 |
Stability 0.001
|
| 25 |
-
Validity 0.499</title></rect><rect x='137.6' y='274.1' width='22.4' height='53.9' fill='#5ba4cd' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='160.0' y='274.1' width='22.4' height='53.9' fill='#db7626' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='160.0' x2='160.0' y1='245.0' y2='309.7'/><line x1='155.0' x2='165.0' y1='245.0' y2='245.0'/><line x1='155.0' x2='165.0' y1='309.7' y2='309.7'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(164.0 342.0) rotate(-35)'>LaplacianScore</text><rect class='sa-hit' x='128.0' y='18' width='64' height='414'><title>LaplacianScore (
|
| 26 |
Elo 1025 [95% CI 959β1079]
|
| 27 |
Stability 0.644
|
| 28 |
-
Validity 0.850</title></rect><rect x='201.6' y='169.5' width='22.4' height='158.5' fill='#c7e0ee' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='224.0' y='169.5' width='22.4' height='158.5' fill='#e59c63' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='224.0' x2='224.0' y1='145.8' y2='191.6'/><line x1='219.0' x2='229.0' y1='145.8' y2='145.8'/><line x1='219.0' x2='229.0' y1='191.6' y2='191.6'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(228.0 342.0) rotate(-35)'>LOCO</text><rect class='sa-hit' x='192.0' y='18' width='64' height='414'><title>LOCO (
|
| 29 |
Elo 1219 [95% CI 1178β1263]
|
| 30 |
Stability 0.220
|
| 31 |
-
Validity 0.613</title></rect><rect x='265.6' y='165.2' width='22.4' height='162.8' fill='#d2e6f2' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='288.0' y='165.2' width='22.4' height='162.8' fill='#e7a26c' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='288.0' x2='288.0' y1='137.7' y2='190.5'/><line x1='283.0' x2='293.0' y1='137.7' y2='137.7'/><line x1='283.0' x2='293.0' y1='190.5' y2='190.5'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(292.0 342.0) rotate(-35)'>SBE</text><rect class='sa-hit' x='256.0' y='18' width='64' height='414'><title>SBE (
|
| 32 |
Elo 1227 [95% CI 1180β1278]
|
| 33 |
Stability 0.175
|
| 34 |
-
Validity 0.578</title></rect><rect x='329.6' y='149.5' width='22.4' height='178.5' fill='#b8d8ea' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='352.0' y='149.5' width='22.4' height='178.5' fill='#e39456' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='352.0' x2='352.0' y1='125.3' y2='171.1'/><line x1='347.0' x2='357.0' y1='125.3' y2='125.3'/><line x1='347.0' x2='357.0' y1='171.1' y2='171.1'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(356.0 342.0) rotate(-35)'>SFS</text><rect class='sa-hit' x='320.0' y='18' width='64' height='414'><title>SFS (
|
| 35 |
Elo 1256 [95% CI 1216β1301]
|
| 36 |
Stability 0.278
|
| 37 |
-
Validity 0.664</title></rect><rect x='393.6' y='149.0' width='22.4' height='179.0' fill='#5aa4cd' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='416.0' y='149.0' width='22.4' height='179.0' fill='#dd7d31' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='416.0' x2='416.0' y1='128.0' y2='172.7'/><line x1='411.0' x2='421.0' y1='128.0' y2='128.0'/><line x1='411.0' x2='421.0' y1='172.7' y2='172.7'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(420.0 342.0) rotate(-35)'>(R)ReliefF</text><rect class='sa-hit' x='384.0' y='18' width='64' height='414'><title>(R)ReliefF (
|
| 38 |
Elo 1257 [95% CI 1213β1296]
|
| 39 |
Stability 0.647
|
| 40 |
-
Validity 0.808</title></rect><rect x='457.6' y='143.1' width='22.4' height='184.9' fill='#6caed2' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='480.0' y='143.1' width='22.4' height='184.9' fill='#e18c49' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='480.0' x2='480.0' y1='122.1' y2='165.2'/><line x1='475.0' x2='485.0' y1='122.1' y2='122.1'/><line x1='475.0' x2='485.0' y1='165.2' y2='165.2'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(484.0 342.0) rotate(-35)'>mRMR</text><rect class='sa-hit' x='448.0' y='18' width='64' height='414'><title>mRMR (
|
| 41 |
Elo 1268 [95% CI 1227β1307]
|
| 42 |
Stability 0.578
|
| 43 |
-
Validity 0.712</title></rect><rect x='521.6' y='140.4' width='22.4' height='187.6' fill='#519fcb' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='544.0' y='140.4' width='22.4' height='187.6' fill='#d96c16' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='544.0' x2='544.0' y1='122.1' y2='159.3'/><line x1='539.0' x2='549.0' y1='122.1' y2='122.1'/><line x1='539.0' x2='549.0' y1='159.3' y2='159.3'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(548.0 342.0) rotate(-35)'>GainRatio</text><rect class='sa-hit' x='512.0' y='18' width='64' height='414'><title>GainRatio (
|
| 44 |
Elo 1273 [95% CI 1238β1307]
|
| 45 |
Stability 0.681
|
| 46 |
-
Validity 0.915</title></rect><rect x='585.6' y='126.9' width='22.4' height='201.1' fill='#4498c6' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='608.0' y='126.9' width='22.4' height='201.1' fill='#d86911' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='608.0' x2='608.0' y1='106.4' y2='146.3'/><line x1='603.0' x2='613.0' y1='106.4' y2='106.4'/><line x1='603.0' x2='613.0' y1='146.3' y2='146.3'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(612.0 342.0) rotate(-35)'>F-Test</text><rect class='sa-hit' x='576.0' y='18' width='64' height='414'><title>F-Test (
|
| 47 |
Elo 1298 [95% CI 1262β1336]
|
| 48 |
Stability 0.734
|
| 49 |
-
Validity 0.932</title></rect><rect x='649.6' y='122.1' width='22.4' height='205.9' fill='#9ac7e1' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='672.0' y='122.1' width='22.4' height='205.9' fill='#ebb488' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='672.0' x2='672.0' y1='102.1' y2='143.1'/><line x1='667.0' x2='677.0' y1='102.1' y2='102.1'/><line x1='667.0' x2='677.0' y1='143.1' y2='143.1'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(676.0 342.0) rotate(-35)'>MarkovBlanket</text><rect class='sa-hit' x='640.0' y='18' width='64' height='414'><title>MarkovBlanket (
|
| 50 |
Elo 1307 [95% CI 1268β1344]
|
| 51 |
Stability 0.396
|
| 52 |
-
Validity 0.467</title></rect><rect x='713.6' y='117.2' width='22.4' height='210.8' fill='#53a0cb' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='736.0' y='117.2' width='22.4' height='210.8' fill='#db7423' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='736.0' x2='736.0' y1='98.3' y2='134.5'/><line x1='731.0' x2='741.0' y1='98.3' y2='98.3'/><line x1='731.0' x2='741.0' y1='134.5' y2='134.5'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(740.0 342.0) rotate(-35)'>MI</text><rect class='sa-hit' x='704.0' y='18' width='64' height='414'><title>MI (
|
| 53 |
Elo 1316 [95% CI 1284β1351]
|
| 54 |
Stability 0.673
|
| 55 |
-
Validity 0.863</title></rect><rect x='777.6' y='113.4' width='22.4' height='214.6' fill='#93c3df' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='800.0' y='113.4' width='22.4' height='214.6' fill='#dc7a2d' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='800.0' x2='800.0' y1='89.7' y2='135.5'/><line x1='795.0' x2='805.0' y1='89.7' y2='89.7'/><line x1='795.0' x2='805.0' y1='135.5' y2='135.5'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(804.0 342.0) rotate(-35)'>Lasso</text><rect class='sa-hit' x='768.0' y='18' width='64' height='414'><title>Lasso (
|
| 56 |
Elo 1323 [95% CI 1282β1367]
|
| 57 |
Stability 0.422
|
| 58 |
-
Validity 0.825</title></rect><rect x='841.6' y='107.0' width='22.4' height='221.0' fill='#8bbfdc' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='864.0' y='107.0' width='22.4' height='221.0' fill='#e4985c' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='864.0' x2='864.0' y1='84.3' y2='128.0'/><line x1='859.0' x2='869.0' y1='84.3' y2='84.3'/><line x1='859.0' x2='869.0' y1='128.0' y2='128.0'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(868.0 342.0) rotate(-35)'>CART</text><rect class='sa-hit' x='832.0' y='18' width='64' height='414'><title>CART (
|
| 59 |
Elo 1335 [95% CI 1296β1377]
|
| 60 |
Stability 0.456
|
| 61 |
-
Validity 0.638</title></rect><rect x='905.6' y='101.0' width='22.4' height='227.0' fill='#7bb6d7' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='928.0' y='101.0' width='22.4' height='227.0' fill='#da7220' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='928.0' x2='928.0' y1='78.9' y2='124.2'/><line x1='923.0' x2='933.0' y1='78.9' y2='78.9'/><line x1='923.0' x2='933.0' y1='124.2' y2='124.2'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(932.0 342.0) rotate(-35)'>ElasticNet</text><rect class='sa-hit' x='896.0' y='18' width='64' height='414'><title>ElasticNet (
|
| 62 |
Elo 1346 [95% CI 1303β1387]
|
| 63 |
Stability 0.519
|
| 64 |
-
Validity 0.873</title></rect><rect x='969.6' y='90.2' width='22.4' height='237.8' fill='#58a3cd' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='992.0' y='90.2' width='22.4' height='237.8' fill='#e08842' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='992.0' x2='992.0' y1='67.1' y2='112.3'/><line x1='987.0' x2='997.0' y1='67.1' y2='67.1'/><line x1='987.0' x2='997.0' y1='112.3' y2='112.3'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(996.0 342.0) rotate(-35)'>RFImportance</text><rect class='sa-hit' x='960.0' y='18' width='64' height='414'><title>RFImportance (
|
| 65 |
Elo 1366 [95% CI 1325β1409]
|
| 66 |
Stability 0.653
|
| 67 |
-
Validity 0.741</title></rect><line class='sa-ink' x1='64' x2='1024' y1='328.0' y2='328.0' stroke-width='1' opacity='0.4'/><rect x='1070' y='40.0' width='12' height='107.0' fill='url(#sa-g-stab)' stroke='#000' stroke-width='0.6'/><text class='sa-text' x='1076' y='30.0' text-anchor='middle' font-size='12'>stable</text><text class='sa-text' x='1076' y='165.0' text-anchor='middle' font-size='12'>unstable</text><text class='sa-muted' x='1088' y='151.0' font-size='11'>0.0</text><text class='sa-muted' x='1088' y='97.5' font-size='11'>0.5</text><text class='sa-muted' x='1088' y='44.0' font-size='11'>1.0</text><rect x='1070' y='207.0' width='12' height='107.0' fill='url(#sa-g-val)' stroke='#000' stroke-width='0.6'/><text class='sa-text' x='1076' y='197.0' text-anchor='middle' font-size='12'>valid</text><text class='sa-text' x='1076' y='332.0' text-anchor='middle' font-size='12'>invalid</text><text class='sa-muted' x='1088' y='318.0' font-size='11'>0.0</text><text class='sa-muted' x='1088' y='264.5' font-size='11'>0.5</text><text class='sa-muted' x='1088' y='211.0' font-size='11'>1.0</text><rect x='1070' y='352.0' width='12' height='12' fill='url(#sa-na)' stroke='#000' stroke-width='0.6'/><text class='sa-muted' x='1088' y='362.0' font-size='11'>no result</text></svg><script type='text/plain' id='sa-csv'>Method,
|
| 68 |
-
RFImportance,
|
| 69 |
-
ElasticNet,
|
| 70 |
-
CART,
|
| 71 |
-
Lasso,
|
| 72 |
-
MI,
|
| 73 |
-
MarkovBlanket,
|
| 74 |
-
F-Test,
|
| 75 |
-
GainRatio,
|
| 76 |
-
mRMR,
|
| 77 |
-
(R)ReliefF,
|
| 78 |
-
SFS,
|
| 79 |
-
SBE,
|
| 80 |
-
LOCO,
|
| 81 |
-
LaplacianScore,
|
| 82 |
-
Random,Baseline,1000.0,953.0,1042.0,0.001,0.499
|
| 83 |
</script><script>
|
| 84 |
function saDownload(name, blob) {
|
| 85 |
var a = document.createElement('a');
|
|
|
|
| 19 |
.sa-ink { stroke:var(--fg); }
|
| 20 |
.sa-hit { fill:transparent; }
|
| 21 |
.sa-hit:hover { fill:var(--fg); fill-opacity:0.06; }
|
| 22 |
+
</style><defs><pattern id='sa-na' width='6' height='6' patternUnits='userSpaceOnUse' patternTransform='rotate(45)'><rect width='6' height='6' fill='#e5e7eb'/><line x1='0' y1='0' x2='0' y2='6' stroke='#9ca3af' stroke-width='2'/></pattern><linearGradient id='sa-g-stab' x1='0' y1='1' x2='0' y2='0'><stop offset='0' stop-color='#ffffff'/><stop offset='1' stop-color='#0072B2'/></linearGradient><linearGradient id='sa-g-val' x1='0' y1='1' x2='0' y2='0'><stop offset='0' stop-color='#ffffff'/><stop offset='1' stop-color='#D55E00'/></linearGradient></defs><line class='sa-grid' x1='64' x2='1024' y1='287.6' y2='287.6' stroke-width='1'/><text class='sa-muted' x='56' y='291.6' text-anchor='end' font-size='12'>1000</text><line class='sa-grid' x1='64' x2='1024' y1='233.7' y2='233.7' stroke-width='1'/><text class='sa-muted' x='56' y='237.7' text-anchor='end' font-size='12'>1100</text><line class='sa-grid' x1='64' x2='1024' y1='179.7' y2='179.7' stroke-width='1'/><text class='sa-muted' x='56' y='183.7' text-anchor='end' font-size='12'>1200</text><line class='sa-grid' x1='64' x2='1024' y1='125.8' y2='125.8' stroke-width='1'/><text class='sa-muted' x='56' y='129.8' text-anchor='end' font-size='12'>1300</text><line class='sa-grid' x1='64' x2='1024' y1='71.9' y2='71.9' stroke-width='1'/><text class='sa-muted' x='56' y='75.9' text-anchor='end' font-size='12'>1400</text><line class='sa-grid' x1='64' x2='1024' y1='18.0' y2='18.0' stroke-width='1'/><text class='sa-muted' x='56' y='22.0' text-anchor='end' font-size='12'>1500</text><text class='sa-text' transform='translate(16 173) rotate(-90)' text-anchor='middle' font-size='13'>Elo</text><line class='sa-ink' x1='64' x2='1024' y1='287.6' y2='287.6' stroke-width='1' stroke-dasharray='4 4' opacity='0.5'/><rect x='73.6' y='287.6' width='22.4' height='40.4' fill='#ffffff' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='96.0' y='287.6' width='22.4' height='40.4' fill='#eaaf80' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='96.0' x2='96.0' y1='264.9' y2='312.9'/><line x1='91.0' x2='101.0' y1='264.9' y2='264.9'/><line x1='91.0' x2='101.0' y1='312.9' y2='312.9'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(100.0 342.0) rotate(-35)'>Random</text><rect class='sa-hit' x='64.0' y='18' width='64' height='414'><title>Random (Baseline, baseline)
|
| 23 |
Elo 1000 [95% CI 953β1042]
|
| 24 |
Stability 0.001
|
| 25 |
+
Validity 0.499</title></rect><rect x='137.6' y='274.1' width='22.4' height='53.9' fill='#5ba4cd' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='160.0' y='274.1' width='22.4' height='53.9' fill='#db7626' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='160.0' x2='160.0' y1='245.0' y2='309.7'/><line x1='155.0' x2='165.0' y1='245.0' y2='245.0'/><line x1='155.0' x2='165.0' y1='309.7' y2='309.7'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(164.0 342.0) rotate(-35)'>LaplacianScore</text><rect class='sa-hit' x='128.0' y='18' width='64' height='414'><title>LaplacianScore (Distance, non-search)
|
| 26 |
Elo 1025 [95% CI 959β1079]
|
| 27 |
Stability 0.644
|
| 28 |
+
Validity 0.850</title></rect><rect x='201.6' y='169.5' width='22.4' height='158.5' fill='#c7e0ee' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='224.0' y='169.5' width='22.4' height='158.5' fill='#e59c63' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='224.0' x2='224.0' y1='145.8' y2='191.6'/><line x1='219.0' x2='229.0' y1='145.8' y2='145.8'/><line x1='219.0' x2='229.0' y1='191.6' y2='191.6'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(228.0 342.0) rotate(-35)'>LOCO</text><rect class='sa-hit' x='192.0' y='18' width='64' height='414'><title>LOCO (Leave-one-out, search)
|
| 29 |
Elo 1219 [95% CI 1178β1263]
|
| 30 |
Stability 0.220
|
| 31 |
+
Validity 0.613</title></rect><rect x='265.6' y='165.2' width='22.4' height='162.8' fill='#d2e6f2' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='288.0' y='165.2' width='22.4' height='162.8' fill='#e7a26c' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='288.0' x2='288.0' y1='137.7' y2='190.5'/><line x1='283.0' x2='293.0' y1='137.7' y2='137.7'/><line x1='283.0' x2='293.0' y1='190.5' y2='190.5'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(292.0 342.0) rotate(-35)'>SBE</text><rect class='sa-hit' x='256.0' y='18' width='64' height='414'><title>SBE (Backward search, search)
|
| 32 |
Elo 1227 [95% CI 1180β1278]
|
| 33 |
Stability 0.175
|
| 34 |
+
Validity 0.578</title></rect><rect x='329.6' y='149.5' width='22.4' height='178.5' fill='#b8d8ea' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='352.0' y='149.5' width='22.4' height='178.5' fill='#e39456' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='352.0' x2='352.0' y1='125.3' y2='171.1'/><line x1='347.0' x2='357.0' y1='125.3' y2='125.3'/><line x1='347.0' x2='357.0' y1='171.1' y2='171.1'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(356.0 342.0) rotate(-35)'>SFS</text><rect class='sa-hit' x='320.0' y='18' width='64' height='414'><title>SFS (Forward search, search)
|
| 35 |
Elo 1256 [95% CI 1216β1301]
|
| 36 |
Stability 0.278
|
| 37 |
+
Validity 0.664</title></rect><rect x='393.6' y='149.0' width='22.4' height='179.0' fill='#5aa4cd' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='416.0' y='149.0' width='22.4' height='179.0' fill='#dd7d31' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='416.0' x2='416.0' y1='128.0' y2='172.7'/><line x1='411.0' x2='421.0' y1='128.0' y2='128.0'/><line x1='411.0' x2='421.0' y1='172.7' y2='172.7'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(420.0 342.0) rotate(-35)'>(R)ReliefF</text><rect class='sa-hit' x='384.0' y='18' width='64' height='414'><title>(R)ReliefF (Distance, non-search)
|
| 38 |
Elo 1257 [95% CI 1213β1296]
|
| 39 |
Stability 0.647
|
| 40 |
+
Validity 0.808</title></rect><rect x='457.6' y='143.1' width='22.4' height='184.9' fill='#6caed2' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='480.0' y='143.1' width='22.4' height='184.9' fill='#e18c49' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='480.0' x2='480.0' y1='122.1' y2='165.2'/><line x1='475.0' x2='485.0' y1='122.1' y2='122.1'/><line x1='475.0' x2='485.0' y1='165.2' y2='165.2'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(484.0 342.0) rotate(-35)'>mRMR</text><rect class='sa-hit' x='448.0' y='18' width='64' height='414'><title>mRMR (Information theory, non-search)
|
| 41 |
Elo 1268 [95% CI 1227β1307]
|
| 42 |
Stability 0.578
|
| 43 |
+
Validity 0.712</title></rect><rect x='521.6' y='140.4' width='22.4' height='187.6' fill='#519fcb' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='544.0' y='140.4' width='22.4' height='187.6' fill='#d96c16' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='544.0' x2='544.0' y1='122.1' y2='159.3'/><line x1='539.0' x2='549.0' y1='122.1' y2='122.1'/><line x1='539.0' x2='549.0' y1='159.3' y2='159.3'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(548.0 342.0) rotate(-35)'>GainRatio</text><rect class='sa-hit' x='512.0' y='18' width='64' height='414'><title>GainRatio (Information theory, non-search)
|
| 44 |
Elo 1273 [95% CI 1238β1307]
|
| 45 |
Stability 0.681
|
| 46 |
+
Validity 0.915</title></rect><rect x='585.6' y='126.9' width='22.4' height='201.1' fill='#4498c6' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='608.0' y='126.9' width='22.4' height='201.1' fill='#d86911' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='608.0' x2='608.0' y1='106.4' y2='146.3'/><line x1='603.0' x2='613.0' y1='106.4' y2='106.4'/><line x1='603.0' x2='613.0' y1='146.3' y2='146.3'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(612.0 342.0) rotate(-35)'>F-Test</text><rect class='sa-hit' x='576.0' y='18' width='64' height='414'><title>F-Test (Correlation, non-search)
|
| 47 |
Elo 1298 [95% CI 1262β1336]
|
| 48 |
Stability 0.734
|
| 49 |
+
Validity 0.932</title></rect><rect x='649.6' y='122.1' width='22.4' height='205.9' fill='#9ac7e1' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='672.0' y='122.1' width='22.4' height='205.9' fill='#ebb488' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='672.0' x2='672.0' y1='102.1' y2='143.1'/><line x1='667.0' x2='677.0' y1='102.1' y2='102.1'/><line x1='667.0' x2='677.0' y1='143.1' y2='143.1'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(676.0 342.0) rotate(-35)'>MarkovBlanket</text><rect class='sa-hit' x='640.0' y='18' width='64' height='414'><title>MarkovBlanket (Backward search, search)
|
| 50 |
Elo 1307 [95% CI 1268β1344]
|
| 51 |
Stability 0.396
|
| 52 |
+
Validity 0.467</title></rect><rect x='713.6' y='117.2' width='22.4' height='210.8' fill='#53a0cb' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='736.0' y='117.2' width='22.4' height='210.8' fill='#db7423' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='736.0' x2='736.0' y1='98.3' y2='134.5'/><line x1='731.0' x2='741.0' y1='98.3' y2='98.3'/><line x1='731.0' x2='741.0' y1='134.5' y2='134.5'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(740.0 342.0) rotate(-35)'>MI</text><rect class='sa-hit' x='704.0' y='18' width='64' height='414'><title>MI (Information theory, non-search)
|
| 53 |
Elo 1316 [95% CI 1284β1351]
|
| 54 |
Stability 0.673
|
| 55 |
+
Validity 0.863</title></rect><rect x='777.6' y='113.4' width='22.4' height='214.6' fill='#93c3df' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='800.0' y='113.4' width='22.4' height='214.6' fill='#dc7a2d' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='800.0' x2='800.0' y1='89.7' y2='135.5'/><line x1='795.0' x2='805.0' y1='89.7' y2='89.7'/><line x1='795.0' x2='805.0' y1='135.5' y2='135.5'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(804.0 342.0) rotate(-35)'>Lasso</text><rect class='sa-hit' x='768.0' y='18' width='64' height='414'><title>Lasso (Regularization, non-search)
|
| 56 |
Elo 1323 [95% CI 1282β1367]
|
| 57 |
Stability 0.422
|
| 58 |
+
Validity 0.825</title></rect><rect x='841.6' y='107.0' width='22.4' height='221.0' fill='#8bbfdc' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='864.0' y='107.0' width='22.4' height='221.0' fill='#e4985c' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='864.0' x2='864.0' y1='84.3' y2='128.0'/><line x1='859.0' x2='869.0' y1='84.3' y2='84.3'/><line x1='859.0' x2='869.0' y1='128.0' y2='128.0'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(868.0 342.0) rotate(-35)'>CART</text><rect class='sa-hit' x='832.0' y='18' width='64' height='414'><title>CART (Tree, non-search)
|
| 59 |
Elo 1335 [95% CI 1296β1377]
|
| 60 |
Stability 0.456
|
| 61 |
+
Validity 0.638</title></rect><rect x='905.6' y='101.0' width='22.4' height='227.0' fill='#7bb6d7' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='928.0' y='101.0' width='22.4' height='227.0' fill='#da7220' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='928.0' x2='928.0' y1='78.9' y2='124.2'/><line x1='923.0' x2='933.0' y1='78.9' y2='78.9'/><line x1='923.0' x2='933.0' y1='124.2' y2='124.2'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(932.0 342.0) rotate(-35)'>ElasticNet</text><rect class='sa-hit' x='896.0' y='18' width='64' height='414'><title>ElasticNet (Regularization, non-search)
|
| 62 |
Elo 1346 [95% CI 1303β1387]
|
| 63 |
Stability 0.519
|
| 64 |
+
Validity 0.873</title></rect><rect x='969.6' y='90.2' width='22.4' height='237.8' fill='#58a3cd' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='992.0' y='90.2' width='22.4' height='237.8' fill='#e08842' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='992.0' x2='992.0' y1='67.1' y2='112.3'/><line x1='987.0' x2='997.0' y1='67.1' y2='67.1'/><line x1='987.0' x2='997.0' y1='112.3' y2='112.3'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(996.0 342.0) rotate(-35)'>RFImportance</text><rect class='sa-hit' x='960.0' y='18' width='64' height='414'><title>RFImportance (Tree, non-search)
|
| 65 |
Elo 1366 [95% CI 1325β1409]
|
| 66 |
Stability 0.653
|
| 67 |
+
Validity 0.741</title></rect><line class='sa-ink' x1='64' x2='1024' y1='328.0' y2='328.0' stroke-width='1' opacity='0.4'/><rect x='1070' y='40.0' width='12' height='107.0' fill='url(#sa-g-stab)' stroke='#000' stroke-width='0.6'/><text class='sa-text' x='1076' y='30.0' text-anchor='middle' font-size='12'>stable</text><text class='sa-text' x='1076' y='165.0' text-anchor='middle' font-size='12'>unstable</text><text class='sa-muted' x='1088' y='151.0' font-size='11'>0.0</text><text class='sa-muted' x='1088' y='97.5' font-size='11'>0.5</text><text class='sa-muted' x='1088' y='44.0' font-size='11'>1.0</text><rect x='1070' y='207.0' width='12' height='107.0' fill='url(#sa-g-val)' stroke='#000' stroke-width='0.6'/><text class='sa-text' x='1076' y='197.0' text-anchor='middle' font-size='12'>valid</text><text class='sa-text' x='1076' y='332.0' text-anchor='middle' font-size='12'>invalid</text><text class='sa-muted' x='1088' y='318.0' font-size='11'>0.0</text><text class='sa-muted' x='1088' y='264.5' font-size='11'>0.5</text><text class='sa-muted' x='1088' y='211.0' font-size='11'>1.0</text><rect x='1070' y='352.0' width='12' height='12' fill='url(#sa-na)' stroke='#000' stroke-width='0.6'/><text class='sa-muted' x='1088' y='362.0' font-size='11'>no result</text></svg><script type='text/plain' id='sa-csv'>Method,Category,Subgroup,Elo,Elo CI low,Elo CI high,Stability,Validity
|
| 68 |
+
RFImportance,Non-search,Tree,1366.0,1325.0,1409.0,0.653,0.741
|
| 69 |
+
ElasticNet,Non-search,Regularization,1346.0,1303.0,1387.0,0.519,0.873
|
| 70 |
+
CART,Non-search,Tree,1335.0,1296.0,1377.0,0.456,0.638
|
| 71 |
+
Lasso,Non-search,Regularization,1323.0,1282.0,1367.0,0.422,0.825
|
| 72 |
+
MI,Non-search,Information theory,1316.0,1284.0,1351.0,0.673,0.863
|
| 73 |
+
MarkovBlanket,Search,Backward search,1307.0,1268.0,1344.0,0.396,0.467
|
| 74 |
+
F-Test,Non-search,Correlation,1298.0,1262.0,1336.0,0.734,0.932
|
| 75 |
+
GainRatio,Non-search,Information theory,1273.0,1238.0,1307.0,0.681,0.915
|
| 76 |
+
mRMR,Non-search,Information theory,1268.0,1227.0,1307.0,0.578,0.712
|
| 77 |
+
(R)ReliefF,Non-search,Distance,1257.0,1213.0,1296.0,0.647,0.808
|
| 78 |
+
SFS,Search,Forward search,1256.0,1216.0,1301.0,0.278,0.664
|
| 79 |
+
SBE,Search,Backward search,1227.0,1180.0,1278.0,0.175,0.578
|
| 80 |
+
LOCO,Search,Leave-one-out,1219.0,1178.0,1263.0,0.22,0.613
|
| 81 |
+
LaplacianScore,Non-search,Distance,1025.0,959.0,1079.0,0.644,0.85
|
| 82 |
+
Random,Baseline,Baseline,1000.0,953.0,1042.0,0.001,0.499
|
| 83 |
</script><script>
|
| 84 |
function saDownload(name, blob) {
|
| 85 |
var a = document.createElement('a');
|
data/subsets/tasks_all/models_all/budgets_1/leaderboard_table.html
CHANGED
|
@@ -26,23 +26,25 @@ td.method { font-weight:600; }
|
|
| 26 |
td .sub { color:var(--muted); font-size:12px; font-weight:400; margin-left:6px; }
|
| 27 |
td.na { color:var(--muted); }
|
| 28 |
.pill { display:inline-block; padding:0 8px; border:1px solid var(--line); border-radius:999px;
|
| 29 |
-
font-size:12px; color:var(--
|
| 30 |
-
</style></head><body><div class='toolbar'><span class='hint'>Click a column to sort. Hover a header for its definition.</span><button onclick="saCsv('selectarena_leaderboard.csv')">CSV</button></div><div class='wrap'><table id='sa-table'><thead><tr><th class='num'>#</th><th class=''>Method</th><th class='' title='Filter: scores features from the data alone. Wrapper: searches feature subsets by refitting a proxy model. Embedded: reads the selection off a fitted model.'>Type</th><th class='num' title='Bradley-Terry Elo from pairwise matches within each dataset, budget and downstream model. Random = 1000. +/β give the 95% bootstrap interval.'>Elo β</th><th class='num' title='Mean rank of the downstream error.'>Avg. rank β</th><th class='num' title='Share of pairwise comparisons won, in %. Ties count half.'>Win rate β</th><th class='num' title='Mean relative gap (error β best error) / error to the best method, in %.'>Improvability β</th><th class='num' title='Mean ratio of the error to the same model trained on all features. Below 1: the selection helped.'>Error vs. all β</th><th class='num' title='Agreement of the selected features across bootstrap resamples (Nogueira et al., 2018).'>Stability β</th><th class='num' title='Share of the budget filled with original features when as many noise features are added.'>Validity β</th><th class='num' title='Median wall-clock time of the selection.'>Time (s) β</th><th class='num' title='Share of runs where the method ran out of time and random features filled the budget.'>Fallback (%)</th><th class='num'>#Datasets</th></tr></thead><tbody><tr><td class='num' data-sort='1'>1</td><td class='method' data-sort='RFImportance'>RFImportance</td><td class='' data-sort='Embedded'><span class='pill'>π³ Embedded</span></td><td class='num' data-sort='1366.0'>1366<span class='sub'>+43 / β41</span></td><td class='num' data-sort='6.31'>6.31</td><td class='num' data-sort='62.1'>62.1</td><td class='num' data-sort='17.8'>17.8</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.653'>0.653</td><td class='num' data-sort='0.741'>0.741</td><td class='num' data-sort='0.2'>0.2</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='2'>2</td><td class='method' data-sort='ElasticNet'>ElasticNet</td><td class='' data-sort='Embedded'><span class='pill'>π³ Embedded</span></td><td class='num' data-sort='1346.0'>1346<span class='sub'>+41 / β43</span></td><td class='num' data-sort='6.51'>6.51</td><td class='num' data-sort='60.6'>60.6</td><td class='num' data-sort='20.6'>20.6</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.519'>0.519</td><td class='num' data-sort='0.873'>0.873</td><td class='num' data-sort='7.6'>7.6</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='3'>3</td><td class='method' data-sort='CART'>CART</td><td class='' data-sort='Embedded'><span class='pill'>π³ Embedded</span></td><td class='num' data-sort='1335.0'>1335<span class='sub'>+42 / β39</span></td><td class='num' data-sort='6.63'>6.63</td><td class='num' data-sort='59.8'>59.8</td><td class='num' data-sort='20.9'>20.9</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.456'>0.456</td><td class='num' data-sort='0.638'>0.638</td><td class='num' data-sort='0.1'>0.1</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='4'>4</td><td class='method' data-sort='Lasso'>Lasso</td><td class='' data-sort='Embedded'><span class='pill'>π³ Embedded</span></td><td class='num' data-sort='1323.0'>1323<span class='sub'>+44 / β41</span></td><td class='num' data-sort='6.78'>6.78</td><td class='num' data-sort='58.7'>58.7</td><td class='num' data-sort='20.2'>20.2</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.422'>0.422</td><td class='num' data-sort='0.825'>0.825</td><td class='num' data-sort='2.3'>2.3</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='5'>5</td><td class='method' data-sort='MI'>MI</td><td class='' data-sort='Filter'><span class='pill'>π Filter</span></td><td class='num' data-sort='1316.0'>1316<span class='sub'>+35 / β32</span></td><td class='num' data-sort='7.01'>7.01</td><td class='num' data-sort='57.1'>57.1</td><td class='num' data-sort='21.2'>21.2</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.673'>0.673</td><td class='num' data-sort='0.863'>0.863</td><td class='num' data-sort='0.4'>0.4</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='6'>6</td><td class='method' data-sort='MarkovBlanket'>MarkovBlanket</td><td class='' data-sort='Filter'><span class='pill'>π Filter</span></td><td class='num' data-sort='1307.0'>1307<span class='sub'>+37 / β39</span></td><td class='num' data-sort='7.15'>7.15</td><td class='num' data-sort='56.0'>56.0</td><td class='num' data-sort='22.8'>22.8</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.396'>0.396</td><td class='num' data-sort='0.467'>0.467</td><td class='num' data-sort='0.2'>0.2</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='7'>7</td><td class='method' data-sort='F-Test'>F-Test</td><td class='' data-sort='Filter'><span class='pill'>π Filter</span></td><td class='num' data-sort='1298.0'>1298<span class='sub'>+38 / β36</span></td><td class='num' data-sort='7.39'>7.39</td><td class='num' data-sort='54.3'>54.3</td><td class='num' data-sort='23.2'>23.2</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.734'>0.734</td><td class='num' data-sort='0.932'>0.932</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='8'>8</td><td class='method' data-sort='GainRatio'>GainRatio</td><td class='' data-sort='Filter'><span class='pill'>π Filter</span></td><td class='num' data-sort='1273.0'>1273<span class='sub'>+34 / β35</span></td><td class='num' data-sort='7.82'>7.82</td><td class='num' data-sort='51.3'>51.3</td><td class='num' data-sort='25.1'>25.1</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.681'>0.681</td><td class='num' data-sort='0.915'>0.915</td><td class='num' data-sort='0.2'>0.2</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='9'>9</td><td class='method' data-sort='mRMR'>mRMR</td><td class='' data-sort='Filter'><span class='pill'>π Filter</span></td><td class='num' data-sort='1268.0'>1268<span class='sub'>+39 / β41</span></td><td class='num' data-sort='7.89'>7.89</td><td class='num' data-sort='50.8'>50.8</td><td class='num' data-sort='23.8'>23.8</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.578'>0.578</td><td class='num' data-sort='0.712'>0.712</td><td class='num' data-sort='0.4'>0.4</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='10'>10</td><td class='method' data-sort='(R)ReliefF'>(R)ReliefF</td><td class='' data-sort='Filter'><span class='pill'>π Filter</span></td><td class='num' data-sort='1257.0'>1257<span class='sub'>+39 / β44</span></td><td class='num' data-sort='8.18'>8.18</td><td class='num' data-sort='48.7'>48.7</td><td class='num' data-sort='24.4'>24.4</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.647'>0.647</td><td class='num' data-sort='0.808'>0.808</td><td class='num' data-sort='14.3'>14.3</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='11'>11</td><td class='method' data-sort='SFS'>SFS</td><td class='' data-sort='Wrapper'><span class='pill'>π Wrapper</span></td><td class='num' data-sort='1256.0'>1256<span class='sub'>+45 / β40</span></td><td class='num' data-sort='7.91'>7.91</td><td class='num' data-sort='50.6'>50.6</td><td class='num' data-sort='25.0'>25.0</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.278'>0.278</td><td class='num' data-sort='0.664'>0.664</td><td class='num' data-sort='1634.9'>1,634.9</td><td class='num' data-sort='43.8'>43.8</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='12'>12</td><td class='method' data-sort='SBE'>SBE</td><td class='' data-sort='Wrapper'><span class='pill'>π Wrapper</span></td><td class='num' data-sort='1227.0'>1227<span class='sub'>+51 / β47</span></td><td class='num' data-sort='8.37'>8.37</td><td class='num' data-sort='47.3'>47.3</td><td class='num' data-sort='28.2'>28.2</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.175'>0.175</td><td class='num' data-sort='0.578'>0.578</td><td class='num' data-sort='3166.2'>3,166.2</td><td class='num' data-sort='48.9'>48.9</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='13'>13</td><td class='method' data-sort='LOCO'>LOCO</td><td class='' data-sort='Wrapper'><span class='pill'>π Wrapper</span></td><td class='num' data-sort='1219.0'>1219<span class='sub'>+44 / β41</span></td><td class='num' data-sort='8.54'>8.54</td><td class='num' data-sort='46.1'>46.1</td><td class='num' data-sort='27.9'>27.9</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.22'>0.220</td><td class='num' data-sort='0.613'>0.613</td><td class='num' data-sort='237.3'>237.3</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='14'>14</td><td class='method' data-sort='LaplacianScore'>LaplacianScore</td><td class='' data-sort='Filter'><span class='pill'>π Filter</span></td><td class='num' data-sort='1025.0'>1025<span class='sub'>+54 / β66</span></td><td class='num' data-sort='11.85'>11.85</td><td class='num' data-sort='22.5'>22.5</td><td class='num' data-sort='42.2'>42.2</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.644'>0.644</td><td class='num' data-sort='0.85'>0.850</td><td class='num' data-sort='0.2'>0.2</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='15'>15</td><td class='method' data-sort='Random'>Random</td><td class='' data-sort='Baseline'><span class='pill'>βͺ Baseline</span></td><td class='num' data-sort='1000.0'>1000<span class='sub'>+42 / β47</span></td><td class='num' data-sort='11.64'>11.64</td><td class='num' data-sort='24.0'>24.0</td><td class='num' data-sort='39.7'>39.7</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.001'>0.001</td><td class='num' data-sort='0.499'>0.499</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='100.0'>100.0</td><td class='num' data-sort='71'>71</td></tr></tbody></table></div><script type='text/plain' id='sa-csv'>Rank,Type,TypeName,Method,Elo,Elo CI low,Elo CI high,Avg. rank,Win rate (%),Improvability (%),Error vs. all features,Stability,Validity,Selection time (s),Fallback rate (%),#Datasets,#Comparisons
|
| 31 |
-
|
| 32 |
-
2,π³,Embedded,ElasticNet,1346.0,1303.0,1387.0,6.51,60.6,20.6,,0.519,0.873,7.6,0.0,71,1065
|
| 33 |
-
|
| 34 |
-
|
| 35 |
-
|
| 36 |
-
|
| 37 |
-
|
| 38 |
-
|
| 39 |
-
|
| 40 |
-
|
| 41 |
-
|
| 42 |
-
|
| 43 |
-
|
| 44 |
-
|
| 45 |
-
|
|
|
|
|
|
|
| 46 |
</script><script>
|
| 47 |
function saDownload(name, blob) {
|
| 48 |
var a = document.createElement('a');
|
|
|
|
| 26 |
td .sub { color:var(--muted); font-size:12px; font-weight:400; margin-left:6px; }
|
| 27 |
td.na { color:var(--muted); }
|
| 28 |
.pill { display:inline-block; padding:0 8px; border:1px solid var(--line); border-radius:999px;
|
| 29 |
+
font-size:12px; color:var(--fg); }
|
| 30 |
+
.pill.cat-search { border-color:var(--accent); }
|
| 31 |
+
.pill.cat-baseline { color:var(--muted); }
|
| 32 |
+
</style></head><body><div class='toolbar'><span class='hint'>Click a column to sort. Hover a header for its definition.</span><button onclick="saCsv('selectarena_leaderboard.csv')">CSV</button></div><div class='wrap'><table id='sa-table'><thead><tr><th class='num'>#</th><th class=''>Method</th><th class='' title='Taxonomy of the paper (Appendix A). Search methods evaluate many feature subsets with the model (forward, backward, leave-one-out); non-search methods use a proxy instead (correlation, distance, information theory, tree, regularization).'>Type</th><th class='num' title='Bradley-Terry Elo from pairwise matches within each dataset, budget and downstream model. Random = 1000. +/β give the 95% bootstrap interval.'>Elo β</th><th class='num' title='Mean rank of the downstream error.'>Avg. rank β</th><th class='num' title='Share of pairwise comparisons won, in %. Ties count half.'>Win rate β</th><th class='num' title='Mean relative gap (error β best error) / error to the best method, in %.'>Improvability β</th><th class='num' title='Mean ratio of the error to the same model trained on all features. Below 1: the selection helped.'>Error vs. all β</th><th class='num' title='Agreement of the selected features across bootstrap resamples (Nogueira et al., 2018).'>Stability β</th><th class='num' title='Share of the budget filled with original features when as many noise features are added.'>Validity β</th><th class='num' title='Median wall-clock time of the selection.'>Time (s) β</th><th class='num' title='Share of runs where the method ran out of time and random features filled the budget.'>Fallback (%)</th><th class='num'>#Datasets</th></tr></thead><tbody><tr><td class='num' data-sort='1'>1</td><td class='method' data-sort='RFImportance'>RFImportance</td><td class='' data-sort='Tree'><span class='pill cat-non-search'>Tree</span><span class='sub'>non-search</span></td><td class='num' data-sort='1366.0'>1366<span class='sub'>+43 / β41</span></td><td class='num' data-sort='6.31'>6.31</td><td class='num' data-sort='62.1'>62.1</td><td class='num' data-sort='17.8'>17.8</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.653'>0.653</td><td class='num' data-sort='0.741'>0.741</td><td class='num' data-sort='0.2'>0.2</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='2'>2</td><td class='method' data-sort='ElasticNet'>ElasticNet</td><td class='' data-sort='Regularization'><span class='pill cat-non-search'>Regularization</span><span class='sub'>non-search</span></td><td class='num' data-sort='1346.0'>1346<span class='sub'>+41 / β43</span></td><td class='num' data-sort='6.51'>6.51</td><td class='num' data-sort='60.6'>60.6</td><td class='num' data-sort='20.6'>20.6</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.519'>0.519</td><td class='num' data-sort='0.873'>0.873</td><td class='num' data-sort='7.6'>7.6</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='3'>3</td><td class='method' data-sort='CART'>CART</td><td class='' data-sort='Tree'><span class='pill cat-non-search'>Tree</span><span class='sub'>non-search</span></td><td class='num' data-sort='1335.0'>1335<span class='sub'>+42 / β39</span></td><td class='num' data-sort='6.63'>6.63</td><td class='num' data-sort='59.8'>59.8</td><td class='num' data-sort='20.9'>20.9</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.456'>0.456</td><td class='num' data-sort='0.638'>0.638</td><td class='num' data-sort='0.1'>0.1</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='4'>4</td><td class='method' data-sort='Lasso'>Lasso</td><td class='' data-sort='Regularization'><span class='pill cat-non-search'>Regularization</span><span class='sub'>non-search</span></td><td class='num' data-sort='1323.0'>1323<span class='sub'>+44 / β41</span></td><td class='num' data-sort='6.78'>6.78</td><td class='num' data-sort='58.7'>58.7</td><td class='num' data-sort='20.2'>20.2</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.422'>0.422</td><td class='num' data-sort='0.825'>0.825</td><td class='num' data-sort='2.3'>2.3</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='5'>5</td><td class='method' data-sort='MI'>MI</td><td class='' data-sort='Information theory'><span class='pill cat-non-search'>Information theory</span><span class='sub'>non-search</span></td><td class='num' data-sort='1316.0'>1316<span class='sub'>+35 / β32</span></td><td class='num' data-sort='7.01'>7.01</td><td class='num' data-sort='57.1'>57.1</td><td class='num' data-sort='21.2'>21.2</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.673'>0.673</td><td class='num' data-sort='0.863'>0.863</td><td class='num' data-sort='0.4'>0.4</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='6'>6</td><td class='method' data-sort='MarkovBlanket'>MarkovBlanket</td><td class='' data-sort='Backward search'><span class='pill cat-search'>Backward search</span><span class='sub'>search</span></td><td class='num' data-sort='1307.0'>1307<span class='sub'>+37 / β39</span></td><td class='num' data-sort='7.15'>7.15</td><td class='num' data-sort='56.0'>56.0</td><td class='num' data-sort='22.8'>22.8</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.396'>0.396</td><td class='num' data-sort='0.467'>0.467</td><td class='num' data-sort='0.2'>0.2</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='7'>7</td><td class='method' data-sort='F-Test'>F-Test</td><td class='' data-sort='Correlation'><span class='pill cat-non-search'>Correlation</span><span class='sub'>non-search</span></td><td class='num' data-sort='1298.0'>1298<span class='sub'>+38 / β36</span></td><td class='num' data-sort='7.39'>7.39</td><td class='num' data-sort='54.3'>54.3</td><td class='num' data-sort='23.2'>23.2</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.734'>0.734</td><td class='num' data-sort='0.932'>0.932</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='8'>8</td><td class='method' data-sort='GainRatio'>GainRatio</td><td class='' data-sort='Information theory'><span class='pill cat-non-search'>Information theory</span><span class='sub'>non-search</span></td><td class='num' data-sort='1273.0'>1273<span class='sub'>+34 / β35</span></td><td class='num' data-sort='7.82'>7.82</td><td class='num' data-sort='51.3'>51.3</td><td class='num' data-sort='25.1'>25.1</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.681'>0.681</td><td class='num' data-sort='0.915'>0.915</td><td class='num' data-sort='0.2'>0.2</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='9'>9</td><td class='method' data-sort='mRMR'>mRMR</td><td class='' data-sort='Information theory'><span class='pill cat-non-search'>Information theory</span><span class='sub'>non-search</span></td><td class='num' data-sort='1268.0'>1268<span class='sub'>+39 / β41</span></td><td class='num' data-sort='7.89'>7.89</td><td class='num' data-sort='50.8'>50.8</td><td class='num' data-sort='23.8'>23.8</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.578'>0.578</td><td class='num' data-sort='0.712'>0.712</td><td class='num' data-sort='0.4'>0.4</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='10'>10</td><td class='method' data-sort='(R)ReliefF'>(R)ReliefF</td><td class='' data-sort='Distance'><span class='pill cat-non-search'>Distance</span><span class='sub'>non-search</span></td><td class='num' data-sort='1257.0'>1257<span class='sub'>+39 / β44</span></td><td class='num' data-sort='8.18'>8.18</td><td class='num' data-sort='48.7'>48.7</td><td class='num' data-sort='24.4'>24.4</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.647'>0.647</td><td class='num' data-sort='0.808'>0.808</td><td class='num' data-sort='14.3'>14.3</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='11'>11</td><td class='method' data-sort='SFS'>SFS</td><td class='' data-sort='Forward search'><span class='pill cat-search'>Forward search</span><span class='sub'>search</span></td><td class='num' data-sort='1256.0'>1256<span class='sub'>+45 / β40</span></td><td class='num' data-sort='7.91'>7.91</td><td class='num' data-sort='50.6'>50.6</td><td class='num' data-sort='25.0'>25.0</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.278'>0.278</td><td class='num' data-sort='0.664'>0.664</td><td class='num' data-sort='1634.9'>1,634.9</td><td class='num' data-sort='43.8'>43.8</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='12'>12</td><td class='method' data-sort='SBE'>SBE</td><td class='' data-sort='Backward search'><span class='pill cat-search'>Backward search</span><span class='sub'>search</span></td><td class='num' data-sort='1227.0'>1227<span class='sub'>+51 / β47</span></td><td class='num' data-sort='8.37'>8.37</td><td class='num' data-sort='47.3'>47.3</td><td class='num' data-sort='28.2'>28.2</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.175'>0.175</td><td class='num' data-sort='0.578'>0.578</td><td class='num' data-sort='3166.2'>3,166.2</td><td class='num' data-sort='48.9'>48.9</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='13'>13</td><td class='method' data-sort='LOCO'>LOCO</td><td class='' data-sort='Leave-one-out'><span class='pill cat-search'>Leave-one-out</span><span class='sub'>search</span></td><td class='num' data-sort='1219.0'>1219<span class='sub'>+44 / β41</span></td><td class='num' data-sort='8.54'>8.54</td><td class='num' data-sort='46.1'>46.1</td><td class='num' data-sort='27.9'>27.9</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.22'>0.220</td><td class='num' data-sort='0.613'>0.613</td><td class='num' data-sort='237.3'>237.3</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='14'>14</td><td class='method' data-sort='LaplacianScore'>LaplacianScore</td><td class='' data-sort='Distance'><span class='pill cat-non-search'>Distance</span><span class='sub'>non-search</span></td><td class='num' data-sort='1025.0'>1025<span class='sub'>+54 / β66</span></td><td class='num' data-sort='11.85'>11.85</td><td class='num' data-sort='22.5'>22.5</td><td class='num' data-sort='42.2'>42.2</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.644'>0.644</td><td class='num' data-sort='0.85'>0.850</td><td class='num' data-sort='0.2'>0.2</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='15'>15</td><td class='method' data-sort='Random'>Random</td><td class='' data-sort='Baseline'><span class='pill cat-baseline'>Baseline</span></td><td class='num' data-sort='1000.0'>1000<span class='sub'>+42 / β47</span></td><td class='num' data-sort='11.64'>11.64</td><td class='num' data-sort='24.0'>24.0</td><td class='num' data-sort='39.7'>39.7</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.001'>0.001</td><td class='num' data-sort='0.499'>0.499</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='100.0'>100.0</td><td class='num' data-sort='71'>71</td></tr></tbody></table></div><script type='text/plain' id='sa-csv'>Rank,Method,Category,Subgroup,Elo,Elo CI low,Elo CI high,Avg. rank,Win rate (%),Improvability (%),Error vs. all features,Stability,Validity,Selection time (s),Fallback rate (%),#Datasets,#Comparisons
|
| 33 |
+
1,RFImportance,Non-search,Tree,1366.0,1325.0,1409.0,6.31,62.1,17.8,,0.653,0.741,0.2,0.0,71,1065
|
| 34 |
+
2,ElasticNet,Non-search,Regularization,1346.0,1303.0,1387.0,6.51,60.6,20.6,,0.519,0.873,7.6,0.0,71,1065
|
| 35 |
+
3,CART,Non-search,Tree,1335.0,1296.0,1377.0,6.63,59.8,20.9,,0.456,0.638,0.1,0.0,71,1065
|
| 36 |
+
4,Lasso,Non-search,Regularization,1323.0,1282.0,1367.0,6.78,58.7,20.2,,0.422,0.825,2.3,0.0,71,1065
|
| 37 |
+
5,MI,Non-search,Information theory,1316.0,1284.0,1351.0,7.01,57.1,21.2,,0.673,0.863,0.4,0.0,71,1065
|
| 38 |
+
6,MarkovBlanket,Search,Backward search,1307.0,1268.0,1344.0,7.15,56.0,22.8,,0.396,0.467,0.2,0.0,71,1065
|
| 39 |
+
7,F-Test,Non-search,Correlation,1298.0,1262.0,1336.0,7.39,54.3,23.2,,0.734,0.932,0.0,0.0,71,1065
|
| 40 |
+
8,GainRatio,Non-search,Information theory,1273.0,1238.0,1307.0,7.82,51.3,25.1,,0.681,0.915,0.2,0.0,71,1065
|
| 41 |
+
9,mRMR,Non-search,Information theory,1268.0,1227.0,1307.0,7.89,50.8,23.8,,0.578,0.712,0.4,0.0,71,1065
|
| 42 |
+
10,(R)ReliefF,Non-search,Distance,1257.0,1213.0,1296.0,8.18,48.7,24.4,,0.647,0.808,14.3,0.0,71,1065
|
| 43 |
+
11,SFS,Search,Forward search,1256.0,1216.0,1301.0,7.91,50.6,25.0,,0.278,0.664,1634.9,43.8,71,1065
|
| 44 |
+
12,SBE,Search,Backward search,1227.0,1180.0,1278.0,8.37,47.3,28.2,,0.175,0.578,3166.2,48.9,71,1065
|
| 45 |
+
13,LOCO,Search,Leave-one-out,1219.0,1178.0,1263.0,8.54,46.1,27.9,,0.22,0.613,237.3,0.0,71,1065
|
| 46 |
+
14,LaplacianScore,Non-search,Distance,1025.0,959.0,1079.0,11.85,22.5,42.2,,0.644,0.85,0.2,0.0,71,1065
|
| 47 |
+
15,Random,Baseline,Baseline,1000.0,953.0,1042.0,11.64,24.0,39.7,,0.001,0.499,0.0,100.0,71,1065
|
| 48 |
</script><script>
|
| 49 |
function saDownload(name, blob) {
|
| 50 |
var a = document.createElement('a');
|
data/subsets/tasks_all/models_all/budgets_1/website_leaderboard.csv
CHANGED
|
@@ -1,16 +1,16 @@
|
|
| 1 |
-
Rank,
|
| 2 |
-
1,
|
| 3 |
-
2,
|
| 4 |
-
3,
|
| 5 |
-
4,
|
| 6 |
-
5,
|
| 7 |
-
6,
|
| 8 |
-
7,
|
| 9 |
-
8,
|
| 10 |
-
9,
|
| 11 |
-
10,
|
| 12 |
-
11,
|
| 13 |
-
12,
|
| 14 |
-
13,
|
| 15 |
-
14,
|
| 16 |
-
15,
|
|
|
|
| 1 |
+
Rank,Method,Category,Subgroup,Elo,Elo CI low,Elo CI high,Avg. rank,Win rate (%),Improvability (%),Error vs. all features,Stability,Validity,Selection time (s),Fallback rate (%),#Datasets,#Comparisons
|
| 2 |
+
1,RFImportance,Non-search,Tree,1366.0,1325.0,1409.0,6.31,62.1,17.8,,0.653,0.741,0.2,0.0,71,1065
|
| 3 |
+
2,ElasticNet,Non-search,Regularization,1346.0,1303.0,1387.0,6.51,60.6,20.6,,0.519,0.873,7.6,0.0,71,1065
|
| 4 |
+
3,CART,Non-search,Tree,1335.0,1296.0,1377.0,6.63,59.8,20.9,,0.456,0.638,0.1,0.0,71,1065
|
| 5 |
+
4,Lasso,Non-search,Regularization,1323.0,1282.0,1367.0,6.78,58.7,20.2,,0.422,0.825,2.3,0.0,71,1065
|
| 6 |
+
5,MI,Non-search,Information theory,1316.0,1284.0,1351.0,7.01,57.1,21.2,,0.673,0.863,0.4,0.0,71,1065
|
| 7 |
+
6,MarkovBlanket,Search,Backward search,1307.0,1268.0,1344.0,7.15,56.0,22.8,,0.396,0.467,0.2,0.0,71,1065
|
| 8 |
+
7,F-Test,Non-search,Correlation,1298.0,1262.0,1336.0,7.39,54.3,23.2,,0.734,0.932,0.0,0.0,71,1065
|
| 9 |
+
8,GainRatio,Non-search,Information theory,1273.0,1238.0,1307.0,7.82,51.3,25.1,,0.681,0.915,0.2,0.0,71,1065
|
| 10 |
+
9,mRMR,Non-search,Information theory,1268.0,1227.0,1307.0,7.89,50.8,23.8,,0.578,0.712,0.4,0.0,71,1065
|
| 11 |
+
10,(R)ReliefF,Non-search,Distance,1257.0,1213.0,1296.0,8.18,48.7,24.4,,0.647,0.808,14.3,0.0,71,1065
|
| 12 |
+
11,SFS,Search,Forward search,1256.0,1216.0,1301.0,7.91,50.6,25.0,,0.278,0.664,1634.9,43.8,71,1065
|
| 13 |
+
12,SBE,Search,Backward search,1227.0,1180.0,1278.0,8.37,47.3,28.2,,0.175,0.578,3166.2,48.9,71,1065
|
| 14 |
+
13,LOCO,Search,Leave-one-out,1219.0,1178.0,1263.0,8.54,46.1,27.9,,0.22,0.613,237.3,0.0,71,1065
|
| 15 |
+
14,LaplacianScore,Non-search,Distance,1025.0,959.0,1079.0,11.85,22.5,42.2,,0.644,0.85,0.2,0.0,71,1065
|
| 16 |
+
15,Random,Baseline,Baseline,1000.0,953.0,1042.0,11.64,24.0,39.7,,0.001,0.499,0.0,100.0,71,1065
|
data/subsets/tasks_all/models_all/budgets_2/elo_explorer.html
CHANGED
|
@@ -19,67 +19,67 @@ svg { display:block; width:100%; height:auto; }
|
|
| 19 |
.sa-ink { stroke:var(--fg); }
|
| 20 |
.sa-hit { fill:transparent; }
|
| 21 |
.sa-hit:hover { fill:var(--fg); fill-opacity:0.06; }
|
| 22 |
-
</style><defs><pattern id='sa-na' width='6' height='6' patternUnits='userSpaceOnUse' patternTransform='rotate(45)'><rect width='6' height='6' fill='#e5e7eb'/><line x1='0' y1='0' x2='0' y2='6' stroke='#9ca3af' stroke-width='2'/></pattern><linearGradient id='sa-g-stab' x1='0' y1='1' x2='0' y2='0'><stop offset='0' stop-color='#ffffff'/><stop offset='1' stop-color='#0072B2'/></linearGradient><linearGradient id='sa-g-val' x1='0' y1='1' x2='0' y2='0'><stop offset='0' stop-color='#ffffff'/><stop offset='1' stop-color='#D55E00'/></linearGradient></defs><line class='sa-grid' x1='64' x2='1024' y1='287.6' y2='287.6' stroke-width='1'/><text class='sa-muted' x='56' y='291.6' text-anchor='end' font-size='12'>900</text><line class='sa-grid' x1='64' x2='1024' y1='233.7' y2='233.7' stroke-width='1'/><text class='sa-muted' x='56' y='237.7' text-anchor='end' font-size='12'>1000</text><line class='sa-grid' x1='64' x2='1024' y1='179.7' y2='179.7' stroke-width='1'/><text class='sa-muted' x='56' y='183.7' text-anchor='end' font-size='12'>1100</text><line class='sa-grid' x1='64' x2='1024' y1='125.8' y2='125.8' stroke-width='1'/><text class='sa-muted' x='56' y='129.8' text-anchor='end' font-size='12'>1200</text><line class='sa-grid' x1='64' x2='1024' y1='71.9' y2='71.9' stroke-width='1'/><text class='sa-muted' x='56' y='75.9' text-anchor='end' font-size='12'>1300</text><line class='sa-grid' x1='64' x2='1024' y1='18.0' y2='18.0' stroke-width='1'/><text class='sa-muted' x='56' y='22.0' text-anchor='end' font-size='12'>1400</text><text class='sa-text' transform='translate(16 173) rotate(-90)' text-anchor='middle' font-size='13'>Elo</text><line class='sa-ink' x1='64' x2='1024' y1='233.7' y2='233.7' stroke-width='1' stroke-dasharray='4 4' opacity='0.5'/><rect x='73.6' y='271.4' width='22.4' height='56.6' fill='#57a2cc' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='96.0' y='271.4' width='22.4' height='56.6' fill='#dc7a2c' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='96.0' x2='96.0' y1='241.2' y2='309.1'/><line x1='91.0' x2='101.0' y1='241.2' y2='241.2'/><line x1='91.0' x2='101.0' y1='309.1' y2='309.1'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(100.0 342.0) rotate(-35)'>LaplacianScore</text><rect class='sa-hit' x='64.0' y='18' width='64' height='414'><title>LaplacianScore (
|
| 23 |
Elo 930 [95% CI 860β986]
|
| 24 |
Stability 0.660
|
| 25 |
-
Validity 0.827</title></rect><rect x='137.6' y='233.7' width='22.4' height='94.3' fill='#ffffff' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='160.0' y='233.7' width='22.4' height='94.3' fill='#e9ac7c' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='160.0' x2='160.0' y1='211.5' y2='260.1'/><line x1='155.0' x2='165.0' y1='211.5' y2='211.5'/><line x1='155.0' x2='165.0' y1='260.1' y2='260.1'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(164.0 342.0) rotate(-35)'>Random</text><rect class='sa-hit' x='128.0' y='18' width='64' height='414'><title>Random (Baseline)
|
| 26 |
Elo 1000 [95% CI 951β1041]
|
| 27 |
Stability -0.001
|
| 28 |
-
Validity 0.513</title></rect><rect x='201.6' y='149.0' width='22.4' height='179.0' fill='#d8eaf3' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='224.0' y='149.0' width='22.4' height='179.0' fill='#e7a26c' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='224.0' x2='224.0' y1='125.8' y2='170.6'/><line x1='219.0' x2='229.0' y1='125.8' y2='125.8'/><line x1='219.0' x2='229.0' y1='170.6' y2='170.6'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(228.0 342.0) rotate(-35)'>LOCO</text><rect class='sa-hit' x='192.0' y='18' width='64' height='414'><title>LOCO (
|
| 29 |
Elo 1157 [95% CI 1117β1200]
|
| 30 |
Stability 0.151
|
| 31 |
-
Validity 0.575</title></rect><rect x='265.6' y='139.3' width='22.4' height='188.7' fill='#c2dded' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='288.0' y='139.3' width='22.4' height='188.7' fill='#e59a5f' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='288.0' x2='288.0' y1='118.3' y2='159.3'/><line x1='283.0' x2='293.0' y1='118.3' y2='118.3'/><line x1='283.0' x2='293.0' y1='159.3' y2='159.3'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(292.0 342.0) rotate(-35)'>SFS</text><rect class='sa-hit' x='256.0' y='18' width='64' height='414'><title>SFS (
|
| 32 |
Elo 1175 [95% CI 1138β1214]
|
| 33 |
Stability 0.239
|
| 34 |
-
Validity 0.627</title></rect><rect x='329.6' y='136.1' width='22.4' height='191.9' fill='#62a8d0' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='352.0' y='136.1' width='22.4' height='191.9' fill='#e39557' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='352.0' x2='352.0' y1='116.7' y2='156.0'/><line x1='347.0' x2='357.0' y1='116.7' y2='116.7'/><line x1='347.0' x2='357.0' y1='156.0' y2='156.0'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(356.0 342.0) rotate(-35)'>mRMR</text><rect class='sa-hit' x='320.0' y='18' width='64' height='414'><title>mRMR (
|
| 35 |
Elo 1181 [95% CI 1144β1217]
|
| 36 |
Stability 0.614
|
| 37 |
-
Validity 0.660</title></rect><rect x='393.6' y='133.4' width='22.4' height='194.6' fill='#daebf4' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='416.0' y='133.4' width='22.4' height='194.6' fill='#e8a571' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='416.0' x2='416.0' y1='108.6' y2='155.5'/><line x1='411.0' x2='421.0' y1='108.6' y2='108.6'/><line x1='411.0' x2='421.0' y1='155.5' y2='155.5'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(420.0 342.0) rotate(-35)'>SBE</text><rect class='sa-hit' x='384.0' y='18' width='64' height='414'><title>SBE (
|
| 38 |
Elo 1186 [95% CI 1145β1232]
|
| 39 |
Stability 0.144
|
| 40 |
-
Validity 0.556</title></rect><rect x='457.6' y='129.1' width='22.4' height='198.9' fill='#54a1cb' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='480.0' y='129.1' width='22.4' height='198.9' fill='#db7525' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='480.0' x2='480.0' y1='108.6' y2='147.9'/><line x1='475.0' x2='485.0' y1='108.6' y2='108.6'/><line x1='475.0' x2='485.0' y1='147.9' y2='147.9'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(484.0 342.0) rotate(-35)'>GainRatio</text><rect class='sa-hit' x='448.0' y='18' width='64' height='414'><title>GainRatio (
|
| 41 |
Elo 1194 [95% CI 1159β1232]
|
| 42 |
Stability 0.670
|
| 43 |
-
Validity 0.856</title></rect><rect x='521.6' y='120.4' width='22.4' height='207.6' fill='#53a0cb' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='544.0' y='120.4' width='22.4' height='207.6' fill='#df853e' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='544.0' x2='544.0' y1='101.0' y2='140.4'/><line x1='539.0' x2='549.0' y1='101.0' y2='101.0'/><line x1='539.0' x2='549.0' y1='140.4' y2='140.4'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(548.0 342.0) rotate(-35)'>(R)ReliefF</text><rect class='sa-hit' x='512.0' y='18' width='64' height='414'><title>(R)ReliefF (
|
| 44 |
Elo 1210 [95% CI 1173β1246]
|
| 45 |
Stability 0.673
|
| 46 |
-
Validity 0.757</title></rect><rect x='585.6' y='113.4' width='22.4' height='214.6' fill='#4b9bc9' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='608.0' y='113.4' width='22.4' height='214.6' fill='#da711f' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='608.0' x2='608.0' y1='95.6' y2='132.3'/><line x1='603.0' x2='613.0' y1='95.6' y2='95.6'/><line x1='603.0' x2='613.0' y1='132.3' y2='132.3'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(612.0 342.0) rotate(-35)'>F-Test</text><rect class='sa-hit' x='576.0' y='18' width='64' height='414'><title>F-Test (
|
| 47 |
Elo 1223 [95% CI 1188β1256]
|
| 48 |
Stability 0.706
|
| 49 |
-
Validity 0.879</title></rect><rect x='649.6' y='105.9' width='22.4' height='222.1' fill='#96c5df' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='672.0' y='105.9' width='22.4' height='222.1' fill='#ecb78d' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='672.0' x2='672.0' y1='88.1' y2='125.8'/><line x1='667.0' x2='677.0' y1='88.1' y2='88.1'/><line x1='667.0' x2='677.0' y1='125.8' y2='125.8'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(676.0 342.0) rotate(-35)'>MarkovBlanket</text><rect class='sa-hit' x='640.0' y='18' width='64' height='414'><title>MarkovBlanket (
|
| 50 |
Elo 1237 [95% CI 1200β1270]
|
| 51 |
Stability 0.413
|
| 52 |
-
Validity 0.447</title></rect><rect x='713.6' y='99.4' width='22.4' height='228.6' fill='#9cc8e1' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='736.0' y='99.4' width='22.4' height='228.6' fill='#e08945' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='736.0' x2='736.0' y1='77.3' y2='120.4'/><line x1='731.0' x2='741.0' y1='77.3' y2='77.3'/><line x1='731.0' x2='741.0' y1='120.4' y2='120.4'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(740.0 342.0) rotate(-35)'>Lasso</text><rect class='sa-hit' x='704.0' y='18' width='64' height='414'><title>Lasso (
|
| 53 |
Elo 1249 [95% CI 1210β1290]
|
| 54 |
Stability 0.389
|
| 55 |
-
Validity 0.730</title></rect><rect x='777.6' y='87.0' width='22.4' height='241.0' fill='#519fca' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='800.0' y='87.0' width='22.4' height='241.0' fill='#dd7c2f' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='800.0' x2='800.0' y1='64.4' y2='106.4'/><line x1='795.0' x2='805.0' y1='64.4' y2='64.4'/><line x1='795.0' x2='805.0' y1='106.4' y2='106.4'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(804.0 342.0) rotate(-35)'>MI</text><rect class='sa-hit' x='768.0' y='18' width='64' height='414'><title>MI (
|
| 56 |
Elo 1272 [95% CI 1236β1314]
|
| 57 |
Stability 0.683
|
| 58 |
-
Validity 0.815</title></rect><rect x='841.6' y='86.5' width='22.4' height='241.5' fill='#84bbda' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='864.0' y='86.5' width='22.4' height='241.5' fill='#de8239' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='864.0' x2='864.0' y1='64.9' y2='107.5'/><line x1='859.0' x2='869.0' y1='64.9' y2='64.9'/><line x1='859.0' x2='869.0' y1='107.5' y2='107.5'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(868.0 342.0) rotate(-35)'>ElasticNet</text><rect class='sa-hit' x='832.0' y='18' width='64' height='414'><title>ElasticNet (
|
| 59 |
Elo 1273 [95% CI 1234β1313]
|
| 60 |
Stability 0.481
|
| 61 |
-
Validity 0.777</title></rect><rect x='905.6' y='83.8' width='22.4' height='244.2' fill='#8ec0dd' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='928.0' y='83.8' width='22.4' height='244.2' fill='#e7a36d' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='928.0' x2='928.0' y1='63.3' y2='105.3'/><line x1='923.0' x2='933.0' y1='63.3' y2='63.3'/><line x1='923.0' x2='933.0' y1='105.3' y2='105.3'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(932.0 342.0) rotate(-35)'>CART</text><rect class='sa-hit' x='896.0' y='18' width='64' height='414'><title>CART (
|
| 62 |
Elo 1278 [95% CI 1238β1316]
|
| 63 |
Stability 0.444
|
| 64 |
-
Validity 0.572</title></rect><rect x='969.6' y='58.4' width='22.4' height='269.6' fill='#53a0cb' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='992.0' y='58.4' width='22.4' height='269.6' fill='#e2904f' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='992.0' x2='992.0' y1='34.2' y2='80.5'/><line x1='987.0' x2='997.0' y1='34.2' y2='34.2'/><line x1='987.0' x2='997.0' y1='80.5' y2='80.5'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(996.0 342.0) rotate(-35)'>RFImportance</text><rect class='sa-hit' x='960.0' y='18' width='64' height='414'><title>RFImportance (
|
| 65 |
Elo 1325 [95% CI 1284β1370]
|
| 66 |
Stability 0.676
|
| 67 |
-
Validity 0.691</title></rect><line class='sa-ink' x1='64' x2='1024' y1='328.0' y2='328.0' stroke-width='1' opacity='0.4'/><rect x='1070' y='40.0' width='12' height='107.0' fill='url(#sa-g-stab)' stroke='#000' stroke-width='0.6'/><text class='sa-text' x='1076' y='30.0' text-anchor='middle' font-size='12'>stable</text><text class='sa-text' x='1076' y='165.0' text-anchor='middle' font-size='12'>unstable</text><text class='sa-muted' x='1088' y='151.0' font-size='11'>0.0</text><text class='sa-muted' x='1088' y='97.5' font-size='11'>0.5</text><text class='sa-muted' x='1088' y='44.0' font-size='11'>1.0</text><rect x='1070' y='207.0' width='12' height='107.0' fill='url(#sa-g-val)' stroke='#000' stroke-width='0.6'/><text class='sa-text' x='1076' y='197.0' text-anchor='middle' font-size='12'>valid</text><text class='sa-text' x='1076' y='332.0' text-anchor='middle' font-size='12'>invalid</text><text class='sa-muted' x='1088' y='318.0' font-size='11'>0.0</text><text class='sa-muted' x='1088' y='264.5' font-size='11'>0.5</text><text class='sa-muted' x='1088' y='211.0' font-size='11'>1.0</text><rect x='1070' y='352.0' width='12' height='12' fill='url(#sa-na)' stroke='#000' stroke-width='0.6'/><text class='sa-muted' x='1088' y='362.0' font-size='11'>no result</text></svg><script type='text/plain' id='sa-csv'>Method,
|
| 68 |
-
RFImportance,
|
| 69 |
-
CART,
|
| 70 |
-
ElasticNet,
|
| 71 |
-
MI,
|
| 72 |
-
Lasso,
|
| 73 |
-
MarkovBlanket,
|
| 74 |
-
F-Test,
|
| 75 |
-
(R)ReliefF,
|
| 76 |
-
GainRatio,
|
| 77 |
-
SBE,
|
| 78 |
-
mRMR,
|
| 79 |
-
SFS,
|
| 80 |
-
LOCO,
|
| 81 |
-
Random,Baseline,1000.0,951.0,1041.0,-0.001,0.513
|
| 82 |
-
LaplacianScore,
|
| 83 |
</script><script>
|
| 84 |
function saDownload(name, blob) {
|
| 85 |
var a = document.createElement('a');
|
|
|
|
| 19 |
.sa-ink { stroke:var(--fg); }
|
| 20 |
.sa-hit { fill:transparent; }
|
| 21 |
.sa-hit:hover { fill:var(--fg); fill-opacity:0.06; }
|
| 22 |
+
</style><defs><pattern id='sa-na' width='6' height='6' patternUnits='userSpaceOnUse' patternTransform='rotate(45)'><rect width='6' height='6' fill='#e5e7eb'/><line x1='0' y1='0' x2='0' y2='6' stroke='#9ca3af' stroke-width='2'/></pattern><linearGradient id='sa-g-stab' x1='0' y1='1' x2='0' y2='0'><stop offset='0' stop-color='#ffffff'/><stop offset='1' stop-color='#0072B2'/></linearGradient><linearGradient id='sa-g-val' x1='0' y1='1' x2='0' y2='0'><stop offset='0' stop-color='#ffffff'/><stop offset='1' stop-color='#D55E00'/></linearGradient></defs><line class='sa-grid' x1='64' x2='1024' y1='287.6' y2='287.6' stroke-width='1'/><text class='sa-muted' x='56' y='291.6' text-anchor='end' font-size='12'>900</text><line class='sa-grid' x1='64' x2='1024' y1='233.7' y2='233.7' stroke-width='1'/><text class='sa-muted' x='56' y='237.7' text-anchor='end' font-size='12'>1000</text><line class='sa-grid' x1='64' x2='1024' y1='179.7' y2='179.7' stroke-width='1'/><text class='sa-muted' x='56' y='183.7' text-anchor='end' font-size='12'>1100</text><line class='sa-grid' x1='64' x2='1024' y1='125.8' y2='125.8' stroke-width='1'/><text class='sa-muted' x='56' y='129.8' text-anchor='end' font-size='12'>1200</text><line class='sa-grid' x1='64' x2='1024' y1='71.9' y2='71.9' stroke-width='1'/><text class='sa-muted' x='56' y='75.9' text-anchor='end' font-size='12'>1300</text><line class='sa-grid' x1='64' x2='1024' y1='18.0' y2='18.0' stroke-width='1'/><text class='sa-muted' x='56' y='22.0' text-anchor='end' font-size='12'>1400</text><text class='sa-text' transform='translate(16 173) rotate(-90)' text-anchor='middle' font-size='13'>Elo</text><line class='sa-ink' x1='64' x2='1024' y1='233.7' y2='233.7' stroke-width='1' stroke-dasharray='4 4' opacity='0.5'/><rect x='73.6' y='271.4' width='22.4' height='56.6' fill='#57a2cc' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='96.0' y='271.4' width='22.4' height='56.6' fill='#dc7a2c' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='96.0' x2='96.0' y1='241.2' y2='309.1'/><line x1='91.0' x2='101.0' y1='241.2' y2='241.2'/><line x1='91.0' x2='101.0' y1='309.1' y2='309.1'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(100.0 342.0) rotate(-35)'>LaplacianScore</text><rect class='sa-hit' x='64.0' y='18' width='64' height='414'><title>LaplacianScore (Distance, non-search)
|
| 23 |
Elo 930 [95% CI 860β986]
|
| 24 |
Stability 0.660
|
| 25 |
+
Validity 0.827</title></rect><rect x='137.6' y='233.7' width='22.4' height='94.3' fill='#ffffff' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='160.0' y='233.7' width='22.4' height='94.3' fill='#e9ac7c' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='160.0' x2='160.0' y1='211.5' y2='260.1'/><line x1='155.0' x2='165.0' y1='211.5' y2='211.5'/><line x1='155.0' x2='165.0' y1='260.1' y2='260.1'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(164.0 342.0) rotate(-35)'>Random</text><rect class='sa-hit' x='128.0' y='18' width='64' height='414'><title>Random (Baseline, baseline)
|
| 26 |
Elo 1000 [95% CI 951β1041]
|
| 27 |
Stability -0.001
|
| 28 |
+
Validity 0.513</title></rect><rect x='201.6' y='149.0' width='22.4' height='179.0' fill='#d8eaf3' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='224.0' y='149.0' width='22.4' height='179.0' fill='#e7a26c' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='224.0' x2='224.0' y1='125.8' y2='170.6'/><line x1='219.0' x2='229.0' y1='125.8' y2='125.8'/><line x1='219.0' x2='229.0' y1='170.6' y2='170.6'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(228.0 342.0) rotate(-35)'>LOCO</text><rect class='sa-hit' x='192.0' y='18' width='64' height='414'><title>LOCO (Leave-one-out, search)
|
| 29 |
Elo 1157 [95% CI 1117β1200]
|
| 30 |
Stability 0.151
|
| 31 |
+
Validity 0.575</title></rect><rect x='265.6' y='139.3' width='22.4' height='188.7' fill='#c2dded' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='288.0' y='139.3' width='22.4' height='188.7' fill='#e59a5f' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='288.0' x2='288.0' y1='118.3' y2='159.3'/><line x1='283.0' x2='293.0' y1='118.3' y2='118.3'/><line x1='283.0' x2='293.0' y1='159.3' y2='159.3'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(292.0 342.0) rotate(-35)'>SFS</text><rect class='sa-hit' x='256.0' y='18' width='64' height='414'><title>SFS (Forward search, search)
|
| 32 |
Elo 1175 [95% CI 1138β1214]
|
| 33 |
Stability 0.239
|
| 34 |
+
Validity 0.627</title></rect><rect x='329.6' y='136.1' width='22.4' height='191.9' fill='#62a8d0' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='352.0' y='136.1' width='22.4' height='191.9' fill='#e39557' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='352.0' x2='352.0' y1='116.7' y2='156.0'/><line x1='347.0' x2='357.0' y1='116.7' y2='116.7'/><line x1='347.0' x2='357.0' y1='156.0' y2='156.0'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(356.0 342.0) rotate(-35)'>mRMR</text><rect class='sa-hit' x='320.0' y='18' width='64' height='414'><title>mRMR (Information theory, non-search)
|
| 35 |
Elo 1181 [95% CI 1144β1217]
|
| 36 |
Stability 0.614
|
| 37 |
+
Validity 0.660</title></rect><rect x='393.6' y='133.4' width='22.4' height='194.6' fill='#daebf4' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='416.0' y='133.4' width='22.4' height='194.6' fill='#e8a571' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='416.0' x2='416.0' y1='108.6' y2='155.5'/><line x1='411.0' x2='421.0' y1='108.6' y2='108.6'/><line x1='411.0' x2='421.0' y1='155.5' y2='155.5'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(420.0 342.0) rotate(-35)'>SBE</text><rect class='sa-hit' x='384.0' y='18' width='64' height='414'><title>SBE (Backward search, search)
|
| 38 |
Elo 1186 [95% CI 1145β1232]
|
| 39 |
Stability 0.144
|
| 40 |
+
Validity 0.556</title></rect><rect x='457.6' y='129.1' width='22.4' height='198.9' fill='#54a1cb' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='480.0' y='129.1' width='22.4' height='198.9' fill='#db7525' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='480.0' x2='480.0' y1='108.6' y2='147.9'/><line x1='475.0' x2='485.0' y1='108.6' y2='108.6'/><line x1='475.0' x2='485.0' y1='147.9' y2='147.9'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(484.0 342.0) rotate(-35)'>GainRatio</text><rect class='sa-hit' x='448.0' y='18' width='64' height='414'><title>GainRatio (Information theory, non-search)
|
| 41 |
Elo 1194 [95% CI 1159β1232]
|
| 42 |
Stability 0.670
|
| 43 |
+
Validity 0.856</title></rect><rect x='521.6' y='120.4' width='22.4' height='207.6' fill='#53a0cb' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='544.0' y='120.4' width='22.4' height='207.6' fill='#df853e' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='544.0' x2='544.0' y1='101.0' y2='140.4'/><line x1='539.0' x2='549.0' y1='101.0' y2='101.0'/><line x1='539.0' x2='549.0' y1='140.4' y2='140.4'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(548.0 342.0) rotate(-35)'>(R)ReliefF</text><rect class='sa-hit' x='512.0' y='18' width='64' height='414'><title>(R)ReliefF (Distance, non-search)
|
| 44 |
Elo 1210 [95% CI 1173β1246]
|
| 45 |
Stability 0.673
|
| 46 |
+
Validity 0.757</title></rect><rect x='585.6' y='113.4' width='22.4' height='214.6' fill='#4b9bc9' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='608.0' y='113.4' width='22.4' height='214.6' fill='#da711f' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='608.0' x2='608.0' y1='95.6' y2='132.3'/><line x1='603.0' x2='613.0' y1='95.6' y2='95.6'/><line x1='603.0' x2='613.0' y1='132.3' y2='132.3'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(612.0 342.0) rotate(-35)'>F-Test</text><rect class='sa-hit' x='576.0' y='18' width='64' height='414'><title>F-Test (Correlation, non-search)
|
| 47 |
Elo 1223 [95% CI 1188β1256]
|
| 48 |
Stability 0.706
|
| 49 |
+
Validity 0.879</title></rect><rect x='649.6' y='105.9' width='22.4' height='222.1' fill='#96c5df' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='672.0' y='105.9' width='22.4' height='222.1' fill='#ecb78d' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='672.0' x2='672.0' y1='88.1' y2='125.8'/><line x1='667.0' x2='677.0' y1='88.1' y2='88.1'/><line x1='667.0' x2='677.0' y1='125.8' y2='125.8'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(676.0 342.0) rotate(-35)'>MarkovBlanket</text><rect class='sa-hit' x='640.0' y='18' width='64' height='414'><title>MarkovBlanket (Backward search, search)
|
| 50 |
Elo 1237 [95% CI 1200β1270]
|
| 51 |
Stability 0.413
|
| 52 |
+
Validity 0.447</title></rect><rect x='713.6' y='99.4' width='22.4' height='228.6' fill='#9cc8e1' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='736.0' y='99.4' width='22.4' height='228.6' fill='#e08945' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='736.0' x2='736.0' y1='77.3' y2='120.4'/><line x1='731.0' x2='741.0' y1='77.3' y2='77.3'/><line x1='731.0' x2='741.0' y1='120.4' y2='120.4'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(740.0 342.0) rotate(-35)'>Lasso</text><rect class='sa-hit' x='704.0' y='18' width='64' height='414'><title>Lasso (Regularization, non-search)
|
| 53 |
Elo 1249 [95% CI 1210β1290]
|
| 54 |
Stability 0.389
|
| 55 |
+
Validity 0.730</title></rect><rect x='777.6' y='87.0' width='22.4' height='241.0' fill='#519fca' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='800.0' y='87.0' width='22.4' height='241.0' fill='#dd7c2f' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='800.0' x2='800.0' y1='64.4' y2='106.4'/><line x1='795.0' x2='805.0' y1='64.4' y2='64.4'/><line x1='795.0' x2='805.0' y1='106.4' y2='106.4'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(804.0 342.0) rotate(-35)'>MI</text><rect class='sa-hit' x='768.0' y='18' width='64' height='414'><title>MI (Information theory, non-search)
|
| 56 |
Elo 1272 [95% CI 1236β1314]
|
| 57 |
Stability 0.683
|
| 58 |
+
Validity 0.815</title></rect><rect x='841.6' y='86.5' width='22.4' height='241.5' fill='#84bbda' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='864.0' y='86.5' width='22.4' height='241.5' fill='#de8239' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='864.0' x2='864.0' y1='64.9' y2='107.5'/><line x1='859.0' x2='869.0' y1='64.9' y2='64.9'/><line x1='859.0' x2='869.0' y1='107.5' y2='107.5'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(868.0 342.0) rotate(-35)'>ElasticNet</text><rect class='sa-hit' x='832.0' y='18' width='64' height='414'><title>ElasticNet (Regularization, non-search)
|
| 59 |
Elo 1273 [95% CI 1234β1313]
|
| 60 |
Stability 0.481
|
| 61 |
+
Validity 0.777</title></rect><rect x='905.6' y='83.8' width='22.4' height='244.2' fill='#8ec0dd' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='928.0' y='83.8' width='22.4' height='244.2' fill='#e7a36d' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='928.0' x2='928.0' y1='63.3' y2='105.3'/><line x1='923.0' x2='933.0' y1='63.3' y2='63.3'/><line x1='923.0' x2='933.0' y1='105.3' y2='105.3'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(932.0 342.0) rotate(-35)'>CART</text><rect class='sa-hit' x='896.0' y='18' width='64' height='414'><title>CART (Tree, non-search)
|
| 62 |
Elo 1278 [95% CI 1238β1316]
|
| 63 |
Stability 0.444
|
| 64 |
+
Validity 0.572</title></rect><rect x='969.6' y='58.4' width='22.4' height='269.6' fill='#53a0cb' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='992.0' y='58.4' width='22.4' height='269.6' fill='#e2904f' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='992.0' x2='992.0' y1='34.2' y2='80.5'/><line x1='987.0' x2='997.0' y1='34.2' y2='34.2'/><line x1='987.0' x2='997.0' y1='80.5' y2='80.5'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(996.0 342.0) rotate(-35)'>RFImportance</text><rect class='sa-hit' x='960.0' y='18' width='64' height='414'><title>RFImportance (Tree, non-search)
|
| 65 |
Elo 1325 [95% CI 1284β1370]
|
| 66 |
Stability 0.676
|
| 67 |
+
Validity 0.691</title></rect><line class='sa-ink' x1='64' x2='1024' y1='328.0' y2='328.0' stroke-width='1' opacity='0.4'/><rect x='1070' y='40.0' width='12' height='107.0' fill='url(#sa-g-stab)' stroke='#000' stroke-width='0.6'/><text class='sa-text' x='1076' y='30.0' text-anchor='middle' font-size='12'>stable</text><text class='sa-text' x='1076' y='165.0' text-anchor='middle' font-size='12'>unstable</text><text class='sa-muted' x='1088' y='151.0' font-size='11'>0.0</text><text class='sa-muted' x='1088' y='97.5' font-size='11'>0.5</text><text class='sa-muted' x='1088' y='44.0' font-size='11'>1.0</text><rect x='1070' y='207.0' width='12' height='107.0' fill='url(#sa-g-val)' stroke='#000' stroke-width='0.6'/><text class='sa-text' x='1076' y='197.0' text-anchor='middle' font-size='12'>valid</text><text class='sa-text' x='1076' y='332.0' text-anchor='middle' font-size='12'>invalid</text><text class='sa-muted' x='1088' y='318.0' font-size='11'>0.0</text><text class='sa-muted' x='1088' y='264.5' font-size='11'>0.5</text><text class='sa-muted' x='1088' y='211.0' font-size='11'>1.0</text><rect x='1070' y='352.0' width='12' height='12' fill='url(#sa-na)' stroke='#000' stroke-width='0.6'/><text class='sa-muted' x='1088' y='362.0' font-size='11'>no result</text></svg><script type='text/plain' id='sa-csv'>Method,Category,Subgroup,Elo,Elo CI low,Elo CI high,Stability,Validity
|
| 68 |
+
RFImportance,Non-search,Tree,1325.0,1284.0,1370.0,0.676,0.691
|
| 69 |
+
CART,Non-search,Tree,1278.0,1238.0,1316.0,0.444,0.572
|
| 70 |
+
ElasticNet,Non-search,Regularization,1273.0,1234.0,1313.0,0.481,0.777
|
| 71 |
+
MI,Non-search,Information theory,1272.0,1236.0,1314.0,0.683,0.815
|
| 72 |
+
Lasso,Non-search,Regularization,1249.0,1210.0,1290.0,0.389,0.73
|
| 73 |
+
MarkovBlanket,Search,Backward search,1237.0,1200.0,1270.0,0.413,0.447
|
| 74 |
+
F-Test,Non-search,Correlation,1223.0,1188.0,1256.0,0.706,0.879
|
| 75 |
+
(R)ReliefF,Non-search,Distance,1210.0,1173.0,1246.0,0.673,0.757
|
| 76 |
+
GainRatio,Non-search,Information theory,1194.0,1159.0,1232.0,0.67,0.856
|
| 77 |
+
SBE,Search,Backward search,1186.0,1145.0,1232.0,0.144,0.556
|
| 78 |
+
mRMR,Non-search,Information theory,1181.0,1144.0,1217.0,0.614,0.66
|
| 79 |
+
SFS,Search,Forward search,1175.0,1138.0,1214.0,0.239,0.627
|
| 80 |
+
LOCO,Search,Leave-one-out,1157.0,1117.0,1200.0,0.151,0.575
|
| 81 |
+
Random,Baseline,Baseline,1000.0,951.0,1041.0,-0.001,0.513
|
| 82 |
+
LaplacianScore,Non-search,Distance,930.0,860.0,986.0,0.66,0.827
|
| 83 |
</script><script>
|
| 84 |
function saDownload(name, blob) {
|
| 85 |
var a = document.createElement('a');
|
data/subsets/tasks_all/models_all/budgets_2/leaderboard_table.html
CHANGED
|
@@ -26,23 +26,25 @@ td.method { font-weight:600; }
|
|
| 26 |
td .sub { color:var(--muted); font-size:12px; font-weight:400; margin-left:6px; }
|
| 27 |
td.na { color:var(--muted); }
|
| 28 |
.pill { display:inline-block; padding:0 8px; border:1px solid var(--line); border-radius:999px;
|
| 29 |
-
font-size:12px; color:var(--
|
| 30 |
-
</style></head><body><div class='toolbar'><span class='hint'>Click a column to sort. Hover a header for its definition.</span><button onclick="saCsv('selectarena_leaderboard.csv')">CSV</button></div><div class='wrap'><table id='sa-table'><thead><tr><th class='num'>#</th><th class=''>Method</th><th class='' title='Filter: scores features from the data alone. Wrapper: searches feature subsets by refitting a proxy model. Embedded: reads the selection off a fitted model.'>Type</th><th class='num' title='Bradley-Terry Elo from pairwise matches within each dataset, budget and downstream model. Random = 1000. +/β give the 95% bootstrap interval.'>Elo β</th><th class='num' title='Mean rank of the downstream error.'>Avg. rank β</th><th class='num' title='Share of pairwise comparisons won, in %. Ties count half.'>Win rate β</th><th class='num' title='Mean relative gap (error β best error) / error to the best method, in %.'>Improvability β</th><th class='num' title='Mean ratio of the error to the same model trained on all features. Below 1: the selection helped.'>Error vs. all β</th><th class='num' title='Agreement of the selected features across bootstrap resamples (Nogueira et al., 2018).'>Stability β</th><th class='num' title='Share of the budget filled with original features when as many noise features are added.'>Validity β</th><th class='num' title='Median wall-clock time of the selection.'>Time (s) β</th><th class='num' title='Share of runs where the method ran out of time and random features filled the budget.'>Fallback (%)</th><th class='num'>#Datasets</th></tr></thead><tbody><tr><td class='num' data-sort='1'>1</td><td class='method' data-sort='RFImportance'>RFImportance</td><td class='' data-sort='Embedded'><span class='pill'>π³ Embedded</span></td><td class='num' data-sort='1325.0'>1325<span class='sub'>+45 / β41</span></td><td class='num' data-sort='6.04'>6.04</td><td class='num' data-sort='64.0'>64.0</td><td class='num' data-sort='14.6'>14.6</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.676'>0.676</td><td class='num' data-sort='0.691'>0.691</td><td class='num' data-sort='0.2'>0.2</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='2'>2</td><td class='method' data-sort='CART'>CART</td><td class='' data-sort='Embedded'><span class='pill'>π³ Embedded</span></td><td class='num' data-sort='1278.0'>1278<span class='sub'>+38 / β40</span></td><td class='num' data-sort='6.67'>6.67</td><td class='num' data-sort='59.5'>59.5</td><td class='num' data-sort='18.5'>18.5</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.444'>0.444</td><td class='num' data-sort='0.572'>0.572</td><td class='num' data-sort='0.1'>0.1</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='3'>3</td><td class='method' data-sort='ElasticNet'>ElasticNet</td><td class='' data-sort='Embedded'><span class='pill'>π³ Embedded</span></td><td class='num' data-sort='1273.0'>1273<span class='sub'>+40 / β39</span></td><td class='num' data-sort='6.64'>6.64</td><td class='num' data-sort='59.7'>59.7</td><td class='num' data-sort='18.1'>18.1</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.481'>0.481</td><td class='num' data-sort='0.777'>0.777</td><td class='num' data-sort='7.6'>7.6</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='4'>4</td><td class='method' data-sort='MI'>MI</td><td class='' data-sort='Filter'><span class='pill'>π Filter</span></td><td class='num' data-sort='1272.0'>1272<span class='sub'>+42 / β36</span></td><td class='num' data-sort='6.82'>6.82</td><td class='num' data-sort='58.4'>58.4</td><td class='num' data-sort='18.3'>18.3</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.683'>0.683</td><td class='num' data-sort='0.815'>0.815</td><td class='num' data-sort='0.4'>0.4</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='5'>5</td><td class='method' data-sort='Lasso'>Lasso</td><td class='' data-sort='Embedded'><span class='pill'>π³ Embedded</span></td><td class='num' data-sort='1249.0'>1249<span class='sub'>+41 / β39</span></td><td class='num' data-sort='6.97'>6.97</td><td class='num' data-sort='57.4'>57.4</td><td class='num' data-sort='18.3'>18.3</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.389'>0.389</td><td class='num' data-sort='0.73'>0.730</td><td class='num' data-sort='2.0'>2.0</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='6'>6</td><td class='method' data-sort='MarkovBlanket'>MarkovBlanket</td><td class='' data-sort='Filter'><span class='pill'>π Filter</span></td><td class='num' data-sort='1237.0'>1237<span class='sub'>+33 / β37</span></td><td class='num' data-sort='7.5'>7.50</td><td class='num' data-sort='53.6'>53.6</td><td class='num' data-sort='21.0'>21.0</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.413'>0.413</td><td class='num' data-sort='0.447'>0.447</td><td class='num' data-sort='0.1'>0.1</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='7'>7</td><td class='method' data-sort='F-Test'>F-Test</td><td class='' data-sort='Filter'><span class='pill'>π Filter</span></td><td class='num' data-sort='1223.0'>1223<span class='sub'>+33 / β35</span></td><td class='num' data-sort='7.49'>7.49</td><td class='num' data-sort='53.6'>53.6</td><td class='num' data-sort='20.5'>20.5</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.706'>0.706</td><td class='num' data-sort='0.879'>0.879</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='8'>8</td><td class='method' data-sort='(R)ReliefF'>(R)ReliefF</td><td class='' data-sort='Filter'><span class='pill'>π Filter</span></td><td class='num' data-sort='1210.0'>1210<span class='sub'>+36 / β37</span></td><td class='num' data-sort='7.86'>7.86</td><td class='num' data-sort='51.0'>51.0</td><td class='num' data-sort='20.2'>20.2</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.673'>0.673</td><td class='num' data-sort='0.757'>0.757</td><td class='num' data-sort='14.4'>14.4</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='9'>9</td><td class='method' data-sort='GainRatio'>GainRatio</td><td class='' data-sort='Filter'><span class='pill'>π Filter</span></td><td class='num' data-sort='1194.0'>1194<span class='sub'>+38 / β35</span></td><td class='num' data-sort='8.22'>8.22</td><td class='num' data-sort='48.4'>48.4</td><td class='num' data-sort='22.8'>22.8</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.67'>0.670</td><td class='num' data-sort='0.856'>0.856</td><td class='num' data-sort='0.2'>0.2</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='10'>10</td><td class='method' data-sort='SBE'>SBE</td><td class='' data-sort='Wrapper'><span class='pill'>π Wrapper</span></td><td class='num' data-sort='1186.0'>1186<span class='sub'>+46 / β41</span></td><td class='num' data-sort='8.15'>8.15</td><td class='num' data-sort='48.9'>48.9</td><td class='num' data-sort='24.7'>24.7</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.144'>0.144</td><td class='num' data-sort='0.556'>0.556</td><td class='num' data-sort='2621.7'>2,621.7</td><td class='num' data-sort='47.8'>47.8</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='11'>11</td><td class='method' data-sort='mRMR'>mRMR</td><td class='' data-sort='Filter'><span class='pill'>π Filter</span></td><td class='num' data-sort='1181.0'>1181<span class='sub'>+36 / β37</span></td><td class='num' data-sort='8.22'>8.22</td><td class='num' data-sort='48.4'>48.4</td><td class='num' data-sort='21.5'>21.5</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.614'>0.614</td><td class='num' data-sort='0.66'>0.660</td><td class='num' data-sort='0.5'>0.5</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='12'>12</td><td class='method' data-sort='SFS'>SFS</td><td class='' data-sort='Wrapper'><span class='pill'>π Wrapper</span></td><td class='num' data-sort='1175.0'>1175<span class='sub'>+39 / β37</span></td><td class='num' data-sort='8.11'>8.11</td><td class='num' data-sort='49.2'>49.2</td><td class='num' data-sort='23.1'>23.1</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.239'>0.239</td><td class='num' data-sort='0.627'>0.627</td><td class='num' data-sort='2398.9'>2,398.9</td><td class='num' data-sort='47.2'>47.2</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='13'>13</td><td class='method' data-sort='LOCO'>LOCO</td><td class='' data-sort='Wrapper'><span class='pill'>π Wrapper</span></td><td class='num' data-sort='1157.0'>1157<span class='sub'>+43 / β40</span></td><td class='num' data-sort='8.42'>8.42</td><td class='num' data-sort='47.0'>47.0</td><td class='num' data-sort='25.1'>25.1</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.151'>0.151</td><td class='num' data-sort='0.575'>0.575</td><td class='num' data-sort='241.3'>241.3</td><td class='num' data-sort='6.8'>6.8</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='14'>14</td><td class='method' data-sort='Random'>Random</td><td class='' data-sort='Baseline'><span class='pill'>βͺ Baseline</span></td><td class='num' data-sort='1000.0'>1000<span class='sub'>+41 / β49</span></td><td class='num' data-sort='10.82'>10.82</td><td class='num' data-sort='29.9'>29.9</td><td class='num' data-sort='33.5'>33.5</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='-0.001'>-0.001</td><td class='num' data-sort='0.513'>0.513</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='100.0'>100.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='15'>15</td><td class='method' data-sort='LaplacianScore'>LaplacianScore</td><td class='' data-sort='Filter'><span class='pill'>π Filter</span></td><td class='num' data-sort='930.0'>930<span class='sub'>+56 / β70</span></td><td class='num' data-sort='12.08'>12.08</td><td class='num' data-sort='20.9'>20.9</td><td class='num' data-sort='39.5'>39.5</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.66'>0.660</td><td class='num' data-sort='0.827'>0.827</td><td class='num' data-sort='0.2'>0.2</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr></tbody></table></div><script type='text/plain' id='sa-csv'>Rank,Type,TypeName,Method,Elo,Elo CI low,Elo CI high,Avg. rank,Win rate (%),Improvability (%),Error vs. all features,Stability,Validity,Selection time (s),Fallback rate (%),#Datasets,#Comparisons
|
| 31 |
-
|
| 32 |
-
2,π³,Embedded,CART,1278.0,1238.0,1316.0,6.67,59.5,18.5,,0.444,0.572,0.1,0.0,71,1065
|
| 33 |
-
|
| 34 |
-
|
| 35 |
-
|
| 36 |
-
|
| 37 |
-
|
| 38 |
-
|
| 39 |
-
|
| 40 |
-
|
| 41 |
-
|
| 42 |
-
|
| 43 |
-
|
| 44 |
-
|
| 45 |
-
|
|
|
|
|
|
|
| 46 |
</script><script>
|
| 47 |
function saDownload(name, blob) {
|
| 48 |
var a = document.createElement('a');
|
|
|
|
| 26 |
td .sub { color:var(--muted); font-size:12px; font-weight:400; margin-left:6px; }
|
| 27 |
td.na { color:var(--muted); }
|
| 28 |
.pill { display:inline-block; padding:0 8px; border:1px solid var(--line); border-radius:999px;
|
| 29 |
+
font-size:12px; color:var(--fg); }
|
| 30 |
+
.pill.cat-search { border-color:var(--accent); }
|
| 31 |
+
.pill.cat-baseline { color:var(--muted); }
|
| 32 |
+
</style></head><body><div class='toolbar'><span class='hint'>Click a column to sort. Hover a header for its definition.</span><button onclick="saCsv('selectarena_leaderboard.csv')">CSV</button></div><div class='wrap'><table id='sa-table'><thead><tr><th class='num'>#</th><th class=''>Method</th><th class='' title='Taxonomy of the paper (Appendix A). Search methods evaluate many feature subsets with the model (forward, backward, leave-one-out); non-search methods use a proxy instead (correlation, distance, information theory, tree, regularization).'>Type</th><th class='num' title='Bradley-Terry Elo from pairwise matches within each dataset, budget and downstream model. Random = 1000. +/β give the 95% bootstrap interval.'>Elo β</th><th class='num' title='Mean rank of the downstream error.'>Avg. rank β</th><th class='num' title='Share of pairwise comparisons won, in %. Ties count half.'>Win rate β</th><th class='num' title='Mean relative gap (error β best error) / error to the best method, in %.'>Improvability β</th><th class='num' title='Mean ratio of the error to the same model trained on all features. Below 1: the selection helped.'>Error vs. all β</th><th class='num' title='Agreement of the selected features across bootstrap resamples (Nogueira et al., 2018).'>Stability β</th><th class='num' title='Share of the budget filled with original features when as many noise features are added.'>Validity β</th><th class='num' title='Median wall-clock time of the selection.'>Time (s) β</th><th class='num' title='Share of runs where the method ran out of time and random features filled the budget.'>Fallback (%)</th><th class='num'>#Datasets</th></tr></thead><tbody><tr><td class='num' data-sort='1'>1</td><td class='method' data-sort='RFImportance'>RFImportance</td><td class='' data-sort='Tree'><span class='pill cat-non-search'>Tree</span><span class='sub'>non-search</span></td><td class='num' data-sort='1325.0'>1325<span class='sub'>+45 / β41</span></td><td class='num' data-sort='6.04'>6.04</td><td class='num' data-sort='64.0'>64.0</td><td class='num' data-sort='14.6'>14.6</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.676'>0.676</td><td class='num' data-sort='0.691'>0.691</td><td class='num' data-sort='0.2'>0.2</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='2'>2</td><td class='method' data-sort='CART'>CART</td><td class='' data-sort='Tree'><span class='pill cat-non-search'>Tree</span><span class='sub'>non-search</span></td><td class='num' data-sort='1278.0'>1278<span class='sub'>+38 / β40</span></td><td class='num' data-sort='6.67'>6.67</td><td class='num' data-sort='59.5'>59.5</td><td class='num' data-sort='18.5'>18.5</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.444'>0.444</td><td class='num' data-sort='0.572'>0.572</td><td class='num' data-sort='0.1'>0.1</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='3'>3</td><td class='method' data-sort='ElasticNet'>ElasticNet</td><td class='' data-sort='Regularization'><span class='pill cat-non-search'>Regularization</span><span class='sub'>non-search</span></td><td class='num' data-sort='1273.0'>1273<span class='sub'>+40 / β39</span></td><td class='num' data-sort='6.64'>6.64</td><td class='num' data-sort='59.7'>59.7</td><td class='num' data-sort='18.1'>18.1</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.481'>0.481</td><td class='num' data-sort='0.777'>0.777</td><td class='num' data-sort='7.6'>7.6</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='4'>4</td><td class='method' data-sort='MI'>MI</td><td class='' data-sort='Information theory'><span class='pill cat-non-search'>Information theory</span><span class='sub'>non-search</span></td><td class='num' data-sort='1272.0'>1272<span class='sub'>+42 / β36</span></td><td class='num' data-sort='6.82'>6.82</td><td class='num' data-sort='58.4'>58.4</td><td class='num' data-sort='18.3'>18.3</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.683'>0.683</td><td class='num' data-sort='0.815'>0.815</td><td class='num' data-sort='0.4'>0.4</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='5'>5</td><td class='method' data-sort='Lasso'>Lasso</td><td class='' data-sort='Regularization'><span class='pill cat-non-search'>Regularization</span><span class='sub'>non-search</span></td><td class='num' data-sort='1249.0'>1249<span class='sub'>+41 / β39</span></td><td class='num' data-sort='6.97'>6.97</td><td class='num' data-sort='57.4'>57.4</td><td class='num' data-sort='18.3'>18.3</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.389'>0.389</td><td class='num' data-sort='0.73'>0.730</td><td class='num' data-sort='2.0'>2.0</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='6'>6</td><td class='method' data-sort='MarkovBlanket'>MarkovBlanket</td><td class='' data-sort='Backward search'><span class='pill cat-search'>Backward search</span><span class='sub'>search</span></td><td class='num' data-sort='1237.0'>1237<span class='sub'>+33 / β37</span></td><td class='num' data-sort='7.5'>7.50</td><td class='num' data-sort='53.6'>53.6</td><td class='num' data-sort='21.0'>21.0</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.413'>0.413</td><td class='num' data-sort='0.447'>0.447</td><td class='num' data-sort='0.1'>0.1</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='7'>7</td><td class='method' data-sort='F-Test'>F-Test</td><td class='' data-sort='Correlation'><span class='pill cat-non-search'>Correlation</span><span class='sub'>non-search</span></td><td class='num' data-sort='1223.0'>1223<span class='sub'>+33 / β35</span></td><td class='num' data-sort='7.49'>7.49</td><td class='num' data-sort='53.6'>53.6</td><td class='num' data-sort='20.5'>20.5</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.706'>0.706</td><td class='num' data-sort='0.879'>0.879</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='8'>8</td><td class='method' data-sort='(R)ReliefF'>(R)ReliefF</td><td class='' data-sort='Distance'><span class='pill cat-non-search'>Distance</span><span class='sub'>non-search</span></td><td class='num' data-sort='1210.0'>1210<span class='sub'>+36 / β37</span></td><td class='num' data-sort='7.86'>7.86</td><td class='num' data-sort='51.0'>51.0</td><td class='num' data-sort='20.2'>20.2</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.673'>0.673</td><td class='num' data-sort='0.757'>0.757</td><td class='num' data-sort='14.4'>14.4</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='9'>9</td><td class='method' data-sort='GainRatio'>GainRatio</td><td class='' data-sort='Information theory'><span class='pill cat-non-search'>Information theory</span><span class='sub'>non-search</span></td><td class='num' data-sort='1194.0'>1194<span class='sub'>+38 / β35</span></td><td class='num' data-sort='8.22'>8.22</td><td class='num' data-sort='48.4'>48.4</td><td class='num' data-sort='22.8'>22.8</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.67'>0.670</td><td class='num' data-sort='0.856'>0.856</td><td class='num' data-sort='0.2'>0.2</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='10'>10</td><td class='method' data-sort='SBE'>SBE</td><td class='' data-sort='Backward search'><span class='pill cat-search'>Backward search</span><span class='sub'>search</span></td><td class='num' data-sort='1186.0'>1186<span class='sub'>+46 / β41</span></td><td class='num' data-sort='8.15'>8.15</td><td class='num' data-sort='48.9'>48.9</td><td class='num' data-sort='24.7'>24.7</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.144'>0.144</td><td class='num' data-sort='0.556'>0.556</td><td class='num' data-sort='2621.7'>2,621.7</td><td class='num' data-sort='47.8'>47.8</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='11'>11</td><td class='method' data-sort='mRMR'>mRMR</td><td class='' data-sort='Information theory'><span class='pill cat-non-search'>Information theory</span><span class='sub'>non-search</span></td><td class='num' data-sort='1181.0'>1181<span class='sub'>+36 / β37</span></td><td class='num' data-sort='8.22'>8.22</td><td class='num' data-sort='48.4'>48.4</td><td class='num' data-sort='21.5'>21.5</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.614'>0.614</td><td class='num' data-sort='0.66'>0.660</td><td class='num' data-sort='0.5'>0.5</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='12'>12</td><td class='method' data-sort='SFS'>SFS</td><td class='' data-sort='Forward search'><span class='pill cat-search'>Forward search</span><span class='sub'>search</span></td><td class='num' data-sort='1175.0'>1175<span class='sub'>+39 / β37</span></td><td class='num' data-sort='8.11'>8.11</td><td class='num' data-sort='49.2'>49.2</td><td class='num' data-sort='23.1'>23.1</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.239'>0.239</td><td class='num' data-sort='0.627'>0.627</td><td class='num' data-sort='2398.9'>2,398.9</td><td class='num' data-sort='47.2'>47.2</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='13'>13</td><td class='method' data-sort='LOCO'>LOCO</td><td class='' data-sort='Leave-one-out'><span class='pill cat-search'>Leave-one-out</span><span class='sub'>search</span></td><td class='num' data-sort='1157.0'>1157<span class='sub'>+43 / β40</span></td><td class='num' data-sort='8.42'>8.42</td><td class='num' data-sort='47.0'>47.0</td><td class='num' data-sort='25.1'>25.1</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.151'>0.151</td><td class='num' data-sort='0.575'>0.575</td><td class='num' data-sort='241.3'>241.3</td><td class='num' data-sort='6.8'>6.8</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='14'>14</td><td class='method' data-sort='Random'>Random</td><td class='' data-sort='Baseline'><span class='pill cat-baseline'>Baseline</span></td><td class='num' data-sort='1000.0'>1000<span class='sub'>+41 / β49</span></td><td class='num' data-sort='10.82'>10.82</td><td class='num' data-sort='29.9'>29.9</td><td class='num' data-sort='33.5'>33.5</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='-0.001'>-0.001</td><td class='num' data-sort='0.513'>0.513</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='100.0'>100.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='15'>15</td><td class='method' data-sort='LaplacianScore'>LaplacianScore</td><td class='' data-sort='Distance'><span class='pill cat-non-search'>Distance</span><span class='sub'>non-search</span></td><td class='num' data-sort='930.0'>930<span class='sub'>+56 / β70</span></td><td class='num' data-sort='12.08'>12.08</td><td class='num' data-sort='20.9'>20.9</td><td class='num' data-sort='39.5'>39.5</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.66'>0.660</td><td class='num' data-sort='0.827'>0.827</td><td class='num' data-sort='0.2'>0.2</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr></tbody></table></div><script type='text/plain' id='sa-csv'>Rank,Method,Category,Subgroup,Elo,Elo CI low,Elo CI high,Avg. rank,Win rate (%),Improvability (%),Error vs. all features,Stability,Validity,Selection time (s),Fallback rate (%),#Datasets,#Comparisons
|
| 33 |
+
1,RFImportance,Non-search,Tree,1325.0,1284.0,1370.0,6.04,64.0,14.6,,0.676,0.691,0.2,0.0,71,1065
|
| 34 |
+
2,CART,Non-search,Tree,1278.0,1238.0,1316.0,6.67,59.5,18.5,,0.444,0.572,0.1,0.0,71,1065
|
| 35 |
+
3,ElasticNet,Non-search,Regularization,1273.0,1234.0,1313.0,6.64,59.7,18.1,,0.481,0.777,7.6,0.0,71,1065
|
| 36 |
+
4,MI,Non-search,Information theory,1272.0,1236.0,1314.0,6.82,58.4,18.3,,0.683,0.815,0.4,0.0,71,1065
|
| 37 |
+
5,Lasso,Non-search,Regularization,1249.0,1210.0,1290.0,6.97,57.4,18.3,,0.389,0.73,2.0,0.0,71,1065
|
| 38 |
+
6,MarkovBlanket,Search,Backward search,1237.0,1200.0,1270.0,7.5,53.6,21.0,,0.413,0.447,0.1,0.0,71,1065
|
| 39 |
+
7,F-Test,Non-search,Correlation,1223.0,1188.0,1256.0,7.49,53.6,20.5,,0.706,0.879,0.0,0.0,71,1065
|
| 40 |
+
8,(R)ReliefF,Non-search,Distance,1210.0,1173.0,1246.0,7.86,51.0,20.2,,0.673,0.757,14.4,0.0,71,1065
|
| 41 |
+
9,GainRatio,Non-search,Information theory,1194.0,1159.0,1232.0,8.22,48.4,22.8,,0.67,0.856,0.2,0.0,71,1065
|
| 42 |
+
10,SBE,Search,Backward search,1186.0,1145.0,1232.0,8.15,48.9,24.7,,0.144,0.556,2621.7,47.8,71,1065
|
| 43 |
+
11,mRMR,Non-search,Information theory,1181.0,1144.0,1217.0,8.22,48.4,21.5,,0.614,0.66,0.5,0.0,71,1065
|
| 44 |
+
12,SFS,Search,Forward search,1175.0,1138.0,1214.0,8.11,49.2,23.1,,0.239,0.627,2398.9,47.2,71,1065
|
| 45 |
+
13,LOCO,Search,Leave-one-out,1157.0,1117.0,1200.0,8.42,47.0,25.1,,0.151,0.575,241.3,6.8,71,1065
|
| 46 |
+
14,Random,Baseline,Baseline,1000.0,951.0,1041.0,10.82,29.9,33.5,,-0.001,0.513,0.0,100.0,71,1065
|
| 47 |
+
15,LaplacianScore,Non-search,Distance,930.0,860.0,986.0,12.08,20.9,39.5,,0.66,0.827,0.2,0.0,71,1065
|
| 48 |
</script><script>
|
| 49 |
function saDownload(name, blob) {
|
| 50 |
var a = document.createElement('a');
|
data/subsets/tasks_all/models_all/budgets_2/website_leaderboard.csv
CHANGED
|
@@ -1,16 +1,16 @@
|
|
| 1 |
-
Rank,
|
| 2 |
-
1,
|
| 3 |
-
2,
|
| 4 |
-
3,
|
| 5 |
-
4,
|
| 6 |
-
5,
|
| 7 |
-
6,
|
| 8 |
-
7,
|
| 9 |
-
8,
|
| 10 |
-
9,
|
| 11 |
-
10,
|
| 12 |
-
11,
|
| 13 |
-
12,
|
| 14 |
-
13,
|
| 15 |
-
14,
|
| 16 |
-
15,
|
|
|
|
| 1 |
+
Rank,Method,Category,Subgroup,Elo,Elo CI low,Elo CI high,Avg. rank,Win rate (%),Improvability (%),Error vs. all features,Stability,Validity,Selection time (s),Fallback rate (%),#Datasets,#Comparisons
|
| 2 |
+
1,RFImportance,Non-search,Tree,1325.0,1284.0,1370.0,6.04,64.0,14.6,,0.676,0.691,0.2,0.0,71,1065
|
| 3 |
+
2,CART,Non-search,Tree,1278.0,1238.0,1316.0,6.67,59.5,18.5,,0.444,0.572,0.1,0.0,71,1065
|
| 4 |
+
3,ElasticNet,Non-search,Regularization,1273.0,1234.0,1313.0,6.64,59.7,18.1,,0.481,0.777,7.6,0.0,71,1065
|
| 5 |
+
4,MI,Non-search,Information theory,1272.0,1236.0,1314.0,6.82,58.4,18.3,,0.683,0.815,0.4,0.0,71,1065
|
| 6 |
+
5,Lasso,Non-search,Regularization,1249.0,1210.0,1290.0,6.97,57.4,18.3,,0.389,0.73,2.0,0.0,71,1065
|
| 7 |
+
6,MarkovBlanket,Search,Backward search,1237.0,1200.0,1270.0,7.5,53.6,21.0,,0.413,0.447,0.1,0.0,71,1065
|
| 8 |
+
7,F-Test,Non-search,Correlation,1223.0,1188.0,1256.0,7.49,53.6,20.5,,0.706,0.879,0.0,0.0,71,1065
|
| 9 |
+
8,(R)ReliefF,Non-search,Distance,1210.0,1173.0,1246.0,7.86,51.0,20.2,,0.673,0.757,14.4,0.0,71,1065
|
| 10 |
+
9,GainRatio,Non-search,Information theory,1194.0,1159.0,1232.0,8.22,48.4,22.8,,0.67,0.856,0.2,0.0,71,1065
|
| 11 |
+
10,SBE,Search,Backward search,1186.0,1145.0,1232.0,8.15,48.9,24.7,,0.144,0.556,2621.7,47.8,71,1065
|
| 12 |
+
11,mRMR,Non-search,Information theory,1181.0,1144.0,1217.0,8.22,48.4,21.5,,0.614,0.66,0.5,0.0,71,1065
|
| 13 |
+
12,SFS,Search,Forward search,1175.0,1138.0,1214.0,8.11,49.2,23.1,,0.239,0.627,2398.9,47.2,71,1065
|
| 14 |
+
13,LOCO,Search,Leave-one-out,1157.0,1117.0,1200.0,8.42,47.0,25.1,,0.151,0.575,241.3,6.8,71,1065
|
| 15 |
+
14,Random,Baseline,Baseline,1000.0,951.0,1041.0,10.82,29.9,33.5,,-0.001,0.513,0.0,100.0,71,1065
|
| 16 |
+
15,LaplacianScore,Non-search,Distance,930.0,860.0,986.0,12.08,20.9,39.5,,0.66,0.827,0.2,0.0,71,1065
|
data/subsets/tasks_all/models_all/budgets_3/elo_explorer.html
CHANGED
|
@@ -19,67 +19,67 @@ svg { display:block; width:100%; height:auto; }
|
|
| 19 |
.sa-ink { stroke:var(--fg); }
|
| 20 |
.sa-hit { fill:transparent; }
|
| 21 |
.sa-hit:hover { fill:var(--fg); fill-opacity:0.06; }
|
| 22 |
-
</style><defs><pattern id='sa-na' width='6' height='6' patternUnits='userSpaceOnUse' patternTransform='rotate(45)'><rect width='6' height='6' fill='#e5e7eb'/><line x1='0' y1='0' x2='0' y2='6' stroke='#9ca3af' stroke-width='2'/></pattern><linearGradient id='sa-g-stab' x1='0' y1='1' x2='0' y2='0'><stop offset='0' stop-color='#ffffff'/><stop offset='1' stop-color='#0072B2'/></linearGradient><linearGradient id='sa-g-val' x1='0' y1='1' x2='0' y2='0'><stop offset='0' stop-color='#ffffff'/><stop offset='1' stop-color='#D55E00'/></linearGradient></defs><line class='sa-grid' x1='64' x2='1024' y1='287.6' y2='287.6' stroke-width='1'/><text class='sa-muted' x='56' y='291.6' text-anchor='end' font-size='12'>900</text><line class='sa-grid' x1='64' x2='1024' y1='233.7' y2='233.7' stroke-width='1'/><text class='sa-muted' x='56' y='237.7' text-anchor='end' font-size='12'>1000</text><line class='sa-grid' x1='64' x2='1024' y1='179.7' y2='179.7' stroke-width='1'/><text class='sa-muted' x='56' y='183.7' text-anchor='end' font-size='12'>1100</text><line class='sa-grid' x1='64' x2='1024' y1='125.8' y2='125.8' stroke-width='1'/><text class='sa-muted' x='56' y='129.8' text-anchor='end' font-size='12'>1200</text><line class='sa-grid' x1='64' x2='1024' y1='71.9' y2='71.9' stroke-width='1'/><text class='sa-muted' x='56' y='75.9' text-anchor='end' font-size='12'>1300</text><line class='sa-grid' x1='64' x2='1024' y1='18.0' y2='18.0' stroke-width='1'/><text class='sa-muted' x='56' y='22.0' text-anchor='end' font-size='12'>1400</text><text class='sa-text' transform='translate(16 173) rotate(-90)' text-anchor='middle' font-size='13'>Elo</text><line class='sa-ink' x1='64' x2='1024' y1='233.7' y2='233.7' stroke-width='1' stroke-dasharray='4 4' opacity='0.5'/><rect x='73.6' y='271.9' width='22.4' height='56.1' fill='#4f9eca' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='96.0' y='271.9' width='22.4' height='56.1' fill='#df853e' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='96.0' x2='96.0' y1='242.8' y2='303.7'/><line x1='91.0' x2='101.0' y1='242.8' y2='242.8'/><line x1='91.0' x2='101.0' y1='303.7' y2='303.7'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(100.0 342.0) rotate(-35)'>LaplacianScore</text><rect class='sa-hit' x='64.0' y='18' width='64' height='414'><title>LaplacianScore (
|
| 23 |
Elo 929 [95% CI 870β983]
|
| 24 |
Stability 0.689
|
| 25 |
-
Validity 0.758</title></rect><rect x='137.6' y='233.7' width='22.4' height='94.3' fill='#feffff' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='160.0' y='233.7' width='22.4' height='94.3' fill='#eaaf81' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='160.0' x2='160.0' y1='212.1' y2='256.3'/><line x1='155.0' x2='165.0' y1='212.1' y2='212.1'/><line x1='155.0' x2='165.0' y1='256.3' y2='256.3'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(164.0 342.0) rotate(-35)'>Random</text><rect class='sa-hit' x='128.0' y='18' width='64' height='414'><title>Random (Baseline)
|
| 26 |
Elo 1000 [95% CI 958β1040]
|
| 27 |
Stability 0.002
|
| 28 |
-
Validity 0.494</title></rect><rect x='201.6' y='166.8' width='22.4' height='161.2' fill='#c6dfee' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='224.0' y='166.8' width='22.4' height='161.2' fill='#e7a46e' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='224.0' x2='224.0' y1='147.4' y2='186.2'/><line x1='219.0' x2='229.0' y1='147.4' y2='147.4'/><line x1='219.0' x2='229.0' y1='186.2' y2='186.2'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(228.0 342.0) rotate(-35)'>SFS</text><rect class='sa-hit' x='192.0' y='18' width='64' height='414'><title>SFS (
|
| 29 |
Elo 1124 [95% CI 1088β1160]
|
| 30 |
Stability 0.225
|
| 31 |
-
Validity 0.568</title></rect><rect x='265.6' y='165.2' width='22.4' height='162.8' fill='#e2eff6' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='288.0' y='165.2' width='22.4' height='162.8' fill='#e8a774' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='288.0' x2='288.0' y1='145.8' y2='184.6'/><line x1='283.0' x2='293.0' y1='145.8' y2='145.8'/><line x1='283.0' x2='293.0' y1='184.6' y2='184.6'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(292.0 342.0) rotate(-35)'>LOCO</text><rect class='sa-hit' x='256.0' y='18' width='64' height='414'><title>LOCO (
|
| 32 |
Elo 1127 [95% CI 1091β1163]
|
| 33 |
Stability 0.114
|
| 34 |
-
Validity 0.544</title></rect><rect x='329.6' y='161.4' width='22.4' height='166.6' fill='#5ca5ce' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='352.0' y='161.4' width='22.4' height='166.6' fill='#df863f' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='352.0' x2='352.0' y1='142.5' y2='178.7'/><line x1='347.0' x2='357.0' y1='142.5' y2='142.5'/><line x1='347.0' x2='357.0' y1='178.7' y2='178.7'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(356.0 342.0) rotate(-35)'>GainRatio</text><rect class='sa-hit' x='320.0' y='18' width='64' height='414'><title>GainRatio (
|
| 35 |
Elo 1134 [95% CI 1102β1169]
|
| 36 |
Stability 0.641
|
| 37 |
-
Validity 0.751</title></rect><rect x='393.6' y='150.1' width='22.4' height='177.9' fill='#e7f2f8' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='416.0' y='150.1' width='22.4' height='177.9' fill='#e9aa78' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='416.0' x2='416.0' y1='123.7' y2='174.9'/><line x1='411.0' x2='421.0' y1='123.7' y2='123.7'/><line x1='411.0' x2='421.0' y1='174.9' y2='174.9'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(420.0 342.0) rotate(-35)'>SBE</text><rect class='sa-hit' x='384.0' y='18' width='64' height='414'><title>SBE (
|
| 38 |
Elo 1155 [95% CI 1109β1204]
|
| 39 |
Stability 0.095
|
| 40 |
-
Validity 0.530</title></rect><rect x='457.6' y='150.1' width='22.4' height='177.9' fill='#58a3cc' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='480.0' y='150.1' width='22.4' height='177.9' fill='#e6a068' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='480.0' x2='480.0' y1='131.2' y2='170.6'/><line x1='475.0' x2='485.0' y1='131.2' y2='131.2'/><line x1='475.0' x2='485.0' y1='170.6' y2='170.6'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(484.0 342.0) rotate(-35)'>mRMR</text><rect class='sa-hit' x='448.0' y='18' width='64' height='414'><title>mRMR (
|
| 41 |
Elo 1155 [95% CI 1117β1190]
|
| 42 |
Stability 0.656
|
| 43 |
-
Validity 0.593</title></rect><rect x='521.6' y='148.5' width='22.4' height='179.5' fill='#90c2de' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='544.0' y='148.5' width='22.4' height='179.5' fill='#ebb386' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='544.0' x2='544.0' y1='130.1' y2='168.4'/><line x1='539.0' x2='549.0' y1='130.1' y2='130.1'/><line x1='539.0' x2='549.0' y1='168.4' y2='168.4'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(548.0 342.0) rotate(-35)'>MarkovBlanket</text><rect class='sa-hit' x='512.0' y='18' width='64' height='414'><title>MarkovBlanket (
|
| 44 |
Elo 1158 [95% CI 1121β1192]
|
| 45 |
Stability 0.434
|
| 46 |
-
Validity 0.473</title></rect><rect x='585.6' y='140.9' width='22.4' height='187.1' fill='#58a3cc' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='608.0' y='140.9' width='22.4' height='187.1' fill='#e39455' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='608.0' x2='608.0' y1='121.5' y2='159.8'/><line x1='603.0' x2='613.0' y1='121.5' y2='121.5'/><line x1='603.0' x2='613.0' y1='159.8' y2='159.8'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(612.0 342.0) rotate(-35)'>(R)ReliefF</text><rect class='sa-hit' x='576.0' y='18' width='64' height='414'><title>(R)ReliefF (
|
| 47 |
Elo 1172 [95% CI 1137β1208]
|
| 48 |
Stability 0.656
|
| 49 |
-
Validity 0.667</title></rect><rect x='649.6' y='131.8' width='22.4' height='196.2' fill='#55a1cc' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='672.0' y='131.8' width='22.4' height='196.2' fill='#de8138' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='672.0' x2='672.0' y1='112.3' y2='148.5'/><line x1='667.0' x2='677.0' y1='112.3' y2='112.3'/><line x1='667.0' x2='677.0' y1='148.5' y2='148.5'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(676.0 342.0) rotate(-35)'>F-Test</text><rect class='sa-hit' x='640.0' y='18' width='64' height='414'><title>F-Test (
|
| 50 |
Elo 1189 [95% CI 1158β1225]
|
| 51 |
Stability 0.668
|
| 52 |
-
Validity 0.781</title></rect><rect x='713.6' y='130.1' width='22.4' height='197.9' fill='#aad0e5' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='736.0' y='130.1' width='22.4' height='197.9' fill='#e4975b' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='736.0' x2='736.0' y1='108.0' y2='152.2'/><line x1='731.0' x2='741.0' y1='108.0' y2='108.0'/><line x1='731.0' x2='741.0' y1='152.2' y2='152.2'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(740.0 342.0) rotate(-35)'>Lasso</text><rect class='sa-hit' x='704.0' y='18' width='64' height='414'><title>Lasso (
|
| 53 |
Elo 1192 [95% CI 1151β1233]
|
| 54 |
Stability 0.334
|
| 55 |
-
Validity 0.643</title></rect><rect x='777.6' y='119.4' width='22.4' height='208.6' fill='#94c4df' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='800.0' y='119.4' width='22.4' height='208.6' fill='#e39455' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='800.0' x2='800.0' y1='96.7' y2='142.0'/><line x1='795.0' x2='805.0' y1='96.7' y2='96.7'/><line x1='795.0' x2='805.0' y1='142.0' y2='142.0'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(804.0 342.0) rotate(-35)'>ElasticNet</text><rect class='sa-hit' x='768.0' y='18' width='64' height='414'><title>ElasticNet (
|
| 56 |
Elo 1212 [95% CI 1170β1254]
|
| 57 |
Stability 0.419
|
| 58 |
-
Validity 0.665</title></rect><rect x='841.6' y='116.1' width='22.4' height='211.9' fill='#5aa3cd' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='864.0' y='116.1' width='22.4' height='211.9' fill='#e18b47' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='864.0' x2='864.0' y1='96.2' y2='134.5'/><line x1='859.0' x2='869.0' y1='96.2' y2='96.2'/><line x1='859.0' x2='869.0' y1='134.5' y2='134.5'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(868.0 342.0) rotate(-35)'>MI</text><rect class='sa-hit' x='832.0' y='18' width='64' height='414'><title>MI (
|
| 59 |
Elo 1218 [95% CI 1184β1255]
|
| 60 |
Stability 0.649
|
| 61 |
-
Validity 0.723</title></rect><rect x='905.6' y='108.6' width='22.4' height='219.4' fill='#98c6e0' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='928.0' y='108.6' width='22.4' height='219.4' fill='#eaad7e' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='928.0' x2='928.0' y1='89.2' y2='129.1'/><line x1='923.0' x2='933.0' y1='89.2' y2='89.2'/><line x1='923.0' x2='933.0' y1='129.1' y2='129.1'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(932.0 342.0) rotate(-35)'>CART</text><rect class='sa-hit' x='896.0' y='18' width='64' height='414'><title>CART (
|
| 62 |
Elo 1232 [95% CI 1194β1268]
|
| 63 |
Stability 0.402
|
| 64 |
-
Validity 0.507</title></rect><rect x='969.6' y='78.4' width='22.4' height='249.6' fill='#56a2cc' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='992.0' y='78.4' width='22.4' height='249.6' fill='#e69e66' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='992.0' x2='992.0' y1='54.7' y2='99.9'/><line x1='987.0' x2='997.0' y1='54.7' y2='54.7'/><line x1='987.0' x2='997.0' y1='99.9' y2='99.9'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(996.0 342.0) rotate(-35)'>RFImportance</text><rect class='sa-hit' x='960.0' y='18' width='64' height='414'><title>RFImportance (
|
| 65 |
Elo 1288 [95% CI 1248β1332]
|
| 66 |
Stability 0.661
|
| 67 |
-
Validity 0.600</title></rect><line class='sa-ink' x1='64' x2='1024' y1='328.0' y2='328.0' stroke-width='1' opacity='0.4'/><rect x='1070' y='40.0' width='12' height='107.0' fill='url(#sa-g-stab)' stroke='#000' stroke-width='0.6'/><text class='sa-text' x='1076' y='30.0' text-anchor='middle' font-size='12'>stable</text><text class='sa-text' x='1076' y='165.0' text-anchor='middle' font-size='12'>unstable</text><text class='sa-muted' x='1088' y='151.0' font-size='11'>0.0</text><text class='sa-muted' x='1088' y='97.5' font-size='11'>0.5</text><text class='sa-muted' x='1088' y='44.0' font-size='11'>1.0</text><rect x='1070' y='207.0' width='12' height='107.0' fill='url(#sa-g-val)' stroke='#000' stroke-width='0.6'/><text class='sa-text' x='1076' y='197.0' text-anchor='middle' font-size='12'>valid</text><text class='sa-text' x='1076' y='332.0' text-anchor='middle' font-size='12'>invalid</text><text class='sa-muted' x='1088' y='318.0' font-size='11'>0.0</text><text class='sa-muted' x='1088' y='264.5' font-size='11'>0.5</text><text class='sa-muted' x='1088' y='211.0' font-size='11'>1.0</text><rect x='1070' y='352.0' width='12' height='12' fill='url(#sa-na)' stroke='#000' stroke-width='0.6'/><text class='sa-muted' x='1088' y='362.0' font-size='11'>no result</text></svg><script type='text/plain' id='sa-csv'>Method,
|
| 68 |
-
RFImportance,
|
| 69 |
-
CART,
|
| 70 |
-
MI,
|
| 71 |
-
ElasticNet,
|
| 72 |
-
Lasso,
|
| 73 |
-
F-Test,
|
| 74 |
-
(R)ReliefF,
|
| 75 |
-
MarkovBlanket,
|
| 76 |
-
SBE,
|
| 77 |
-
mRMR,
|
| 78 |
-
GainRatio,
|
| 79 |
-
LOCO,
|
| 80 |
-
SFS,
|
| 81 |
-
Random,Baseline,1000.0,958.0,1040.0,0.002,0.494
|
| 82 |
-
LaplacianScore,
|
| 83 |
</script><script>
|
| 84 |
function saDownload(name, blob) {
|
| 85 |
var a = document.createElement('a');
|
|
|
|
| 19 |
.sa-ink { stroke:var(--fg); }
|
| 20 |
.sa-hit { fill:transparent; }
|
| 21 |
.sa-hit:hover { fill:var(--fg); fill-opacity:0.06; }
|
| 22 |
+
</style><defs><pattern id='sa-na' width='6' height='6' patternUnits='userSpaceOnUse' patternTransform='rotate(45)'><rect width='6' height='6' fill='#e5e7eb'/><line x1='0' y1='0' x2='0' y2='6' stroke='#9ca3af' stroke-width='2'/></pattern><linearGradient id='sa-g-stab' x1='0' y1='1' x2='0' y2='0'><stop offset='0' stop-color='#ffffff'/><stop offset='1' stop-color='#0072B2'/></linearGradient><linearGradient id='sa-g-val' x1='0' y1='1' x2='0' y2='0'><stop offset='0' stop-color='#ffffff'/><stop offset='1' stop-color='#D55E00'/></linearGradient></defs><line class='sa-grid' x1='64' x2='1024' y1='287.6' y2='287.6' stroke-width='1'/><text class='sa-muted' x='56' y='291.6' text-anchor='end' font-size='12'>900</text><line class='sa-grid' x1='64' x2='1024' y1='233.7' y2='233.7' stroke-width='1'/><text class='sa-muted' x='56' y='237.7' text-anchor='end' font-size='12'>1000</text><line class='sa-grid' x1='64' x2='1024' y1='179.7' y2='179.7' stroke-width='1'/><text class='sa-muted' x='56' y='183.7' text-anchor='end' font-size='12'>1100</text><line class='sa-grid' x1='64' x2='1024' y1='125.8' y2='125.8' stroke-width='1'/><text class='sa-muted' x='56' y='129.8' text-anchor='end' font-size='12'>1200</text><line class='sa-grid' x1='64' x2='1024' y1='71.9' y2='71.9' stroke-width='1'/><text class='sa-muted' x='56' y='75.9' text-anchor='end' font-size='12'>1300</text><line class='sa-grid' x1='64' x2='1024' y1='18.0' y2='18.0' stroke-width='1'/><text class='sa-muted' x='56' y='22.0' text-anchor='end' font-size='12'>1400</text><text class='sa-text' transform='translate(16 173) rotate(-90)' text-anchor='middle' font-size='13'>Elo</text><line class='sa-ink' x1='64' x2='1024' y1='233.7' y2='233.7' stroke-width='1' stroke-dasharray='4 4' opacity='0.5'/><rect x='73.6' y='271.9' width='22.4' height='56.1' fill='#4f9eca' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='96.0' y='271.9' width='22.4' height='56.1' fill='#df853e' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='96.0' x2='96.0' y1='242.8' y2='303.7'/><line x1='91.0' x2='101.0' y1='242.8' y2='242.8'/><line x1='91.0' x2='101.0' y1='303.7' y2='303.7'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(100.0 342.0) rotate(-35)'>LaplacianScore</text><rect class='sa-hit' x='64.0' y='18' width='64' height='414'><title>LaplacianScore (Distance, non-search)
|
| 23 |
Elo 929 [95% CI 870β983]
|
| 24 |
Stability 0.689
|
| 25 |
+
Validity 0.758</title></rect><rect x='137.6' y='233.7' width='22.4' height='94.3' fill='#feffff' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='160.0' y='233.7' width='22.4' height='94.3' fill='#eaaf81' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='160.0' x2='160.0' y1='212.1' y2='256.3'/><line x1='155.0' x2='165.0' y1='212.1' y2='212.1'/><line x1='155.0' x2='165.0' y1='256.3' y2='256.3'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(164.0 342.0) rotate(-35)'>Random</text><rect class='sa-hit' x='128.0' y='18' width='64' height='414'><title>Random (Baseline, baseline)
|
| 26 |
Elo 1000 [95% CI 958β1040]
|
| 27 |
Stability 0.002
|
| 28 |
+
Validity 0.494</title></rect><rect x='201.6' y='166.8' width='22.4' height='161.2' fill='#c6dfee' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='224.0' y='166.8' width='22.4' height='161.2' fill='#e7a46e' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='224.0' x2='224.0' y1='147.4' y2='186.2'/><line x1='219.0' x2='229.0' y1='147.4' y2='147.4'/><line x1='219.0' x2='229.0' y1='186.2' y2='186.2'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(228.0 342.0) rotate(-35)'>SFS</text><rect class='sa-hit' x='192.0' y='18' width='64' height='414'><title>SFS (Forward search, search)
|
| 29 |
Elo 1124 [95% CI 1088β1160]
|
| 30 |
Stability 0.225
|
| 31 |
+
Validity 0.568</title></rect><rect x='265.6' y='165.2' width='22.4' height='162.8' fill='#e2eff6' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='288.0' y='165.2' width='22.4' height='162.8' fill='#e8a774' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='288.0' x2='288.0' y1='145.8' y2='184.6'/><line x1='283.0' x2='293.0' y1='145.8' y2='145.8'/><line x1='283.0' x2='293.0' y1='184.6' y2='184.6'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(292.0 342.0) rotate(-35)'>LOCO</text><rect class='sa-hit' x='256.0' y='18' width='64' height='414'><title>LOCO (Leave-one-out, search)
|
| 32 |
Elo 1127 [95% CI 1091β1163]
|
| 33 |
Stability 0.114
|
| 34 |
+
Validity 0.544</title></rect><rect x='329.6' y='161.4' width='22.4' height='166.6' fill='#5ca5ce' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='352.0' y='161.4' width='22.4' height='166.6' fill='#df863f' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='352.0' x2='352.0' y1='142.5' y2='178.7'/><line x1='347.0' x2='357.0' y1='142.5' y2='142.5'/><line x1='347.0' x2='357.0' y1='178.7' y2='178.7'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(356.0 342.0) rotate(-35)'>GainRatio</text><rect class='sa-hit' x='320.0' y='18' width='64' height='414'><title>GainRatio (Information theory, non-search)
|
| 35 |
Elo 1134 [95% CI 1102β1169]
|
| 36 |
Stability 0.641
|
| 37 |
+
Validity 0.751</title></rect><rect x='393.6' y='150.1' width='22.4' height='177.9' fill='#e7f2f8' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='416.0' y='150.1' width='22.4' height='177.9' fill='#e9aa78' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='416.0' x2='416.0' y1='123.7' y2='174.9'/><line x1='411.0' x2='421.0' y1='123.7' y2='123.7'/><line x1='411.0' x2='421.0' y1='174.9' y2='174.9'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(420.0 342.0) rotate(-35)'>SBE</text><rect class='sa-hit' x='384.0' y='18' width='64' height='414'><title>SBE (Backward search, search)
|
| 38 |
Elo 1155 [95% CI 1109β1204]
|
| 39 |
Stability 0.095
|
| 40 |
+
Validity 0.530</title></rect><rect x='457.6' y='150.1' width='22.4' height='177.9' fill='#58a3cc' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='480.0' y='150.1' width='22.4' height='177.9' fill='#e6a068' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='480.0' x2='480.0' y1='131.2' y2='170.6'/><line x1='475.0' x2='485.0' y1='131.2' y2='131.2'/><line x1='475.0' x2='485.0' y1='170.6' y2='170.6'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(484.0 342.0) rotate(-35)'>mRMR</text><rect class='sa-hit' x='448.0' y='18' width='64' height='414'><title>mRMR (Information theory, non-search)
|
| 41 |
Elo 1155 [95% CI 1117β1190]
|
| 42 |
Stability 0.656
|
| 43 |
+
Validity 0.593</title></rect><rect x='521.6' y='148.5' width='22.4' height='179.5' fill='#90c2de' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='544.0' y='148.5' width='22.4' height='179.5' fill='#ebb386' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='544.0' x2='544.0' y1='130.1' y2='168.4'/><line x1='539.0' x2='549.0' y1='130.1' y2='130.1'/><line x1='539.0' x2='549.0' y1='168.4' y2='168.4'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(548.0 342.0) rotate(-35)'>MarkovBlanket</text><rect class='sa-hit' x='512.0' y='18' width='64' height='414'><title>MarkovBlanket (Backward search, search)
|
| 44 |
Elo 1158 [95% CI 1121β1192]
|
| 45 |
Stability 0.434
|
| 46 |
+
Validity 0.473</title></rect><rect x='585.6' y='140.9' width='22.4' height='187.1' fill='#58a3cc' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='608.0' y='140.9' width='22.4' height='187.1' fill='#e39455' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='608.0' x2='608.0' y1='121.5' y2='159.8'/><line x1='603.0' x2='613.0' y1='121.5' y2='121.5'/><line x1='603.0' x2='613.0' y1='159.8' y2='159.8'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(612.0 342.0) rotate(-35)'>(R)ReliefF</text><rect class='sa-hit' x='576.0' y='18' width='64' height='414'><title>(R)ReliefF (Distance, non-search)
|
| 47 |
Elo 1172 [95% CI 1137β1208]
|
| 48 |
Stability 0.656
|
| 49 |
+
Validity 0.667</title></rect><rect x='649.6' y='131.8' width='22.4' height='196.2' fill='#55a1cc' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='672.0' y='131.8' width='22.4' height='196.2' fill='#de8138' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='672.0' x2='672.0' y1='112.3' y2='148.5'/><line x1='667.0' x2='677.0' y1='112.3' y2='112.3'/><line x1='667.0' x2='677.0' y1='148.5' y2='148.5'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(676.0 342.0) rotate(-35)'>F-Test</text><rect class='sa-hit' x='640.0' y='18' width='64' height='414'><title>F-Test (Correlation, non-search)
|
| 50 |
Elo 1189 [95% CI 1158β1225]
|
| 51 |
Stability 0.668
|
| 52 |
+
Validity 0.781</title></rect><rect x='713.6' y='130.1' width='22.4' height='197.9' fill='#aad0e5' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='736.0' y='130.1' width='22.4' height='197.9' fill='#e4975b' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='736.0' x2='736.0' y1='108.0' y2='152.2'/><line x1='731.0' x2='741.0' y1='108.0' y2='108.0'/><line x1='731.0' x2='741.0' y1='152.2' y2='152.2'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(740.0 342.0) rotate(-35)'>Lasso</text><rect class='sa-hit' x='704.0' y='18' width='64' height='414'><title>Lasso (Regularization, non-search)
|
| 53 |
Elo 1192 [95% CI 1151β1233]
|
| 54 |
Stability 0.334
|
| 55 |
+
Validity 0.643</title></rect><rect x='777.6' y='119.4' width='22.4' height='208.6' fill='#94c4df' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='800.0' y='119.4' width='22.4' height='208.6' fill='#e39455' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='800.0' x2='800.0' y1='96.7' y2='142.0'/><line x1='795.0' x2='805.0' y1='96.7' y2='96.7'/><line x1='795.0' x2='805.0' y1='142.0' y2='142.0'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(804.0 342.0) rotate(-35)'>ElasticNet</text><rect class='sa-hit' x='768.0' y='18' width='64' height='414'><title>ElasticNet (Regularization, non-search)
|
| 56 |
Elo 1212 [95% CI 1170β1254]
|
| 57 |
Stability 0.419
|
| 58 |
+
Validity 0.665</title></rect><rect x='841.6' y='116.1' width='22.4' height='211.9' fill='#5aa3cd' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='864.0' y='116.1' width='22.4' height='211.9' fill='#e18b47' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='864.0' x2='864.0' y1='96.2' y2='134.5'/><line x1='859.0' x2='869.0' y1='96.2' y2='96.2'/><line x1='859.0' x2='869.0' y1='134.5' y2='134.5'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(868.0 342.0) rotate(-35)'>MI</text><rect class='sa-hit' x='832.0' y='18' width='64' height='414'><title>MI (Information theory, non-search)
|
| 59 |
Elo 1218 [95% CI 1184β1255]
|
| 60 |
Stability 0.649
|
| 61 |
+
Validity 0.723</title></rect><rect x='905.6' y='108.6' width='22.4' height='219.4' fill='#98c6e0' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='928.0' y='108.6' width='22.4' height='219.4' fill='#eaad7e' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='928.0' x2='928.0' y1='89.2' y2='129.1'/><line x1='923.0' x2='933.0' y1='89.2' y2='89.2'/><line x1='923.0' x2='933.0' y1='129.1' y2='129.1'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(932.0 342.0) rotate(-35)'>CART</text><rect class='sa-hit' x='896.0' y='18' width='64' height='414'><title>CART (Tree, non-search)
|
| 62 |
Elo 1232 [95% CI 1194β1268]
|
| 63 |
Stability 0.402
|
| 64 |
+
Validity 0.507</title></rect><rect x='969.6' y='78.4' width='22.4' height='249.6' fill='#56a2cc' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='992.0' y='78.4' width='22.4' height='249.6' fill='#e69e66' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='992.0' x2='992.0' y1='54.7' y2='99.9'/><line x1='987.0' x2='997.0' y1='54.7' y2='54.7'/><line x1='987.0' x2='997.0' y1='99.9' y2='99.9'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(996.0 342.0) rotate(-35)'>RFImportance</text><rect class='sa-hit' x='960.0' y='18' width='64' height='414'><title>RFImportance (Tree, non-search)
|
| 65 |
Elo 1288 [95% CI 1248β1332]
|
| 66 |
Stability 0.661
|
| 67 |
+
Validity 0.600</title></rect><line class='sa-ink' x1='64' x2='1024' y1='328.0' y2='328.0' stroke-width='1' opacity='0.4'/><rect x='1070' y='40.0' width='12' height='107.0' fill='url(#sa-g-stab)' stroke='#000' stroke-width='0.6'/><text class='sa-text' x='1076' y='30.0' text-anchor='middle' font-size='12'>stable</text><text class='sa-text' x='1076' y='165.0' text-anchor='middle' font-size='12'>unstable</text><text class='sa-muted' x='1088' y='151.0' font-size='11'>0.0</text><text class='sa-muted' x='1088' y='97.5' font-size='11'>0.5</text><text class='sa-muted' x='1088' y='44.0' font-size='11'>1.0</text><rect x='1070' y='207.0' width='12' height='107.0' fill='url(#sa-g-val)' stroke='#000' stroke-width='0.6'/><text class='sa-text' x='1076' y='197.0' text-anchor='middle' font-size='12'>valid</text><text class='sa-text' x='1076' y='332.0' text-anchor='middle' font-size='12'>invalid</text><text class='sa-muted' x='1088' y='318.0' font-size='11'>0.0</text><text class='sa-muted' x='1088' y='264.5' font-size='11'>0.5</text><text class='sa-muted' x='1088' y='211.0' font-size='11'>1.0</text><rect x='1070' y='352.0' width='12' height='12' fill='url(#sa-na)' stroke='#000' stroke-width='0.6'/><text class='sa-muted' x='1088' y='362.0' font-size='11'>no result</text></svg><script type='text/plain' id='sa-csv'>Method,Category,Subgroup,Elo,Elo CI low,Elo CI high,Stability,Validity
|
| 68 |
+
RFImportance,Non-search,Tree,1288.0,1248.0,1332.0,0.661,0.6
|
| 69 |
+
CART,Non-search,Tree,1232.0,1194.0,1268.0,0.402,0.507
|
| 70 |
+
MI,Non-search,Information theory,1218.0,1184.0,1255.0,0.649,0.723
|
| 71 |
+
ElasticNet,Non-search,Regularization,1212.0,1170.0,1254.0,0.419,0.665
|
| 72 |
+
Lasso,Non-search,Regularization,1192.0,1151.0,1233.0,0.334,0.643
|
| 73 |
+
F-Test,Non-search,Correlation,1189.0,1158.0,1225.0,0.668,0.781
|
| 74 |
+
(R)ReliefF,Non-search,Distance,1172.0,1137.0,1208.0,0.656,0.667
|
| 75 |
+
MarkovBlanket,Search,Backward search,1158.0,1121.0,1192.0,0.434,0.473
|
| 76 |
+
SBE,Search,Backward search,1155.0,1109.0,1204.0,0.095,0.53
|
| 77 |
+
mRMR,Non-search,Information theory,1155.0,1117.0,1190.0,0.656,0.593
|
| 78 |
+
GainRatio,Non-search,Information theory,1134.0,1102.0,1169.0,0.641,0.751
|
| 79 |
+
LOCO,Search,Leave-one-out,1127.0,1091.0,1163.0,0.114,0.544
|
| 80 |
+
SFS,Search,Forward search,1124.0,1088.0,1160.0,0.225,0.568
|
| 81 |
+
Random,Baseline,Baseline,1000.0,958.0,1040.0,0.002,0.494
|
| 82 |
+
LaplacianScore,Non-search,Distance,929.0,870.0,983.0,0.689,0.758
|
| 83 |
</script><script>
|
| 84 |
function saDownload(name, blob) {
|
| 85 |
var a = document.createElement('a');
|
data/subsets/tasks_all/models_all/budgets_3/leaderboard_table.html
CHANGED
|
@@ -26,23 +26,25 @@ td.method { font-weight:600; }
|
|
| 26 |
td .sub { color:var(--muted); font-size:12px; font-weight:400; margin-left:6px; }
|
| 27 |
td.na { color:var(--muted); }
|
| 28 |
.pill { display:inline-block; padding:0 8px; border:1px solid var(--line); border-radius:999px;
|
| 29 |
-
font-size:12px; color:var(--
|
| 30 |
-
</style></head><body><div class='toolbar'><span class='hint'>Click a column to sort. Hover a header for its definition.</span><button onclick="saCsv('selectarena_leaderboard.csv')">CSV</button></div><div class='wrap'><table id='sa-table'><thead><tr><th class='num'>#</th><th class=''>Method</th><th class='' title='Filter: scores features from the data alone. Wrapper: searches feature subsets by refitting a proxy model. Embedded: reads the selection off a fitted model.'>Type</th><th class='num' title='Bradley-Terry Elo from pairwise matches within each dataset, budget and downstream model. Random = 1000. +/β give the 95% bootstrap interval.'>Elo β</th><th class='num' title='Mean rank of the downstream error.'>Avg. rank β</th><th class='num' title='Share of pairwise comparisons won, in %. Ties count half.'>Win rate β</th><th class='num' title='Mean relative gap (error β best error) / error to the best method, in %.'>Improvability β</th><th class='num' title='Mean ratio of the error to the same model trained on all features. Below 1: the selection helped.'>Error vs. all β</th><th class='num' title='Agreement of the selected features across bootstrap resamples (Nogueira et al., 2018).'>Stability β</th><th class='num' title='Share of the budget filled with original features when as many noise features are added.'>Validity β</th><th class='num' title='Median wall-clock time of the selection.'>Time (s) β</th><th class='num' title='Share of runs where the method ran out of time and random features filled the budget.'>Fallback (%)</th><th class='num'>#Datasets</th></tr></thead><tbody><tr><td class='num' data-sort='1'>1</td><td class='method' data-sort='RFImportance'>RFImportance</td><td class='' data-sort='Embedded'><span class='pill'>π³ Embedded</span></td><td class='num' data-sort='1288.0'>1288<span class='sub'>+44 / β40</span></td><td class='num' data-sort='5.9'>5.90</td><td class='num' data-sort='65.0'>65.0</td><td class='num' data-sort='12.4'>12.4</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.661'>0.661</td><td class='num' data-sort='0.6'>0.600</td><td class='num' data-sort='0.2'>0.2</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='2'>2</td><td class='method' data-sort='CART'>CART</td><td class='' data-sort='Embedded'><span class='pill'>π³ Embedded</span></td><td class='num' data-sort='1232.0'>1232<span class='sub'>+36 / β38</span></td><td class='num' data-sort='6.65'>6.65</td><td class='num' data-sort='59.7'>59.7</td><td class='num' data-sort='15.6'>15.6</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.402'>0.402</td><td class='num' data-sort='0.507'>0.507</td><td class='num' data-sort='0.1'>0.1</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='3'>3</td><td class='method' data-sort='MI'>MI</td><td class='' data-sort='Filter'><span class='pill'>π Filter</span></td><td class='num' data-sort='1218.0'>1218<span class='sub'>+37 / β34</span></td><td class='num' data-sort='6.94'>6.94</td><td class='num' data-sort='57.6'>57.6</td><td class='num' data-sort='15.8'>15.8</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.649'>0.649</td><td class='num' data-sort='0.723'>0.723</td><td class='num' data-sort='0.4'>0.4</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='4'>4</td><td class='method' data-sort='ElasticNet'>ElasticNet</td><td class='' data-sort='Embedded'><span class='pill'>π³ Embedded</span></td><td class='num' data-sort='1212.0'>1212<span class='sub'>+42 / β42</span></td><td class='num' data-sort='7.02'>7.02</td><td class='num' data-sort='57.0'>57.0</td><td class='num' data-sort='15.9'>15.9</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.419'>0.419</td><td class='num' data-sort='0.665'>0.665</td><td class='num' data-sort='7.6'>7.6</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='5'>5</td><td class='method' data-sort='Lasso'>Lasso</td><td class='' data-sort='Embedded'><span class='pill'>π³ Embedded</span></td><td class='num' data-sort='1192.0'>1192<span class='sub'>+41 / β41</span></td><td class='num' data-sort='7.27'>7.27</td><td class='num' data-sort='55.2'>55.2</td><td class='num' data-sort='16.4'>16.4</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.334'>0.334</td><td class='num' data-sort='0.643'>0.643</td><td class='num' data-sort='2.0'>2.0</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='6'>6</td><td class='method' data-sort='F-Test'>F-Test</td><td class='' data-sort='Filter'><span class='pill'>π Filter</span></td><td class='num' data-sort='1189.0'>1189<span class='sub'>+36 / β31</span></td><td class='num' data-sort='7.37'>7.37</td><td class='num' data-sort='54.5'>54.5</td><td class='num' data-sort='15.9'>15.9</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.668'>0.668</td><td class='num' data-sort='0.781'>0.781</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='7'>7</td><td class='method' data-sort='(R)ReliefF'>(R)ReliefF</td><td class='' data-sort='Filter'><span class='pill'>π Filter</span></td><td class='num' data-sort='1172.0'>1172<span class='sub'>+36 / β35</span></td><td class='num' data-sort='7.8'>7.80</td><td class='num' data-sort='51.4'>51.4</td><td class='num' data-sort='17.0'>17.0</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.656'>0.656</td><td class='num' data-sort='0.667'>0.667</td><td class='num' data-sort='14.3'>14.3</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='8'>8</td><td class='method' data-sort='MarkovBlanket'>MarkovBlanket</td><td class='' data-sort='Filter'><span class='pill'>π Filter</span></td><td class='num' data-sort='1158.0'>1158<span class='sub'>+34 / β37</span></td><td class='num' data-sort='8.15'>8.15</td><td class='num' data-sort='48.9'>48.9</td><td class='num' data-sort='18.6'>18.6</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.434'>0.434</td><td class='num' data-sort='0.473'>0.473</td><td class='num' data-sort='0.1'>0.1</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='9'>9</td><td class='method' data-sort='SBE'>SBE</td><td class='' data-sort='Wrapper'><span class='pill'>π Wrapper</span></td><td class='num' data-sort='1155.0'>1155<span class='sub'>+49 / β46</span></td><td class='num' data-sort='8.03'>8.03</td><td class='num' data-sort='49.8'>49.8</td><td class='num' data-sort='19.9'>19.9</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.095'>0.095</td><td class='num' data-sort='0.53'>0.530</td><td class='num' data-sort='1890.2'>1,890.2</td><td class='num' data-sort='44.9'>44.9</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='10'>10</td><td class='method' data-sort='mRMR'>mRMR</td><td class='' data-sort='Filter'><span class='pill'>π Filter</span></td><td class='num' data-sort='1155.0'>1155<span class='sub'>+35 / β38</span></td><td class='num' data-sort='8.09'>8.09</td><td class='num' data-sort='49.4'>49.4</td><td class='num' data-sort='17.2'>17.2</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.656'>0.656</td><td class='num' data-sort='0.593'>0.593</td><td class='num' data-sort='0.5'>0.5</td><td class='num' data-sort='7.0'>7.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='11'>11</td><td class='method' data-sort='GainRatio'>GainRatio</td><td class='' data-sort='Filter'><span class='pill'>π Filter</span></td><td class='num' data-sort='1134.0'>1134<span class='sub'>+35 / β32</span></td><td class='num' data-sort='8.44'>8.44</td><td class='num' data-sort='46.9'>46.9</td><td class='num' data-sort='18.3'>18.3</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.641'>0.641</td><td class='num' data-sort='0.751'>0.751</td><td class='num' data-sort='0.2'>0.2</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='12'>12</td><td class='method' data-sort='LOCO'>LOCO</td><td class='' data-sort='Wrapper'><span class='pill'>π Wrapper</span></td><td class='num' data-sort='1127.0'>1127<span class='sub'>+36 / β36</span></td><td class='num' data-sort='8.21'>8.21</td><td class='num' data-sort='48.5'>48.5</td><td class='num' data-sort='20.4'>20.4</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.114'>0.114</td><td class='num' data-sort='0.544'>0.544</td><td class='num' data-sort='246.0'>246.0</td><td class='num' data-sort='12.3'>12.3</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='13'>13</td><td class='method' data-sort='SFS'>SFS</td><td class='' data-sort='Wrapper'><span class='pill'>π Wrapper</span></td><td class='num' data-sort='1124.0'>1124<span class='sub'>+36 / β36</span></td><td class='num' data-sort='8.32'>8.32</td><td class='num' data-sort='47.7'>47.7</td><td class='num' data-sort='20.2'>20.2</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.225'>0.225</td><td class='num' data-sort='0.568'>0.568</td><td class='num' data-sort='2771.3'>2,771.3</td><td class='num' data-sort='48.7'>48.7</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='14'>14</td><td class='method' data-sort='Random'>Random</td><td class='' data-sort='Baseline'><span class='pill'>βͺ Baseline</span></td><td class='num' data-sort='1000.0'>1000<span class='sub'>+40 / β42</span></td><td class='num' data-sort='10.09'>10.09</td><td class='num' data-sort='35.1'>35.1</td><td class='num' data-sort='25.9'>25.9</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.002'>0.002</td><td class='num' data-sort='0.494'>0.494</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='100.0'>100.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='15'>15</td><td class='method' data-sort='LaplacianScore'>LaplacianScore</td><td class='' data-sort='Filter'><span class='pill'>π Filter</span></td><td class='num' data-sort='929.0'>929<span class='sub'>+54 / β59</span></td><td class='num' data-sort='11.73'>11.73</td><td class='num' data-sort='23.4'>23.4</td><td class='num' data-sort='33.1'>33.1</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.689'>0.689</td><td class='num' data-sort='0.758'>0.758</td><td class='num' data-sort='0.2'>0.2</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr></tbody></table></div><script type='text/plain' id='sa-csv'>Rank,Type,TypeName,Method,Elo,Elo CI low,Elo CI high,Avg. rank,Win rate (%),Improvability (%),Error vs. all features,Stability,Validity,Selection time (s),Fallback rate (%),#Datasets,#Comparisons
|
| 31 |
-
|
| 32 |
-
2,π³,Embedded,CART,1232.0,1194.0,1268.0,6.65,59.7,15.6,,0.402,0.507,0.1,0.0,71,1065
|
| 33 |
-
|
| 34 |
-
|
| 35 |
-
|
| 36 |
-
|
| 37 |
-
|
| 38 |
-
|
| 39 |
-
|
| 40 |
-
|
| 41 |
-
|
| 42 |
-
|
| 43 |
-
|
| 44 |
-
|
| 45 |
-
|
|
|
|
|
|
|
| 46 |
</script><script>
|
| 47 |
function saDownload(name, blob) {
|
| 48 |
var a = document.createElement('a');
|
|
|
|
| 26 |
td .sub { color:var(--muted); font-size:12px; font-weight:400; margin-left:6px; }
|
| 27 |
td.na { color:var(--muted); }
|
| 28 |
.pill { display:inline-block; padding:0 8px; border:1px solid var(--line); border-radius:999px;
|
| 29 |
+
font-size:12px; color:var(--fg); }
|
| 30 |
+
.pill.cat-search { border-color:var(--accent); }
|
| 31 |
+
.pill.cat-baseline { color:var(--muted); }
|
| 32 |
+
</style></head><body><div class='toolbar'><span class='hint'>Click a column to sort. Hover a header for its definition.</span><button onclick="saCsv('selectarena_leaderboard.csv')">CSV</button></div><div class='wrap'><table id='sa-table'><thead><tr><th class='num'>#</th><th class=''>Method</th><th class='' title='Taxonomy of the paper (Appendix A). Search methods evaluate many feature subsets with the model (forward, backward, leave-one-out); non-search methods use a proxy instead (correlation, distance, information theory, tree, regularization).'>Type</th><th class='num' title='Bradley-Terry Elo from pairwise matches within each dataset, budget and downstream model. Random = 1000. +/β give the 95% bootstrap interval.'>Elo β</th><th class='num' title='Mean rank of the downstream error.'>Avg. rank β</th><th class='num' title='Share of pairwise comparisons won, in %. Ties count half.'>Win rate β</th><th class='num' title='Mean relative gap (error β best error) / error to the best method, in %.'>Improvability β</th><th class='num' title='Mean ratio of the error to the same model trained on all features. Below 1: the selection helped.'>Error vs. all β</th><th class='num' title='Agreement of the selected features across bootstrap resamples (Nogueira et al., 2018).'>Stability β</th><th class='num' title='Share of the budget filled with original features when as many noise features are added.'>Validity β</th><th class='num' title='Median wall-clock time of the selection.'>Time (s) β</th><th class='num' title='Share of runs where the method ran out of time and random features filled the budget.'>Fallback (%)</th><th class='num'>#Datasets</th></tr></thead><tbody><tr><td class='num' data-sort='1'>1</td><td class='method' data-sort='RFImportance'>RFImportance</td><td class='' data-sort='Tree'><span class='pill cat-non-search'>Tree</span><span class='sub'>non-search</span></td><td class='num' data-sort='1288.0'>1288<span class='sub'>+44 / β40</span></td><td class='num' data-sort='5.9'>5.90</td><td class='num' data-sort='65.0'>65.0</td><td class='num' data-sort='12.4'>12.4</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.661'>0.661</td><td class='num' data-sort='0.6'>0.600</td><td class='num' data-sort='0.2'>0.2</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='2'>2</td><td class='method' data-sort='CART'>CART</td><td class='' data-sort='Tree'><span class='pill cat-non-search'>Tree</span><span class='sub'>non-search</span></td><td class='num' data-sort='1232.0'>1232<span class='sub'>+36 / β38</span></td><td class='num' data-sort='6.65'>6.65</td><td class='num' data-sort='59.7'>59.7</td><td class='num' data-sort='15.6'>15.6</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.402'>0.402</td><td class='num' data-sort='0.507'>0.507</td><td class='num' data-sort='0.1'>0.1</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='3'>3</td><td class='method' data-sort='MI'>MI</td><td class='' data-sort='Information theory'><span class='pill cat-non-search'>Information theory</span><span class='sub'>non-search</span></td><td class='num' data-sort='1218.0'>1218<span class='sub'>+37 / β34</span></td><td class='num' data-sort='6.94'>6.94</td><td class='num' data-sort='57.6'>57.6</td><td class='num' data-sort='15.8'>15.8</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.649'>0.649</td><td class='num' data-sort='0.723'>0.723</td><td class='num' data-sort='0.4'>0.4</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='4'>4</td><td class='method' data-sort='ElasticNet'>ElasticNet</td><td class='' data-sort='Regularization'><span class='pill cat-non-search'>Regularization</span><span class='sub'>non-search</span></td><td class='num' data-sort='1212.0'>1212<span class='sub'>+42 / β42</span></td><td class='num' data-sort='7.02'>7.02</td><td class='num' data-sort='57.0'>57.0</td><td class='num' data-sort='15.9'>15.9</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.419'>0.419</td><td class='num' data-sort='0.665'>0.665</td><td class='num' data-sort='7.6'>7.6</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='5'>5</td><td class='method' data-sort='Lasso'>Lasso</td><td class='' data-sort='Regularization'><span class='pill cat-non-search'>Regularization</span><span class='sub'>non-search</span></td><td class='num' data-sort='1192.0'>1192<span class='sub'>+41 / β41</span></td><td class='num' data-sort='7.27'>7.27</td><td class='num' data-sort='55.2'>55.2</td><td class='num' data-sort='16.4'>16.4</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.334'>0.334</td><td class='num' data-sort='0.643'>0.643</td><td class='num' data-sort='2.0'>2.0</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='6'>6</td><td class='method' data-sort='F-Test'>F-Test</td><td class='' data-sort='Correlation'><span class='pill cat-non-search'>Correlation</span><span class='sub'>non-search</span></td><td class='num' data-sort='1189.0'>1189<span class='sub'>+36 / β31</span></td><td class='num' data-sort='7.37'>7.37</td><td class='num' data-sort='54.5'>54.5</td><td class='num' data-sort='15.9'>15.9</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.668'>0.668</td><td class='num' data-sort='0.781'>0.781</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='7'>7</td><td class='method' data-sort='(R)ReliefF'>(R)ReliefF</td><td class='' data-sort='Distance'><span class='pill cat-non-search'>Distance</span><span class='sub'>non-search</span></td><td class='num' data-sort='1172.0'>1172<span class='sub'>+36 / β35</span></td><td class='num' data-sort='7.8'>7.80</td><td class='num' data-sort='51.4'>51.4</td><td class='num' data-sort='17.0'>17.0</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.656'>0.656</td><td class='num' data-sort='0.667'>0.667</td><td class='num' data-sort='14.3'>14.3</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='8'>8</td><td class='method' data-sort='MarkovBlanket'>MarkovBlanket</td><td class='' data-sort='Backward search'><span class='pill cat-search'>Backward search</span><span class='sub'>search</span></td><td class='num' data-sort='1158.0'>1158<span class='sub'>+34 / β37</span></td><td class='num' data-sort='8.15'>8.15</td><td class='num' data-sort='48.9'>48.9</td><td class='num' data-sort='18.6'>18.6</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.434'>0.434</td><td class='num' data-sort='0.473'>0.473</td><td class='num' data-sort='0.1'>0.1</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='9'>9</td><td class='method' data-sort='SBE'>SBE</td><td class='' data-sort='Backward search'><span class='pill cat-search'>Backward search</span><span class='sub'>search</span></td><td class='num' data-sort='1155.0'>1155<span class='sub'>+49 / β46</span></td><td class='num' data-sort='8.03'>8.03</td><td class='num' data-sort='49.8'>49.8</td><td class='num' data-sort='19.9'>19.9</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.095'>0.095</td><td class='num' data-sort='0.53'>0.530</td><td class='num' data-sort='1890.2'>1,890.2</td><td class='num' data-sort='44.9'>44.9</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='10'>10</td><td class='method' data-sort='mRMR'>mRMR</td><td class='' data-sort='Information theory'><span class='pill cat-non-search'>Information theory</span><span class='sub'>non-search</span></td><td class='num' data-sort='1155.0'>1155<span class='sub'>+35 / β38</span></td><td class='num' data-sort='8.09'>8.09</td><td class='num' data-sort='49.4'>49.4</td><td class='num' data-sort='17.2'>17.2</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.656'>0.656</td><td class='num' data-sort='0.593'>0.593</td><td class='num' data-sort='0.5'>0.5</td><td class='num' data-sort='7.0'>7.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='11'>11</td><td class='method' data-sort='GainRatio'>GainRatio</td><td class='' data-sort='Information theory'><span class='pill cat-non-search'>Information theory</span><span class='sub'>non-search</span></td><td class='num' data-sort='1134.0'>1134<span class='sub'>+35 / β32</span></td><td class='num' data-sort='8.44'>8.44</td><td class='num' data-sort='46.9'>46.9</td><td class='num' data-sort='18.3'>18.3</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.641'>0.641</td><td class='num' data-sort='0.751'>0.751</td><td class='num' data-sort='0.2'>0.2</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='12'>12</td><td class='method' data-sort='LOCO'>LOCO</td><td class='' data-sort='Leave-one-out'><span class='pill cat-search'>Leave-one-out</span><span class='sub'>search</span></td><td class='num' data-sort='1127.0'>1127<span class='sub'>+36 / β36</span></td><td class='num' data-sort='8.21'>8.21</td><td class='num' data-sort='48.5'>48.5</td><td class='num' data-sort='20.4'>20.4</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.114'>0.114</td><td class='num' data-sort='0.544'>0.544</td><td class='num' data-sort='246.0'>246.0</td><td class='num' data-sort='12.3'>12.3</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='13'>13</td><td class='method' data-sort='SFS'>SFS</td><td class='' data-sort='Forward search'><span class='pill cat-search'>Forward search</span><span class='sub'>search</span></td><td class='num' data-sort='1124.0'>1124<span class='sub'>+36 / β36</span></td><td class='num' data-sort='8.32'>8.32</td><td class='num' data-sort='47.7'>47.7</td><td class='num' data-sort='20.2'>20.2</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.225'>0.225</td><td class='num' data-sort='0.568'>0.568</td><td class='num' data-sort='2771.3'>2,771.3</td><td class='num' data-sort='48.7'>48.7</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='14'>14</td><td class='method' data-sort='Random'>Random</td><td class='' data-sort='Baseline'><span class='pill cat-baseline'>Baseline</span></td><td class='num' data-sort='1000.0'>1000<span class='sub'>+40 / β42</span></td><td class='num' data-sort='10.09'>10.09</td><td class='num' data-sort='35.1'>35.1</td><td class='num' data-sort='25.9'>25.9</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.002'>0.002</td><td class='num' data-sort='0.494'>0.494</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='100.0'>100.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='15'>15</td><td class='method' data-sort='LaplacianScore'>LaplacianScore</td><td class='' data-sort='Distance'><span class='pill cat-non-search'>Distance</span><span class='sub'>non-search</span></td><td class='num' data-sort='929.0'>929<span class='sub'>+54 / β59</span></td><td class='num' data-sort='11.73'>11.73</td><td class='num' data-sort='23.4'>23.4</td><td class='num' data-sort='33.1'>33.1</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.689'>0.689</td><td class='num' data-sort='0.758'>0.758</td><td class='num' data-sort='0.2'>0.2</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr></tbody></table></div><script type='text/plain' id='sa-csv'>Rank,Method,Category,Subgroup,Elo,Elo CI low,Elo CI high,Avg. rank,Win rate (%),Improvability (%),Error vs. all features,Stability,Validity,Selection time (s),Fallback rate (%),#Datasets,#Comparisons
|
| 33 |
+
1,RFImportance,Non-search,Tree,1288.0,1248.0,1332.0,5.9,65.0,12.4,,0.661,0.6,0.2,0.0,71,1065
|
| 34 |
+
2,CART,Non-search,Tree,1232.0,1194.0,1268.0,6.65,59.7,15.6,,0.402,0.507,0.1,0.0,71,1065
|
| 35 |
+
3,MI,Non-search,Information theory,1218.0,1184.0,1255.0,6.94,57.6,15.8,,0.649,0.723,0.4,0.0,71,1065
|
| 36 |
+
4,ElasticNet,Non-search,Regularization,1212.0,1170.0,1254.0,7.02,57.0,15.9,,0.419,0.665,7.6,0.0,71,1065
|
| 37 |
+
5,Lasso,Non-search,Regularization,1192.0,1151.0,1233.0,7.27,55.2,16.4,,0.334,0.643,2.0,0.0,71,1065
|
| 38 |
+
6,F-Test,Non-search,Correlation,1189.0,1158.0,1225.0,7.37,54.5,15.9,,0.668,0.781,0.0,0.0,71,1065
|
| 39 |
+
7,(R)ReliefF,Non-search,Distance,1172.0,1137.0,1208.0,7.8,51.4,17.0,,0.656,0.667,14.3,0.0,71,1065
|
| 40 |
+
8,MarkovBlanket,Search,Backward search,1158.0,1121.0,1192.0,8.15,48.9,18.6,,0.434,0.473,0.1,0.0,71,1065
|
| 41 |
+
9,SBE,Search,Backward search,1155.0,1109.0,1204.0,8.03,49.8,19.9,,0.095,0.53,1890.2,44.9,71,1065
|
| 42 |
+
10,mRMR,Non-search,Information theory,1155.0,1117.0,1190.0,8.09,49.4,17.2,,0.656,0.593,0.5,7.0,71,1065
|
| 43 |
+
11,GainRatio,Non-search,Information theory,1134.0,1102.0,1169.0,8.44,46.9,18.3,,0.641,0.751,0.2,0.0,71,1065
|
| 44 |
+
12,LOCO,Search,Leave-one-out,1127.0,1091.0,1163.0,8.21,48.5,20.4,,0.114,0.544,246.0,12.3,71,1065
|
| 45 |
+
13,SFS,Search,Forward search,1124.0,1088.0,1160.0,8.32,47.7,20.2,,0.225,0.568,2771.3,48.7,71,1065
|
| 46 |
+
14,Random,Baseline,Baseline,1000.0,958.0,1040.0,10.09,35.1,25.9,,0.002,0.494,0.0,100.0,71,1065
|
| 47 |
+
15,LaplacianScore,Non-search,Distance,929.0,870.0,983.0,11.73,23.4,33.1,,0.689,0.758,0.2,0.0,71,1065
|
| 48 |
</script><script>
|
| 49 |
function saDownload(name, blob) {
|
| 50 |
var a = document.createElement('a');
|
data/subsets/tasks_all/models_all/budgets_3/website_leaderboard.csv
CHANGED
|
@@ -1,16 +1,16 @@
|
|
| 1 |
-
Rank,
|
| 2 |
-
1,
|
| 3 |
-
2,
|
| 4 |
-
3,
|
| 5 |
-
4,
|
| 6 |
-
5,
|
| 7 |
-
6,
|
| 8 |
-
7,
|
| 9 |
-
8,
|
| 10 |
-
9,
|
| 11 |
-
10,
|
| 12 |
-
11,
|
| 13 |
-
12,
|
| 14 |
-
13,
|
| 15 |
-
14,
|
| 16 |
-
15,
|
|
|
|
| 1 |
+
Rank,Method,Category,Subgroup,Elo,Elo CI low,Elo CI high,Avg. rank,Win rate (%),Improvability (%),Error vs. all features,Stability,Validity,Selection time (s),Fallback rate (%),#Datasets,#Comparisons
|
| 2 |
+
1,RFImportance,Non-search,Tree,1288.0,1248.0,1332.0,5.9,65.0,12.4,,0.661,0.6,0.2,0.0,71,1065
|
| 3 |
+
2,CART,Non-search,Tree,1232.0,1194.0,1268.0,6.65,59.7,15.6,,0.402,0.507,0.1,0.0,71,1065
|
| 4 |
+
3,MI,Non-search,Information theory,1218.0,1184.0,1255.0,6.94,57.6,15.8,,0.649,0.723,0.4,0.0,71,1065
|
| 5 |
+
4,ElasticNet,Non-search,Regularization,1212.0,1170.0,1254.0,7.02,57.0,15.9,,0.419,0.665,7.6,0.0,71,1065
|
| 6 |
+
5,Lasso,Non-search,Regularization,1192.0,1151.0,1233.0,7.27,55.2,16.4,,0.334,0.643,2.0,0.0,71,1065
|
| 7 |
+
6,F-Test,Non-search,Correlation,1189.0,1158.0,1225.0,7.37,54.5,15.9,,0.668,0.781,0.0,0.0,71,1065
|
| 8 |
+
7,(R)ReliefF,Non-search,Distance,1172.0,1137.0,1208.0,7.8,51.4,17.0,,0.656,0.667,14.3,0.0,71,1065
|
| 9 |
+
8,MarkovBlanket,Search,Backward search,1158.0,1121.0,1192.0,8.15,48.9,18.6,,0.434,0.473,0.1,0.0,71,1065
|
| 10 |
+
9,SBE,Search,Backward search,1155.0,1109.0,1204.0,8.03,49.8,19.9,,0.095,0.53,1890.2,44.9,71,1065
|
| 11 |
+
10,mRMR,Non-search,Information theory,1155.0,1117.0,1190.0,8.09,49.4,17.2,,0.656,0.593,0.5,7.0,71,1065
|
| 12 |
+
11,GainRatio,Non-search,Information theory,1134.0,1102.0,1169.0,8.44,46.9,18.3,,0.641,0.751,0.2,0.0,71,1065
|
| 13 |
+
12,LOCO,Search,Leave-one-out,1127.0,1091.0,1163.0,8.21,48.5,20.4,,0.114,0.544,246.0,12.3,71,1065
|
| 14 |
+
13,SFS,Search,Forward search,1124.0,1088.0,1160.0,8.32,47.7,20.2,,0.225,0.568,2771.3,48.7,71,1065
|
| 15 |
+
14,Random,Baseline,Baseline,1000.0,958.0,1040.0,10.09,35.1,25.9,,0.002,0.494,0.0,100.0,71,1065
|
| 16 |
+
15,LaplacianScore,Non-search,Distance,929.0,870.0,983.0,11.73,23.4,33.1,,0.689,0.758,0.2,0.0,71,1065
|
data/subsets/tasks_all/models_all/budgets_4/elo_explorer.html
CHANGED
|
@@ -19,67 +19,67 @@ svg { display:block; width:100%; height:auto; }
|
|
| 19 |
.sa-ink { stroke:var(--fg); }
|
| 20 |
.sa-hit { fill:transparent; }
|
| 21 |
.sa-hit:hover { fill:var(--fg); fill-opacity:0.06; }
|
| 22 |
-
</style><defs><pattern id='sa-na' width='6' height='6' patternUnits='userSpaceOnUse' patternTransform='rotate(45)'><rect width='6' height='6' fill='#e5e7eb'/><line x1='0' y1='0' x2='0' y2='6' stroke='#9ca3af' stroke-width='2'/></pattern><linearGradient id='sa-g-stab' x1='0' y1='1' x2='0' y2='0'><stop offset='0' stop-color='#ffffff'/><stop offset='1' stop-color='#0072B2'/></linearGradient><linearGradient id='sa-g-val' x1='0' y1='1' x2='0' y2='0'><stop offset='0' stop-color='#ffffff'/><stop offset='1' stop-color='#D55E00'/></linearGradient></defs><line class='sa-grid' x1='64' x2='1024' y1='309.8' y2='309.8' stroke-width='1'/><text class='sa-muted' x='56' y='313.8' text-anchor='end' font-size='12'>900</text><line class='sa-grid' x1='64' x2='1024' y1='236.8' y2='236.8' stroke-width='1'/><text class='sa-muted' x='56' y='240.8' text-anchor='end' font-size='12'>1000</text><line class='sa-grid' x1='64' x2='1024' y1='163.9' y2='163.9' stroke-width='1'/><text class='sa-muted' x='56' y='167.9' text-anchor='end' font-size='12'>1100</text><line class='sa-grid' x1='64' x2='1024' y1='90.9' y2='90.9' stroke-width='1'/><text class='sa-muted' x='56' y='94.9' text-anchor='end' font-size='12'>1200</text><line class='sa-grid' x1='64' x2='1024' y1='18.0' y2='18.0' stroke-width='1'/><text class='sa-muted' x='56' y='22.0' text-anchor='end' font-size='12'>1300</text><text class='sa-text' transform='translate(16 173) rotate(-90)' text-anchor='middle' font-size='13'>Elo</text><line class='sa-ink' x1='64' x2='1024' y1='236.8' y2='236.8' stroke-width='1' stroke-dasharray='4 4' opacity='0.5'/><rect x='73.6' y='251.4' width='22.4' height='76.6' fill='#529fcb' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='96.0' y='251.4' width='22.4' height='76.6' fill='#e28e4c' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='96.0' x2='96.0' y1='216.4' y2='285.7'/><line x1='91.0' x2='101.0' y1='216.4' y2='216.4'/><line x1='91.0' x2='101.0' y1='285.7' y2='285.7'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(100.0 342.0) rotate(-35)'>LaplacianScore</text><rect class='sa-hit' x='64.0' y='18' width='64' height='414'><title>LaplacianScore (
|
| 23 |
Elo 980 [95% CI 933β1028]
|
| 24 |
Stability 0.678
|
| 25 |
-
Validity 0.701</title></rect><rect x='137.6' y='236.8' width='22.4' height='91.2' fill='#feffff' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='160.0' y='236.8' width='22.4' height='91.2' fill='#eaae80' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='160.0' x2='160.0' y1='208.4' y2='268.2'/><line x1='155.0' x2='165.0' y1='208.4' y2='208.4'/><line x1='155.0' x2='165.0' y1='268.2' y2='268.2'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(164.0 342.0) rotate(-35)'>Random</text><rect class='sa-hit' x='128.0' y='18' width='64' height='414'><title>Random (Baseline)
|
| 26 |
Elo 1000 [95% CI 957β1039]
|
| 27 |
Stability 0.002
|
| 28 |
-
Validity 0.500</title></rect><rect x='201.6' y='169.7' width='22.4' height='158.3' fill='#d2e6f1' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='224.0' y='169.7' width='22.4' height='158.3' fill='#e9aa79' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='224.0' x2='224.0' y1='144.9' y2='193.1'/><line x1='219.0' x2='229.0' y1='144.9' y2='144.9'/><line x1='219.0' x2='229.0' y1='193.1' y2='193.1'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(228.0 342.0) rotate(-35)'>SFS</text><rect class='sa-hit' x='192.0' y='18' width='64' height='414'><title>SFS (
|
| 29 |
Elo 1092 [95% CI 1060β1126]
|
| 30 |
Stability 0.176
|
| 31 |
-
Validity 0.525</title></rect><rect x='265.6' y='166.1' width='22.4' height='161.9' fill='#ebf4f9' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='288.0' y='166.1' width='22.4' height='161.9' fill='#e9ab7a' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='288.0' x2='288.0' y1='141.3' y2='189.4'/><line x1='283.0' x2='293.0' y1='141.3' y2='141.3'/><line x1='283.0' x2='293.0' y1='189.4' y2='189.4'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(292.0 342.0) rotate(-35)'>LOCO</text><rect class='sa-hit' x='256.0' y='18' width='64' height='414'><title>LOCO (
|
| 32 |
Elo 1097 [95% CI 1065β1131]
|
| 33 |
Stability 0.080
|
| 34 |
-
Validity 0.520</title></rect><rect x='329.6' y='166.1' width='22.4' height='161.9' fill='#73b1d5' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='352.0' y='166.1' width='22.4' height='161.9' fill='#e39557' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='352.0' x2='352.0' y1='139.1' y2='190.9'/><line x1='347.0' x2='357.0' y1='139.1' y2='139.1'/><line x1='347.0' x2='357.0' y1='190.9' y2='190.9'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(356.0 342.0) rotate(-35)'>GainRatio</text><rect class='sa-hit' x='320.0' y='18' width='64' height='414'><title>GainRatio (
|
| 35 |
Elo 1097 [95% CI 1063β1134]
|
| 36 |
Stability 0.550
|
| 37 |
-
Validity 0.659</title></rect><rect x='393.6' y='154.4' width='22.4' height='173.6' fill='#f0f7fa' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='416.0' y='154.4' width='22.4' height='173.6' fill='#e9ac7b' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='416.0' x2='416.0' y1='126.0' y2='181.4'/><line x1='411.0' x2='421.0' y1='126.0' y2='126.0'/><line x1='411.0' x2='421.0' y1='181.4' y2='181.4'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(420.0 342.0) rotate(-35)'>SBE</text><rect class='sa-hit' x='384.0' y='18' width='64' height='414'><title>SBE (
|
| 38 |
Elo 1113 [95% CI 1076β1152]
|
| 39 |
Stability 0.059
|
| 40 |
-
Validity 0.517</title></rect><rect x='457.6' y='152.9' width='22.4' height='175.1' fill='#98c6e0' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='480.0' y='152.9' width='22.4' height='175.1' fill='#ecb489' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='480.0' x2='480.0' y1='126.7' y2='180.7'/><line x1='475.0' x2='485.0' y1='126.7' y2='126.7'/><line x1='475.0' x2='485.0' y1='180.7' y2='180.7'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(484.0 342.0) rotate(-35)'>MarkovBlanket</text><rect class='sa-hit' x='448.0' y='18' width='64' height='414'><title>MarkovBlanket (
|
| 41 |
Elo 1115 [95% CI 1077β1151]
|
| 42 |
Stability 0.404
|
| 43 |
-
Validity 0.463</title></rect><rect x='521.6' y='141.3' width='22.4' height='186.7' fill='#59a3cd' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='544.0' y='141.3' width='22.4' height='186.7' fill='#e9aa78' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='544.0' x2='544.0' y1='117.2' y2='166.1'/><line x1='539.0' x2='549.0' y1='117.2' y2='117.2'/><line x1='539.0' x2='549.0' y1='166.1' y2='166.1'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(548.0 342.0) rotate(-35)'>mRMR</text><rect class='sa-hit' x='512.0' y='18' width='64' height='414'><title>mRMR (
|
| 44 |
Elo 1131 [95% CI 1097β1164]
|
| 45 |
Stability 0.651
|
| 46 |
-
Validity 0.530</title></rect><rect x='585.6' y='134.7' width='22.4' height='193.3' fill='#c0dcec' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='608.0' y='134.7' width='22.4' height='193.3' fill='#e7a46f' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='608.0' x2='608.0' y1='109.2' y2='161.0'/><line x1='603.0' x2='613.0' y1='109.2' y2='109.2'/><line x1='603.0' x2='613.0' y1='161.0' y2='161.0'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(612.0 342.0) rotate(-35)'>Lasso</text><rect class='sa-hit' x='576.0' y='18' width='64' height='414'><title>Lasso (
|
| 47 |
Elo 1140 [95% CI 1104β1175]
|
| 48 |
Stability 0.246
|
| 49 |
-
Validity 0.563</title></rect><rect x='649.6' y='116.5' width='22.4' height='211.5' fill='#63a9d0' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='672.0' y='116.5' width='22.4' height='211.5' fill='#e69f68' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='672.0' x2='672.0' y1='91.7' y2='141.3'/><line x1='667.0' x2='677.0' y1='91.7' y2='91.7'/><line x1='667.0' x2='677.0' y1='141.3' y2='141.3'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(676.0 342.0) rotate(-35)'>(R)ReliefF</text><rect class='sa-hit' x='640.0' y='18' width='64' height='414'><title>(R)ReliefF (
|
| 50 |
Elo 1165 [95% CI 1131β1199]
|
| 51 |
Stability 0.612
|
| 52 |
-
Validity 0.594</title></rect><rect x='713.6' y='114.3' width='22.4' height='213.7' fill='#70b0d4' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='736.0' y='114.3' width='22.4' height='213.7' fill='#e2904f' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='736.0' x2='736.0' y1='85.8' y2='140.5'/><line x1='731.0' x2='741.0' y1='85.8' y2='85.8'/><line x1='731.0' x2='741.0' y1='140.5' y2='140.5'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(740.0 342.0) rotate(-35)'>F-Test</text><rect class='sa-hit' x='704.0' y='18' width='64' height='414'><title>F-Test (
|
| 53 |
Elo 1168 [95% CI 1132β1207]
|
| 54 |
Stability 0.562
|
| 55 |
-
Validity 0.689</title></rect><rect x='777.6' y='112.1' width='22.4' height='215.9' fill='#69acd2' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='800.0' y='112.1' width='22.4' height='215.9' fill='#e59a5f' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='800.0' x2='800.0' y1='88.8' y2='134.7'/><line x1='795.0' x2='805.0' y1='88.8' y2='88.8'/><line x1='795.0' x2='805.0' y1='134.7' y2='134.7'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(804.0 342.0) rotate(-35)'>MI</text><rect class='sa-hit' x='768.0' y='18' width='64' height='414'><title>MI (
|
| 56 |
Elo 1171 [95% CI 1140β1203]
|
| 57 |
Stability 0.587
|
| 58 |
-
Validity 0.627</title></rect><rect x='841.6' y='111.4' width='22.4' height='216.6' fill='#b0d4e7' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='864.0' y='111.4' width='22.4' height='216.6' fill='#e7a46e' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='864.0' x2='864.0' y1='87.3' y2='136.2'/><line x1='859.0' x2='869.0' y1='87.3' y2='87.3'/><line x1='859.0' x2='869.0' y1='136.2' y2='136.2'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(868.0 342.0) rotate(-35)'>ElasticNet</text><rect class='sa-hit' x='832.0' y='18' width='64' height='414'><title>ElasticNet (
|
| 59 |
Elo 1172 [95% CI 1138β1205]
|
| 60 |
Stability 0.308
|
| 61 |
-
Validity 0.568</title></rect><rect x='905.6' y='99.0' width='22.4' height='229.0' fill='#acd1e6' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='928.0' y='99.0' width='22.4' height='229.0' fill='#ebb184' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='928.0' x2='928.0' y1='73.4' y2='126.7'/><line x1='923.0' x2='933.0' y1='73.4' y2='73.4'/><line x1='923.0' x2='933.0' y1='126.7' y2='126.7'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(932.0 342.0) rotate(-35)'>CART</text><rect class='sa-hit' x='896.0' y='18' width='64' height='414'><title>CART (
|
| 62 |
Elo 1189 [95% CI 1151β1224]
|
| 63 |
Stability 0.324
|
| 64 |
-
Validity 0.482</title></rect><rect x='969.6' y='54.5' width='22.4' height='273.5' fill='#67abd1' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='992.0' y='54.5' width='22.4' height='273.5' fill='#eaad7e' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='992.0' x2='992.0' y1='30.4' y2='77.1'/><line x1='987.0' x2='997.0' y1='30.4' y2='30.4'/><line x1='987.0' x2='997.0' y1='77.1' y2='77.1'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(996.0 342.0) rotate(-35)'>RFImportance</text><rect class='sa-hit' x='960.0' y='18' width='64' height='414'><title>RFImportance (
|
| 65 |
Elo 1250 [95% CI 1219β1283]
|
| 66 |
Stability 0.597
|
| 67 |
-
Validity 0.507</title></rect><line class='sa-ink' x1='64' x2='1024' y1='328.0' y2='328.0' stroke-width='1' opacity='0.4'/><rect x='1070' y='40.0' width='12' height='107.0' fill='url(#sa-g-stab)' stroke='#000' stroke-width='0.6'/><text class='sa-text' x='1076' y='30.0' text-anchor='middle' font-size='12'>stable</text><text class='sa-text' x='1076' y='165.0' text-anchor='middle' font-size='12'>unstable</text><text class='sa-muted' x='1088' y='151.0' font-size='11'>0.0</text><text class='sa-muted' x='1088' y='97.5' font-size='11'>0.5</text><text class='sa-muted' x='1088' y='44.0' font-size='11'>1.0</text><rect x='1070' y='207.0' width='12' height='107.0' fill='url(#sa-g-val)' stroke='#000' stroke-width='0.6'/><text class='sa-text' x='1076' y='197.0' text-anchor='middle' font-size='12'>valid</text><text class='sa-text' x='1076' y='332.0' text-anchor='middle' font-size='12'>invalid</text><text class='sa-muted' x='1088' y='318.0' font-size='11'>0.0</text><text class='sa-muted' x='1088' y='264.5' font-size='11'>0.5</text><text class='sa-muted' x='1088' y='211.0' font-size='11'>1.0</text><rect x='1070' y='352.0' width='12' height='12' fill='url(#sa-na)' stroke='#000' stroke-width='0.6'/><text class='sa-muted' x='1088' y='362.0' font-size='11'>no result</text></svg><script type='text/plain' id='sa-csv'>Method,
|
| 68 |
-
RFImportance,
|
| 69 |
-
CART,
|
| 70 |
-
ElasticNet,
|
| 71 |
-
MI,
|
| 72 |
-
F-Test,
|
| 73 |
-
(R)ReliefF,
|
| 74 |
-
Lasso,
|
| 75 |
-
mRMR,
|
| 76 |
-
MarkovBlanket,
|
| 77 |
-
SBE,
|
| 78 |
-
LOCO,
|
| 79 |
-
GainRatio,
|
| 80 |
-
SFS,
|
| 81 |
-
Random,Baseline,1000.0,957.0,1039.0,0.002,0.5
|
| 82 |
-
LaplacianScore,
|
| 83 |
</script><script>
|
| 84 |
function saDownload(name, blob) {
|
| 85 |
var a = document.createElement('a');
|
|
|
|
| 19 |
.sa-ink { stroke:var(--fg); }
|
| 20 |
.sa-hit { fill:transparent; }
|
| 21 |
.sa-hit:hover { fill:var(--fg); fill-opacity:0.06; }
|
| 22 |
+
</style><defs><pattern id='sa-na' width='6' height='6' patternUnits='userSpaceOnUse' patternTransform='rotate(45)'><rect width='6' height='6' fill='#e5e7eb'/><line x1='0' y1='0' x2='0' y2='6' stroke='#9ca3af' stroke-width='2'/></pattern><linearGradient id='sa-g-stab' x1='0' y1='1' x2='0' y2='0'><stop offset='0' stop-color='#ffffff'/><stop offset='1' stop-color='#0072B2'/></linearGradient><linearGradient id='sa-g-val' x1='0' y1='1' x2='0' y2='0'><stop offset='0' stop-color='#ffffff'/><stop offset='1' stop-color='#D55E00'/></linearGradient></defs><line class='sa-grid' x1='64' x2='1024' y1='309.8' y2='309.8' stroke-width='1'/><text class='sa-muted' x='56' y='313.8' text-anchor='end' font-size='12'>900</text><line class='sa-grid' x1='64' x2='1024' y1='236.8' y2='236.8' stroke-width='1'/><text class='sa-muted' x='56' y='240.8' text-anchor='end' font-size='12'>1000</text><line class='sa-grid' x1='64' x2='1024' y1='163.9' y2='163.9' stroke-width='1'/><text class='sa-muted' x='56' y='167.9' text-anchor='end' font-size='12'>1100</text><line class='sa-grid' x1='64' x2='1024' y1='90.9' y2='90.9' stroke-width='1'/><text class='sa-muted' x='56' y='94.9' text-anchor='end' font-size='12'>1200</text><line class='sa-grid' x1='64' x2='1024' y1='18.0' y2='18.0' stroke-width='1'/><text class='sa-muted' x='56' y='22.0' text-anchor='end' font-size='12'>1300</text><text class='sa-text' transform='translate(16 173) rotate(-90)' text-anchor='middle' font-size='13'>Elo</text><line class='sa-ink' x1='64' x2='1024' y1='236.8' y2='236.8' stroke-width='1' stroke-dasharray='4 4' opacity='0.5'/><rect x='73.6' y='251.4' width='22.4' height='76.6' fill='#529fcb' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='96.0' y='251.4' width='22.4' height='76.6' fill='#e28e4c' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='96.0' x2='96.0' y1='216.4' y2='285.7'/><line x1='91.0' x2='101.0' y1='216.4' y2='216.4'/><line x1='91.0' x2='101.0' y1='285.7' y2='285.7'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(100.0 342.0) rotate(-35)'>LaplacianScore</text><rect class='sa-hit' x='64.0' y='18' width='64' height='414'><title>LaplacianScore (Distance, non-search)
|
| 23 |
Elo 980 [95% CI 933β1028]
|
| 24 |
Stability 0.678
|
| 25 |
+
Validity 0.701</title></rect><rect x='137.6' y='236.8' width='22.4' height='91.2' fill='#feffff' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='160.0' y='236.8' width='22.4' height='91.2' fill='#eaae80' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='160.0' x2='160.0' y1='208.4' y2='268.2'/><line x1='155.0' x2='165.0' y1='208.4' y2='208.4'/><line x1='155.0' x2='165.0' y1='268.2' y2='268.2'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(164.0 342.0) rotate(-35)'>Random</text><rect class='sa-hit' x='128.0' y='18' width='64' height='414'><title>Random (Baseline, baseline)
|
| 26 |
Elo 1000 [95% CI 957β1039]
|
| 27 |
Stability 0.002
|
| 28 |
+
Validity 0.500</title></rect><rect x='201.6' y='169.7' width='22.4' height='158.3' fill='#d2e6f1' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='224.0' y='169.7' width='22.4' height='158.3' fill='#e9aa79' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='224.0' x2='224.0' y1='144.9' y2='193.1'/><line x1='219.0' x2='229.0' y1='144.9' y2='144.9'/><line x1='219.0' x2='229.0' y1='193.1' y2='193.1'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(228.0 342.0) rotate(-35)'>SFS</text><rect class='sa-hit' x='192.0' y='18' width='64' height='414'><title>SFS (Forward search, search)
|
| 29 |
Elo 1092 [95% CI 1060β1126]
|
| 30 |
Stability 0.176
|
| 31 |
+
Validity 0.525</title></rect><rect x='265.6' y='166.1' width='22.4' height='161.9' fill='#ebf4f9' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='288.0' y='166.1' width='22.4' height='161.9' fill='#e9ab7a' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='288.0' x2='288.0' y1='141.3' y2='189.4'/><line x1='283.0' x2='293.0' y1='141.3' y2='141.3'/><line x1='283.0' x2='293.0' y1='189.4' y2='189.4'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(292.0 342.0) rotate(-35)'>LOCO</text><rect class='sa-hit' x='256.0' y='18' width='64' height='414'><title>LOCO (Leave-one-out, search)
|
| 32 |
Elo 1097 [95% CI 1065β1131]
|
| 33 |
Stability 0.080
|
| 34 |
+
Validity 0.520</title></rect><rect x='329.6' y='166.1' width='22.4' height='161.9' fill='#73b1d5' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='352.0' y='166.1' width='22.4' height='161.9' fill='#e39557' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='352.0' x2='352.0' y1='139.1' y2='190.9'/><line x1='347.0' x2='357.0' y1='139.1' y2='139.1'/><line x1='347.0' x2='357.0' y1='190.9' y2='190.9'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(356.0 342.0) rotate(-35)'>GainRatio</text><rect class='sa-hit' x='320.0' y='18' width='64' height='414'><title>GainRatio (Information theory, non-search)
|
| 35 |
Elo 1097 [95% CI 1063β1134]
|
| 36 |
Stability 0.550
|
| 37 |
+
Validity 0.659</title></rect><rect x='393.6' y='154.4' width='22.4' height='173.6' fill='#f0f7fa' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='416.0' y='154.4' width='22.4' height='173.6' fill='#e9ac7b' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='416.0' x2='416.0' y1='126.0' y2='181.4'/><line x1='411.0' x2='421.0' y1='126.0' y2='126.0'/><line x1='411.0' x2='421.0' y1='181.4' y2='181.4'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(420.0 342.0) rotate(-35)'>SBE</text><rect class='sa-hit' x='384.0' y='18' width='64' height='414'><title>SBE (Backward search, search)
|
| 38 |
Elo 1113 [95% CI 1076β1152]
|
| 39 |
Stability 0.059
|
| 40 |
+
Validity 0.517</title></rect><rect x='457.6' y='152.9' width='22.4' height='175.1' fill='#98c6e0' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='480.0' y='152.9' width='22.4' height='175.1' fill='#ecb489' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='480.0' x2='480.0' y1='126.7' y2='180.7'/><line x1='475.0' x2='485.0' y1='126.7' y2='126.7'/><line x1='475.0' x2='485.0' y1='180.7' y2='180.7'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(484.0 342.0) rotate(-35)'>MarkovBlanket</text><rect class='sa-hit' x='448.0' y='18' width='64' height='414'><title>MarkovBlanket (Backward search, search)
|
| 41 |
Elo 1115 [95% CI 1077β1151]
|
| 42 |
Stability 0.404
|
| 43 |
+
Validity 0.463</title></rect><rect x='521.6' y='141.3' width='22.4' height='186.7' fill='#59a3cd' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='544.0' y='141.3' width='22.4' height='186.7' fill='#e9aa78' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='544.0' x2='544.0' y1='117.2' y2='166.1'/><line x1='539.0' x2='549.0' y1='117.2' y2='117.2'/><line x1='539.0' x2='549.0' y1='166.1' y2='166.1'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(548.0 342.0) rotate(-35)'>mRMR</text><rect class='sa-hit' x='512.0' y='18' width='64' height='414'><title>mRMR (Information theory, non-search)
|
| 44 |
Elo 1131 [95% CI 1097β1164]
|
| 45 |
Stability 0.651
|
| 46 |
+
Validity 0.530</title></rect><rect x='585.6' y='134.7' width='22.4' height='193.3' fill='#c0dcec' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='608.0' y='134.7' width='22.4' height='193.3' fill='#e7a46f' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='608.0' x2='608.0' y1='109.2' y2='161.0'/><line x1='603.0' x2='613.0' y1='109.2' y2='109.2'/><line x1='603.0' x2='613.0' y1='161.0' y2='161.0'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(612.0 342.0) rotate(-35)'>Lasso</text><rect class='sa-hit' x='576.0' y='18' width='64' height='414'><title>Lasso (Regularization, non-search)
|
| 47 |
Elo 1140 [95% CI 1104β1175]
|
| 48 |
Stability 0.246
|
| 49 |
+
Validity 0.563</title></rect><rect x='649.6' y='116.5' width='22.4' height='211.5' fill='#63a9d0' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='672.0' y='116.5' width='22.4' height='211.5' fill='#e69f68' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='672.0' x2='672.0' y1='91.7' y2='141.3'/><line x1='667.0' x2='677.0' y1='91.7' y2='91.7'/><line x1='667.0' x2='677.0' y1='141.3' y2='141.3'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(676.0 342.0) rotate(-35)'>(R)ReliefF</text><rect class='sa-hit' x='640.0' y='18' width='64' height='414'><title>(R)ReliefF (Distance, non-search)
|
| 50 |
Elo 1165 [95% CI 1131β1199]
|
| 51 |
Stability 0.612
|
| 52 |
+
Validity 0.594</title></rect><rect x='713.6' y='114.3' width='22.4' height='213.7' fill='#70b0d4' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='736.0' y='114.3' width='22.4' height='213.7' fill='#e2904f' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='736.0' x2='736.0' y1='85.8' y2='140.5'/><line x1='731.0' x2='741.0' y1='85.8' y2='85.8'/><line x1='731.0' x2='741.0' y1='140.5' y2='140.5'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(740.0 342.0) rotate(-35)'>F-Test</text><rect class='sa-hit' x='704.0' y='18' width='64' height='414'><title>F-Test (Correlation, non-search)
|
| 53 |
Elo 1168 [95% CI 1132β1207]
|
| 54 |
Stability 0.562
|
| 55 |
+
Validity 0.689</title></rect><rect x='777.6' y='112.1' width='22.4' height='215.9' fill='#69acd2' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='800.0' y='112.1' width='22.4' height='215.9' fill='#e59a5f' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='800.0' x2='800.0' y1='88.8' y2='134.7'/><line x1='795.0' x2='805.0' y1='88.8' y2='88.8'/><line x1='795.0' x2='805.0' y1='134.7' y2='134.7'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(804.0 342.0) rotate(-35)'>MI</text><rect class='sa-hit' x='768.0' y='18' width='64' height='414'><title>MI (Information theory, non-search)
|
| 56 |
Elo 1171 [95% CI 1140β1203]
|
| 57 |
Stability 0.587
|
| 58 |
+
Validity 0.627</title></rect><rect x='841.6' y='111.4' width='22.4' height='216.6' fill='#b0d4e7' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='864.0' y='111.4' width='22.4' height='216.6' fill='#e7a46e' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='864.0' x2='864.0' y1='87.3' y2='136.2'/><line x1='859.0' x2='869.0' y1='87.3' y2='87.3'/><line x1='859.0' x2='869.0' y1='136.2' y2='136.2'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(868.0 342.0) rotate(-35)'>ElasticNet</text><rect class='sa-hit' x='832.0' y='18' width='64' height='414'><title>ElasticNet (Regularization, non-search)
|
| 59 |
Elo 1172 [95% CI 1138β1205]
|
| 60 |
Stability 0.308
|
| 61 |
+
Validity 0.568</title></rect><rect x='905.6' y='99.0' width='22.4' height='229.0' fill='#acd1e6' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='928.0' y='99.0' width='22.4' height='229.0' fill='#ebb184' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='928.0' x2='928.0' y1='73.4' y2='126.7'/><line x1='923.0' x2='933.0' y1='73.4' y2='73.4'/><line x1='923.0' x2='933.0' y1='126.7' y2='126.7'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(932.0 342.0) rotate(-35)'>CART</text><rect class='sa-hit' x='896.0' y='18' width='64' height='414'><title>CART (Tree, non-search)
|
| 62 |
Elo 1189 [95% CI 1151β1224]
|
| 63 |
Stability 0.324
|
| 64 |
+
Validity 0.482</title></rect><rect x='969.6' y='54.5' width='22.4' height='273.5' fill='#67abd1' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='992.0' y='54.5' width='22.4' height='273.5' fill='#eaad7e' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='992.0' x2='992.0' y1='30.4' y2='77.1'/><line x1='987.0' x2='997.0' y1='30.4' y2='30.4'/><line x1='987.0' x2='997.0' y1='77.1' y2='77.1'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(996.0 342.0) rotate(-35)'>RFImportance</text><rect class='sa-hit' x='960.0' y='18' width='64' height='414'><title>RFImportance (Tree, non-search)
|
| 65 |
Elo 1250 [95% CI 1219β1283]
|
| 66 |
Stability 0.597
|
| 67 |
+
Validity 0.507</title></rect><line class='sa-ink' x1='64' x2='1024' y1='328.0' y2='328.0' stroke-width='1' opacity='0.4'/><rect x='1070' y='40.0' width='12' height='107.0' fill='url(#sa-g-stab)' stroke='#000' stroke-width='0.6'/><text class='sa-text' x='1076' y='30.0' text-anchor='middle' font-size='12'>stable</text><text class='sa-text' x='1076' y='165.0' text-anchor='middle' font-size='12'>unstable</text><text class='sa-muted' x='1088' y='151.0' font-size='11'>0.0</text><text class='sa-muted' x='1088' y='97.5' font-size='11'>0.5</text><text class='sa-muted' x='1088' y='44.0' font-size='11'>1.0</text><rect x='1070' y='207.0' width='12' height='107.0' fill='url(#sa-g-val)' stroke='#000' stroke-width='0.6'/><text class='sa-text' x='1076' y='197.0' text-anchor='middle' font-size='12'>valid</text><text class='sa-text' x='1076' y='332.0' text-anchor='middle' font-size='12'>invalid</text><text class='sa-muted' x='1088' y='318.0' font-size='11'>0.0</text><text class='sa-muted' x='1088' y='264.5' font-size='11'>0.5</text><text class='sa-muted' x='1088' y='211.0' font-size='11'>1.0</text><rect x='1070' y='352.0' width='12' height='12' fill='url(#sa-na)' stroke='#000' stroke-width='0.6'/><text class='sa-muted' x='1088' y='362.0' font-size='11'>no result</text></svg><script type='text/plain' id='sa-csv'>Method,Category,Subgroup,Elo,Elo CI low,Elo CI high,Stability,Validity
|
| 68 |
+
RFImportance,Non-search,Tree,1250.0,1219.0,1283.0,0.597,0.507
|
| 69 |
+
CART,Non-search,Tree,1189.0,1151.0,1224.0,0.324,0.482
|
| 70 |
+
ElasticNet,Non-search,Regularization,1172.0,1138.0,1205.0,0.308,0.568
|
| 71 |
+
MI,Non-search,Information theory,1171.0,1140.0,1203.0,0.587,0.627
|
| 72 |
+
F-Test,Non-search,Correlation,1168.0,1132.0,1207.0,0.562,0.689
|
| 73 |
+
(R)ReliefF,Non-search,Distance,1165.0,1131.0,1199.0,0.612,0.594
|
| 74 |
+
Lasso,Non-search,Regularization,1140.0,1104.0,1175.0,0.246,0.563
|
| 75 |
+
mRMR,Non-search,Information theory,1131.0,1097.0,1164.0,0.651,0.53
|
| 76 |
+
MarkovBlanket,Search,Backward search,1115.0,1077.0,1151.0,0.404,0.463
|
| 77 |
+
SBE,Search,Backward search,1113.0,1076.0,1152.0,0.059,0.517
|
| 78 |
+
LOCO,Search,Leave-one-out,1097.0,1065.0,1131.0,0.08,0.52
|
| 79 |
+
GainRatio,Non-search,Information theory,1097.0,1063.0,1134.0,0.55,0.659
|
| 80 |
+
SFS,Search,Forward search,1092.0,1060.0,1126.0,0.176,0.525
|
| 81 |
+
Random,Baseline,Baseline,1000.0,957.0,1039.0,0.002,0.5
|
| 82 |
+
LaplacianScore,Non-search,Distance,980.0,933.0,1028.0,0.678,0.701
|
| 83 |
</script><script>
|
| 84 |
function saDownload(name, blob) {
|
| 85 |
var a = document.createElement('a');
|
data/subsets/tasks_all/models_all/budgets_4/leaderboard_table.html
CHANGED
|
@@ -26,23 +26,25 @@ td.method { font-weight:600; }
|
|
| 26 |
td .sub { color:var(--muted); font-size:12px; font-weight:400; margin-left:6px; }
|
| 27 |
td.na { color:var(--muted); }
|
| 28 |
.pill { display:inline-block; padding:0 8px; border:1px solid var(--line); border-radius:999px;
|
| 29 |
-
font-size:12px; color:var(--
|
| 30 |
-
</style></head><body><div class='toolbar'><span class='hint'>Click a column to sort. Hover a header for its definition.</span><button onclick="saCsv('selectarena_leaderboard.csv')">CSV</button></div><div class='wrap'><table id='sa-table'><thead><tr><th class='num'>#</th><th class=''>Method</th><th class='' title='Filter: scores features from the data alone. Wrapper: searches feature subsets by refitting a proxy model. Embedded: reads the selection off a fitted model.'>Type</th><th class='num' title='Bradley-Terry Elo from pairwise matches within each dataset, budget and downstream model. Random = 1000. +/β give the 95% bootstrap interval.'>Elo β</th><th class='num' title='Mean rank of the downstream error.'>Avg. rank β</th><th class='num' title='Share of pairwise comparisons won, in %. Ties count half.'>Win rate β</th><th class='num' title='Mean relative gap (error β best error) / error to the best method, in %.'>Improvability β</th><th class='num' title='Mean ratio of the error to the same model trained on all features. Below 1: the selection helped.'>Error vs. all β</th><th class='num' title='Agreement of the selected features across bootstrap resamples (Nogueira et al., 2018).'>Stability β</th><th class='num' title='Share of the budget filled with original features when as many noise features are added.'>Validity β</th><th class='num' title='Median wall-clock time of the selection.'>Time (s) β</th><th class='num' title='Share of runs where the method ran out of time and random features filled the budget.'>Fallback (%)</th><th class='num'>#Datasets</th></tr></thead><tbody><tr><td class='num' data-sort='1'>1</td><td class='method' data-sort='RFImportance'>RFImportance</td><td class='' data-sort='Embedded'><span class='pill'>π³ Embedded</span></td><td class='num' data-sort='1250.0'>1250<span class='sub'>+33 / β31</span></td><td class='num' data-sort='6.2'>6.20</td><td class='num' data-sort='62.9'>62.9</td><td class='num' data-sort='10.7'>10.7</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.597'>0.597</td><td class='num' data-sort='0.507'>0.507</td><td class='num' data-sort='0.2'>0.2</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='2'>2</td><td class='method' data-sort='CART'>CART</td><td class='' data-sort='Embedded'><span class='pill'>π³ Embedded</span></td><td class='num' data-sort='1189.0'>1189<span class='sub'>+35 / β38</span></td><td class='num' data-sort='6.97'>6.97</td><td class='num' data-sort='57.4'>57.4</td><td class='num' data-sort='12.6'>12.6</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.324'>0.324</td><td class='num' data-sort='0.482'>0.482</td><td class='num' data-sort='0.1'>0.1</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='3'>3</td><td class='method' data-sort='ElasticNet'>ElasticNet</td><td class='' data-sort='Embedded'><span class='pill'>π³ Embedded</span></td><td class='num' data-sort='1172.0'>1172<span class='sub'>+33 / β34</span></td><td class='num' data-sort='7.21'>7.21</td><td class='num' data-sort='55.6'>55.6</td><td class='num' data-sort='12.4'>12.4</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.308'>0.308</td><td class='num' data-sort='0.568'>0.568</td><td class='num' data-sort='7.7'>7.7</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='4'>4</td><td class='method' data-sort='MI'>MI</td><td class='' data-sort='Filter'><span class='pill'>π Filter</span></td><td class='num' data-sort='1171.0'>1171<span class='sub'>+32 / β31</span></td><td class='num' data-sort='7.24'>7.24</td><td class='num' data-sort='55.4'>55.4</td><td class='num' data-sort='12.6'>12.6</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.587'>0.587</td><td class='num' data-sort='0.627'>0.627</td><td class='num' data-sort='0.4'>0.4</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='5'>5</td><td class='method' data-sort='F-Test'>F-Test</td><td class='' data-sort='Filter'><span class='pill'>π Filter</span></td><td class='num' data-sort='1168.0'>1168<span class='sub'>+39 / β36</span></td><td class='num' data-sort='7.39'>7.39</td><td class='num' data-sort='54.3'>54.3</td><td class='num' data-sort='11.8'>11.8</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.562'>0.562</td><td class='num' data-sort='0.689'>0.689</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='6'>6</td><td class='method' data-sort='(R)ReliefF'>(R)ReliefF</td><td class='' data-sort='Filter'><span class='pill'>π Filter</span></td><td class='num' data-sort='1165.0'>1165<span class='sub'>+34 / β34</span></td><td class='num' data-sort='7.67'>7.67</td><td class='num' data-sort='52.3'>52.3</td><td class='num' data-sort='13.0'>13.0</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.612'>0.612</td><td class='num' data-sort='0.594'>0.594</td><td class='num' data-sort='14.4'>14.4</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='7'>7</td><td class='method' data-sort='Lasso'>Lasso</td><td class='' data-sort='Embedded'><span class='pill'>π³ Embedded</span></td><td class='num' data-sort='1140.0'>1140<span class='sub'>+35 / β36</span></td><td class='num' data-sort='7.56'>7.56</td><td class='num' data-sort='53.1'>53.1</td><td class='num' data-sort='13.1'>13.1</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.246'>0.246</td><td class='num' data-sort='0.563'>0.563</td><td class='num' data-sort='2.0'>2.0</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='8'>8</td><td class='method' data-sort='mRMR'>mRMR</td><td class='' data-sort='Filter'><span class='pill'>π Filter</span></td><td class='num' data-sort='1131.0'>1131<span class='sub'>+33 / β34</span></td><td class='num' data-sort='7.89'>7.89</td><td class='num' data-sort='50.8'>50.8</td><td class='num' data-sort='12.5'>12.5</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.651'>0.651</td><td class='num' data-sort='0.53'>0.530</td><td class='num' data-sort='0.6'>0.6</td><td class='num' data-sort='9.9'>9.9</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='9'>9</td><td class='method' data-sort='MarkovBlanket'>MarkovBlanket</td><td class='' data-sort='Filter'><span class='pill'>π Filter</span></td><td class='num' data-sort='1115.0'>1115<span class='sub'>+36 / β38</span></td><td class='num' data-sort='8.26'>8.26</td><td class='num' data-sort='48.1'>48.1</td><td class='num' data-sort='14.0'>14.0</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.404'>0.404</td><td class='num' data-sort='0.463'>0.463</td><td class='num' data-sort='0.1'>0.1</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='10'>10</td><td class='method' data-sort='SBE'>SBE</td><td class='' data-sort='Wrapper'><span class='pill'>π Wrapper</span></td><td class='num' data-sort='1113.0'>1113<span class='sub'>+39 / β37</span></td><td class='num' data-sort='8.31'>8.31</td><td class='num' data-sort='47.8'>47.8</td><td class='num' data-sort='15.3'>15.3</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.059'>0.059</td><td class='num' data-sort='0.517'>0.517</td><td class='num' data-sort='989.9'>989.9</td><td class='num' data-sort='41.5'>41.5</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='11'>11</td><td class='method' data-sort='LOCO'>LOCO</td><td class='' data-sort='Wrapper'><span class='pill'>π Wrapper</span></td><td class='num' data-sort='1097.0'>1097<span class='sub'>+34 / β32</span></td><td class='num' data-sort='8.26'>8.26</td><td class='num' data-sort='48.2'>48.2</td><td class='num' data-sort='15.5'>15.5</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.08'>0.080</td><td class='num' data-sort='0.52'>0.520</td><td class='num' data-sort='243.3'>243.3</td><td class='num' data-sort='18.7'>18.7</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='12'>12</td><td class='method' data-sort='GainRatio'>GainRatio</td><td class='' data-sort='Filter'><span class='pill'>π Filter</span></td><td class='num' data-sort='1097.0'>1097<span class='sub'>+37 / β34</span></td><td class='num' data-sort='8.5'>8.50</td><td class='num' data-sort='46.4'>46.4</td><td class='num' data-sort='14.9'>14.9</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.55'>0.550</td><td class='num' data-sort='0.659'>0.659</td><td class='num' data-sort='0.2'>0.2</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='13'>13</td><td class='method' data-sort='SFS'>SFS</td><td class='' data-sort='Wrapper'><span class='pill'>π Wrapper</span></td><td class='num' data-sort='1092.0'>1092<span class='sub'>+34 / β32</span></td><td class='num' data-sort='8.46'>8.46</td><td class='num' data-sort='46.7'>46.7</td><td class='num' data-sort='15.4'>15.4</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.176'>0.176</td><td class='num' data-sort='0.525'>0.525</td><td class='num' data-sort='2953.5'>2,953.5</td><td class='num' data-sort='48.8'>48.8</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='14'>14</td><td class='method' data-sort='Random'>Random</td><td class='' data-sort='Baseline'><span class='pill'>βͺ Baseline</span></td><td class='num' data-sort='1000.0'>1000<span class='sub'>+39 / β43</span></td><td class='num' data-sort='9.67'>9.67</td><td class='num' data-sort='38.1'>38.1</td><td class='num' data-sort='18.9'>18.9</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.002'>0.002</td><td class='num' data-sort='0.5'>0.500</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='100.0'>100.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='15'>15</td><td class='method' data-sort='LaplacianScore'>LaplacianScore</td><td class='' data-sort='Filter'><span class='pill'>π Filter</span></td><td class='num' data-sort='980.0'>980<span class='sub'>+48 / β47</span></td><td class='num' data-sort='10.39'>10.39</td><td class='num' data-sort='32.9'>32.9</td><td class='num' data-sort='23.1'>23.1</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.678'>0.678</td><td class='num' data-sort='0.701'>0.701</td><td class='num' data-sort='0.2'>0.2</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr></tbody></table></div><script type='text/plain' id='sa-csv'>Rank,Type,TypeName,Method,Elo,Elo CI low,Elo CI high,Avg. rank,Win rate (%),Improvability (%),Error vs. all features,Stability,Validity,Selection time (s),Fallback rate (%),#Datasets,#Comparisons
|
| 31 |
-
|
| 32 |
-
2,π³,Embedded,CART,1189.0,1151.0,1224.0,6.97,57.4,12.6,,0.324,0.482,0.1,0.0,71,1065
|
| 33 |
-
|
| 34 |
-
|
| 35 |
-
|
| 36 |
-
|
| 37 |
-
|
| 38 |
-
|
| 39 |
-
|
| 40 |
-
|
| 41 |
-
|
| 42 |
-
|
| 43 |
-
|
| 44 |
-
|
| 45 |
-
|
|
|
|
|
|
|
| 46 |
</script><script>
|
| 47 |
function saDownload(name, blob) {
|
| 48 |
var a = document.createElement('a');
|
|
|
|
| 26 |
td .sub { color:var(--muted); font-size:12px; font-weight:400; margin-left:6px; }
|
| 27 |
td.na { color:var(--muted); }
|
| 28 |
.pill { display:inline-block; padding:0 8px; border:1px solid var(--line); border-radius:999px;
|
| 29 |
+
font-size:12px; color:var(--fg); }
|
| 30 |
+
.pill.cat-search { border-color:var(--accent); }
|
| 31 |
+
.pill.cat-baseline { color:var(--muted); }
|
| 32 |
+
</style></head><body><div class='toolbar'><span class='hint'>Click a column to sort. Hover a header for its definition.</span><button onclick="saCsv('selectarena_leaderboard.csv')">CSV</button></div><div class='wrap'><table id='sa-table'><thead><tr><th class='num'>#</th><th class=''>Method</th><th class='' title='Taxonomy of the paper (Appendix A). Search methods evaluate many feature subsets with the model (forward, backward, leave-one-out); non-search methods use a proxy instead (correlation, distance, information theory, tree, regularization).'>Type</th><th class='num' title='Bradley-Terry Elo from pairwise matches within each dataset, budget and downstream model. Random = 1000. +/β give the 95% bootstrap interval.'>Elo β</th><th class='num' title='Mean rank of the downstream error.'>Avg. rank β</th><th class='num' title='Share of pairwise comparisons won, in %. Ties count half.'>Win rate β</th><th class='num' title='Mean relative gap (error β best error) / error to the best method, in %.'>Improvability β</th><th class='num' title='Mean ratio of the error to the same model trained on all features. Below 1: the selection helped.'>Error vs. all β</th><th class='num' title='Agreement of the selected features across bootstrap resamples (Nogueira et al., 2018).'>Stability β</th><th class='num' title='Share of the budget filled with original features when as many noise features are added.'>Validity β</th><th class='num' title='Median wall-clock time of the selection.'>Time (s) β</th><th class='num' title='Share of runs where the method ran out of time and random features filled the budget.'>Fallback (%)</th><th class='num'>#Datasets</th></tr></thead><tbody><tr><td class='num' data-sort='1'>1</td><td class='method' data-sort='RFImportance'>RFImportance</td><td class='' data-sort='Tree'><span class='pill cat-non-search'>Tree</span><span class='sub'>non-search</span></td><td class='num' data-sort='1250.0'>1250<span class='sub'>+33 / β31</span></td><td class='num' data-sort='6.2'>6.20</td><td class='num' data-sort='62.9'>62.9</td><td class='num' data-sort='10.7'>10.7</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.597'>0.597</td><td class='num' data-sort='0.507'>0.507</td><td class='num' data-sort='0.2'>0.2</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='2'>2</td><td class='method' data-sort='CART'>CART</td><td class='' data-sort='Tree'><span class='pill cat-non-search'>Tree</span><span class='sub'>non-search</span></td><td class='num' data-sort='1189.0'>1189<span class='sub'>+35 / β38</span></td><td class='num' data-sort='6.97'>6.97</td><td class='num' data-sort='57.4'>57.4</td><td class='num' data-sort='12.6'>12.6</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.324'>0.324</td><td class='num' data-sort='0.482'>0.482</td><td class='num' data-sort='0.1'>0.1</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='3'>3</td><td class='method' data-sort='ElasticNet'>ElasticNet</td><td class='' data-sort='Regularization'><span class='pill cat-non-search'>Regularization</span><span class='sub'>non-search</span></td><td class='num' data-sort='1172.0'>1172<span class='sub'>+33 / β34</span></td><td class='num' data-sort='7.21'>7.21</td><td class='num' data-sort='55.6'>55.6</td><td class='num' data-sort='12.4'>12.4</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.308'>0.308</td><td class='num' data-sort='0.568'>0.568</td><td class='num' data-sort='7.7'>7.7</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='4'>4</td><td class='method' data-sort='MI'>MI</td><td class='' data-sort='Information theory'><span class='pill cat-non-search'>Information theory</span><span class='sub'>non-search</span></td><td class='num' data-sort='1171.0'>1171<span class='sub'>+32 / β31</span></td><td class='num' data-sort='7.24'>7.24</td><td class='num' data-sort='55.4'>55.4</td><td class='num' data-sort='12.6'>12.6</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.587'>0.587</td><td class='num' data-sort='0.627'>0.627</td><td class='num' data-sort='0.4'>0.4</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='5'>5</td><td class='method' data-sort='F-Test'>F-Test</td><td class='' data-sort='Correlation'><span class='pill cat-non-search'>Correlation</span><span class='sub'>non-search</span></td><td class='num' data-sort='1168.0'>1168<span class='sub'>+39 / β36</span></td><td class='num' data-sort='7.39'>7.39</td><td class='num' data-sort='54.3'>54.3</td><td class='num' data-sort='11.8'>11.8</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.562'>0.562</td><td class='num' data-sort='0.689'>0.689</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='6'>6</td><td class='method' data-sort='(R)ReliefF'>(R)ReliefF</td><td class='' data-sort='Distance'><span class='pill cat-non-search'>Distance</span><span class='sub'>non-search</span></td><td class='num' data-sort='1165.0'>1165<span class='sub'>+34 / β34</span></td><td class='num' data-sort='7.67'>7.67</td><td class='num' data-sort='52.3'>52.3</td><td class='num' data-sort='13.0'>13.0</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.612'>0.612</td><td class='num' data-sort='0.594'>0.594</td><td class='num' data-sort='14.4'>14.4</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='7'>7</td><td class='method' data-sort='Lasso'>Lasso</td><td class='' data-sort='Regularization'><span class='pill cat-non-search'>Regularization</span><span class='sub'>non-search</span></td><td class='num' data-sort='1140.0'>1140<span class='sub'>+35 / β36</span></td><td class='num' data-sort='7.56'>7.56</td><td class='num' data-sort='53.1'>53.1</td><td class='num' data-sort='13.1'>13.1</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.246'>0.246</td><td class='num' data-sort='0.563'>0.563</td><td class='num' data-sort='2.0'>2.0</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='8'>8</td><td class='method' data-sort='mRMR'>mRMR</td><td class='' data-sort='Information theory'><span class='pill cat-non-search'>Information theory</span><span class='sub'>non-search</span></td><td class='num' data-sort='1131.0'>1131<span class='sub'>+33 / β34</span></td><td class='num' data-sort='7.89'>7.89</td><td class='num' data-sort='50.8'>50.8</td><td class='num' data-sort='12.5'>12.5</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.651'>0.651</td><td class='num' data-sort='0.53'>0.530</td><td class='num' data-sort='0.6'>0.6</td><td class='num' data-sort='9.9'>9.9</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='9'>9</td><td class='method' data-sort='MarkovBlanket'>MarkovBlanket</td><td class='' data-sort='Backward search'><span class='pill cat-search'>Backward search</span><span class='sub'>search</span></td><td class='num' data-sort='1115.0'>1115<span class='sub'>+36 / β38</span></td><td class='num' data-sort='8.26'>8.26</td><td class='num' data-sort='48.1'>48.1</td><td class='num' data-sort='14.0'>14.0</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.404'>0.404</td><td class='num' data-sort='0.463'>0.463</td><td class='num' data-sort='0.1'>0.1</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='10'>10</td><td class='method' data-sort='SBE'>SBE</td><td class='' data-sort='Backward search'><span class='pill cat-search'>Backward search</span><span class='sub'>search</span></td><td class='num' data-sort='1113.0'>1113<span class='sub'>+39 / β37</span></td><td class='num' data-sort='8.31'>8.31</td><td class='num' data-sort='47.8'>47.8</td><td class='num' data-sort='15.3'>15.3</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.059'>0.059</td><td class='num' data-sort='0.517'>0.517</td><td class='num' data-sort='989.9'>989.9</td><td class='num' data-sort='41.5'>41.5</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='11'>11</td><td class='method' data-sort='LOCO'>LOCO</td><td class='' data-sort='Leave-one-out'><span class='pill cat-search'>Leave-one-out</span><span class='sub'>search</span></td><td class='num' data-sort='1097.0'>1097<span class='sub'>+34 / β32</span></td><td class='num' data-sort='8.26'>8.26</td><td class='num' data-sort='48.2'>48.2</td><td class='num' data-sort='15.5'>15.5</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.08'>0.080</td><td class='num' data-sort='0.52'>0.520</td><td class='num' data-sort='243.3'>243.3</td><td class='num' data-sort='18.7'>18.7</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='12'>12</td><td class='method' data-sort='GainRatio'>GainRatio</td><td class='' data-sort='Information theory'><span class='pill cat-non-search'>Information theory</span><span class='sub'>non-search</span></td><td class='num' data-sort='1097.0'>1097<span class='sub'>+37 / β34</span></td><td class='num' data-sort='8.5'>8.50</td><td class='num' data-sort='46.4'>46.4</td><td class='num' data-sort='14.9'>14.9</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.55'>0.550</td><td class='num' data-sort='0.659'>0.659</td><td class='num' data-sort='0.2'>0.2</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='13'>13</td><td class='method' data-sort='SFS'>SFS</td><td class='' data-sort='Forward search'><span class='pill cat-search'>Forward search</span><span class='sub'>search</span></td><td class='num' data-sort='1092.0'>1092<span class='sub'>+34 / β32</span></td><td class='num' data-sort='8.46'>8.46</td><td class='num' data-sort='46.7'>46.7</td><td class='num' data-sort='15.4'>15.4</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.176'>0.176</td><td class='num' data-sort='0.525'>0.525</td><td class='num' data-sort='2953.5'>2,953.5</td><td class='num' data-sort='48.8'>48.8</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='14'>14</td><td class='method' data-sort='Random'>Random</td><td class='' data-sort='Baseline'><span class='pill cat-baseline'>Baseline</span></td><td class='num' data-sort='1000.0'>1000<span class='sub'>+39 / β43</span></td><td class='num' data-sort='9.67'>9.67</td><td class='num' data-sort='38.1'>38.1</td><td class='num' data-sort='18.9'>18.9</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.002'>0.002</td><td class='num' data-sort='0.5'>0.500</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='100.0'>100.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='15'>15</td><td class='method' data-sort='LaplacianScore'>LaplacianScore</td><td class='' data-sort='Distance'><span class='pill cat-non-search'>Distance</span><span class='sub'>non-search</span></td><td class='num' data-sort='980.0'>980<span class='sub'>+48 / β47</span></td><td class='num' data-sort='10.39'>10.39</td><td class='num' data-sort='32.9'>32.9</td><td class='num' data-sort='23.1'>23.1</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.678'>0.678</td><td class='num' data-sort='0.701'>0.701</td><td class='num' data-sort='0.2'>0.2</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr></tbody></table></div><script type='text/plain' id='sa-csv'>Rank,Method,Category,Subgroup,Elo,Elo CI low,Elo CI high,Avg. rank,Win rate (%),Improvability (%),Error vs. all features,Stability,Validity,Selection time (s),Fallback rate (%),#Datasets,#Comparisons
|
| 33 |
+
1,RFImportance,Non-search,Tree,1250.0,1219.0,1283.0,6.2,62.9,10.7,,0.597,0.507,0.2,0.0,71,1065
|
| 34 |
+
2,CART,Non-search,Tree,1189.0,1151.0,1224.0,6.97,57.4,12.6,,0.324,0.482,0.1,0.0,71,1065
|
| 35 |
+
3,ElasticNet,Non-search,Regularization,1172.0,1138.0,1205.0,7.21,55.6,12.4,,0.308,0.568,7.7,0.0,71,1065
|
| 36 |
+
4,MI,Non-search,Information theory,1171.0,1140.0,1203.0,7.24,55.4,12.6,,0.587,0.627,0.4,0.0,71,1065
|
| 37 |
+
5,F-Test,Non-search,Correlation,1168.0,1132.0,1207.0,7.39,54.3,11.8,,0.562,0.689,0.0,0.0,71,1065
|
| 38 |
+
6,(R)ReliefF,Non-search,Distance,1165.0,1131.0,1199.0,7.67,52.3,13.0,,0.612,0.594,14.4,0.0,71,1065
|
| 39 |
+
7,Lasso,Non-search,Regularization,1140.0,1104.0,1175.0,7.56,53.1,13.1,,0.246,0.563,2.0,0.0,71,1065
|
| 40 |
+
8,mRMR,Non-search,Information theory,1131.0,1097.0,1164.0,7.89,50.8,12.5,,0.651,0.53,0.6,9.9,71,1065
|
| 41 |
+
9,MarkovBlanket,Search,Backward search,1115.0,1077.0,1151.0,8.26,48.1,14.0,,0.404,0.463,0.1,0.0,71,1065
|
| 42 |
+
10,SBE,Search,Backward search,1113.0,1076.0,1152.0,8.31,47.8,15.3,,0.059,0.517,989.9,41.5,71,1064
|
| 43 |
+
11,LOCO,Search,Leave-one-out,1097.0,1065.0,1131.0,8.26,48.2,15.5,,0.08,0.52,243.3,18.7,71,1065
|
| 44 |
+
12,GainRatio,Non-search,Information theory,1097.0,1063.0,1134.0,8.5,46.4,14.9,,0.55,0.659,0.2,0.0,71,1065
|
| 45 |
+
13,SFS,Search,Forward search,1092.0,1060.0,1126.0,8.46,46.7,15.4,,0.176,0.525,2953.5,48.8,71,1064
|
| 46 |
+
14,Random,Baseline,Baseline,1000.0,957.0,1039.0,9.67,38.1,18.9,,0.002,0.5,0.0,100.0,71,1065
|
| 47 |
+
15,LaplacianScore,Non-search,Distance,980.0,933.0,1028.0,10.39,32.9,23.1,,0.678,0.701,0.2,0.0,71,1065
|
| 48 |
</script><script>
|
| 49 |
function saDownload(name, blob) {
|
| 50 |
var a = document.createElement('a');
|
data/subsets/tasks_all/models_all/budgets_4/website_leaderboard.csv
CHANGED
|
@@ -1,16 +1,16 @@
|
|
| 1 |
-
Rank,
|
| 2 |
-
1,
|
| 3 |
-
2,
|
| 4 |
-
3,
|
| 5 |
-
4,
|
| 6 |
-
5,
|
| 7 |
-
6,
|
| 8 |
-
7,
|
| 9 |
-
8,
|
| 10 |
-
9,
|
| 11 |
-
10,
|
| 12 |
-
11,
|
| 13 |
-
12,
|
| 14 |
-
13,
|
| 15 |
-
14,
|
| 16 |
-
15,
|
|
|
|
| 1 |
+
Rank,Method,Category,Subgroup,Elo,Elo CI low,Elo CI high,Avg. rank,Win rate (%),Improvability (%),Error vs. all features,Stability,Validity,Selection time (s),Fallback rate (%),#Datasets,#Comparisons
|
| 2 |
+
1,RFImportance,Non-search,Tree,1250.0,1219.0,1283.0,6.2,62.9,10.7,,0.597,0.507,0.2,0.0,71,1065
|
| 3 |
+
2,CART,Non-search,Tree,1189.0,1151.0,1224.0,6.97,57.4,12.6,,0.324,0.482,0.1,0.0,71,1065
|
| 4 |
+
3,ElasticNet,Non-search,Regularization,1172.0,1138.0,1205.0,7.21,55.6,12.4,,0.308,0.568,7.7,0.0,71,1065
|
| 5 |
+
4,MI,Non-search,Information theory,1171.0,1140.0,1203.0,7.24,55.4,12.6,,0.587,0.627,0.4,0.0,71,1065
|
| 6 |
+
5,F-Test,Non-search,Correlation,1168.0,1132.0,1207.0,7.39,54.3,11.8,,0.562,0.689,0.0,0.0,71,1065
|
| 7 |
+
6,(R)ReliefF,Non-search,Distance,1165.0,1131.0,1199.0,7.67,52.3,13.0,,0.612,0.594,14.4,0.0,71,1065
|
| 8 |
+
7,Lasso,Non-search,Regularization,1140.0,1104.0,1175.0,7.56,53.1,13.1,,0.246,0.563,2.0,0.0,71,1065
|
| 9 |
+
8,mRMR,Non-search,Information theory,1131.0,1097.0,1164.0,7.89,50.8,12.5,,0.651,0.53,0.6,9.9,71,1065
|
| 10 |
+
9,MarkovBlanket,Search,Backward search,1115.0,1077.0,1151.0,8.26,48.1,14.0,,0.404,0.463,0.1,0.0,71,1065
|
| 11 |
+
10,SBE,Search,Backward search,1113.0,1076.0,1152.0,8.31,47.8,15.3,,0.059,0.517,989.9,41.5,71,1064
|
| 12 |
+
11,LOCO,Search,Leave-one-out,1097.0,1065.0,1131.0,8.26,48.2,15.5,,0.08,0.52,243.3,18.7,71,1065
|
| 13 |
+
12,GainRatio,Non-search,Information theory,1097.0,1063.0,1134.0,8.5,46.4,14.9,,0.55,0.659,0.2,0.0,71,1065
|
| 14 |
+
13,SFS,Search,Forward search,1092.0,1060.0,1126.0,8.46,46.7,15.4,,0.176,0.525,2953.5,48.8,71,1064
|
| 15 |
+
14,Random,Baseline,Baseline,1000.0,957.0,1039.0,9.67,38.1,18.9,,0.002,0.5,0.0,100.0,71,1065
|
| 16 |
+
15,LaplacianScore,Non-search,Distance,980.0,933.0,1028.0,10.39,32.9,23.1,,0.678,0.701,0.2,0.0,71,1065
|
data/subsets/tasks_all/models_all/budgets_all/elo_explorer.html
CHANGED
|
@@ -19,67 +19,67 @@ svg { display:block; width:100%; height:auto; }
|
|
| 19 |
.sa-ink { stroke:var(--fg); }
|
| 20 |
.sa-hit { fill:transparent; }
|
| 21 |
.sa-hit:hover { fill:var(--fg); fill-opacity:0.06; }
|
| 22 |
-
</style><defs><pattern id='sa-na' width='6' height='6' patternUnits='userSpaceOnUse' patternTransform='rotate(45)'><rect width='6' height='6' fill='#e5e7eb'/><line x1='0' y1='0' x2='0' y2='6' stroke='#9ca3af' stroke-width='2'/></pattern><linearGradient id='sa-g-stab' x1='0' y1='1' x2='0' y2='0'><stop offset='0' stop-color='#ffffff'/><stop offset='1' stop-color='#0072B2'/></linearGradient><linearGradient id='sa-g-val' x1='0' y1='1' x2='0' y2='0'><stop offset='0' stop-color='#ffffff'/><stop offset='1' stop-color='#D55E00'/></linearGradient></defs><line class='sa-grid' x1='64' x2='1024' y1='313.2' y2='313.2' stroke-width='1'/><text class='sa-muted' x='56' y='317.2' text-anchor='end' font-size='12'>900</text><line class='sa-grid' x1='64' x2='1024' y1='254.2' y2='254.2' stroke-width='1'/><text class='sa-muted' x='56' y='258.2' text-anchor='end' font-size='12'>1000</text><line class='sa-grid' x1='64' x2='1024' y1='195.1' y2='195.1' stroke-width='1'/><text class='sa-muted' x='56' y='199.1' text-anchor='end' font-size='12'>1100</text><line class='sa-grid' x1='64' x2='1024' y1='136.1' y2='136.1' stroke-width='1'/><text class='sa-muted' x='56' y='140.1' text-anchor='end' font-size='12'>1200</text><line class='sa-grid' x1='64' x2='1024' y1='77.0' y2='77.0' stroke-width='1'/><text class='sa-muted' x='56' y='81.0' text-anchor='end' font-size='12'>1300</text><line class='sa-grid' x1='64' x2='1024' y1='18.0' y2='18.0' stroke-width='1'/><text class='sa-muted' x='56' y='22.0' text-anchor='end' font-size='12'>1400</text><text class='sa-text' transform='translate(16 173) rotate(-90)' text-anchor='middle' font-size='13'>Elo</text><line class='sa-ink' x1='64' x2='1024' y1='254.2' y2='254.2' stroke-width='1' stroke-dasharray='4 4' opacity='0.5'/><rect x='73.6' y='274.9' width='22.4' height='53.1' fill='#59a3cd' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='96.0' y='274.9' width='22.4' height='53.1' fill='#de7f34' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='96.0' x2='96.0' y1='246.5' y2='305.6'/><line x1='91.0' x2='101.0' y1='246.5' y2='246.5'/><line x1='91.0' x2='101.0' y1='305.6' y2='305.6'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(100.0 342.0) rotate(-35)'>LaplacianScore</text><rect class='sa-hit' x='64.0' y='18' width='64' height='414'><title>LaplacianScore (
|
| 23 |
Elo 965 [95% CI 913β1013]
|
| 24 |
Stability 0.652
|
| 25 |
-
Validity 0.797</title></rect><rect x='137.6' y='254.2' width='22.4' height='73.8' fill='#ffffff' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='160.0' y='254.2' width='22.4' height='73.8' fill='#eaae80' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='160.0' x2='160.0' y1='236.5' y2='273.7'/><line x1='155.0' x2='165.0' y1='236.5' y2='236.5'/><line x1='155.0' x2='165.0' y1='273.7' y2='273.7'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(164.0 342.0) rotate(-35)'>Random</text><rect class='sa-hit' x='128.0' y='18' width='64' height='414'><title>Random (Baseline)
|
| 26 |
Elo 1000 [95% CI 967β1030]
|
| 27 |
Stability 0.001
|
| 28 |
-
Validity 0.500</title></rect><rect x='201.6' y='165.0' width='22.4' height='163.0' fill='#d3e7f2' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='224.0' y='165.0' width='22.4' height='163.0' fill='#e7a16a' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='224.0' x2='224.0' y1='146.1' y2='183.3'/><line x1='219.0' x2='229.0' y1='146.1' y2='146.1'/><line x1='219.0' x2='229.0' y1='183.3' y2='183.3'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(228.0 342.0) rotate(-35)'>LOCO</text><rect class='sa-hit' x='192.0' y='18' width='64' height='414'><title>LOCO (
|
| 29 |
Elo 1151 [95% CI 1120β1183]
|
| 30 |
Stability 0.173
|
| 31 |
-
Validity 0.583</title></rect><rect x='265.6' y='154.4' width='22.4' height='173.6' fill='#ddecf5' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='288.0' y='154.4' width='22.4' height='173.6' fill='#e7a570' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='288.0' x2='288.0' y1='129.6' y2='175.7'/><line x1='283.0' x2='293.0' y1='129.6' y2='129.6'/><line x1='283.0' x2='293.0' y1='175.7' y2='175.7'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(292.0 342.0) rotate(-35)'>SBE</text><rect class='sa-hit' x='256.0' y='18' width='64' height='414'><title>SBE (
|
| 32 |
Elo 1169 [95% CI 1133β1211]
|
| 33 |
Stability 0.132
|
| 34 |
-
Validity 0.560</title></rect><rect x='329.6' y='149.1' width='22.4' height='178.9' fill='#bbd9ea' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='352.0' y='149.1' width='22.4' height='178.9' fill='#e59a5f' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='352.0' x2='352.0' y1='129.6' y2='164.4'/><line x1='347.0' x2='357.0' y1='129.6' y2='129.6'/><line x1='347.0' x2='357.0' y1='164.4' y2='164.4'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(356.0 342.0) rotate(-35)'>SFS</text><rect class='sa-hit' x='320.0' y='18' width='64' height='414'><title>SFS (
|
| 35 |
Elo 1178 [95% CI 1152β1211]
|
| 36 |
Stability 0.267
|
| 37 |
-
Validity 0.627</title></rect><rect x='393.6' y='146.1' width='22.4' height='181.9' fill='#5ba4cd' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='416.0' y='146.1' width='22.4' height='181.9' fill='#dc7a2d' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='416.0' x2='416.0' y1='129.0' y2='161.5'/><line x1='411.0' x2='421.0' y1='129.0' y2='129.0'/><line x1='411.0' x2='421.0' y1='161.5' y2='161.5'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(420.0 342.0) rotate(-35)'>GainRatio</text><rect class='sa-hit' x='384.0' y='18' width='64' height='414'><title>GainRatio (
|
| 38 |
Elo 1183 [95% CI 1157β1212]
|
| 39 |
Stability 0.644
|
| 40 |
-
Validity 0.825</title></rect><rect x='457.6' y='140.2' width='22.4' height='187.8' fill='#65aad0' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='480.0' y='140.2' width='22.4' height='187.8' fill='#e39658' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='480.0' x2='480.0' y1='123.7' y2='156.8'/><line x1='475.0' x2='485.0' y1='123.7' y2='123.7'/><line x1='475.0' x2='485.0' y1='156.8' y2='156.8'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(484.0 342.0) rotate(-35)'>mRMR</text><rect class='sa-hit' x='448.0' y='18' width='64' height='414'><title>mRMR (
|
| 41 |
Elo 1193 [95% CI 1165β1221]
|
| 42 |
Stability 0.604
|
| 43 |
-
Validity 0.655</title></rect><rect x='521.6' y='137.3' width='22.4' height='190.7' fill='#5aa4cd' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='544.0' y='137.3' width='22.4' height='190.7' fill='#e08842' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='544.0' x2='544.0' y1='119.6' y2='155.0'/><line x1='539.0' x2='549.0' y1='119.6' y2='119.6'/><line x1='539.0' x2='549.0' y1='155.0' y2='155.0'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(548.0 342.0) rotate(-35)'>(R)ReliefF</text><rect class='sa-hit' x='512.0' y='18' width='64' height='414'><title>(R)ReliefF (
|
| 44 |
Elo 1198 [95% CI 1168β1228]
|
| 45 |
Stability 0.646
|
| 46 |
-
Validity 0.740</title></rect><rect x='585.6' y='124.3' width='22.4' height='203.7' fill='#96c5df' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='608.0' y='124.3' width='22.4' height='203.7' fill='#ecb489' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='608.0' x2='608.0' y1='109.5' y2='141.4'/><line x1='603.0' x2='613.0' y1='109.5' y2='109.5'/><line x1='603.0' x2='613.0' y1='141.4' y2='141.4'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(612.0 342.0) rotate(-35)'>MarkovBlanket</text><rect class='sa-hit' x='576.0' y='18' width='64' height='414'><title>MarkovBlanket (
|
| 47 |
Elo 1220 [95% CI 1191β1245]
|
| 48 |
Stability 0.413
|
| 49 |
-
Validity 0.463</title></rect><rect x='649.6' y='120.7' width='22.4' height='207.3' fill='#a3cce3' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='672.0' y='120.7' width='22.4' height='207.3' fill='#e08945' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='672.0' x2='672.0' y1='102.4' y2='138.5'/><line x1='667.0' x2='677.0' y1='102.4' y2='102.4'/><line x1='667.0' x2='677.0' y1='138.5' y2='138.5'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(676.0 342.0) rotate(-35)'>Lasso</text><rect class='sa-hit' x='640.0' y='18' width='64' height='414'><title>Lasso (
|
| 50 |
Elo 1226 [95% CI 1196β1257]
|
| 51 |
Stability 0.362
|
| 52 |
-
Validity 0.730</title></rect><rect x='713.6' y='119.0' width='22.4' height='209.0' fill='#529fcb' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='736.0' y='119.0' width='22.4' height='209.0' fill='#db7727' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='736.0' x2='736.0' y1='103.0' y2='134.3'/><line x1='731.0' x2='741.0' y1='103.0' y2='103.0'/><line x1='731.0' x2='741.0' y1='134.3' y2='134.3'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(740.0 342.0) rotate(-35)'>F-Test</text><rect class='sa-hit' x='704.0' y='18' width='64' height='414'><title>F-Test (
|
| 53 |
Elo 1229 [95% CI 1203β1256]
|
| 54 |
Stability 0.679
|
| 55 |
-
Validity 0.846</title></rect><rect x='777.6' y='107.8' width='22.4' height='220.2' fill='#5aa4cd' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='800.0' y='107.8' width='22.4' height='220.2' fill='#de8137' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='800.0' x2='800.0' y1='91.2' y2='121.9'/><line x1='795.0' x2='805.0' y1='91.2' y2='91.2'/><line x1='795.0' x2='805.0' y1='121.9' y2='121.9'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(804.0 342.0) rotate(-35)'>MI</text><rect class='sa-hit' x='768.0' y='18' width='64' height='414'><title>MI (
|
| 56 |
Elo 1248 [95% CI 1224β1276]
|
| 57 |
Stability 0.647
|
| 58 |
-
Validity 0.785</title></rect><rect x='841.6' y='105.4' width='22.4' height='222.6' fill='#8dc0dd' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='864.0' y='105.4' width='22.4' height='222.6' fill='#df853d' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='864.0' x2='864.0' y1='86.5' y2='124.9'/><line x1='859.0' x2='869.0' y1='86.5' y2='86.5'/><line x1='859.0' x2='869.0' y1='124.9' y2='124.9'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(868.0 342.0) rotate(-35)'>ElasticNet</text><rect class='sa-hit' x='832.0' y='18' width='64' height='414'><title>ElasticNet (
|
| 59 |
Elo 1252 [95% CI 1219β1284]
|
| 60 |
Stability 0.447
|
| 61 |
-
Validity 0.760</title></rect><rect x='905.6' y='97.7' width='22.4' height='230.3' fill='#93c3df' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='928.0' y='97.7' width='22.4' height='230.3' fill='#e6a16a' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='928.0' x2='928.0' y1='78.2' y2='117.2'/><line x1='923.0' x2='933.0' y1='78.2' y2='78.2'/><line x1='923.0' x2='933.0' y1='117.2' y2='117.2'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(932.0 342.0) rotate(-35)'>CART</text><rect class='sa-hit' x='896.0' y='18' width='64' height='414'><title>CART (
|
| 62 |
Elo 1265 [95% CI 1232β1298]
|
| 63 |
Stability 0.422
|
| 64 |
-
Validity 0.586</title></rect><rect x='969.6' y='71.1' width='22.4' height='256.9' fill='#5ba4cd' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='992.0' y='71.1' width='22.4' height='256.9' fill='#e39354' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='992.0' x2='992.0' y1='52.8' y2='88.9'/><line x1='987.0' x2='997.0' y1='52.8' y2='52.8'/><line x1='987.0' x2='997.0' y1='88.9' y2='88.9'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(996.0 342.0) rotate(-35)'>RFImportance</text><rect class='sa-hit' x='960.0' y='18' width='64' height='414'><title>RFImportance (
|
| 65 |
Elo 1310 [95% CI 1280β1341]
|
| 66 |
Stability 0.643
|
| 67 |
-
Validity 0.669</title></rect><line class='sa-ink' x1='64' x2='1024' y1='328.0' y2='328.0' stroke-width='1' opacity='0.4'/><rect x='1070' y='40.0' width='12' height='107.0' fill='url(#sa-g-stab)' stroke='#000' stroke-width='0.6'/><text class='sa-text' x='1076' y='30.0' text-anchor='middle' font-size='12'>stable</text><text class='sa-text' x='1076' y='165.0' text-anchor='middle' font-size='12'>unstable</text><text class='sa-muted' x='1088' y='151.0' font-size='11'>0.0</text><text class='sa-muted' x='1088' y='97.5' font-size='11'>0.5</text><text class='sa-muted' x='1088' y='44.0' font-size='11'>1.0</text><rect x='1070' y='207.0' width='12' height='107.0' fill='url(#sa-g-val)' stroke='#000' stroke-width='0.6'/><text class='sa-text' x='1076' y='197.0' text-anchor='middle' font-size='12'>valid</text><text class='sa-text' x='1076' y='332.0' text-anchor='middle' font-size='12'>invalid</text><text class='sa-muted' x='1088' y='318.0' font-size='11'>0.0</text><text class='sa-muted' x='1088' y='264.5' font-size='11'>0.5</text><text class='sa-muted' x='1088' y='211.0' font-size='11'>1.0</text><rect x='1070' y='352.0' width='12' height='12' fill='url(#sa-na)' stroke='#000' stroke-width='0.6'/><text class='sa-muted' x='1088' y='362.0' font-size='11'>no result</text></svg><script type='text/plain' id='sa-csv'>Method,
|
| 68 |
-
RFImportance,
|
| 69 |
-
CART,
|
| 70 |
-
ElasticNet,
|
| 71 |
-
MI,
|
| 72 |
-
F-Test,
|
| 73 |
-
Lasso,
|
| 74 |
-
MarkovBlanket,
|
| 75 |
-
(R)ReliefF,
|
| 76 |
-
mRMR,
|
| 77 |
-
GainRatio,
|
| 78 |
-
SFS,
|
| 79 |
-
SBE,
|
| 80 |
-
LOCO,
|
| 81 |
-
Random,Baseline,1000.0,967.0,1030.0,0.001,0.5
|
| 82 |
-
LaplacianScore,
|
| 83 |
</script><script>
|
| 84 |
function saDownload(name, blob) {
|
| 85 |
var a = document.createElement('a');
|
|
|
|
| 19 |
.sa-ink { stroke:var(--fg); }
|
| 20 |
.sa-hit { fill:transparent; }
|
| 21 |
.sa-hit:hover { fill:var(--fg); fill-opacity:0.06; }
|
| 22 |
+
</style><defs><pattern id='sa-na' width='6' height='6' patternUnits='userSpaceOnUse' patternTransform='rotate(45)'><rect width='6' height='6' fill='#e5e7eb'/><line x1='0' y1='0' x2='0' y2='6' stroke='#9ca3af' stroke-width='2'/></pattern><linearGradient id='sa-g-stab' x1='0' y1='1' x2='0' y2='0'><stop offset='0' stop-color='#ffffff'/><stop offset='1' stop-color='#0072B2'/></linearGradient><linearGradient id='sa-g-val' x1='0' y1='1' x2='0' y2='0'><stop offset='0' stop-color='#ffffff'/><stop offset='1' stop-color='#D55E00'/></linearGradient></defs><line class='sa-grid' x1='64' x2='1024' y1='313.2' y2='313.2' stroke-width='1'/><text class='sa-muted' x='56' y='317.2' text-anchor='end' font-size='12'>900</text><line class='sa-grid' x1='64' x2='1024' y1='254.2' y2='254.2' stroke-width='1'/><text class='sa-muted' x='56' y='258.2' text-anchor='end' font-size='12'>1000</text><line class='sa-grid' x1='64' x2='1024' y1='195.1' y2='195.1' stroke-width='1'/><text class='sa-muted' x='56' y='199.1' text-anchor='end' font-size='12'>1100</text><line class='sa-grid' x1='64' x2='1024' y1='136.1' y2='136.1' stroke-width='1'/><text class='sa-muted' x='56' y='140.1' text-anchor='end' font-size='12'>1200</text><line class='sa-grid' x1='64' x2='1024' y1='77.0' y2='77.0' stroke-width='1'/><text class='sa-muted' x='56' y='81.0' text-anchor='end' font-size='12'>1300</text><line class='sa-grid' x1='64' x2='1024' y1='18.0' y2='18.0' stroke-width='1'/><text class='sa-muted' x='56' y='22.0' text-anchor='end' font-size='12'>1400</text><text class='sa-text' transform='translate(16 173) rotate(-90)' text-anchor='middle' font-size='13'>Elo</text><line class='sa-ink' x1='64' x2='1024' y1='254.2' y2='254.2' stroke-width='1' stroke-dasharray='4 4' opacity='0.5'/><rect x='73.6' y='274.9' width='22.4' height='53.1' fill='#59a3cd' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='96.0' y='274.9' width='22.4' height='53.1' fill='#de7f34' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='96.0' x2='96.0' y1='246.5' y2='305.6'/><line x1='91.0' x2='101.0' y1='246.5' y2='246.5'/><line x1='91.0' x2='101.0' y1='305.6' y2='305.6'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(100.0 342.0) rotate(-35)'>LaplacianScore</text><rect class='sa-hit' x='64.0' y='18' width='64' height='414'><title>LaplacianScore (Distance, non-search)
|
| 23 |
Elo 965 [95% CI 913β1013]
|
| 24 |
Stability 0.652
|
| 25 |
+
Validity 0.797</title></rect><rect x='137.6' y='254.2' width='22.4' height='73.8' fill='#ffffff' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='160.0' y='254.2' width='22.4' height='73.8' fill='#eaae80' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='160.0' x2='160.0' y1='236.5' y2='273.7'/><line x1='155.0' x2='165.0' y1='236.5' y2='236.5'/><line x1='155.0' x2='165.0' y1='273.7' y2='273.7'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(164.0 342.0) rotate(-35)'>Random</text><rect class='sa-hit' x='128.0' y='18' width='64' height='414'><title>Random (Baseline, baseline)
|
| 26 |
Elo 1000 [95% CI 967β1030]
|
| 27 |
Stability 0.001
|
| 28 |
+
Validity 0.500</title></rect><rect x='201.6' y='165.0' width='22.4' height='163.0' fill='#d3e7f2' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='224.0' y='165.0' width='22.4' height='163.0' fill='#e7a16a' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='224.0' x2='224.0' y1='146.1' y2='183.3'/><line x1='219.0' x2='229.0' y1='146.1' y2='146.1'/><line x1='219.0' x2='229.0' y1='183.3' y2='183.3'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(228.0 342.0) rotate(-35)'>LOCO</text><rect class='sa-hit' x='192.0' y='18' width='64' height='414'><title>LOCO (Leave-one-out, search)
|
| 29 |
Elo 1151 [95% CI 1120β1183]
|
| 30 |
Stability 0.173
|
| 31 |
+
Validity 0.583</title></rect><rect x='265.6' y='154.4' width='22.4' height='173.6' fill='#ddecf5' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='288.0' y='154.4' width='22.4' height='173.6' fill='#e7a570' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='288.0' x2='288.0' y1='129.6' y2='175.7'/><line x1='283.0' x2='293.0' y1='129.6' y2='129.6'/><line x1='283.0' x2='293.0' y1='175.7' y2='175.7'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(292.0 342.0) rotate(-35)'>SBE</text><rect class='sa-hit' x='256.0' y='18' width='64' height='414'><title>SBE (Backward search, search)
|
| 32 |
Elo 1169 [95% CI 1133β1211]
|
| 33 |
Stability 0.132
|
| 34 |
+
Validity 0.560</title></rect><rect x='329.6' y='149.1' width='22.4' height='178.9' fill='#bbd9ea' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='352.0' y='149.1' width='22.4' height='178.9' fill='#e59a5f' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='352.0' x2='352.0' y1='129.6' y2='164.4'/><line x1='347.0' x2='357.0' y1='129.6' y2='129.6'/><line x1='347.0' x2='357.0' y1='164.4' y2='164.4'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(356.0 342.0) rotate(-35)'>SFS</text><rect class='sa-hit' x='320.0' y='18' width='64' height='414'><title>SFS (Forward search, search)
|
| 35 |
Elo 1178 [95% CI 1152β1211]
|
| 36 |
Stability 0.267
|
| 37 |
+
Validity 0.627</title></rect><rect x='393.6' y='146.1' width='22.4' height='181.9' fill='#5ba4cd' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='416.0' y='146.1' width='22.4' height='181.9' fill='#dc7a2d' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='416.0' x2='416.0' y1='129.0' y2='161.5'/><line x1='411.0' x2='421.0' y1='129.0' y2='129.0'/><line x1='411.0' x2='421.0' y1='161.5' y2='161.5'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(420.0 342.0) rotate(-35)'>GainRatio</text><rect class='sa-hit' x='384.0' y='18' width='64' height='414'><title>GainRatio (Information theory, non-search)
|
| 38 |
Elo 1183 [95% CI 1157β1212]
|
| 39 |
Stability 0.644
|
| 40 |
+
Validity 0.825</title></rect><rect x='457.6' y='140.2' width='22.4' height='187.8' fill='#65aad0' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='480.0' y='140.2' width='22.4' height='187.8' fill='#e39658' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='480.0' x2='480.0' y1='123.7' y2='156.8'/><line x1='475.0' x2='485.0' y1='123.7' y2='123.7'/><line x1='475.0' x2='485.0' y1='156.8' y2='156.8'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(484.0 342.0) rotate(-35)'>mRMR</text><rect class='sa-hit' x='448.0' y='18' width='64' height='414'><title>mRMR (Information theory, non-search)
|
| 41 |
Elo 1193 [95% CI 1165β1221]
|
| 42 |
Stability 0.604
|
| 43 |
+
Validity 0.655</title></rect><rect x='521.6' y='137.3' width='22.4' height='190.7' fill='#5aa4cd' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='544.0' y='137.3' width='22.4' height='190.7' fill='#e08842' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='544.0' x2='544.0' y1='119.6' y2='155.0'/><line x1='539.0' x2='549.0' y1='119.6' y2='119.6'/><line x1='539.0' x2='549.0' y1='155.0' y2='155.0'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(548.0 342.0) rotate(-35)'>(R)ReliefF</text><rect class='sa-hit' x='512.0' y='18' width='64' height='414'><title>(R)ReliefF (Distance, non-search)
|
| 44 |
Elo 1198 [95% CI 1168β1228]
|
| 45 |
Stability 0.646
|
| 46 |
+
Validity 0.740</title></rect><rect x='585.6' y='124.3' width='22.4' height='203.7' fill='#96c5df' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='608.0' y='124.3' width='22.4' height='203.7' fill='#ecb489' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='608.0' x2='608.0' y1='109.5' y2='141.4'/><line x1='603.0' x2='613.0' y1='109.5' y2='109.5'/><line x1='603.0' x2='613.0' y1='141.4' y2='141.4'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(612.0 342.0) rotate(-35)'>MarkovBlanket</text><rect class='sa-hit' x='576.0' y='18' width='64' height='414'><title>MarkovBlanket (Backward search, search)
|
| 47 |
Elo 1220 [95% CI 1191β1245]
|
| 48 |
Stability 0.413
|
| 49 |
+
Validity 0.463</title></rect><rect x='649.6' y='120.7' width='22.4' height='207.3' fill='#a3cce3' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='672.0' y='120.7' width='22.4' height='207.3' fill='#e08945' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='672.0' x2='672.0' y1='102.4' y2='138.5'/><line x1='667.0' x2='677.0' y1='102.4' y2='102.4'/><line x1='667.0' x2='677.0' y1='138.5' y2='138.5'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(676.0 342.0) rotate(-35)'>Lasso</text><rect class='sa-hit' x='640.0' y='18' width='64' height='414'><title>Lasso (Regularization, non-search)
|
| 50 |
Elo 1226 [95% CI 1196β1257]
|
| 51 |
Stability 0.362
|
| 52 |
+
Validity 0.730</title></rect><rect x='713.6' y='119.0' width='22.4' height='209.0' fill='#529fcb' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='736.0' y='119.0' width='22.4' height='209.0' fill='#db7727' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='736.0' x2='736.0' y1='103.0' y2='134.3'/><line x1='731.0' x2='741.0' y1='103.0' y2='103.0'/><line x1='731.0' x2='741.0' y1='134.3' y2='134.3'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(740.0 342.0) rotate(-35)'>F-Test</text><rect class='sa-hit' x='704.0' y='18' width='64' height='414'><title>F-Test (Correlation, non-search)
|
| 53 |
Elo 1229 [95% CI 1203β1256]
|
| 54 |
Stability 0.679
|
| 55 |
+
Validity 0.846</title></rect><rect x='777.6' y='107.8' width='22.4' height='220.2' fill='#5aa4cd' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='800.0' y='107.8' width='22.4' height='220.2' fill='#de8137' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='800.0' x2='800.0' y1='91.2' y2='121.9'/><line x1='795.0' x2='805.0' y1='91.2' y2='91.2'/><line x1='795.0' x2='805.0' y1='121.9' y2='121.9'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(804.0 342.0) rotate(-35)'>MI</text><rect class='sa-hit' x='768.0' y='18' width='64' height='414'><title>MI (Information theory, non-search)
|
| 56 |
Elo 1248 [95% CI 1224β1276]
|
| 57 |
Stability 0.647
|
| 58 |
+
Validity 0.785</title></rect><rect x='841.6' y='105.4' width='22.4' height='222.6' fill='#8dc0dd' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='864.0' y='105.4' width='22.4' height='222.6' fill='#df853d' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='864.0' x2='864.0' y1='86.5' y2='124.9'/><line x1='859.0' x2='869.0' y1='86.5' y2='86.5'/><line x1='859.0' x2='869.0' y1='124.9' y2='124.9'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(868.0 342.0) rotate(-35)'>ElasticNet</text><rect class='sa-hit' x='832.0' y='18' width='64' height='414'><title>ElasticNet (Regularization, non-search)
|
| 59 |
Elo 1252 [95% CI 1219β1284]
|
| 60 |
Stability 0.447
|
| 61 |
+
Validity 0.760</title></rect><rect x='905.6' y='97.7' width='22.4' height='230.3' fill='#93c3df' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='928.0' y='97.7' width='22.4' height='230.3' fill='#e6a16a' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='928.0' x2='928.0' y1='78.2' y2='117.2'/><line x1='923.0' x2='933.0' y1='78.2' y2='78.2'/><line x1='923.0' x2='933.0' y1='117.2' y2='117.2'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(932.0 342.0) rotate(-35)'>CART</text><rect class='sa-hit' x='896.0' y='18' width='64' height='414'><title>CART (Tree, non-search)
|
| 62 |
Elo 1265 [95% CI 1232β1298]
|
| 63 |
Stability 0.422
|
| 64 |
+
Validity 0.586</title></rect><rect x='969.6' y='71.1' width='22.4' height='256.9' fill='#5ba4cd' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='992.0' y='71.1' width='22.4' height='256.9' fill='#e39354' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='992.0' x2='992.0' y1='52.8' y2='88.9'/><line x1='987.0' x2='997.0' y1='52.8' y2='52.8'/><line x1='987.0' x2='997.0' y1='88.9' y2='88.9'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(996.0 342.0) rotate(-35)'>RFImportance</text><rect class='sa-hit' x='960.0' y='18' width='64' height='414'><title>RFImportance (Tree, non-search)
|
| 65 |
Elo 1310 [95% CI 1280β1341]
|
| 66 |
Stability 0.643
|
| 67 |
+
Validity 0.669</title></rect><line class='sa-ink' x1='64' x2='1024' y1='328.0' y2='328.0' stroke-width='1' opacity='0.4'/><rect x='1070' y='40.0' width='12' height='107.0' fill='url(#sa-g-stab)' stroke='#000' stroke-width='0.6'/><text class='sa-text' x='1076' y='30.0' text-anchor='middle' font-size='12'>stable</text><text class='sa-text' x='1076' y='165.0' text-anchor='middle' font-size='12'>unstable</text><text class='sa-muted' x='1088' y='151.0' font-size='11'>0.0</text><text class='sa-muted' x='1088' y='97.5' font-size='11'>0.5</text><text class='sa-muted' x='1088' y='44.0' font-size='11'>1.0</text><rect x='1070' y='207.0' width='12' height='107.0' fill='url(#sa-g-val)' stroke='#000' stroke-width='0.6'/><text class='sa-text' x='1076' y='197.0' text-anchor='middle' font-size='12'>valid</text><text class='sa-text' x='1076' y='332.0' text-anchor='middle' font-size='12'>invalid</text><text class='sa-muted' x='1088' y='318.0' font-size='11'>0.0</text><text class='sa-muted' x='1088' y='264.5' font-size='11'>0.5</text><text class='sa-muted' x='1088' y='211.0' font-size='11'>1.0</text><rect x='1070' y='352.0' width='12' height='12' fill='url(#sa-na)' stroke='#000' stroke-width='0.6'/><text class='sa-muted' x='1088' y='362.0' font-size='11'>no result</text></svg><script type='text/plain' id='sa-csv'>Method,Category,Subgroup,Elo,Elo CI low,Elo CI high,Stability,Validity
|
| 68 |
+
RFImportance,Non-search,Tree,1310.0,1280.0,1341.0,0.643,0.669
|
| 69 |
+
CART,Non-search,Tree,1265.0,1232.0,1298.0,0.422,0.586
|
| 70 |
+
ElasticNet,Non-search,Regularization,1252.0,1219.0,1284.0,0.447,0.76
|
| 71 |
+
MI,Non-search,Information theory,1248.0,1224.0,1276.0,0.647,0.785
|
| 72 |
+
F-Test,Non-search,Correlation,1229.0,1203.0,1256.0,0.679,0.846
|
| 73 |
+
Lasso,Non-search,Regularization,1226.0,1196.0,1257.0,0.362,0.73
|
| 74 |
+
MarkovBlanket,Search,Backward search,1220.0,1191.0,1245.0,0.413,0.463
|
| 75 |
+
(R)ReliefF,Non-search,Distance,1198.0,1168.0,1228.0,0.646,0.74
|
| 76 |
+
mRMR,Non-search,Information theory,1193.0,1165.0,1221.0,0.604,0.655
|
| 77 |
+
GainRatio,Non-search,Information theory,1183.0,1157.0,1212.0,0.644,0.825
|
| 78 |
+
SFS,Search,Forward search,1178.0,1152.0,1211.0,0.267,0.627
|
| 79 |
+
SBE,Search,Backward search,1169.0,1133.0,1211.0,0.132,0.56
|
| 80 |
+
LOCO,Search,Leave-one-out,1151.0,1120.0,1183.0,0.173,0.583
|
| 81 |
+
Random,Baseline,Baseline,1000.0,967.0,1030.0,0.001,0.5
|
| 82 |
+
LaplacianScore,Non-search,Distance,965.0,913.0,1013.0,0.652,0.797
|
| 83 |
</script><script>
|
| 84 |
function saDownload(name, blob) {
|
| 85 |
var a = document.createElement('a');
|
data/subsets/tasks_all/models_all/budgets_all/leaderboard_table.html
CHANGED
|
@@ -26,23 +26,25 @@ td.method { font-weight:600; }
|
|
| 26 |
td .sub { color:var(--muted); font-size:12px; font-weight:400; margin-left:6px; }
|
| 27 |
td.na { color:var(--muted); }
|
| 28 |
.pill { display:inline-block; padding:0 8px; border:1px solid var(--line); border-radius:999px;
|
| 29 |
-
font-size:12px; color:var(--
|
| 30 |
-
</style></head><body><div class='toolbar'><span class='hint'>Click a column to sort. Hover a header for its definition.</span><button onclick="saCsv('selectarena_leaderboard.csv')">CSV</button></div><div class='wrap'><table id='sa-table'><thead><tr><th class='num'>#</th><th class=''>Method</th><th class='' title='Filter: scores features from the data alone. Wrapper: searches feature subsets by refitting a proxy model. Embedded: reads the selection off a fitted model.'>Type</th><th class='num' title='Bradley-Terry Elo from pairwise matches within each dataset, budget and downstream model. Random = 1000. +/β give the 95% bootstrap interval.'>Elo β</th><th class='num' title='Mean rank of the downstream error.'>Avg. rank β</th><th class='num' title='Share of pairwise comparisons won, in %. Ties count half.'>Win rate β</th><th class='num' title='Mean relative gap (error β best error) / error to the best method, in %.'>Improvability β</th><th class='num' title='Mean ratio of the error to the same model trained on all features. Below 1: the selection helped.'>Error vs. all β</th><th class='num' title='Agreement of the selected features across bootstrap resamples (Nogueira et al., 2018).'>Stability β</th><th class='num' title='Share of the budget filled with original features when as many noise features are added.'>Validity β</th><th class='num' title='Median wall-clock time of the selection.'>Time (s) β</th><th class='num' title='Share of runs where the method ran out of time and random features filled the budget.'>Fallback (%)</th><th class='num'>#Datasets</th></tr></thead><tbody><tr><td class='num' data-sort='1'>1</td><td class='method' data-sort='RFImportance'>RFImportance</td><td class='' data-sort='Embedded'><span class='pill'>π³ Embedded</span></td><td class='num' data-sort='1310.0'>1310<span class='sub'>+31 / β30</span></td><td class='num' data-sort='6.15'>6.15</td><td class='num' data-sort='63.2'>63.2</td><td class='num' data-sort='15.1'>15.1</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.643'>0.643</td><td class='num' data-sort='0.669'>0.669</td><td class='num' data-sort='0.2'>0.2</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='2'>2</td><td class='method' data-sort='CART'>CART</td><td class='' data-sort='Embedded'><span class='pill'>π³ Embedded</span></td><td class='num' data-sort='1265.0'>1265<span class='sub'>+33 / β33</span></td><td class='num' data-sort='6.69'>6.69</td><td class='num' data-sort='59.3'>59.3</td><td class='num' data-sort='18.0'>18.0</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.422'>0.422</td><td class='num' data-sort='0.586'>0.586</td><td class='num' data-sort='0.1'>0.1</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='3'>3</td><td class='method' data-sort='ElasticNet'>ElasticNet</td><td class='' data-sort='Embedded'><span class='pill'>π³ Embedded</span></td><td class='num' data-sort='1252.0'>1252<span class='sub'>+32 / β33</span></td><td class='num' data-sort='6.87'>6.87</td><td class='num' data-sort='58.1'>58.1</td><td class='num' data-sort='18.3'>18.3</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.447'>0.447</td><td class='num' data-sort='0.76'>0.760</td><td class='num' data-sort='7.6'>7.6</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='4'>4</td><td class='method' data-sort='MI'>MI</td><td class='' data-sort='Filter'><span class='pill'>π Filter</span></td><td class='num' data-sort='1248.0'>1248<span class='sub'>+28 / β24</span></td><td class='num' data-sort='6.99'>6.99</td><td class='num' data-sort='57.2'>57.2</td><td class='num' data-sort='18.2'>18.2</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.647'>0.647</td><td class='num' data-sort='0.785'>0.785</td><td class='num' data-sort='0.4'>0.4</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='5'>5</td><td class='method' data-sort='F-Test'>F-Test</td><td class='' data-sort='Filter'><span class='pill'>π Filter</span></td><td class='num' data-sort='1229.0'>1229<span class='sub'>+27 / β26</span></td><td class='num' data-sort='7.35'>7.35</td><td class='num' data-sort='54.6'>54.6</td><td class='num' data-sort='19.0'>19.0</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.679'>0.679</td><td class='num' data-sort='0.846'>0.846</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='6'>6</td><td class='method' data-sort='Lasso'>Lasso</td><td class='' data-sort='Embedded'><span class='pill'>π³ Embedded</span></td><td class='num' data-sort='1226.0'>1226<span class='sub'>+31 / β30</span></td><td class='num' data-sort='7.2'>7.20</td><td class='num' data-sort='55.7'>55.7</td><td class='num' data-sort='18.7'>18.7</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.362'>0.362</td><td class='num' data-sort='0.73'>0.730</td><td class='num' data-sort='2.1'>2.1</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='7'>7</td><td class='method' data-sort='MarkovBlanket'>MarkovBlanket</td><td class='' data-sort='Filter'><span class='pill'>π Filter</span></td><td class='num' data-sort='1220.0'>1220<span class='sub'>+25 / β29</span></td><td class='num' data-sort='7.55'>7.55</td><td class='num' data-sort='53.2'>53.2</td><td class='num' data-sort='19.6'>19.6</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.413'>0.413</td><td class='num' data-sort='0.463'>0.463</td><td class='num' data-sort='0.1'>0.1</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='8'>8</td><td class='method' data-sort='(R)ReliefF'>(R)ReliefF</td><td class='' data-sort='Filter'><span class='pill'>π Filter</span></td><td class='num' data-sort='1198.0'>1198<span class='sub'>+30 / β30</span></td><td class='num' data-sort='8.0'>8.00</td><td class='num' data-sort='50.0'>50.0</td><td class='num' data-sort='20.6'>20.6</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.646'>0.646</td><td class='num' data-sort='0.74'>0.740</td><td class='num' data-sort='14.3'>14.3</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='9'>9</td><td class='method' data-sort='mRMR'>mRMR</td><td class='' data-sort='Filter'><span class='pill'>π Filter</span></td><td class='num' data-sort='1193.0'>1193<span class='sub'>+28 / β28</span></td><td class='num' data-sort='7.96'>7.96</td><td class='num' data-sort='50.3'>50.3</td><td class='num' data-sort='19.9'>19.9</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.604'>0.604</td><td class='num' data-sort='0.655'>0.655</td><td class='num' data-sort='0.4'>0.4</td><td class='num' data-sort='3.4'>3.4</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='10'>10</td><td class='method' data-sort='GainRatio'>GainRatio</td><td class='' data-sort='Filter'><span class='pill'>π Filter</span></td><td class='num' data-sort='1183.0'>1183<span class='sub'>+29 / β26</span></td><td class='num' data-sort='8.19'>8.19</td><td class='num' data-sort='48.6'>48.6</td><td class='num' data-sort='21.5'>21.5</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.644'>0.644</td><td class='num' data-sort='0.825'>0.825</td><td class='num' data-sort='0.2'>0.2</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='11'>11</td><td class='method' data-sort='SFS'>SFS</td><td class='' data-sort='Wrapper'><span class='pill'>π Wrapper</span></td><td class='num' data-sort='1178.0'>1178<span class='sub'>+33 / β26</span></td><td class='num' data-sort='7.99'>7.99</td><td class='num' data-sort='50.1'>50.1</td><td class='num' data-sort='21.7'>21.7</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.267'>0.267</td><td class='num' data-sort='0.627'>0.627</td><td class='num' data-sort='2105.8'>2,105.8</td><td class='num' data-sort='43.5'>43.5</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='12'>12</td><td class='method' data-sort='SBE'>SBE</td><td class='' data-sort='Wrapper'><span class='pill'>π Wrapper</span></td><td class='num' data-sort='1169.0'>1169<span class='sub'>+42 / β36</span></td><td class='num' data-sort='8.31'>8.31</td><td class='num' data-sort='47.8'>47.8</td><td class='num' data-sort='23.7'>23.7</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.132'>0.132</td><td class='num' data-sort='0.56'>0.560</td><td class='num' data-sort='2495.5'>2,495.5</td><td class='num' data-sort='46.5'>46.5</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='13'>13</td><td class='method' data-sort='LOCO'>LOCO</td><td class='' data-sort='Wrapper'><span class='pill'>π Wrapper</span></td><td class='num' data-sort='1151.0'>1151<span class='sub'>+32 / β31</span></td><td class='num' data-sort='8.39'>8.39</td><td class='num' data-sort='47.2'>47.2</td><td class='num' data-sort='23.8'>23.8</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.173'>0.173</td><td class='num' data-sort='0.583'>0.583</td><td class='num' data-sort='241.5'>241.5</td><td class='num' data-sort='7.6'>7.6</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='14'>14</td><td class='method' data-sort='Random'>Random</td><td class='' data-sort='Baseline'><span class='pill'>βͺ Baseline</span></td><td class='num' data-sort='1000.0'>1000<span class='sub'>+30 / β33</span></td><td class='num' data-sort='10.72'>10.72</td><td class='num' data-sort='30.6'>30.6</td><td class='num' data-sort='32.0'>32.0</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.001'>0.001</td><td class='num' data-sort='0.5'>0.500</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='100.0'>100.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='15'>15</td><td class='method' data-sort='LaplacianScore'>LaplacianScore</td><td class='' data-sort='Filter'><span class='pill'>π Filter</span></td><td class='num' data-sort='965.0'>965<span class='sub'>+48 / β52</span></td><td class='num' data-sort='11.64'>11.64</td><td class='num' data-sort='24.0'>24.0</td><td class='num' data-sort='36.6'>36.6</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.652'>0.652</td><td class='num' data-sort='0.797'>0.797</td><td class='num' data-sort='0.2'>0.2</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr></tbody></table></div><script type='text/plain' id='sa-csv'>Rank,Type,TypeName,Method,Elo,Elo CI low,Elo CI high,Avg. rank,Win rate (%),Improvability (%),Error vs. all features,Stability,Validity,Selection time (s),Fallback rate (%),#Datasets,#Comparisons
|
| 31 |
-
|
| 32 |
-
2,π³,Embedded,CART,1265.0,1232.0,1298.0,6.69,59.3,18.0,,0.422,0.586,0.1,0.0,71,5325
|
| 33 |
-
|
| 34 |
-
|
| 35 |
-
|
| 36 |
-
|
| 37 |
-
|
| 38 |
-
|
| 39 |
-
|
| 40 |
-
|
| 41 |
-
|
| 42 |
-
|
| 43 |
-
|
| 44 |
-
|
| 45 |
-
|
|
|
|
|
|
|
| 46 |
</script><script>
|
| 47 |
function saDownload(name, blob) {
|
| 48 |
var a = document.createElement('a');
|
|
|
|
| 26 |
td .sub { color:var(--muted); font-size:12px; font-weight:400; margin-left:6px; }
|
| 27 |
td.na { color:var(--muted); }
|
| 28 |
.pill { display:inline-block; padding:0 8px; border:1px solid var(--line); border-radius:999px;
|
| 29 |
+
font-size:12px; color:var(--fg); }
|
| 30 |
+
.pill.cat-search { border-color:var(--accent); }
|
| 31 |
+
.pill.cat-baseline { color:var(--muted); }
|
| 32 |
+
</style></head><body><div class='toolbar'><span class='hint'>Click a column to sort. Hover a header for its definition.</span><button onclick="saCsv('selectarena_leaderboard.csv')">CSV</button></div><div class='wrap'><table id='sa-table'><thead><tr><th class='num'>#</th><th class=''>Method</th><th class='' title='Taxonomy of the paper (Appendix A). Search methods evaluate many feature subsets with the model (forward, backward, leave-one-out); non-search methods use a proxy instead (correlation, distance, information theory, tree, regularization).'>Type</th><th class='num' title='Bradley-Terry Elo from pairwise matches within each dataset, budget and downstream model. Random = 1000. +/β give the 95% bootstrap interval.'>Elo β</th><th class='num' title='Mean rank of the downstream error.'>Avg. rank β</th><th class='num' title='Share of pairwise comparisons won, in %. Ties count half.'>Win rate β</th><th class='num' title='Mean relative gap (error β best error) / error to the best method, in %.'>Improvability β</th><th class='num' title='Mean ratio of the error to the same model trained on all features. Below 1: the selection helped.'>Error vs. all β</th><th class='num' title='Agreement of the selected features across bootstrap resamples (Nogueira et al., 2018).'>Stability β</th><th class='num' title='Share of the budget filled with original features when as many noise features are added.'>Validity β</th><th class='num' title='Median wall-clock time of the selection.'>Time (s) β</th><th class='num' title='Share of runs where the method ran out of time and random features filled the budget.'>Fallback (%)</th><th class='num'>#Datasets</th></tr></thead><tbody><tr><td class='num' data-sort='1'>1</td><td class='method' data-sort='RFImportance'>RFImportance</td><td class='' data-sort='Tree'><span class='pill cat-non-search'>Tree</span><span class='sub'>non-search</span></td><td class='num' data-sort='1310.0'>1310<span class='sub'>+31 / β30</span></td><td class='num' data-sort='6.15'>6.15</td><td class='num' data-sort='63.2'>63.2</td><td class='num' data-sort='15.1'>15.1</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.643'>0.643</td><td class='num' data-sort='0.669'>0.669</td><td class='num' data-sort='0.2'>0.2</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='2'>2</td><td class='method' data-sort='CART'>CART</td><td class='' data-sort='Tree'><span class='pill cat-non-search'>Tree</span><span class='sub'>non-search</span></td><td class='num' data-sort='1265.0'>1265<span class='sub'>+33 / β33</span></td><td class='num' data-sort='6.69'>6.69</td><td class='num' data-sort='59.3'>59.3</td><td class='num' data-sort='18.0'>18.0</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.422'>0.422</td><td class='num' data-sort='0.586'>0.586</td><td class='num' data-sort='0.1'>0.1</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='3'>3</td><td class='method' data-sort='ElasticNet'>ElasticNet</td><td class='' data-sort='Regularization'><span class='pill cat-non-search'>Regularization</span><span class='sub'>non-search</span></td><td class='num' data-sort='1252.0'>1252<span class='sub'>+32 / β33</span></td><td class='num' data-sort='6.87'>6.87</td><td class='num' data-sort='58.1'>58.1</td><td class='num' data-sort='18.3'>18.3</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.447'>0.447</td><td class='num' data-sort='0.76'>0.760</td><td class='num' data-sort='7.6'>7.6</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='4'>4</td><td class='method' data-sort='MI'>MI</td><td class='' data-sort='Information theory'><span class='pill cat-non-search'>Information theory</span><span class='sub'>non-search</span></td><td class='num' data-sort='1248.0'>1248<span class='sub'>+28 / β24</span></td><td class='num' data-sort='6.99'>6.99</td><td class='num' data-sort='57.2'>57.2</td><td class='num' data-sort='18.2'>18.2</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.647'>0.647</td><td class='num' data-sort='0.785'>0.785</td><td class='num' data-sort='0.4'>0.4</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='5'>5</td><td class='method' data-sort='F-Test'>F-Test</td><td class='' data-sort='Correlation'><span class='pill cat-non-search'>Correlation</span><span class='sub'>non-search</span></td><td class='num' data-sort='1229.0'>1229<span class='sub'>+27 / β26</span></td><td class='num' data-sort='7.35'>7.35</td><td class='num' data-sort='54.6'>54.6</td><td class='num' data-sort='19.0'>19.0</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.679'>0.679</td><td class='num' data-sort='0.846'>0.846</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='6'>6</td><td class='method' data-sort='Lasso'>Lasso</td><td class='' data-sort='Regularization'><span class='pill cat-non-search'>Regularization</span><span class='sub'>non-search</span></td><td class='num' data-sort='1226.0'>1226<span class='sub'>+31 / β30</span></td><td class='num' data-sort='7.2'>7.20</td><td class='num' data-sort='55.7'>55.7</td><td class='num' data-sort='18.7'>18.7</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.362'>0.362</td><td class='num' data-sort='0.73'>0.730</td><td class='num' data-sort='2.1'>2.1</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='7'>7</td><td class='method' data-sort='MarkovBlanket'>MarkovBlanket</td><td class='' data-sort='Backward search'><span class='pill cat-search'>Backward search</span><span class='sub'>search</span></td><td class='num' data-sort='1220.0'>1220<span class='sub'>+25 / β29</span></td><td class='num' data-sort='7.55'>7.55</td><td class='num' data-sort='53.2'>53.2</td><td class='num' data-sort='19.6'>19.6</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.413'>0.413</td><td class='num' data-sort='0.463'>0.463</td><td class='num' data-sort='0.1'>0.1</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='8'>8</td><td class='method' data-sort='(R)ReliefF'>(R)ReliefF</td><td class='' data-sort='Distance'><span class='pill cat-non-search'>Distance</span><span class='sub'>non-search</span></td><td class='num' data-sort='1198.0'>1198<span class='sub'>+30 / β30</span></td><td class='num' data-sort='8.0'>8.00</td><td class='num' data-sort='50.0'>50.0</td><td class='num' data-sort='20.6'>20.6</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.646'>0.646</td><td class='num' data-sort='0.74'>0.740</td><td class='num' data-sort='14.3'>14.3</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='9'>9</td><td class='method' data-sort='mRMR'>mRMR</td><td class='' data-sort='Information theory'><span class='pill cat-non-search'>Information theory</span><span class='sub'>non-search</span></td><td class='num' data-sort='1193.0'>1193<span class='sub'>+28 / β28</span></td><td class='num' data-sort='7.96'>7.96</td><td class='num' data-sort='50.3'>50.3</td><td class='num' data-sort='19.9'>19.9</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.604'>0.604</td><td class='num' data-sort='0.655'>0.655</td><td class='num' data-sort='0.4'>0.4</td><td class='num' data-sort='3.4'>3.4</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='10'>10</td><td class='method' data-sort='GainRatio'>GainRatio</td><td class='' data-sort='Information theory'><span class='pill cat-non-search'>Information theory</span><span class='sub'>non-search</span></td><td class='num' data-sort='1183.0'>1183<span class='sub'>+29 / β26</span></td><td class='num' data-sort='8.19'>8.19</td><td class='num' data-sort='48.6'>48.6</td><td class='num' data-sort='21.5'>21.5</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.644'>0.644</td><td class='num' data-sort='0.825'>0.825</td><td class='num' data-sort='0.2'>0.2</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='11'>11</td><td class='method' data-sort='SFS'>SFS</td><td class='' data-sort='Forward search'><span class='pill cat-search'>Forward search</span><span class='sub'>search</span></td><td class='num' data-sort='1178.0'>1178<span class='sub'>+33 / β26</span></td><td class='num' data-sort='7.99'>7.99</td><td class='num' data-sort='50.1'>50.1</td><td class='num' data-sort='21.7'>21.7</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.267'>0.267</td><td class='num' data-sort='0.627'>0.627</td><td class='num' data-sort='2105.8'>2,105.8</td><td class='num' data-sort='43.5'>43.5</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='12'>12</td><td class='method' data-sort='SBE'>SBE</td><td class='' data-sort='Backward search'><span class='pill cat-search'>Backward search</span><span class='sub'>search</span></td><td class='num' data-sort='1169.0'>1169<span class='sub'>+42 / β36</span></td><td class='num' data-sort='8.31'>8.31</td><td class='num' data-sort='47.8'>47.8</td><td class='num' data-sort='23.7'>23.7</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.132'>0.132</td><td class='num' data-sort='0.56'>0.560</td><td class='num' data-sort='2495.5'>2,495.5</td><td class='num' data-sort='46.5'>46.5</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='13'>13</td><td class='method' data-sort='LOCO'>LOCO</td><td class='' data-sort='Leave-one-out'><span class='pill cat-search'>Leave-one-out</span><span class='sub'>search</span></td><td class='num' data-sort='1151.0'>1151<span class='sub'>+32 / β31</span></td><td class='num' data-sort='8.39'>8.39</td><td class='num' data-sort='47.2'>47.2</td><td class='num' data-sort='23.8'>23.8</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.173'>0.173</td><td class='num' data-sort='0.583'>0.583</td><td class='num' data-sort='241.5'>241.5</td><td class='num' data-sort='7.6'>7.6</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='14'>14</td><td class='method' data-sort='Random'>Random</td><td class='' data-sort='Baseline'><span class='pill cat-baseline'>Baseline</span></td><td class='num' data-sort='1000.0'>1000<span class='sub'>+30 / β33</span></td><td class='num' data-sort='10.72'>10.72</td><td class='num' data-sort='30.6'>30.6</td><td class='num' data-sort='32.0'>32.0</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.001'>0.001</td><td class='num' data-sort='0.5'>0.500</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='100.0'>100.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='15'>15</td><td class='method' data-sort='LaplacianScore'>LaplacianScore</td><td class='' data-sort='Distance'><span class='pill cat-non-search'>Distance</span><span class='sub'>non-search</span></td><td class='num' data-sort='965.0'>965<span class='sub'>+48 / β52</span></td><td class='num' data-sort='11.64'>11.64</td><td class='num' data-sort='24.0'>24.0</td><td class='num' data-sort='36.6'>36.6</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.652'>0.652</td><td class='num' data-sort='0.797'>0.797</td><td class='num' data-sort='0.2'>0.2</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr></tbody></table></div><script type='text/plain' id='sa-csv'>Rank,Method,Category,Subgroup,Elo,Elo CI low,Elo CI high,Avg. rank,Win rate (%),Improvability (%),Error vs. all features,Stability,Validity,Selection time (s),Fallback rate (%),#Datasets,#Comparisons
|
| 33 |
+
1,RFImportance,Non-search,Tree,1310.0,1280.0,1341.0,6.15,63.2,15.1,,0.643,0.669,0.2,0.0,71,5325
|
| 34 |
+
2,CART,Non-search,Tree,1265.0,1232.0,1298.0,6.69,59.3,18.0,,0.422,0.586,0.1,0.0,71,5325
|
| 35 |
+
3,ElasticNet,Non-search,Regularization,1252.0,1219.0,1284.0,6.87,58.1,18.3,,0.447,0.76,7.6,0.0,71,5325
|
| 36 |
+
4,MI,Non-search,Information theory,1248.0,1224.0,1276.0,6.99,57.2,18.2,,0.647,0.785,0.4,0.0,71,5325
|
| 37 |
+
5,F-Test,Non-search,Correlation,1229.0,1203.0,1256.0,7.35,54.6,19.0,,0.679,0.846,0.0,0.0,71,5325
|
| 38 |
+
6,Lasso,Non-search,Regularization,1226.0,1196.0,1257.0,7.2,55.7,18.7,,0.362,0.73,2.1,0.0,71,5325
|
| 39 |
+
7,MarkovBlanket,Search,Backward search,1220.0,1191.0,1245.0,7.55,53.2,19.6,,0.413,0.463,0.1,0.0,71,5325
|
| 40 |
+
8,(R)ReliefF,Non-search,Distance,1198.0,1168.0,1228.0,8.0,50.0,20.6,,0.646,0.74,14.3,0.0,71,5325
|
| 41 |
+
9,mRMR,Non-search,Information theory,1193.0,1165.0,1221.0,7.96,50.3,19.9,,0.604,0.655,0.4,3.4,71,5325
|
| 42 |
+
10,GainRatio,Non-search,Information theory,1183.0,1157.0,1212.0,8.19,48.6,21.5,,0.644,0.825,0.2,0.0,71,5325
|
| 43 |
+
11,SFS,Search,Forward search,1178.0,1152.0,1211.0,7.99,50.1,21.7,,0.267,0.627,2105.8,43.5,71,5324
|
| 44 |
+
12,SBE,Search,Backward search,1169.0,1133.0,1211.0,8.31,47.8,23.7,,0.132,0.56,2495.5,46.5,71,5324
|
| 45 |
+
13,LOCO,Search,Leave-one-out,1151.0,1120.0,1183.0,8.39,47.2,23.8,,0.173,0.583,241.5,7.6,71,5325
|
| 46 |
+
14,Random,Baseline,Baseline,1000.0,967.0,1030.0,10.72,30.6,32.0,,0.001,0.5,0.0,100.0,71,5325
|
| 47 |
+
15,LaplacianScore,Non-search,Distance,965.0,913.0,1013.0,11.64,24.0,36.6,,0.652,0.797,0.2,0.0,71,5325
|
| 48 |
</script><script>
|
| 49 |
function saDownload(name, blob) {
|
| 50 |
var a = document.createElement('a');
|
data/subsets/tasks_all/models_all/budgets_all/website_leaderboard.csv
CHANGED
|
@@ -1,16 +1,16 @@
|
|
| 1 |
-
Rank,
|
| 2 |
-
1,
|
| 3 |
-
2,
|
| 4 |
-
3,
|
| 5 |
-
4,
|
| 6 |
-
5,
|
| 7 |
-
6,
|
| 8 |
-
7,
|
| 9 |
-
8,
|
| 10 |
-
9,
|
| 11 |
-
10,
|
| 12 |
-
11,
|
| 13 |
-
12,
|
| 14 |
-
13,
|
| 15 |
-
14,
|
| 16 |
-
15,
|
|
|
|
| 1 |
+
Rank,Method,Category,Subgroup,Elo,Elo CI low,Elo CI high,Avg. rank,Win rate (%),Improvability (%),Error vs. all features,Stability,Validity,Selection time (s),Fallback rate (%),#Datasets,#Comparisons
|
| 2 |
+
1,RFImportance,Non-search,Tree,1310.0,1280.0,1341.0,6.15,63.2,15.1,,0.643,0.669,0.2,0.0,71,5325
|
| 3 |
+
2,CART,Non-search,Tree,1265.0,1232.0,1298.0,6.69,59.3,18.0,,0.422,0.586,0.1,0.0,71,5325
|
| 4 |
+
3,ElasticNet,Non-search,Regularization,1252.0,1219.0,1284.0,6.87,58.1,18.3,,0.447,0.76,7.6,0.0,71,5325
|
| 5 |
+
4,MI,Non-search,Information theory,1248.0,1224.0,1276.0,6.99,57.2,18.2,,0.647,0.785,0.4,0.0,71,5325
|
| 6 |
+
5,F-Test,Non-search,Correlation,1229.0,1203.0,1256.0,7.35,54.6,19.0,,0.679,0.846,0.0,0.0,71,5325
|
| 7 |
+
6,Lasso,Non-search,Regularization,1226.0,1196.0,1257.0,7.2,55.7,18.7,,0.362,0.73,2.1,0.0,71,5325
|
| 8 |
+
7,MarkovBlanket,Search,Backward search,1220.0,1191.0,1245.0,7.55,53.2,19.6,,0.413,0.463,0.1,0.0,71,5325
|
| 9 |
+
8,(R)ReliefF,Non-search,Distance,1198.0,1168.0,1228.0,8.0,50.0,20.6,,0.646,0.74,14.3,0.0,71,5325
|
| 10 |
+
9,mRMR,Non-search,Information theory,1193.0,1165.0,1221.0,7.96,50.3,19.9,,0.604,0.655,0.4,3.4,71,5325
|
| 11 |
+
10,GainRatio,Non-search,Information theory,1183.0,1157.0,1212.0,8.19,48.6,21.5,,0.644,0.825,0.2,0.0,71,5325
|
| 12 |
+
11,SFS,Search,Forward search,1178.0,1152.0,1211.0,7.99,50.1,21.7,,0.267,0.627,2105.8,43.5,71,5324
|
| 13 |
+
12,SBE,Search,Backward search,1169.0,1133.0,1211.0,8.31,47.8,23.7,,0.132,0.56,2495.5,46.5,71,5324
|
| 14 |
+
13,LOCO,Search,Leave-one-out,1151.0,1120.0,1183.0,8.39,47.2,23.8,,0.173,0.583,241.5,7.6,71,5325
|
| 15 |
+
14,Random,Baseline,Baseline,1000.0,967.0,1030.0,10.72,30.6,32.0,,0.001,0.5,0.0,100.0,71,5325
|
| 16 |
+
15,LaplacianScore,Non-search,Distance,965.0,913.0,1013.0,11.64,24.0,36.6,,0.652,0.797,0.2,0.0,71,5325
|
data/subsets/tasks_all/models_lightgbm/budgets_0/elo_explorer.html
CHANGED
|
@@ -19,67 +19,67 @@ svg { display:block; width:100%; height:auto; }
|
|
| 19 |
.sa-ink { stroke:var(--fg); }
|
| 20 |
.sa-hit { fill:transparent; }
|
| 21 |
.sa-hit:hover { fill:var(--fg); fill-opacity:0.06; }
|
| 22 |
-
</style><defs><pattern id='sa-na' width='6' height='6' patternUnits='userSpaceOnUse' patternTransform='rotate(45)'><rect width='6' height='6' fill='#e5e7eb'/><line x1='0' y1='0' x2='0' y2='6' stroke='#9ca3af' stroke-width='2'/></pattern><linearGradient id='sa-g-stab' x1='0' y1='1' x2='0' y2='0'><stop offset='0' stop-color='#ffffff'/><stop offset='1' stop-color='#0072B2'/></linearGradient><linearGradient id='sa-g-val' x1='0' y1='1' x2='0' y2='0'><stop offset='0' stop-color='#ffffff'/><stop offset='1' stop-color='#D55E00'/></linearGradient></defs><line class='sa-grid' x1='64' x2='1024' y1='317.3' y2='317.3' stroke-width='1'/><text class='sa-muted' x='56' y='321.3' text-anchor='end' font-size='12'>800</text><line class='sa-grid' x1='64' x2='1024' y1='274.6' y2='274.6' stroke-width='1'/><text class='sa-muted' x='56' y='278.6' text-anchor='end' font-size='12'>900</text><line class='sa-grid' x1='64' x2='1024' y1='231.8' y2='231.8' stroke-width='1'/><text class='sa-muted' x='56' y='235.8' text-anchor='end' font-size='12'>1000</text><line class='sa-grid' x1='64' x2='1024' y1='189.0' y2='189.0' stroke-width='1'/><text class='sa-muted' x='56' y='193.0' text-anchor='end' font-size='12'>1100</text><line class='sa-grid' x1='64' x2='1024' y1='146.3' y2='146.3' stroke-width='1'/><text class='sa-muted' x='56' y='150.3' text-anchor='end' font-size='12'>1200</text><line class='sa-grid' x1='64' x2='1024' y1='103.5' y2='103.5' stroke-width='1'/><text class='sa-muted' x='56' y='107.5' text-anchor='end' font-size='12'>1300</text><line class='sa-grid' x1='64' x2='1024' y1='60.8' y2='60.8' stroke-width='1'/><text class='sa-muted' x='56' y='64.8' text-anchor='end' font-size='12'>1400</text><line class='sa-grid' x1='64' x2='1024' y1='18.0' y2='18.0' stroke-width='1'/><text class='sa-muted' x='56' y='22.0' text-anchor='end' font-size='12'>1500</text><text class='sa-text' transform='translate(16 173) rotate(-90)' text-anchor='middle' font-size='13'>Elo</text><line class='sa-ink' x1='64' x2='1024' y1='231.8' y2='231.8' stroke-width='1' stroke-dasharray='4 4' opacity='0.5'/><rect x='73.6' y='260.0' width='22.4' height='68.0' fill='#69acd2' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='96.0' y='260.0' width='22.4' height='68.0' fill='#db7727' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='96.0' x2='96.0' y1='227.5' y2='300.2'/><line x1='91.0' x2='101.0' y1='227.5' y2='227.5'/><line x1='91.0' x2='101.0' y1='300.2' y2='300.2'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(100.0 342.0) rotate(-35)'>LaplacianScore</text><rect class='sa-hit' x='64.0' y='18' width='64' height='414'><title>LaplacianScore (
|
| 23 |
Elo 934 [95% CI 840β1010]
|
| 24 |
Stability 0.588
|
| 25 |
-
Validity 0.847</title></rect><rect x='137.6' y='231.8' width='22.4' height='96.2' fill='#ffffff' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='160.0' y='231.8' width='22.4' height='96.2' fill='#eab081' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='160.0' x2='160.0' y1='207.8' y2='260.4'/><line x1='155.0' x2='165.0' y1='207.8' y2='207.8'/><line x1='155.0' x2='165.0' y1='260.4' y2='260.4'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(164.0 342.0) rotate(-35)'>Random</text><rect class='sa-hit' x='128.0' y='18' width='64' height='414'><title>Random (Baseline)
|
| 26 |
Elo 1000 [95% CI 933β1056]
|
| 27 |
Stability -0.001
|
| 28 |
-
Validity 0.493</title></rect><rect x='201.6' y='150.1' width='22.4' height='177.9' fill='#5ba4ce' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='224.0' y='150.1' width='22.4' height='177.9' fill='#da7220' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='224.0' x2='224.0' y1='132.2' y2='168.5'/><line x1='219.0' x2='229.0' y1='132.2' y2='132.2'/><line x1='219.0' x2='229.0' y1='168.5' y2='168.5'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(228.0 342.0) rotate(-35)'>(R)ReliefF</text><rect class='sa-hit' x='192.0' y='18' width='64' height='414'><title>(R)ReliefF (
|
| 29 |
Elo 1191 [95% CI 1148β1233]
|
| 30 |
Stability 0.642
|
| 31 |
-
Validity 0.873</title></rect><rect x='265.6' y='148.8' width='22.4' height='179.2' fill='#b3d5e8' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='288.0' y='148.8' width='22.4' height='179.2' fill='#e39455' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='288.0' x2='288.0' y1='129.2' y2='169.4'/><line x1='283.0' x2='293.0' y1='129.2' y2='129.2'/><line x1='283.0' x2='293.0' y1='169.4' y2='169.4'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(292.0 342.0) rotate(-35)'>LOCO</text><rect class='sa-hit' x='256.0' y='18' width='64' height='414'><title>LOCO (
|
| 32 |
Elo 1194 [95% CI 1146β1240]
|
| 33 |
Stability 0.299
|
| 34 |
-
Validity 0.665</title></rect><rect x='329.6' y='132.2' width='22.4' height='195.8' fill='#cde4f0' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='352.0' y='132.2' width='22.4' height='195.8' fill='#e59b61' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='352.0' x2='352.0' y1='106.9' y2='154.4'/><line x1='347.0' x2='357.0' y1='106.9' y2='106.9'/><line x1='347.0' x2='357.0' y1='154.4' y2='154.4'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(356.0 342.0) rotate(-35)'>SBE</text><rect class='sa-hit' x='320.0' y='18' width='64' height='414'><title>SBE (
|
| 35 |
Elo 1233 [95% CI 1181β1292]
|
| 36 |
Stability 0.195
|
| 37 |
-
Validity 0.621</title></rect><rect x='393.6' y='131.7' width='22.4' height='196.3' fill='#529fcb' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='416.0' y='131.7' width='22.4' height='196.3' fill='#d7670f' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='416.0' x2='416.0' y1='112.1' y2='152.7'/><line x1='411.0' x2='421.0' y1='112.1' y2='112.1'/><line x1='411.0' x2='421.0' y1='152.7' y2='152.7'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(420.0 342.0) rotate(-35)'>GainRatio</text><rect class='sa-hit' x='384.0' y='18' width='64' height='414'><title>GainRatio (
|
| 38 |
Elo 1234 [95% CI 1185β1280]
|
| 39 |
Stability 0.679
|
| 40 |
-
Validity 0.942</title></rect><rect x='457.6' y='123.2' width='22.4' height='204.8' fill='#78b5d6' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='480.0' y='123.2' width='22.4' height='204.8' fill='#de8238' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='480.0' x2='480.0' y1='104.8' y2='143.7'/><line x1='475.0' x2='485.0' y1='104.8' y2='104.8'/><line x1='475.0' x2='485.0' y1='143.7' y2='143.7'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(484.0 342.0) rotate(-35)'>mRMR</text><rect class='sa-hit' x='448.0' y='18' width='64' height='414'><title>mRMR (
|
| 41 |
Elo 1254 [95% CI 1206β1297]
|
| 42 |
Stability 0.528
|
| 43 |
-
Validity 0.779</title></rect><rect x='521.6' y='116.3' width='22.4' height='211.7' fill='#94c4df' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='544.0' y='116.3' width='22.4' height='211.7' fill='#da701c' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='544.0' x2='544.0' y1='95.8' y2='133.4'/><line x1='539.0' x2='549.0' y1='95.8' y2='95.8'/><line x1='539.0' x2='549.0' y1='133.4' y2='133.4'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(548.0 342.0) rotate(-35)'>Lasso</text><rect class='sa-hit' x='512.0' y='18' width='64' height='414'><title>Lasso (
|
| 44 |
Elo 1270 [95% CI 1230β1318]
|
| 45 |
Stability 0.421
|
| 46 |
-
Validity 0.890</title></rect><rect x='585.6' y='112.1' width='22.4' height='215.9' fill='#4699c7' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='608.0' y='112.1' width='22.4' height='215.9' fill='#d7660c' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='608.0' x2='608.0' y1='91.1' y2='131.7'/><line x1='603.0' x2='613.0' y1='91.1' y2='91.1'/><line x1='603.0' x2='613.0' y1='131.7' y2='131.7'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(612.0 342.0) rotate(-35)'>F-Test</text><rect class='sa-hit' x='576.0' y='18' width='64' height='414'><title>F-Test (
|
| 47 |
Elo 1280 [95% CI 1234β1329]
|
| 48 |
Stability 0.725
|
| 49 |
-
Validity 0.952</title></rect><rect x='649.6' y='106.9' width='22.4' height='221.1' fill='#5ba4cd' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='672.0' y='106.9' width='22.4' height='221.1' fill='#d96e1a' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='672.0' x2='672.0' y1='89.0' y2='124.9'/><line x1='667.0' x2='677.0' y1='89.0' y2='89.0'/><line x1='667.0' x2='677.0' y1='124.9' y2='124.9'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(676.0 342.0) rotate(-35)'>MI</text><rect class='sa-hit' x='640.0' y='18' width='64' height='414'><title>MI (
|
| 50 |
Elo 1292 [95% CI 1250β1334]
|
| 51 |
Stability 0.645
|
| 52 |
-
Validity 0.898</title></rect><rect x='713.6' y='101.8' width='22.4' height='226.2' fill='#7db7d8' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='736.0' y='101.8' width='22.4' height='226.2' fill='#d96c16' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='736.0' x2='736.0' y1='81.3' y2='119.8'/><line x1='731.0' x2='741.0' y1='81.3' y2='81.3'/><line x1='731.0' x2='741.0' y1='119.8' y2='119.8'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(740.0 342.0) rotate(-35)'>ElasticNet</text><rect class='sa-hit' x='704.0' y='18' width='64' height='414'><title>ElasticNet (
|
| 53 |
Elo 1304 [95% CI 1262β1352]
|
| 54 |
Stability 0.509
|
| 55 |
-
Validity 0.915</title></rect><rect x='777.6' y='98.4' width='22.4' height='229.6' fill='#a1cbe3' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='800.0' y='98.4' width='22.4' height='229.6' fill='#e08640' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='800.0' x2='800.0' y1='74.4' y2='120.6'/><line x1='795.0' x2='805.0' y1='74.4' y2='74.4'/><line x1='795.0' x2='805.0' y1='120.6' y2='120.6'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(804.0 342.0) rotate(-35)'>SFS</text><rect class='sa-hit' x='768.0' y='18' width='64' height='414'><title>SFS (
|
| 56 |
Elo 1312 [95% CI 1260β1368]
|
| 57 |
Stability 0.368
|
| 58 |
-
Validity 0.750</title></rect><rect x='841.6' y='92.8' width='22.4' height='235.2' fill='#94c4df' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='864.0' y='92.8' width='22.4' height='235.2' fill='#ebb488' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='864.0' x2='864.0' y1='71.9' y2='115.1'/><line x1='859.0' x2='869.0' y1='71.9' y2='71.9'/><line x1='859.0' x2='869.0' y1='115.1' y2='115.1'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(868.0 342.0) rotate(-35)'>MarkovBlanket</text><rect class='sa-hit' x='832.0' y='18' width='64' height='414'><title>MarkovBlanket (
|
| 59 |
Elo 1325 [95% CI 1273β1374]
|
| 60 |
Stability 0.418
|
| 61 |
-
Validity 0.465</title></rect><rect x='905.6' y='83.4' width='22.4' height='244.6' fill='#84bbda' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='928.0' y='83.4' width='22.4' height='244.6' fill='#e08945' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='928.0' x2='928.0' y1='60.8' y2='103.5'/><line x1='923.0' x2='933.0' y1='60.8' y2='60.8'/><line x1='923.0' x2='933.0' y1='103.5' y2='103.5'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(932.0 342.0) rotate(-35)'>CART</text><rect class='sa-hit' x='896.0' y='18' width='64' height='414'><title>CART (
|
| 62 |
Elo 1347 [95% CI 1300β1400]
|
| 63 |
Stability 0.484
|
| 64 |
-
Validity 0.731</title></rect><rect x='969.6' y='74.4' width='22.4' height='253.6' fill='#5fa6cf' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='992.0' y='74.4' width='22.4' height='253.6' fill='#dd7d32' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='992.0' x2='992.0' y1='46.6' y2='98.8'/><line x1='987.0' x2='997.0' y1='46.6' y2='46.6'/><line x1='987.0' x2='997.0' y1='98.8' y2='98.8'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(996.0 342.0) rotate(-35)'>RFImportance</text><rect class='sa-hit' x='960.0' y='18' width='64' height='414'><title>RFImportance (
|
| 65 |
Elo 1368 [95% CI 1311β1433]
|
| 66 |
Stability 0.628
|
| 67 |
-
Validity 0.805</title></rect><line class='sa-ink' x1='64' x2='1024' y1='328.0' y2='328.0' stroke-width='1' opacity='0.4'/><rect x='1070' y='40.0' width='12' height='107.0' fill='url(#sa-g-stab)' stroke='#000' stroke-width='0.6'/><text class='sa-text' x='1076' y='30.0' text-anchor='middle' font-size='12'>stable</text><text class='sa-text' x='1076' y='165.0' text-anchor='middle' font-size='12'>unstable</text><text class='sa-muted' x='1088' y='151.0' font-size='11'>0.0</text><text class='sa-muted' x='1088' y='97.5' font-size='11'>0.5</text><text class='sa-muted' x='1088' y='44.0' font-size='11'>1.0</text><rect x='1070' y='207.0' width='12' height='107.0' fill='url(#sa-g-val)' stroke='#000' stroke-width='0.6'/><text class='sa-text' x='1076' y='197.0' text-anchor='middle' font-size='12'>valid</text><text class='sa-text' x='1076' y='332.0' text-anchor='middle' font-size='12'>invalid</text><text class='sa-muted' x='1088' y='318.0' font-size='11'>0.0</text><text class='sa-muted' x='1088' y='264.5' font-size='11'>0.5</text><text class='sa-muted' x='1088' y='211.0' font-size='11'>1.0</text><rect x='1070' y='352.0' width='12' height='12' fill='url(#sa-na)' stroke='#000' stroke-width='0.6'/><text class='sa-muted' x='1088' y='362.0' font-size='11'>no result</text></svg><script type='text/plain' id='sa-csv'>Method,
|
| 68 |
-
RFImportance,
|
| 69 |
-
CART,
|
| 70 |
-
MarkovBlanket,
|
| 71 |
-
SFS,
|
| 72 |
-
ElasticNet,
|
| 73 |
-
MI,
|
| 74 |
-
F-Test,
|
| 75 |
-
Lasso,
|
| 76 |
-
mRMR,
|
| 77 |
-
GainRatio,
|
| 78 |
-
SBE,
|
| 79 |
-
LOCO,
|
| 80 |
-
(R)ReliefF,
|
| 81 |
-
Random,Baseline,1000.0,933.0,1056.0,-0.001,0.493
|
| 82 |
-
LaplacianScore,
|
| 83 |
</script><script>
|
| 84 |
function saDownload(name, blob) {
|
| 85 |
var a = document.createElement('a');
|
|
|
|
| 19 |
.sa-ink { stroke:var(--fg); }
|
| 20 |
.sa-hit { fill:transparent; }
|
| 21 |
.sa-hit:hover { fill:var(--fg); fill-opacity:0.06; }
|
| 22 |
+
</style><defs><pattern id='sa-na' width='6' height='6' patternUnits='userSpaceOnUse' patternTransform='rotate(45)'><rect width='6' height='6' fill='#e5e7eb'/><line x1='0' y1='0' x2='0' y2='6' stroke='#9ca3af' stroke-width='2'/></pattern><linearGradient id='sa-g-stab' x1='0' y1='1' x2='0' y2='0'><stop offset='0' stop-color='#ffffff'/><stop offset='1' stop-color='#0072B2'/></linearGradient><linearGradient id='sa-g-val' x1='0' y1='1' x2='0' y2='0'><stop offset='0' stop-color='#ffffff'/><stop offset='1' stop-color='#D55E00'/></linearGradient></defs><line class='sa-grid' x1='64' x2='1024' y1='317.3' y2='317.3' stroke-width='1'/><text class='sa-muted' x='56' y='321.3' text-anchor='end' font-size='12'>800</text><line class='sa-grid' x1='64' x2='1024' y1='274.6' y2='274.6' stroke-width='1'/><text class='sa-muted' x='56' y='278.6' text-anchor='end' font-size='12'>900</text><line class='sa-grid' x1='64' x2='1024' y1='231.8' y2='231.8' stroke-width='1'/><text class='sa-muted' x='56' y='235.8' text-anchor='end' font-size='12'>1000</text><line class='sa-grid' x1='64' x2='1024' y1='189.0' y2='189.0' stroke-width='1'/><text class='sa-muted' x='56' y='193.0' text-anchor='end' font-size='12'>1100</text><line class='sa-grid' x1='64' x2='1024' y1='146.3' y2='146.3' stroke-width='1'/><text class='sa-muted' x='56' y='150.3' text-anchor='end' font-size='12'>1200</text><line class='sa-grid' x1='64' x2='1024' y1='103.5' y2='103.5' stroke-width='1'/><text class='sa-muted' x='56' y='107.5' text-anchor='end' font-size='12'>1300</text><line class='sa-grid' x1='64' x2='1024' y1='60.8' y2='60.8' stroke-width='1'/><text class='sa-muted' x='56' y='64.8' text-anchor='end' font-size='12'>1400</text><line class='sa-grid' x1='64' x2='1024' y1='18.0' y2='18.0' stroke-width='1'/><text class='sa-muted' x='56' y='22.0' text-anchor='end' font-size='12'>1500</text><text class='sa-text' transform='translate(16 173) rotate(-90)' text-anchor='middle' font-size='13'>Elo</text><line class='sa-ink' x1='64' x2='1024' y1='231.8' y2='231.8' stroke-width='1' stroke-dasharray='4 4' opacity='0.5'/><rect x='73.6' y='260.0' width='22.4' height='68.0' fill='#69acd2' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='96.0' y='260.0' width='22.4' height='68.0' fill='#db7727' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='96.0' x2='96.0' y1='227.5' y2='300.2'/><line x1='91.0' x2='101.0' y1='227.5' y2='227.5'/><line x1='91.0' x2='101.0' y1='300.2' y2='300.2'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(100.0 342.0) rotate(-35)'>LaplacianScore</text><rect class='sa-hit' x='64.0' y='18' width='64' height='414'><title>LaplacianScore (Distance, non-search)
|
| 23 |
Elo 934 [95% CI 840β1010]
|
| 24 |
Stability 0.588
|
| 25 |
+
Validity 0.847</title></rect><rect x='137.6' y='231.8' width='22.4' height='96.2' fill='#ffffff' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='160.0' y='231.8' width='22.4' height='96.2' fill='#eab081' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='160.0' x2='160.0' y1='207.8' y2='260.4'/><line x1='155.0' x2='165.0' y1='207.8' y2='207.8'/><line x1='155.0' x2='165.0' y1='260.4' y2='260.4'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(164.0 342.0) rotate(-35)'>Random</text><rect class='sa-hit' x='128.0' y='18' width='64' height='414'><title>Random (Baseline, baseline)
|
| 26 |
Elo 1000 [95% CI 933β1056]
|
| 27 |
Stability -0.001
|
| 28 |
+
Validity 0.493</title></rect><rect x='201.6' y='150.1' width='22.4' height='177.9' fill='#5ba4ce' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='224.0' y='150.1' width='22.4' height='177.9' fill='#da7220' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='224.0' x2='224.0' y1='132.2' y2='168.5'/><line x1='219.0' x2='229.0' y1='132.2' y2='132.2'/><line x1='219.0' x2='229.0' y1='168.5' y2='168.5'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(228.0 342.0) rotate(-35)'>(R)ReliefF</text><rect class='sa-hit' x='192.0' y='18' width='64' height='414'><title>(R)ReliefF (Distance, non-search)
|
| 29 |
Elo 1191 [95% CI 1148β1233]
|
| 30 |
Stability 0.642
|
| 31 |
+
Validity 0.873</title></rect><rect x='265.6' y='148.8' width='22.4' height='179.2' fill='#b3d5e8' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='288.0' y='148.8' width='22.4' height='179.2' fill='#e39455' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='288.0' x2='288.0' y1='129.2' y2='169.4'/><line x1='283.0' x2='293.0' y1='129.2' y2='129.2'/><line x1='283.0' x2='293.0' y1='169.4' y2='169.4'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(292.0 342.0) rotate(-35)'>LOCO</text><rect class='sa-hit' x='256.0' y='18' width='64' height='414'><title>LOCO (Leave-one-out, search)
|
| 32 |
Elo 1194 [95% CI 1146β1240]
|
| 33 |
Stability 0.299
|
| 34 |
+
Validity 0.665</title></rect><rect x='329.6' y='132.2' width='22.4' height='195.8' fill='#cde4f0' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='352.0' y='132.2' width='22.4' height='195.8' fill='#e59b61' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='352.0' x2='352.0' y1='106.9' y2='154.4'/><line x1='347.0' x2='357.0' y1='106.9' y2='106.9'/><line x1='347.0' x2='357.0' y1='154.4' y2='154.4'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(356.0 342.0) rotate(-35)'>SBE</text><rect class='sa-hit' x='320.0' y='18' width='64' height='414'><title>SBE (Backward search, search)
|
| 35 |
Elo 1233 [95% CI 1181β1292]
|
| 36 |
Stability 0.195
|
| 37 |
+
Validity 0.621</title></rect><rect x='393.6' y='131.7' width='22.4' height='196.3' fill='#529fcb' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='416.0' y='131.7' width='22.4' height='196.3' fill='#d7670f' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='416.0' x2='416.0' y1='112.1' y2='152.7'/><line x1='411.0' x2='421.0' y1='112.1' y2='112.1'/><line x1='411.0' x2='421.0' y1='152.7' y2='152.7'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(420.0 342.0) rotate(-35)'>GainRatio</text><rect class='sa-hit' x='384.0' y='18' width='64' height='414'><title>GainRatio (Information theory, non-search)
|
| 38 |
Elo 1234 [95% CI 1185β1280]
|
| 39 |
Stability 0.679
|
| 40 |
+
Validity 0.942</title></rect><rect x='457.6' y='123.2' width='22.4' height='204.8' fill='#78b5d6' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='480.0' y='123.2' width='22.4' height='204.8' fill='#de8238' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='480.0' x2='480.0' y1='104.8' y2='143.7'/><line x1='475.0' x2='485.0' y1='104.8' y2='104.8'/><line x1='475.0' x2='485.0' y1='143.7' y2='143.7'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(484.0 342.0) rotate(-35)'>mRMR</text><rect class='sa-hit' x='448.0' y='18' width='64' height='414'><title>mRMR (Information theory, non-search)
|
| 41 |
Elo 1254 [95% CI 1206β1297]
|
| 42 |
Stability 0.528
|
| 43 |
+
Validity 0.779</title></rect><rect x='521.6' y='116.3' width='22.4' height='211.7' fill='#94c4df' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='544.0' y='116.3' width='22.4' height='211.7' fill='#da701c' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='544.0' x2='544.0' y1='95.8' y2='133.4'/><line x1='539.0' x2='549.0' y1='95.8' y2='95.8'/><line x1='539.0' x2='549.0' y1='133.4' y2='133.4'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(548.0 342.0) rotate(-35)'>Lasso</text><rect class='sa-hit' x='512.0' y='18' width='64' height='414'><title>Lasso (Regularization, non-search)
|
| 44 |
Elo 1270 [95% CI 1230β1318]
|
| 45 |
Stability 0.421
|
| 46 |
+
Validity 0.890</title></rect><rect x='585.6' y='112.1' width='22.4' height='215.9' fill='#4699c7' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='608.0' y='112.1' width='22.4' height='215.9' fill='#d7660c' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='608.0' x2='608.0' y1='91.1' y2='131.7'/><line x1='603.0' x2='613.0' y1='91.1' y2='91.1'/><line x1='603.0' x2='613.0' y1='131.7' y2='131.7'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(612.0 342.0) rotate(-35)'>F-Test</text><rect class='sa-hit' x='576.0' y='18' width='64' height='414'><title>F-Test (Correlation, non-search)
|
| 47 |
Elo 1280 [95% CI 1234β1329]
|
| 48 |
Stability 0.725
|
| 49 |
+
Validity 0.952</title></rect><rect x='649.6' y='106.9' width='22.4' height='221.1' fill='#5ba4cd' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='672.0' y='106.9' width='22.4' height='221.1' fill='#d96e1a' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='672.0' x2='672.0' y1='89.0' y2='124.9'/><line x1='667.0' x2='677.0' y1='89.0' y2='89.0'/><line x1='667.0' x2='677.0' y1='124.9' y2='124.9'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(676.0 342.0) rotate(-35)'>MI</text><rect class='sa-hit' x='640.0' y='18' width='64' height='414'><title>MI (Information theory, non-search)
|
| 50 |
Elo 1292 [95% CI 1250β1334]
|
| 51 |
Stability 0.645
|
| 52 |
+
Validity 0.898</title></rect><rect x='713.6' y='101.8' width='22.4' height='226.2' fill='#7db7d8' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='736.0' y='101.8' width='22.4' height='226.2' fill='#d96c16' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='736.0' x2='736.0' y1='81.3' y2='119.8'/><line x1='731.0' x2='741.0' y1='81.3' y2='81.3'/><line x1='731.0' x2='741.0' y1='119.8' y2='119.8'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(740.0 342.0) rotate(-35)'>ElasticNet</text><rect class='sa-hit' x='704.0' y='18' width='64' height='414'><title>ElasticNet (Regularization, non-search)
|
| 53 |
Elo 1304 [95% CI 1262β1352]
|
| 54 |
Stability 0.509
|
| 55 |
+
Validity 0.915</title></rect><rect x='777.6' y='98.4' width='22.4' height='229.6' fill='#a1cbe3' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='800.0' y='98.4' width='22.4' height='229.6' fill='#e08640' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='800.0' x2='800.0' y1='74.4' y2='120.6'/><line x1='795.0' x2='805.0' y1='74.4' y2='74.4'/><line x1='795.0' x2='805.0' y1='120.6' y2='120.6'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(804.0 342.0) rotate(-35)'>SFS</text><rect class='sa-hit' x='768.0' y='18' width='64' height='414'><title>SFS (Forward search, search)
|
| 56 |
Elo 1312 [95% CI 1260β1368]
|
| 57 |
Stability 0.368
|
| 58 |
+
Validity 0.750</title></rect><rect x='841.6' y='92.8' width='22.4' height='235.2' fill='#94c4df' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='864.0' y='92.8' width='22.4' height='235.2' fill='#ebb488' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='864.0' x2='864.0' y1='71.9' y2='115.1'/><line x1='859.0' x2='869.0' y1='71.9' y2='71.9'/><line x1='859.0' x2='869.0' y1='115.1' y2='115.1'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(868.0 342.0) rotate(-35)'>MarkovBlanket</text><rect class='sa-hit' x='832.0' y='18' width='64' height='414'><title>MarkovBlanket (Backward search, search)
|
| 59 |
Elo 1325 [95% CI 1273β1374]
|
| 60 |
Stability 0.418
|
| 61 |
+
Validity 0.465</title></rect><rect x='905.6' y='83.4' width='22.4' height='244.6' fill='#84bbda' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='928.0' y='83.4' width='22.4' height='244.6' fill='#e08945' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='928.0' x2='928.0' y1='60.8' y2='103.5'/><line x1='923.0' x2='933.0' y1='60.8' y2='60.8'/><line x1='923.0' x2='933.0' y1='103.5' y2='103.5'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(932.0 342.0) rotate(-35)'>CART</text><rect class='sa-hit' x='896.0' y='18' width='64' height='414'><title>CART (Tree, non-search)
|
| 62 |
Elo 1347 [95% CI 1300β1400]
|
| 63 |
Stability 0.484
|
| 64 |
+
Validity 0.731</title></rect><rect x='969.6' y='74.4' width='22.4' height='253.6' fill='#5fa6cf' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='992.0' y='74.4' width='22.4' height='253.6' fill='#dd7d32' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='992.0' x2='992.0' y1='46.6' y2='98.8'/><line x1='987.0' x2='997.0' y1='46.6' y2='46.6'/><line x1='987.0' x2='997.0' y1='98.8' y2='98.8'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(996.0 342.0) rotate(-35)'>RFImportance</text><rect class='sa-hit' x='960.0' y='18' width='64' height='414'><title>RFImportance (Tree, non-search)
|
| 65 |
Elo 1368 [95% CI 1311β1433]
|
| 66 |
Stability 0.628
|
| 67 |
+
Validity 0.805</title></rect><line class='sa-ink' x1='64' x2='1024' y1='328.0' y2='328.0' stroke-width='1' opacity='0.4'/><rect x='1070' y='40.0' width='12' height='107.0' fill='url(#sa-g-stab)' stroke='#000' stroke-width='0.6'/><text class='sa-text' x='1076' y='30.0' text-anchor='middle' font-size='12'>stable</text><text class='sa-text' x='1076' y='165.0' text-anchor='middle' font-size='12'>unstable</text><text class='sa-muted' x='1088' y='151.0' font-size='11'>0.0</text><text class='sa-muted' x='1088' y='97.5' font-size='11'>0.5</text><text class='sa-muted' x='1088' y='44.0' font-size='11'>1.0</text><rect x='1070' y='207.0' width='12' height='107.0' fill='url(#sa-g-val)' stroke='#000' stroke-width='0.6'/><text class='sa-text' x='1076' y='197.0' text-anchor='middle' font-size='12'>valid</text><text class='sa-text' x='1076' y='332.0' text-anchor='middle' font-size='12'>invalid</text><text class='sa-muted' x='1088' y='318.0' font-size='11'>0.0</text><text class='sa-muted' x='1088' y='264.5' font-size='11'>0.5</text><text class='sa-muted' x='1088' y='211.0' font-size='11'>1.0</text><rect x='1070' y='352.0' width='12' height='12' fill='url(#sa-na)' stroke='#000' stroke-width='0.6'/><text class='sa-muted' x='1088' y='362.0' font-size='11'>no result</text></svg><script type='text/plain' id='sa-csv'>Method,Category,Subgroup,Elo,Elo CI low,Elo CI high,Stability,Validity
|
| 68 |
+
RFImportance,Non-search,Tree,1368.0,1311.0,1433.0,0.628,0.805
|
| 69 |
+
CART,Non-search,Tree,1347.0,1300.0,1400.0,0.484,0.731
|
| 70 |
+
MarkovBlanket,Search,Backward search,1325.0,1273.0,1374.0,0.418,0.465
|
| 71 |
+
SFS,Search,Forward search,1312.0,1260.0,1368.0,0.368,0.75
|
| 72 |
+
ElasticNet,Non-search,Regularization,1304.0,1262.0,1352.0,0.509,0.915
|
| 73 |
+
MI,Non-search,Information theory,1292.0,1250.0,1334.0,0.645,0.898
|
| 74 |
+
F-Test,Non-search,Correlation,1280.0,1234.0,1329.0,0.725,0.952
|
| 75 |
+
Lasso,Non-search,Regularization,1270.0,1230.0,1318.0,0.421,0.89
|
| 76 |
+
mRMR,Non-search,Information theory,1254.0,1206.0,1297.0,0.528,0.779
|
| 77 |
+
GainRatio,Non-search,Information theory,1234.0,1185.0,1280.0,0.679,0.942
|
| 78 |
+
SBE,Search,Backward search,1233.0,1181.0,1292.0,0.195,0.621
|
| 79 |
+
LOCO,Search,Leave-one-out,1194.0,1146.0,1240.0,0.299,0.665
|
| 80 |
+
(R)ReliefF,Non-search,Distance,1191.0,1148.0,1233.0,0.642,0.873
|
| 81 |
+
Random,Baseline,Baseline,1000.0,933.0,1056.0,-0.001,0.493
|
| 82 |
+
LaplacianScore,Non-search,Distance,934.0,840.0,1010.0,0.588,0.847
|
| 83 |
</script><script>
|
| 84 |
function saDownload(name, blob) {
|
| 85 |
var a = document.createElement('a');
|
data/subsets/tasks_all/models_lightgbm/budgets_0/leaderboard_table.html
CHANGED
|
@@ -26,23 +26,25 @@ td.method { font-weight:600; }
|
|
| 26 |
td .sub { color:var(--muted); font-size:12px; font-weight:400; margin-left:6px; }
|
| 27 |
td.na { color:var(--muted); }
|
| 28 |
.pill { display:inline-block; padding:0 8px; border:1px solid var(--line); border-radius:999px;
|
| 29 |
-
font-size:12px; color:var(--
|
| 30 |
-
</style></head><body><div class='toolbar'><span class='hint'>Click a column to sort. Hover a header for its definition.</span><button onclick="saCsv('selectarena_leaderboard.csv')">CSV</button></div><div class='wrap'><table id='sa-table'><thead><tr><th class='num'>#</th><th class=''>Method</th><th class='' title='Filter: scores features from the data alone. Wrapper: searches feature subsets by refitting a proxy model. Embedded: reads the selection off a fitted model.'>Type</th><th class='num' title='Bradley-Terry Elo from pairwise matches within each dataset, budget and downstream model. Random = 1000. +/β give the 95% bootstrap interval.'>Elo β</th><th class='num' title='Mean rank of the downstream error.'>Avg. rank β</th><th class='num' title='Share of pairwise comparisons won, in %. Ties count half.'>Win rate β</th><th class='num' title='Mean relative gap (error β best error) / error to the best method, in %.'>Improvability β</th><th class='num' title='Mean ratio of the error to the same model trained on all features. Below 1: the selection helped.'>Error vs. all β</th><th class='num' title='Agreement of the selected features across bootstrap resamples (Nogueira et al., 2018).'>Stability β</th><th class='num' title='Share of the budget filled with original features when as many noise features are added.'>Validity β</th><th class='num' title='Median wall-clock time of the selection.'>Time (s) β</th><th class='num' title='Share of runs where the method ran out of time and random features filled the budget.'>Fallback (%)</th><th class='num'>#Datasets</th></tr></thead><tbody><tr><td class='num' data-sort='1'>1</td><td class='method' data-sort='RFImportance'>RFImportance</td><td class='' data-sort='Embedded'><span class='pill'>π³ Embedded</span></td><td class='num' data-sort='1368.0'>1368<span class='sub'>+65 / β57</span></td><td class='num' data-sort='6.11'>6.11</td><td class='num' data-sort='63.5'>63.5</td><td class='num' data-sort='18.1'>18.1</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.628'>0.628</td><td class='num' data-sort='0.805'>0.805</td><td class='num' data-sort='0.2'>0.2</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='2'>2</td><td class='method' data-sort='CART'>CART</td><td class='' data-sort='Embedded'><span class='pill'>π³ Embedded</span></td><td class='num' data-sort='1347.0'>1347<span class='sub'>+53 / β47</span></td><td class='num' data-sort='6.29'>6.29</td><td class='num' data-sort='62.2'>62.2</td><td class='num' data-sort='20.1'>20.1</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.484'>0.484</td><td class='num' data-sort='0.731'>0.731</td><td class='num' data-sort='0.1'>0.1</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='3'>3</td><td class='method' data-sort='MarkovBlanket'>MarkovBlanket</td><td class='' data-sort='Filter'><span class='pill'>π Filter</span></td><td class='num' data-sort='1325.0'>1325<span class='sub'>+49 / β52</span></td><td class='num' data-sort='6.54'>6.54</td><td class='num' data-sort='60.4'>60.4</td><td class='num' data-sort='21.1'>21.1</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.418'>0.418</td><td class='num' data-sort='0.465'>0.465</td><td class='num' data-sort='0.1'>0.1</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='4'>4</td><td class='method' data-sort='SFS'>SFS</td><td class='' data-sort='Wrapper'><span class='pill'>π Wrapper</span></td><td class='num' data-sort='1312.0'>1312<span class='sub'>+56 / β52</span></td><td class='num' data-sort='6.76'>6.76</td><td class='num' data-sort='58.8'>58.8</td><td class='num' data-sort='21.5'>21.5</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.368'>0.368</td><td class='num' data-sort='0.75'>0.750</td><td class='num' data-sort='1052.3'>1,052.3</td><td class='num' data-sort='29.1'>29.1</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='5'>5</td><td class='method' data-sort='ElasticNet'>ElasticNet</td><td class='' data-sort='Embedded'><span class='pill'>π³ Embedded</span></td><td class='num' data-sort='1304.0'>1304<span class='sub'>+48 / β42</span></td><td class='num' data-sort='7.01'>7.01</td><td class='num' data-sort='57.1'>57.1</td><td class='num' data-sort='22.5'>22.5</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.509'>0.509</td><td class='num' data-sort='0.915'>0.915</td><td class='num' data-sort='6.1'>6.1</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='6'>6</td><td class='method' data-sort='MI'>MI</td><td class='' data-sort='Filter'><span class='pill'>π Filter</span></td><td class='num' data-sort='1292.0'>1292<span class='sub'>+42 / β42</span></td><td class='num' data-sort='7.12'>7.12</td><td class='num' data-sort='56.3'>56.3</td><td class='num' data-sort='22.0'>22.0</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.645'>0.645</td><td class='num' data-sort='0.898'>0.898</td><td class='num' data-sort='0.4'>0.4</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='7'>7</td><td class='method' data-sort='F-Test'>F-Test</td><td class='' data-sort='Filter'><span class='pill'>π Filter</span></td><td class='num' data-sort='1280.0'>1280<span class='sub'>+49 / β46</span></td><td class='num' data-sort='7.47'>7.47</td><td class='num' data-sort='53.8'>53.8</td><td class='num' data-sort='22.8'>22.8</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.725'>0.725</td><td class='num' data-sort='0.952'>0.952</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='8'>8</td><td class='method' data-sort='Lasso'>Lasso</td><td class='' data-sort='Embedded'><span class='pill'>π³ Embedded</span></td><td class='num' data-sort='1270.0'>1270<span class='sub'>+48 / β40</span></td><td class='num' data-sort='7.48'>7.48</td><td class='num' data-sort='53.7'>53.7</td><td class='num' data-sort='24.0'>24.0</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.421'>0.421</td><td class='num' data-sort='0.89'>0.890</td><td class='num' data-sort='2.0'>2.0</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='9'>9</td><td class='method' data-sort='mRMR'>mRMR</td><td class='' data-sort='Filter'><span class='pill'>π Filter</span></td><td class='num' data-sort='1254.0'>1254<span class='sub'>+43 / β48</span></td><td class='num' data-sort='7.73'>7.73</td><td class='num' data-sort='52.0'>52.0</td><td class='num' data-sort='22.7'>22.7</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.528'>0.528</td><td class='num' data-sort='0.779'>0.779</td><td class='num' data-sort='0.3'>0.3</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='10'>10</td><td class='method' data-sort='GainRatio'>GainRatio</td><td class='' data-sort='Filter'><span class='pill'>π Filter</span></td><td class='num' data-sort='1234.0'>1234<span class='sub'>+46 / β49</span></td><td class='num' data-sort='8.12'>8.12</td><td class='num' data-sort='49.2'>49.2</td><td class='num' data-sort='25.1'>25.1</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.679'>0.679</td><td class='num' data-sort='0.942'>0.942</td><td class='num' data-sort='0.2'>0.2</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='11'>11</td><td class='method' data-sort='SBE'>SBE</td><td class='' data-sort='Wrapper'><span class='pill'>π Wrapper</span></td><td class='num' data-sort='1233.0'>1233<span class='sub'>+59 / β52</span></td><td class='num' data-sort='8.28'>8.28</td><td class='num' data-sort='48.0'>48.0</td><td class='num' data-sort='27.0'>27.0</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.195'>0.195</td><td class='num' data-sort='0.621'>0.621</td><td class='num' data-sort='3579.1'>3,579.1</td><td class='num' data-sort='49.3'>49.3</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='12'>12</td><td class='method' data-sort='LOCO'>LOCO</td><td class='' data-sort='Wrapper'><span class='pill'>π Wrapper</span></td><td class='num' data-sort='1194.0'>1194<span class='sub'>+46 / β48</span></td><td class='num' data-sort='8.42'>8.42</td><td class='num' data-sort='47.0'>47.0</td><td class='num' data-sort='26.9'>26.9</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.299'>0.299</td><td class='num' data-sort='0.665'>0.665</td><td class='num' data-sort='240.9'>240.9</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='13'>13</td><td class='method' data-sort='(R)ReliefF'>(R)ReliefF</td><td class='' data-sort='Filter'><span class='pill'>π Filter</span></td><td class='num' data-sort='1191.0'>1191<span class='sub'>+42 / β43</span></td><td class='num' data-sort='8.79'>8.79</td><td class='num' data-sort='44.4'>44.4</td><td class='num' data-sort='28.4'>28.4</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.642'>0.642</td><td class='num' data-sort='0.873'>0.873</td><td class='num' data-sort='13.2'>13.2</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='14'>14</td><td class='method' data-sort='Random'>Random</td><td class='' data-sort='Baseline'><span class='pill'>βͺ Baseline</span></td><td class='num' data-sort='1000.0'>1000<span class='sub'>+56 / β67</span></td><td class='num' data-sort='11.43'>11.43</td><td class='num' data-sort='25.5'>25.5</td><td class='num' data-sort='39.7'>39.7</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='-0.001'>-0.001</td><td class='num' data-sort='0.493'>0.493</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='100.0'>100.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='15'>15</td><td class='method' data-sort='LaplacianScore'>LaplacianScore</td><td class='' data-sort='Filter'><span class='pill'>π Filter</span></td><td class='num' data-sort='934.0'>934<span class='sub'>+76 / β94</span></td><td class='num' data-sort='12.45'>12.45</td><td class='num' data-sort='18.2'>18.2</td><td class='num' data-sort='44.3'>44.3</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.588'>0.588</td><td class='num' data-sort='0.847'>0.847</td><td class='num' data-sort='0.2'>0.2</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr></tbody></table></div><script type='text/plain' id='sa-csv'>Rank,Type,TypeName,Method,Elo,Elo CI low,Elo CI high,Avg. rank,Win rate (%),Improvability (%),Error vs. all features,Stability,Validity,Selection time (s),Fallback rate (%),#Datasets,#Comparisons
|
| 31 |
-
|
| 32 |
-
2,π³,Embedded,CART,1347.0,1300.0,1400.0,6.29,62.2,20.1,,0.484,0.731,0.1,0.0,71,213
|
| 33 |
-
|
| 34 |
-
|
| 35 |
-
|
| 36 |
-
|
| 37 |
-
|
| 38 |
-
|
| 39 |
-
|
| 40 |
-
|
| 41 |
-
|
| 42 |
-
|
| 43 |
-
|
| 44 |
-
|
| 45 |
-
|
|
|
|
|
|
|
| 46 |
</script><script>
|
| 47 |
function saDownload(name, blob) {
|
| 48 |
var a = document.createElement('a');
|
|
|
|
| 26 |
td .sub { color:var(--muted); font-size:12px; font-weight:400; margin-left:6px; }
|
| 27 |
td.na { color:var(--muted); }
|
| 28 |
.pill { display:inline-block; padding:0 8px; border:1px solid var(--line); border-radius:999px;
|
| 29 |
+
font-size:12px; color:var(--fg); }
|
| 30 |
+
.pill.cat-search { border-color:var(--accent); }
|
| 31 |
+
.pill.cat-baseline { color:var(--muted); }
|
| 32 |
+
</style></head><body><div class='toolbar'><span class='hint'>Click a column to sort. Hover a header for its definition.</span><button onclick="saCsv('selectarena_leaderboard.csv')">CSV</button></div><div class='wrap'><table id='sa-table'><thead><tr><th class='num'>#</th><th class=''>Method</th><th class='' title='Taxonomy of the paper (Appendix A). Search methods evaluate many feature subsets with the model (forward, backward, leave-one-out); non-search methods use a proxy instead (correlation, distance, information theory, tree, regularization).'>Type</th><th class='num' title='Bradley-Terry Elo from pairwise matches within each dataset, budget and downstream model. Random = 1000. +/β give the 95% bootstrap interval.'>Elo β</th><th class='num' title='Mean rank of the downstream error.'>Avg. rank β</th><th class='num' title='Share of pairwise comparisons won, in %. Ties count half.'>Win rate β</th><th class='num' title='Mean relative gap (error β best error) / error to the best method, in %.'>Improvability β</th><th class='num' title='Mean ratio of the error to the same model trained on all features. Below 1: the selection helped.'>Error vs. all β</th><th class='num' title='Agreement of the selected features across bootstrap resamples (Nogueira et al., 2018).'>Stability β</th><th class='num' title='Share of the budget filled with original features when as many noise features are added.'>Validity β</th><th class='num' title='Median wall-clock time of the selection.'>Time (s) β</th><th class='num' title='Share of runs where the method ran out of time and random features filled the budget.'>Fallback (%)</th><th class='num'>#Datasets</th></tr></thead><tbody><tr><td class='num' data-sort='1'>1</td><td class='method' data-sort='RFImportance'>RFImportance</td><td class='' data-sort='Tree'><span class='pill cat-non-search'>Tree</span><span class='sub'>non-search</span></td><td class='num' data-sort='1368.0'>1368<span class='sub'>+65 / β57</span></td><td class='num' data-sort='6.11'>6.11</td><td class='num' data-sort='63.5'>63.5</td><td class='num' data-sort='18.1'>18.1</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.628'>0.628</td><td class='num' data-sort='0.805'>0.805</td><td class='num' data-sort='0.2'>0.2</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='2'>2</td><td class='method' data-sort='CART'>CART</td><td class='' data-sort='Tree'><span class='pill cat-non-search'>Tree</span><span class='sub'>non-search</span></td><td class='num' data-sort='1347.0'>1347<span class='sub'>+53 / β47</span></td><td class='num' data-sort='6.29'>6.29</td><td class='num' data-sort='62.2'>62.2</td><td class='num' data-sort='20.1'>20.1</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.484'>0.484</td><td class='num' data-sort='0.731'>0.731</td><td class='num' data-sort='0.1'>0.1</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='3'>3</td><td class='method' data-sort='MarkovBlanket'>MarkovBlanket</td><td class='' data-sort='Backward search'><span class='pill cat-search'>Backward search</span><span class='sub'>search</span></td><td class='num' data-sort='1325.0'>1325<span class='sub'>+49 / β52</span></td><td class='num' data-sort='6.54'>6.54</td><td class='num' data-sort='60.4'>60.4</td><td class='num' data-sort='21.1'>21.1</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.418'>0.418</td><td class='num' data-sort='0.465'>0.465</td><td class='num' data-sort='0.1'>0.1</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='4'>4</td><td class='method' data-sort='SFS'>SFS</td><td class='' data-sort='Forward search'><span class='pill cat-search'>Forward search</span><span class='sub'>search</span></td><td class='num' data-sort='1312.0'>1312<span class='sub'>+56 / β52</span></td><td class='num' data-sort='6.76'>6.76</td><td class='num' data-sort='58.8'>58.8</td><td class='num' data-sort='21.5'>21.5</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.368'>0.368</td><td class='num' data-sort='0.75'>0.750</td><td class='num' data-sort='1052.3'>1,052.3</td><td class='num' data-sort='29.1'>29.1</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='5'>5</td><td class='method' data-sort='ElasticNet'>ElasticNet</td><td class='' data-sort='Regularization'><span class='pill cat-non-search'>Regularization</span><span class='sub'>non-search</span></td><td class='num' data-sort='1304.0'>1304<span class='sub'>+48 / β42</span></td><td class='num' data-sort='7.01'>7.01</td><td class='num' data-sort='57.1'>57.1</td><td class='num' data-sort='22.5'>22.5</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.509'>0.509</td><td class='num' data-sort='0.915'>0.915</td><td class='num' data-sort='6.1'>6.1</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='6'>6</td><td class='method' data-sort='MI'>MI</td><td class='' data-sort='Information theory'><span class='pill cat-non-search'>Information theory</span><span class='sub'>non-search</span></td><td class='num' data-sort='1292.0'>1292<span class='sub'>+42 / β42</span></td><td class='num' data-sort='7.12'>7.12</td><td class='num' data-sort='56.3'>56.3</td><td class='num' data-sort='22.0'>22.0</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.645'>0.645</td><td class='num' data-sort='0.898'>0.898</td><td class='num' data-sort='0.4'>0.4</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='7'>7</td><td class='method' data-sort='F-Test'>F-Test</td><td class='' data-sort='Correlation'><span class='pill cat-non-search'>Correlation</span><span class='sub'>non-search</span></td><td class='num' data-sort='1280.0'>1280<span class='sub'>+49 / β46</span></td><td class='num' data-sort='7.47'>7.47</td><td class='num' data-sort='53.8'>53.8</td><td class='num' data-sort='22.8'>22.8</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.725'>0.725</td><td class='num' data-sort='0.952'>0.952</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='8'>8</td><td class='method' data-sort='Lasso'>Lasso</td><td class='' data-sort='Regularization'><span class='pill cat-non-search'>Regularization</span><span class='sub'>non-search</span></td><td class='num' data-sort='1270.0'>1270<span class='sub'>+48 / β40</span></td><td class='num' data-sort='7.48'>7.48</td><td class='num' data-sort='53.7'>53.7</td><td class='num' data-sort='24.0'>24.0</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.421'>0.421</td><td class='num' data-sort='0.89'>0.890</td><td class='num' data-sort='2.0'>2.0</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='9'>9</td><td class='method' data-sort='mRMR'>mRMR</td><td class='' data-sort='Information theory'><span class='pill cat-non-search'>Information theory</span><span class='sub'>non-search</span></td><td class='num' data-sort='1254.0'>1254<span class='sub'>+43 / β48</span></td><td class='num' data-sort='7.73'>7.73</td><td class='num' data-sort='52.0'>52.0</td><td class='num' data-sort='22.7'>22.7</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.528'>0.528</td><td class='num' data-sort='0.779'>0.779</td><td class='num' data-sort='0.3'>0.3</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='10'>10</td><td class='method' data-sort='GainRatio'>GainRatio</td><td class='' data-sort='Information theory'><span class='pill cat-non-search'>Information theory</span><span class='sub'>non-search</span></td><td class='num' data-sort='1234.0'>1234<span class='sub'>+46 / β49</span></td><td class='num' data-sort='8.12'>8.12</td><td class='num' data-sort='49.2'>49.2</td><td class='num' data-sort='25.1'>25.1</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.679'>0.679</td><td class='num' data-sort='0.942'>0.942</td><td class='num' data-sort='0.2'>0.2</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='11'>11</td><td class='method' data-sort='SBE'>SBE</td><td class='' data-sort='Backward search'><span class='pill cat-search'>Backward search</span><span class='sub'>search</span></td><td class='num' data-sort='1233.0'>1233<span class='sub'>+59 / β52</span></td><td class='num' data-sort='8.28'>8.28</td><td class='num' data-sort='48.0'>48.0</td><td class='num' data-sort='27.0'>27.0</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.195'>0.195</td><td class='num' data-sort='0.621'>0.621</td><td class='num' data-sort='3579.1'>3,579.1</td><td class='num' data-sort='49.3'>49.3</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='12'>12</td><td class='method' data-sort='LOCO'>LOCO</td><td class='' data-sort='Leave-one-out'><span class='pill cat-search'>Leave-one-out</span><span class='sub'>search</span></td><td class='num' data-sort='1194.0'>1194<span class='sub'>+46 / β48</span></td><td class='num' data-sort='8.42'>8.42</td><td class='num' data-sort='47.0'>47.0</td><td class='num' data-sort='26.9'>26.9</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.299'>0.299</td><td class='num' data-sort='0.665'>0.665</td><td class='num' data-sort='240.9'>240.9</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='13'>13</td><td class='method' data-sort='(R)ReliefF'>(R)ReliefF</td><td class='' data-sort='Distance'><span class='pill cat-non-search'>Distance</span><span class='sub'>non-search</span></td><td class='num' data-sort='1191.0'>1191<span class='sub'>+42 / β43</span></td><td class='num' data-sort='8.79'>8.79</td><td class='num' data-sort='44.4'>44.4</td><td class='num' data-sort='28.4'>28.4</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.642'>0.642</td><td class='num' data-sort='0.873'>0.873</td><td class='num' data-sort='13.2'>13.2</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='14'>14</td><td class='method' data-sort='Random'>Random</td><td class='' data-sort='Baseline'><span class='pill cat-baseline'>Baseline</span></td><td class='num' data-sort='1000.0'>1000<span class='sub'>+56 / β67</span></td><td class='num' data-sort='11.43'>11.43</td><td class='num' data-sort='25.5'>25.5</td><td class='num' data-sort='39.7'>39.7</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='-0.001'>-0.001</td><td class='num' data-sort='0.493'>0.493</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='100.0'>100.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='15'>15</td><td class='method' data-sort='LaplacianScore'>LaplacianScore</td><td class='' data-sort='Distance'><span class='pill cat-non-search'>Distance</span><span class='sub'>non-search</span></td><td class='num' data-sort='934.0'>934<span class='sub'>+76 / β94</span></td><td class='num' data-sort='12.45'>12.45</td><td class='num' data-sort='18.2'>18.2</td><td class='num' data-sort='44.3'>44.3</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.588'>0.588</td><td class='num' data-sort='0.847'>0.847</td><td class='num' data-sort='0.2'>0.2</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr></tbody></table></div><script type='text/plain' id='sa-csv'>Rank,Method,Category,Subgroup,Elo,Elo CI low,Elo CI high,Avg. rank,Win rate (%),Improvability (%),Error vs. all features,Stability,Validity,Selection time (s),Fallback rate (%),#Datasets,#Comparisons
|
| 33 |
+
1,RFImportance,Non-search,Tree,1368.0,1311.0,1433.0,6.11,63.5,18.1,,0.628,0.805,0.2,0.0,71,213
|
| 34 |
+
2,CART,Non-search,Tree,1347.0,1300.0,1400.0,6.29,62.2,20.1,,0.484,0.731,0.1,0.0,71,213
|
| 35 |
+
3,MarkovBlanket,Search,Backward search,1325.0,1273.0,1374.0,6.54,60.4,21.1,,0.418,0.465,0.1,0.0,71,213
|
| 36 |
+
4,SFS,Search,Forward search,1312.0,1260.0,1368.0,6.76,58.8,21.5,,0.368,0.75,1052.3,29.1,71,213
|
| 37 |
+
5,ElasticNet,Non-search,Regularization,1304.0,1262.0,1352.0,7.01,57.1,22.5,,0.509,0.915,6.1,0.0,71,213
|
| 38 |
+
6,MI,Non-search,Information theory,1292.0,1250.0,1334.0,7.12,56.3,22.0,,0.645,0.898,0.4,0.0,71,213
|
| 39 |
+
7,F-Test,Non-search,Correlation,1280.0,1234.0,1329.0,7.47,53.8,22.8,,0.725,0.952,0.0,0.0,71,213
|
| 40 |
+
8,Lasso,Non-search,Regularization,1270.0,1230.0,1318.0,7.48,53.7,24.0,,0.421,0.89,2.0,0.0,71,213
|
| 41 |
+
9,mRMR,Non-search,Information theory,1254.0,1206.0,1297.0,7.73,52.0,22.7,,0.528,0.779,0.3,0.0,71,213
|
| 42 |
+
10,GainRatio,Non-search,Information theory,1234.0,1185.0,1280.0,8.12,49.2,25.1,,0.679,0.942,0.2,0.0,71,213
|
| 43 |
+
11,SBE,Search,Backward search,1233.0,1181.0,1292.0,8.28,48.0,27.0,,0.195,0.621,3579.1,49.3,71,213
|
| 44 |
+
12,LOCO,Search,Leave-one-out,1194.0,1146.0,1240.0,8.42,47.0,26.9,,0.299,0.665,240.9,0.0,71,213
|
| 45 |
+
13,(R)ReliefF,Non-search,Distance,1191.0,1148.0,1233.0,8.79,44.4,28.4,,0.642,0.873,13.2,0.0,71,213
|
| 46 |
+
14,Random,Baseline,Baseline,1000.0,933.0,1056.0,11.43,25.5,39.7,,-0.001,0.493,0.0,100.0,71,213
|
| 47 |
+
15,LaplacianScore,Non-search,Distance,934.0,840.0,1010.0,12.45,18.2,44.3,,0.588,0.847,0.2,0.0,71,213
|
| 48 |
</script><script>
|
| 49 |
function saDownload(name, blob) {
|
| 50 |
var a = document.createElement('a');
|
data/subsets/tasks_all/models_lightgbm/budgets_0/website_leaderboard.csv
CHANGED
|
@@ -1,16 +1,16 @@
|
|
| 1 |
-
Rank,
|
| 2 |
-
1,
|
| 3 |
-
2,
|
| 4 |
-
3,
|
| 5 |
-
4,
|
| 6 |
-
5,
|
| 7 |
-
6,
|
| 8 |
-
7,
|
| 9 |
-
8,
|
| 10 |
-
9,
|
| 11 |
-
10,
|
| 12 |
-
11,
|
| 13 |
-
12,
|
| 14 |
-
13,
|
| 15 |
-
14,
|
| 16 |
-
15,
|
|
|
|
| 1 |
+
Rank,Method,Category,Subgroup,Elo,Elo CI low,Elo CI high,Avg. rank,Win rate (%),Improvability (%),Error vs. all features,Stability,Validity,Selection time (s),Fallback rate (%),#Datasets,#Comparisons
|
| 2 |
+
1,RFImportance,Non-search,Tree,1368.0,1311.0,1433.0,6.11,63.5,18.1,,0.628,0.805,0.2,0.0,71,213
|
| 3 |
+
2,CART,Non-search,Tree,1347.0,1300.0,1400.0,6.29,62.2,20.1,,0.484,0.731,0.1,0.0,71,213
|
| 4 |
+
3,MarkovBlanket,Search,Backward search,1325.0,1273.0,1374.0,6.54,60.4,21.1,,0.418,0.465,0.1,0.0,71,213
|
| 5 |
+
4,SFS,Search,Forward search,1312.0,1260.0,1368.0,6.76,58.8,21.5,,0.368,0.75,1052.3,29.1,71,213
|
| 6 |
+
5,ElasticNet,Non-search,Regularization,1304.0,1262.0,1352.0,7.01,57.1,22.5,,0.509,0.915,6.1,0.0,71,213
|
| 7 |
+
6,MI,Non-search,Information theory,1292.0,1250.0,1334.0,7.12,56.3,22.0,,0.645,0.898,0.4,0.0,71,213
|
| 8 |
+
7,F-Test,Non-search,Correlation,1280.0,1234.0,1329.0,7.47,53.8,22.8,,0.725,0.952,0.0,0.0,71,213
|
| 9 |
+
8,Lasso,Non-search,Regularization,1270.0,1230.0,1318.0,7.48,53.7,24.0,,0.421,0.89,2.0,0.0,71,213
|
| 10 |
+
9,mRMR,Non-search,Information theory,1254.0,1206.0,1297.0,7.73,52.0,22.7,,0.528,0.779,0.3,0.0,71,213
|
| 11 |
+
10,GainRatio,Non-search,Information theory,1234.0,1185.0,1280.0,8.12,49.2,25.1,,0.679,0.942,0.2,0.0,71,213
|
| 12 |
+
11,SBE,Search,Backward search,1233.0,1181.0,1292.0,8.28,48.0,27.0,,0.195,0.621,3579.1,49.3,71,213
|
| 13 |
+
12,LOCO,Search,Leave-one-out,1194.0,1146.0,1240.0,8.42,47.0,26.9,,0.299,0.665,240.9,0.0,71,213
|
| 14 |
+
13,(R)ReliefF,Non-search,Distance,1191.0,1148.0,1233.0,8.79,44.4,28.4,,0.642,0.873,13.2,0.0,71,213
|
| 15 |
+
14,Random,Baseline,Baseline,1000.0,933.0,1056.0,11.43,25.5,39.7,,-0.001,0.493,0.0,100.0,71,213
|
| 16 |
+
15,LaplacianScore,Non-search,Distance,934.0,840.0,1010.0,12.45,18.2,44.3,,0.588,0.847,0.2,0.0,71,213
|
data/subsets/tasks_all/models_lightgbm/budgets_1/elo_explorer.html
CHANGED
|
@@ -19,67 +19,67 @@ svg { display:block; width:100%; height:auto; }
|
|
| 19 |
.sa-ink { stroke:var(--fg); }
|
| 20 |
.sa-hit { fill:transparent; }
|
| 21 |
.sa-hit:hover { fill:var(--fg); fill-opacity:0.06; }
|
| 22 |
-
</style><defs><pattern id='sa-na' width='6' height='6' patternUnits='userSpaceOnUse' patternTransform='rotate(45)'><rect width='6' height='6' fill='#e5e7eb'/><line x1='0' y1='0' x2='0' y2='6' stroke='#9ca3af' stroke-width='2'/></pattern><linearGradient id='sa-g-stab' x1='0' y1='1' x2='0' y2='0'><stop offset='0' stop-color='#ffffff'/><stop offset='1' stop-color='#0072B2'/></linearGradient><linearGradient id='sa-g-val' x1='0' y1='1' x2='0' y2='0'><stop offset='0' stop-color='#ffffff'/><stop offset='1' stop-color='#D55E00'/></linearGradient></defs><line class='sa-grid' x1='64' x2='1024' y1='315.6' y2='315.6' stroke-width='1'/><text class='sa-muted' x='56' y='319.6' text-anchor='end' font-size='12'>900</text><line class='sa-grid' x1='64' x2='1024' y1='266.0' y2='266.0' stroke-width='1'/><text class='sa-muted' x='56' y='270.0' text-anchor='end' font-size='12'>1000</text><line class='sa-grid' x1='64' x2='1024' y1='216.4' y2='216.4' stroke-width='1'/><text class='sa-muted' x='56' y='220.4' text-anchor='end' font-size='12'>1100</text><line class='sa-grid' x1='64' x2='1024' y1='166.8' y2='166.8' stroke-width='1'/><text class='sa-muted' x='56' y='170.8' text-anchor='end' font-size='12'>1200</text><line class='sa-grid' x1='64' x2='1024' y1='117.2' y2='117.2' stroke-width='1'/><text class='sa-muted' x='56' y='121.2' text-anchor='end' font-size='12'>1300</text><line class='sa-grid' x1='64' x2='1024' y1='67.6' y2='67.6' stroke-width='1'/><text class='sa-muted' x='56' y='71.6' text-anchor='end' font-size='12'>1400</text><line class='sa-grid' x1='64' x2='1024' y1='18.0' y2='18.0' stroke-width='1'/><text class='sa-muted' x='56' y='22.0' text-anchor='end' font-size='12'>1500</text><text class='sa-text' transform='translate(16 173) rotate(-90)' text-anchor='middle' font-size='13'>Elo</text><line class='sa-ink' x1='64' x2='1024' y1='266.0' y2='266.0' stroke-width='1' stroke-dasharray='4 4' opacity='0.5'/><rect x='73.6' y='268.5' width='22.4' height='59.5' fill='#5ba4cd' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='96.0' y='268.5' width='22.4' height='59.5' fill='#db7626' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='96.0' x2='96.0' y1='235.7' y2='309.2'/><line x1='91.0' x2='101.0' y1='235.7' y2='235.7'/><line x1='91.0' x2='101.0' y1='309.2' y2='309.2'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(100.0 342.0) rotate(-35)'>LaplacianScore</text><rect class='sa-hit' x='64.0' y='18' width='64' height='414'><title>LaplacianScore (
|
| 23 |
Elo 995 [95% CI 913β1061]
|
| 24 |
Stability 0.644
|
| 25 |
-
Validity 0.850</title></rect><rect x='137.6' y='266.0' width='22.4' height='62.0' fill='#ffffff' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='160.0' y='266.0' width='22.4' height='62.0' fill='#eaaf80' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='160.0' x2='160.0' y1='242.2' y2='293.3'/><line x1='155.0' x2='165.0' y1='242.2' y2='242.2'/><line x1='155.0' x2='165.0' y1='293.3' y2='293.3'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(164.0 342.0) rotate(-35)'>Random</text><rect class='sa-hit' x='128.0' y='18' width='64' height='414'><title>Random (Baseline)
|
| 26 |
Elo 1000 [95% CI 945β1048]
|
| 27 |
Stability 0.001
|
| 28 |
-
Validity 0.499</title></rect><rect x='201.6' y='146.5' width='22.4' height='181.5' fill='#c7e0ee' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='224.0' y='146.5' width='22.4' height='181.5' fill='#e59c63' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='224.0' x2='224.0' y1='117.2' y2='174.7'/><line x1='219.0' x2='229.0' y1='117.2' y2='117.2'/><line x1='219.0' x2='229.0' y1='174.7' y2='174.7'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(228.0 342.0) rotate(-35)'>LOCO</text><rect class='sa-hit' x='192.0' y='18' width='64' height='414'><title>LOCO (
|
| 29 |
Elo 1241 [95% CI 1184β1300]
|
| 30 |
Stability 0.220
|
| 31 |
-
Validity 0.613</title></rect><rect x='265.6' y='137.5' width='22.4' height='190.5' fill='#d2e6f2' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='288.0' y='137.5' width='22.4' height='190.5' fill='#e7a26c' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='288.0' x2='288.0' y1='108.3' y2='166.3'/><line x1='283.0' x2='293.0' y1='108.3' y2='108.3'/><line x1='283.0' x2='293.0' y1='166.3' y2='166.3'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(292.0 342.0) rotate(-35)'>SBE</text><rect class='sa-hit' x='256.0' y='18' width='64' height='414'><title>SBE (
|
| 32 |
Elo 1259 [95% CI 1201β1318]
|
| 33 |
Stability 0.175
|
| 34 |
-
Validity 0.578</title></rect><rect x='329.6' y='129.6' width='22.4' height='198.4' fill='#5aa4cd' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='352.0' y='129.6' width='22.4' height='198.4' fill='#dd7d31' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='352.0' x2='352.0' y1='104.8' y2='155.4'/><line x1='347.0' x2='357.0' y1='104.8' y2='104.8'/><line x1='347.0' x2='357.0' y1='155.4' y2='155.4'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(356.0 342.0) rotate(-35)'>(R)ReliefF</text><rect class='sa-hit' x='320.0' y='18' width='64' height='414'><title>(R)ReliefF (
|
| 35 |
Elo 1275 [95% CI 1223β1325]
|
| 36 |
Stability 0.647
|
| 37 |
-
Validity 0.808</title></rect><rect x='393.6' y='122.7' width='22.4' height='205.3' fill='#4498c6' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='416.0' y='122.7' width='22.4' height='205.3' fill='#d86911' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='416.0' x2='416.0' y1='97.9' y2='146.0'/><line x1='411.0' x2='421.0' y1='97.9' y2='97.9'/><line x1='411.0' x2='421.0' y1='146.0' y2='146.0'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(420.0 342.0) rotate(-35)'>F-Test</text><rect class='sa-hit' x='384.0' y='18' width='64' height='414'><title>F-Test (
|
| 38 |
Elo 1289 [95% CI 1242β1339]
|
| 39 |
Stability 0.734
|
| 40 |
-
Validity 0.932</title></rect><rect x='457.6' y='121.7' width='22.4' height='206.3' fill='#519fcb' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='480.0' y='121.7' width='22.4' height='206.3' fill='#d96c16' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='480.0' x2='480.0' y1='99.3' y2='142.5'/><line x1='475.0' x2='485.0' y1='99.3' y2='99.3'/><line x1='475.0' x2='485.0' y1='142.5' y2='142.5'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(484.0 342.0) rotate(-35)'>GainRatio</text><rect class='sa-hit' x='448.0' y='18' width='64' height='414'><title>GainRatio (
|
| 41 |
Elo 1291 [95% CI 1249β1336]
|
| 42 |
Stability 0.681
|
| 43 |
-
Validity 0.915</title></rect><rect x='521.6' y='120.2' width='22.4' height='207.8' fill='#b8d8ea' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='544.0' y='120.2' width='22.4' height='207.8' fill='#e39456' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='544.0' x2='544.0' y1='95.9' y2='143.0'/><line x1='539.0' x2='549.0' y1='95.9' y2='95.9'/><line x1='539.0' x2='549.0' y1='143.0' y2='143.0'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(548.0 342.0) rotate(-35)'>SFS</text><rect class='sa-hit' x='512.0' y='18' width='64' height='414'><title>SFS (
|
| 44 |
Elo 1294 [95% CI 1248β1343]
|
| 45 |
Stability 0.278
|
| 46 |
-
Validity 0.664</title></rect><rect x='585.6' y='111.7' width='22.4' height='216.3' fill='#6caed2' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='608.0' y='111.7' width='22.4' height='216.3' fill='#e18c49' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='608.0' x2='608.0' y1='88.9' y2='136.0'/><line x1='603.0' x2='613.0' y1='88.9' y2='88.9'/><line x1='603.0' x2='613.0' y1='136.0' y2='136.0'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(612.0 342.0) rotate(-35)'>mRMR</text><rect class='sa-hit' x='576.0' y='18' width='64' height='414'><title>mRMR (
|
| 47 |
Elo 1311 [95% CI 1262β1357]
|
| 48 |
Stability 0.578
|
| 49 |
-
Validity 0.712</title></rect><rect x='649.6' y='104.8' width='22.4' height='223.2' fill='#53a0cb' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='672.0' y='104.8' width='22.4' height='223.2' fill='#db7423' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='672.0' x2='672.0' y1='85.0' y2='122.2'/><line x1='667.0' x2='677.0' y1='85.0' y2='85.0'/><line x1='667.0' x2='677.0' y1='122.2' y2='122.2'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(676.0 342.0) rotate(-35)'>MI</text><rect class='sa-hit' x='640.0' y='18' width='64' height='414'><title>MI (
|
| 50 |
Elo 1325 [95% CI 1290β1365]
|
| 51 |
Stability 0.673
|
| 52 |
-
Validity 0.863</title></rect><rect x='713.6' y='98.4' width='22.4' height='229.6' fill='#93c3df' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='736.0' y='98.4' width='22.4' height='229.6' fill='#dc7a2d' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='736.0' x2='736.0' y1='73.6' y2='123.6'/><line x1='731.0' x2='741.0' y1='73.6' y2='73.6'/><line x1='731.0' x2='741.0' y1='123.6' y2='123.6'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(740.0 342.0) rotate(-35)'>Lasso</text><rect class='sa-hit' x='704.0' y='18' width='64' height='414'><title>Lasso (
|
| 53 |
Elo 1338 [95% CI 1287β1388]
|
| 54 |
Stability 0.422
|
| 55 |
-
Validity 0.825</title></rect><rect x='777.6' y='97.4' width='22.4' height='230.6' fill='#9ac7e1' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='800.0' y='97.4' width='22.4' height='230.6' fill='#ebb488' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='800.0' x2='800.0' y1='75.0' y2='121.2'/><line x1='795.0' x2='805.0' y1='75.0' y2='75.0'/><line x1='795.0' x2='805.0' y1='121.2' y2='121.2'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(804.0 342.0) rotate(-35)'>MarkovBlanket</text><rect class='sa-hit' x='768.0' y='18' width='64' height='414'><title>MarkovBlanket (
|
| 56 |
Elo 1340 [95% CI 1292β1385]
|
| 57 |
Stability 0.396
|
| 58 |
-
Validity 0.467</title></rect><rect x='841.6' y='89.9' width='22.4' height='238.1' fill='#8bbfdc' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='864.0' y='89.9' width='22.4' height='238.1' fill='#e4985c' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='864.0' x2='864.0' y1='65.1' y2='113.7'/><line x1='859.0' x2='869.0' y1='65.1' y2='65.1'/><line x1='859.0' x2='869.0' y1='113.7' y2='113.7'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(868.0 342.0) rotate(-35)'>CART</text><rect class='sa-hit' x='832.0' y='18' width='64' height='414'><title>CART (
|
| 59 |
Elo 1355 [95% CI 1307β1405]
|
| 60 |
Stability 0.456
|
| 61 |
-
Validity 0.638</title></rect><rect x='905.6' y='84.5' width='22.4' height='243.5' fill='#58a3cd' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='928.0' y='84.5' width='22.4' height='243.5' fill='#e08842' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='928.0' x2='928.0' y1='55.7' y2='110.3'/><line x1='923.0' x2='933.0' y1='55.7' y2='55.7'/><line x1='923.0' x2='933.0' y1='110.3' y2='110.3'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(932.0 342.0) rotate(-35)'>RFImportance</text><rect class='sa-hit' x='896.0' y='18' width='64' height='414'><title>RFImportance (
|
| 62 |
Elo 1366 [95% CI 1314β1424]
|
| 63 |
Stability 0.653
|
| 64 |
-
Validity 0.741</title></rect><rect x='969.6' y='84.5' width='22.4' height='243.5' fill='#7bb6d7' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='992.0' y='84.5' width='22.4' height='243.5' fill='#da7220' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='992.0' x2='992.0' y1='62.1' y2='108.3'/><line x1='987.0' x2='997.0' y1='62.1' y2='62.1'/><line x1='987.0' x2='997.0' y1='108.3' y2='108.3'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(996.0 342.0) rotate(-35)'>ElasticNet</text><rect class='sa-hit' x='960.0' y='18' width='64' height='414'><title>ElasticNet (
|
| 65 |
Elo 1366 [95% CI 1318β1411]
|
| 66 |
Stability 0.519
|
| 67 |
-
Validity 0.873</title></rect><line class='sa-ink' x1='64' x2='1024' y1='328.0' y2='328.0' stroke-width='1' opacity='0.4'/><rect x='1070' y='40.0' width='12' height='107.0' fill='url(#sa-g-stab)' stroke='#000' stroke-width='0.6'/><text class='sa-text' x='1076' y='30.0' text-anchor='middle' font-size='12'>stable</text><text class='sa-text' x='1076' y='165.0' text-anchor='middle' font-size='12'>unstable</text><text class='sa-muted' x='1088' y='151.0' font-size='11'>0.0</text><text class='sa-muted' x='1088' y='97.5' font-size='11'>0.5</text><text class='sa-muted' x='1088' y='44.0' font-size='11'>1.0</text><rect x='1070' y='207.0' width='12' height='107.0' fill='url(#sa-g-val)' stroke='#000' stroke-width='0.6'/><text class='sa-text' x='1076' y='197.0' text-anchor='middle' font-size='12'>valid</text><text class='sa-text' x='1076' y='332.0' text-anchor='middle' font-size='12'>invalid</text><text class='sa-muted' x='1088' y='318.0' font-size='11'>0.0</text><text class='sa-muted' x='1088' y='264.5' font-size='11'>0.5</text><text class='sa-muted' x='1088' y='211.0' font-size='11'>1.0</text><rect x='1070' y='352.0' width='12' height='12' fill='url(#sa-na)' stroke='#000' stroke-width='0.6'/><text class='sa-muted' x='1088' y='362.0' font-size='11'>no result</text></svg><script type='text/plain' id='sa-csv'>Method,
|
| 68 |
-
RFImportance,
|
| 69 |
-
ElasticNet,
|
| 70 |
-
CART,
|
| 71 |
-
MarkovBlanket,
|
| 72 |
-
Lasso,
|
| 73 |
-
MI,
|
| 74 |
-
mRMR,
|
| 75 |
-
SFS,
|
| 76 |
-
GainRatio,
|
| 77 |
-
F-Test,
|
| 78 |
-
(R)ReliefF,
|
| 79 |
-
SBE,
|
| 80 |
-
LOCO,
|
| 81 |
-
Random,Baseline,1000.0,945.0,1048.0,0.001,0.499
|
| 82 |
-
LaplacianScore,
|
| 83 |
</script><script>
|
| 84 |
function saDownload(name, blob) {
|
| 85 |
var a = document.createElement('a');
|
|
|
|
| 19 |
.sa-ink { stroke:var(--fg); }
|
| 20 |
.sa-hit { fill:transparent; }
|
| 21 |
.sa-hit:hover { fill:var(--fg); fill-opacity:0.06; }
|
| 22 |
+
</style><defs><pattern id='sa-na' width='6' height='6' patternUnits='userSpaceOnUse' patternTransform='rotate(45)'><rect width='6' height='6' fill='#e5e7eb'/><line x1='0' y1='0' x2='0' y2='6' stroke='#9ca3af' stroke-width='2'/></pattern><linearGradient id='sa-g-stab' x1='0' y1='1' x2='0' y2='0'><stop offset='0' stop-color='#ffffff'/><stop offset='1' stop-color='#0072B2'/></linearGradient><linearGradient id='sa-g-val' x1='0' y1='1' x2='0' y2='0'><stop offset='0' stop-color='#ffffff'/><stop offset='1' stop-color='#D55E00'/></linearGradient></defs><line class='sa-grid' x1='64' x2='1024' y1='315.6' y2='315.6' stroke-width='1'/><text class='sa-muted' x='56' y='319.6' text-anchor='end' font-size='12'>900</text><line class='sa-grid' x1='64' x2='1024' y1='266.0' y2='266.0' stroke-width='1'/><text class='sa-muted' x='56' y='270.0' text-anchor='end' font-size='12'>1000</text><line class='sa-grid' x1='64' x2='1024' y1='216.4' y2='216.4' stroke-width='1'/><text class='sa-muted' x='56' y='220.4' text-anchor='end' font-size='12'>1100</text><line class='sa-grid' x1='64' x2='1024' y1='166.8' y2='166.8' stroke-width='1'/><text class='sa-muted' x='56' y='170.8' text-anchor='end' font-size='12'>1200</text><line class='sa-grid' x1='64' x2='1024' y1='117.2' y2='117.2' stroke-width='1'/><text class='sa-muted' x='56' y='121.2' text-anchor='end' font-size='12'>1300</text><line class='sa-grid' x1='64' x2='1024' y1='67.6' y2='67.6' stroke-width='1'/><text class='sa-muted' x='56' y='71.6' text-anchor='end' font-size='12'>1400</text><line class='sa-grid' x1='64' x2='1024' y1='18.0' y2='18.0' stroke-width='1'/><text class='sa-muted' x='56' y='22.0' text-anchor='end' font-size='12'>1500</text><text class='sa-text' transform='translate(16 173) rotate(-90)' text-anchor='middle' font-size='13'>Elo</text><line class='sa-ink' x1='64' x2='1024' y1='266.0' y2='266.0' stroke-width='1' stroke-dasharray='4 4' opacity='0.5'/><rect x='73.6' y='268.5' width='22.4' height='59.5' fill='#5ba4cd' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='96.0' y='268.5' width='22.4' height='59.5' fill='#db7626' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='96.0' x2='96.0' y1='235.7' y2='309.2'/><line x1='91.0' x2='101.0' y1='235.7' y2='235.7'/><line x1='91.0' x2='101.0' y1='309.2' y2='309.2'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(100.0 342.0) rotate(-35)'>LaplacianScore</text><rect class='sa-hit' x='64.0' y='18' width='64' height='414'><title>LaplacianScore (Distance, non-search)
|
| 23 |
Elo 995 [95% CI 913β1061]
|
| 24 |
Stability 0.644
|
| 25 |
+
Validity 0.850</title></rect><rect x='137.6' y='266.0' width='22.4' height='62.0' fill='#ffffff' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='160.0' y='266.0' width='22.4' height='62.0' fill='#eaaf80' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='160.0' x2='160.0' y1='242.2' y2='293.3'/><line x1='155.0' x2='165.0' y1='242.2' y2='242.2'/><line x1='155.0' x2='165.0' y1='293.3' y2='293.3'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(164.0 342.0) rotate(-35)'>Random</text><rect class='sa-hit' x='128.0' y='18' width='64' height='414'><title>Random (Baseline, baseline)
|
| 26 |
Elo 1000 [95% CI 945β1048]
|
| 27 |
Stability 0.001
|
| 28 |
+
Validity 0.499</title></rect><rect x='201.6' y='146.5' width='22.4' height='181.5' fill='#c7e0ee' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='224.0' y='146.5' width='22.4' height='181.5' fill='#e59c63' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='224.0' x2='224.0' y1='117.2' y2='174.7'/><line x1='219.0' x2='229.0' y1='117.2' y2='117.2'/><line x1='219.0' x2='229.0' y1='174.7' y2='174.7'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(228.0 342.0) rotate(-35)'>LOCO</text><rect class='sa-hit' x='192.0' y='18' width='64' height='414'><title>LOCO (Leave-one-out, search)
|
| 29 |
Elo 1241 [95% CI 1184β1300]
|
| 30 |
Stability 0.220
|
| 31 |
+
Validity 0.613</title></rect><rect x='265.6' y='137.5' width='22.4' height='190.5' fill='#d2e6f2' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='288.0' y='137.5' width='22.4' height='190.5' fill='#e7a26c' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='288.0' x2='288.0' y1='108.3' y2='166.3'/><line x1='283.0' x2='293.0' y1='108.3' y2='108.3'/><line x1='283.0' x2='293.0' y1='166.3' y2='166.3'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(292.0 342.0) rotate(-35)'>SBE</text><rect class='sa-hit' x='256.0' y='18' width='64' height='414'><title>SBE (Backward search, search)
|
| 32 |
Elo 1259 [95% CI 1201β1318]
|
| 33 |
Stability 0.175
|
| 34 |
+
Validity 0.578</title></rect><rect x='329.6' y='129.6' width='22.4' height='198.4' fill='#5aa4cd' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='352.0' y='129.6' width='22.4' height='198.4' fill='#dd7d31' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='352.0' x2='352.0' y1='104.8' y2='155.4'/><line x1='347.0' x2='357.0' y1='104.8' y2='104.8'/><line x1='347.0' x2='357.0' y1='155.4' y2='155.4'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(356.0 342.0) rotate(-35)'>(R)ReliefF</text><rect class='sa-hit' x='320.0' y='18' width='64' height='414'><title>(R)ReliefF (Distance, non-search)
|
| 35 |
Elo 1275 [95% CI 1223β1325]
|
| 36 |
Stability 0.647
|
| 37 |
+
Validity 0.808</title></rect><rect x='393.6' y='122.7' width='22.4' height='205.3' fill='#4498c6' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='416.0' y='122.7' width='22.4' height='205.3' fill='#d86911' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='416.0' x2='416.0' y1='97.9' y2='146.0'/><line x1='411.0' x2='421.0' y1='97.9' y2='97.9'/><line x1='411.0' x2='421.0' y1='146.0' y2='146.0'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(420.0 342.0) rotate(-35)'>F-Test</text><rect class='sa-hit' x='384.0' y='18' width='64' height='414'><title>F-Test (Correlation, non-search)
|
| 38 |
Elo 1289 [95% CI 1242β1339]
|
| 39 |
Stability 0.734
|
| 40 |
+
Validity 0.932</title></rect><rect x='457.6' y='121.7' width='22.4' height='206.3' fill='#519fcb' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='480.0' y='121.7' width='22.4' height='206.3' fill='#d96c16' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='480.0' x2='480.0' y1='99.3' y2='142.5'/><line x1='475.0' x2='485.0' y1='99.3' y2='99.3'/><line x1='475.0' x2='485.0' y1='142.5' y2='142.5'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(484.0 342.0) rotate(-35)'>GainRatio</text><rect class='sa-hit' x='448.0' y='18' width='64' height='414'><title>GainRatio (Information theory, non-search)
|
| 41 |
Elo 1291 [95% CI 1249β1336]
|
| 42 |
Stability 0.681
|
| 43 |
+
Validity 0.915</title></rect><rect x='521.6' y='120.2' width='22.4' height='207.8' fill='#b8d8ea' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='544.0' y='120.2' width='22.4' height='207.8' fill='#e39456' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='544.0' x2='544.0' y1='95.9' y2='143.0'/><line x1='539.0' x2='549.0' y1='95.9' y2='95.9'/><line x1='539.0' x2='549.0' y1='143.0' y2='143.0'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(548.0 342.0) rotate(-35)'>SFS</text><rect class='sa-hit' x='512.0' y='18' width='64' height='414'><title>SFS (Forward search, search)
|
| 44 |
Elo 1294 [95% CI 1248β1343]
|
| 45 |
Stability 0.278
|
| 46 |
+
Validity 0.664</title></rect><rect x='585.6' y='111.7' width='22.4' height='216.3' fill='#6caed2' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='608.0' y='111.7' width='22.4' height='216.3' fill='#e18c49' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='608.0' x2='608.0' y1='88.9' y2='136.0'/><line x1='603.0' x2='613.0' y1='88.9' y2='88.9'/><line x1='603.0' x2='613.0' y1='136.0' y2='136.0'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(612.0 342.0) rotate(-35)'>mRMR</text><rect class='sa-hit' x='576.0' y='18' width='64' height='414'><title>mRMR (Information theory, non-search)
|
| 47 |
Elo 1311 [95% CI 1262β1357]
|
| 48 |
Stability 0.578
|
| 49 |
+
Validity 0.712</title></rect><rect x='649.6' y='104.8' width='22.4' height='223.2' fill='#53a0cb' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='672.0' y='104.8' width='22.4' height='223.2' fill='#db7423' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='672.0' x2='672.0' y1='85.0' y2='122.2'/><line x1='667.0' x2='677.0' y1='85.0' y2='85.0'/><line x1='667.0' x2='677.0' y1='122.2' y2='122.2'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(676.0 342.0) rotate(-35)'>MI</text><rect class='sa-hit' x='640.0' y='18' width='64' height='414'><title>MI (Information theory, non-search)
|
| 50 |
Elo 1325 [95% CI 1290β1365]
|
| 51 |
Stability 0.673
|
| 52 |
+
Validity 0.863</title></rect><rect x='713.6' y='98.4' width='22.4' height='229.6' fill='#93c3df' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='736.0' y='98.4' width='22.4' height='229.6' fill='#dc7a2d' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='736.0' x2='736.0' y1='73.6' y2='123.6'/><line x1='731.0' x2='741.0' y1='73.6' y2='73.6'/><line x1='731.0' x2='741.0' y1='123.6' y2='123.6'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(740.0 342.0) rotate(-35)'>Lasso</text><rect class='sa-hit' x='704.0' y='18' width='64' height='414'><title>Lasso (Regularization, non-search)
|
| 53 |
Elo 1338 [95% CI 1287β1388]
|
| 54 |
Stability 0.422
|
| 55 |
+
Validity 0.825</title></rect><rect x='777.6' y='97.4' width='22.4' height='230.6' fill='#9ac7e1' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='800.0' y='97.4' width='22.4' height='230.6' fill='#ebb488' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='800.0' x2='800.0' y1='75.0' y2='121.2'/><line x1='795.0' x2='805.0' y1='75.0' y2='75.0'/><line x1='795.0' x2='805.0' y1='121.2' y2='121.2'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(804.0 342.0) rotate(-35)'>MarkovBlanket</text><rect class='sa-hit' x='768.0' y='18' width='64' height='414'><title>MarkovBlanket (Backward search, search)
|
| 56 |
Elo 1340 [95% CI 1292β1385]
|
| 57 |
Stability 0.396
|
| 58 |
+
Validity 0.467</title></rect><rect x='841.6' y='89.9' width='22.4' height='238.1' fill='#8bbfdc' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='864.0' y='89.9' width='22.4' height='238.1' fill='#e4985c' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='864.0' x2='864.0' y1='65.1' y2='113.7'/><line x1='859.0' x2='869.0' y1='65.1' y2='65.1'/><line x1='859.0' x2='869.0' y1='113.7' y2='113.7'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(868.0 342.0) rotate(-35)'>CART</text><rect class='sa-hit' x='832.0' y='18' width='64' height='414'><title>CART (Tree, non-search)
|
| 59 |
Elo 1355 [95% CI 1307β1405]
|
| 60 |
Stability 0.456
|
| 61 |
+
Validity 0.638</title></rect><rect x='905.6' y='84.5' width='22.4' height='243.5' fill='#58a3cd' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='928.0' y='84.5' width='22.4' height='243.5' fill='#e08842' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='928.0' x2='928.0' y1='55.7' y2='110.3'/><line x1='923.0' x2='933.0' y1='55.7' y2='55.7'/><line x1='923.0' x2='933.0' y1='110.3' y2='110.3'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(932.0 342.0) rotate(-35)'>RFImportance</text><rect class='sa-hit' x='896.0' y='18' width='64' height='414'><title>RFImportance (Tree, non-search)
|
| 62 |
Elo 1366 [95% CI 1314β1424]
|
| 63 |
Stability 0.653
|
| 64 |
+
Validity 0.741</title></rect><rect x='969.6' y='84.5' width='22.4' height='243.5' fill='#7bb6d7' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='992.0' y='84.5' width='22.4' height='243.5' fill='#da7220' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='992.0' x2='992.0' y1='62.1' y2='108.3'/><line x1='987.0' x2='997.0' y1='62.1' y2='62.1'/><line x1='987.0' x2='997.0' y1='108.3' y2='108.3'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(996.0 342.0) rotate(-35)'>ElasticNet</text><rect class='sa-hit' x='960.0' y='18' width='64' height='414'><title>ElasticNet (Regularization, non-search)
|
| 65 |
Elo 1366 [95% CI 1318β1411]
|
| 66 |
Stability 0.519
|
| 67 |
+
Validity 0.873</title></rect><line class='sa-ink' x1='64' x2='1024' y1='328.0' y2='328.0' stroke-width='1' opacity='0.4'/><rect x='1070' y='40.0' width='12' height='107.0' fill='url(#sa-g-stab)' stroke='#000' stroke-width='0.6'/><text class='sa-text' x='1076' y='30.0' text-anchor='middle' font-size='12'>stable</text><text class='sa-text' x='1076' y='165.0' text-anchor='middle' font-size='12'>unstable</text><text class='sa-muted' x='1088' y='151.0' font-size='11'>0.0</text><text class='sa-muted' x='1088' y='97.5' font-size='11'>0.5</text><text class='sa-muted' x='1088' y='44.0' font-size='11'>1.0</text><rect x='1070' y='207.0' width='12' height='107.0' fill='url(#sa-g-val)' stroke='#000' stroke-width='0.6'/><text class='sa-text' x='1076' y='197.0' text-anchor='middle' font-size='12'>valid</text><text class='sa-text' x='1076' y='332.0' text-anchor='middle' font-size='12'>invalid</text><text class='sa-muted' x='1088' y='318.0' font-size='11'>0.0</text><text class='sa-muted' x='1088' y='264.5' font-size='11'>0.5</text><text class='sa-muted' x='1088' y='211.0' font-size='11'>1.0</text><rect x='1070' y='352.0' width='12' height='12' fill='url(#sa-na)' stroke='#000' stroke-width='0.6'/><text class='sa-muted' x='1088' y='362.0' font-size='11'>no result</text></svg><script type='text/plain' id='sa-csv'>Method,Category,Subgroup,Elo,Elo CI low,Elo CI high,Stability,Validity
|
| 68 |
+
RFImportance,Non-search,Tree,1366.0,1314.0,1424.0,0.653,0.741
|
| 69 |
+
ElasticNet,Non-search,Regularization,1366.0,1318.0,1411.0,0.519,0.873
|
| 70 |
+
CART,Non-search,Tree,1355.0,1307.0,1405.0,0.456,0.638
|
| 71 |
+
MarkovBlanket,Search,Backward search,1340.0,1292.0,1385.0,0.396,0.467
|
| 72 |
+
Lasso,Non-search,Regularization,1338.0,1287.0,1388.0,0.422,0.825
|
| 73 |
+
MI,Non-search,Information theory,1325.0,1290.0,1365.0,0.673,0.863
|
| 74 |
+
mRMR,Non-search,Information theory,1311.0,1262.0,1357.0,0.578,0.712
|
| 75 |
+
SFS,Search,Forward search,1294.0,1248.0,1343.0,0.278,0.664
|
| 76 |
+
GainRatio,Non-search,Information theory,1291.0,1249.0,1336.0,0.681,0.915
|
| 77 |
+
F-Test,Non-search,Correlation,1289.0,1242.0,1339.0,0.734,0.932
|
| 78 |
+
(R)ReliefF,Non-search,Distance,1275.0,1223.0,1325.0,0.647,0.808
|
| 79 |
+
SBE,Search,Backward search,1259.0,1201.0,1318.0,0.175,0.578
|
| 80 |
+
LOCO,Search,Leave-one-out,1241.0,1184.0,1300.0,0.22,0.613
|
| 81 |
+
Random,Baseline,Baseline,1000.0,945.0,1048.0,0.001,0.499
|
| 82 |
+
LaplacianScore,Non-search,Distance,995.0,913.0,1061.0,0.644,0.85
|
| 83 |
</script><script>
|
| 84 |
function saDownload(name, blob) {
|
| 85 |
var a = document.createElement('a');
|
data/subsets/tasks_all/models_lightgbm/budgets_1/leaderboard_table.html
CHANGED
|
@@ -26,23 +26,25 @@ td.method { font-weight:600; }
|
|
| 26 |
td .sub { color:var(--muted); font-size:12px; font-weight:400; margin-left:6px; }
|
| 27 |
td.na { color:var(--muted); }
|
| 28 |
.pill { display:inline-block; padding:0 8px; border:1px solid var(--line); border-radius:999px;
|
| 29 |
-
font-size:12px; color:var(--
|
| 30 |
-
</style></head><body><div class='toolbar'><span class='hint'>Click a column to sort. Hover a header for its definition.</span><button onclick="saCsv('selectarena_leaderboard.csv')">CSV</button></div><div class='wrap'><table id='sa-table'><thead><tr><th class='num'>#</th><th class=''>Method</th><th class='' title='Filter: scores features from the data alone. Wrapper: searches feature subsets by refitting a proxy model. Embedded: reads the selection off a fitted model.'>Type</th><th class='num' title='Bradley-Terry Elo from pairwise matches within each dataset, budget and downstream model. Random = 1000. +/β give the 95% bootstrap interval.'>Elo β</th><th class='num' title='Mean rank of the downstream error.'>Avg. rank β</th><th class='num' title='Share of pairwise comparisons won, in %. Ties count half.'>Win rate β</th><th class='num' title='Mean relative gap (error β best error) / error to the best method, in %.'>Improvability β</th><th class='num' title='Mean ratio of the error to the same model trained on all features. Below 1: the selection helped.'>Error vs. all β</th><th class='num' title='Agreement of the selected features across bootstrap resamples (Nogueira et al., 2018).'>Stability β</th><th class='num' title='Share of the budget filled with original features when as many noise features are added.'>Validity β</th><th class='num' title='Median wall-clock time of the selection.'>Time (s) β</th><th class='num' title='Share of runs where the method ran out of time and random features filled the budget.'>Fallback (%)</th><th class='num'>#Datasets</th></tr></thead><tbody><tr><td class='num' data-sort='1'>1</td><td class='method' data-sort='RFImportance'>RFImportance</td><td class='' data-sort='Embedded'><span class='pill'>π³ Embedded</span></td><td class='num' data-sort='1366.0'>1366<span class='sub'>+58 / β52</span></td><td class='num' data-sort='6.45'>6.45</td><td class='num' data-sort='61.1'>61.1</td><td class='num' data-sort='16.8'>16.8</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.653'>0.653</td><td class='num' data-sort='0.741'>0.741</td><td class='num' data-sort='0.2'>0.2</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='2'>2</td><td class='method' data-sort='ElasticNet'>ElasticNet</td><td class='' data-sort='Embedded'><span class='pill'>π³ Embedded</span></td><td class='num' data-sort='1366.0'>1366<span class='sub'>+45 / β48</span></td><td class='num' data-sort='6.43'>6.43</td><td class='num' data-sort='61.2'>61.2</td><td class='num' data-sort='18.4'>18.4</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.519'>0.519</td><td class='num' data-sort='0.873'>0.873</td><td class='num' data-sort='6.3'>6.3</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='3'>3</td><td class='method' data-sort='CART'>CART</td><td class='' data-sort='Embedded'><span class='pill'>π³ Embedded</span></td><td class='num' data-sort='1355.0'>1355<span class='sub'>+50 / β48</span></td><td class='num' data-sort='6.79'>6.79</td><td class='num' data-sort='58.6'>58.6</td><td class='num' data-sort='19.1'>19.1</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.456'>0.456</td><td class='num' data-sort='0.638'>0.638</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='4'>4</td><td class='method' data-sort='MarkovBlanket'>MarkovBlanket</td><td class='' data-sort='Filter'><span class='pill'>π Filter</span></td><td class='num' data-sort='1340.0'>1340<span class='sub'>+45 / β48</span></td><td class='num' data-sort='7.03'>7.03</td><td class='num' data-sort='56.9'>56.9</td><td class='num' data-sort='20.8'>20.8</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.396'>0.396</td><td class='num' data-sort='0.467'>0.467</td><td class='num' data-sort='0.1'>0.1</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='5'>5</td><td class='method' data-sort='Lasso'>Lasso</td><td class='' data-sort='Embedded'><span class='pill'>π³ Embedded</span></td><td class='num' data-sort='1338.0'>1338<span class='sub'>+50 / β51</span></td><td class='num' data-sort='6.73'>6.73</td><td class='num' data-sort='59.1'>59.1</td><td class='num' data-sort='18.4'>18.4</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.422'>0.422</td><td class='num' data-sort='0.825'>0.825</td><td class='num' data-sort='2.1'>2.1</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='6'>6</td><td class='method' data-sort='MI'>MI</td><td class='' data-sort='Filter'><span class='pill'>π Filter</span></td><td class='num' data-sort='1325.0'>1325<span class='sub'>+40 / β35</span></td><td class='num' data-sort='7.01'>7.01</td><td class='num' data-sort='57.1'>57.1</td><td class='num' data-sort='19.5'>19.5</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.673'>0.673</td><td class='num' data-sort='0.863'>0.863</td><td class='num' data-sort='0.4'>0.4</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='7'>7</td><td class='method' data-sort='mRMR'>mRMR</td><td class='' data-sort='Filter'><span class='pill'>π Filter</span></td><td class='num' data-sort='1311.0'>1311<span class='sub'>+46 / β49</span></td><td class='num' data-sort='7.63'>7.63</td><td class='num' data-sort='52.6'>52.6</td><td class='num' data-sort='21.1'>21.1</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.578'>0.578</td><td class='num' data-sort='0.712'>0.712</td><td class='num' data-sort='0.3'>0.3</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='8'>8</td><td class='method' data-sort='SFS'>SFS</td><td class='' data-sort='Wrapper'><span class='pill'>π Wrapper</span></td><td class='num' data-sort='1294.0'>1294<span class='sub'>+49 / β46</span></td><td class='num' data-sort='7.71'>7.71</td><td class='num' data-sort='52.1'>52.1</td><td class='num' data-sort='22.7'>22.7</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.278'>0.278</td><td class='num' data-sort='0.664'>0.664</td><td class='num' data-sort='1770.9'>1,770.9</td><td class='num' data-sort='44.1'>44.1</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='9'>9</td><td class='method' data-sort='GainRatio'>GainRatio</td><td class='' data-sort='Filter'><span class='pill'>π Filter</span></td><td class='num' data-sort='1291.0'>1291<span class='sub'>+45 / β42</span></td><td class='num' data-sort='7.77'>7.77</td><td class='num' data-sort='51.7'>51.7</td><td class='num' data-sort='22.6'>22.6</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.681'>0.681</td><td class='num' data-sort='0.915'>0.915</td><td class='num' data-sort='0.1'>0.1</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='10'>10</td><td class='method' data-sort='F-Test'>F-Test</td><td class='' data-sort='Filter'><span class='pill'>π Filter</span></td><td class='num' data-sort='1289.0'>1289<span class='sub'>+50 / β47</span></td><td class='num' data-sort='7.73'>7.73</td><td class='num' data-sort='51.9'>51.9</td><td class='num' data-sort='22.0'>22.0</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.734'>0.734</td><td class='num' data-sort='0.932'>0.932</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='11'>11</td><td class='method' data-sort='(R)ReliefF'>(R)ReliefF</td><td class='' data-sort='Filter'><span class='pill'>π Filter</span></td><td class='num' data-sort='1275.0'>1275<span class='sub'>+50 / β52</span></td><td class='num' data-sort='8.04'>8.04</td><td class='num' data-sort='49.7'>49.7</td><td class='num' data-sort='22.5'>22.5</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.647'>0.647</td><td class='num' data-sort='0.808'>0.808</td><td class='num' data-sort='12.8'>12.8</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='12'>12</td><td class='method' data-sort='SBE'>SBE</td><td class='' data-sort='Wrapper'><span class='pill'>π Wrapper</span></td><td class='num' data-sort='1259.0'>1259<span class='sub'>+59 / β58</span></td><td class='num' data-sort='8.01'>8.01</td><td class='num' data-sort='50.0'>50.0</td><td class='num' data-sort='25.5'>25.5</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.175'>0.175</td><td class='num' data-sort='0.578'>0.578</td><td class='num' data-sort='3152.6'>3,152.6</td><td class='num' data-sort='48.8'>48.8</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='13'>13</td><td class='method' data-sort='LOCO'>LOCO</td><td class='' data-sort='Wrapper'><span class='pill'>π Wrapper</span></td><td class='num' data-sort='1241.0'>1241<span class='sub'>+59 / β57</span></td><td class='num' data-sort='8.54'>8.54</td><td class='num' data-sort='46.1'>46.1</td><td class='num' data-sort='25.5'>25.5</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.22'>0.220</td><td class='num' data-sort='0.613'>0.613</td><td class='num' data-sort='239.5'>239.5</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='14'>14</td><td class='method' data-sort='Random'>Random</td><td class='' data-sort='Baseline'><span class='pill'>βͺ Baseline</span></td><td class='num' data-sort='1000.0'>1000<span class='sub'>+48 / β55</span></td><td class='num' data-sort='11.82'>11.82</td><td class='num' data-sort='22.7'>22.7</td><td class='num' data-sort='38.0'>38.0</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.001'>0.001</td><td class='num' data-sort='0.499'>0.499</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='100.0'>100.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='15'>15</td><td class='method' data-sort='LaplacianScore'>LaplacianScore</td><td class='' data-sort='Filter'><span class='pill'>π Filter</span></td><td class='num' data-sort='995.0'>995<span class='sub'>+66 / β82</span></td><td class='num' data-sort='12.32'>12.32</td><td class='num' data-sort='19.1'>19.1</td><td class='num' data-sort='41.7'>41.7</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.644'>0.644</td><td class='num' data-sort='0.85'>0.850</td><td class='num' data-sort='0.2'>0.2</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr></tbody></table></div><script type='text/plain' id='sa-csv'>Rank,Type,TypeName,Method,Elo,Elo CI low,Elo CI high,Avg. rank,Win rate (%),Improvability (%),Error vs. all features,Stability,Validity,Selection time (s),Fallback rate (%),#Datasets,#Comparisons
|
| 31 |
-
|
| 32 |
-
2,π³,Embedded,ElasticNet,1366.0,1318.0,1411.0,6.43,61.2,18.4,,0.519,0.873,6.3,0.0,71,213
|
| 33 |
-
|
| 34 |
-
|
| 35 |
-
|
| 36 |
-
|
| 37 |
-
|
| 38 |
-
|
| 39 |
-
|
| 40 |
-
|
| 41 |
-
|
| 42 |
-
|
| 43 |
-
|
| 44 |
-
|
| 45 |
-
|
|
|
|
|
|
|
| 46 |
</script><script>
|
| 47 |
function saDownload(name, blob) {
|
| 48 |
var a = document.createElement('a');
|
|
|
|
| 26 |
td .sub { color:var(--muted); font-size:12px; font-weight:400; margin-left:6px; }
|
| 27 |
td.na { color:var(--muted); }
|
| 28 |
.pill { display:inline-block; padding:0 8px; border:1px solid var(--line); border-radius:999px;
|
| 29 |
+
font-size:12px; color:var(--fg); }
|
| 30 |
+
.pill.cat-search { border-color:var(--accent); }
|
| 31 |
+
.pill.cat-baseline { color:var(--muted); }
|
| 32 |
+
</style></head><body><div class='toolbar'><span class='hint'>Click a column to sort. Hover a header for its definition.</span><button onclick="saCsv('selectarena_leaderboard.csv')">CSV</button></div><div class='wrap'><table id='sa-table'><thead><tr><th class='num'>#</th><th class=''>Method</th><th class='' title='Taxonomy of the paper (Appendix A). Search methods evaluate many feature subsets with the model (forward, backward, leave-one-out); non-search methods use a proxy instead (correlation, distance, information theory, tree, regularization).'>Type</th><th class='num' title='Bradley-Terry Elo from pairwise matches within each dataset, budget and downstream model. Random = 1000. +/β give the 95% bootstrap interval.'>Elo β</th><th class='num' title='Mean rank of the downstream error.'>Avg. rank β</th><th class='num' title='Share of pairwise comparisons won, in %. Ties count half.'>Win rate β</th><th class='num' title='Mean relative gap (error β best error) / error to the best method, in %.'>Improvability β</th><th class='num' title='Mean ratio of the error to the same model trained on all features. Below 1: the selection helped.'>Error vs. all β</th><th class='num' title='Agreement of the selected features across bootstrap resamples (Nogueira et al., 2018).'>Stability β</th><th class='num' title='Share of the budget filled with original features when as many noise features are added.'>Validity β</th><th class='num' title='Median wall-clock time of the selection.'>Time (s) β</th><th class='num' title='Share of runs where the method ran out of time and random features filled the budget.'>Fallback (%)</th><th class='num'>#Datasets</th></tr></thead><tbody><tr><td class='num' data-sort='1'>1</td><td class='method' data-sort='RFImportance'>RFImportance</td><td class='' data-sort='Tree'><span class='pill cat-non-search'>Tree</span><span class='sub'>non-search</span></td><td class='num' data-sort='1366.0'>1366<span class='sub'>+58 / β52</span></td><td class='num' data-sort='6.45'>6.45</td><td class='num' data-sort='61.1'>61.1</td><td class='num' data-sort='16.8'>16.8</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.653'>0.653</td><td class='num' data-sort='0.741'>0.741</td><td class='num' data-sort='0.2'>0.2</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='2'>2</td><td class='method' data-sort='ElasticNet'>ElasticNet</td><td class='' data-sort='Regularization'><span class='pill cat-non-search'>Regularization</span><span class='sub'>non-search</span></td><td class='num' data-sort='1366.0'>1366<span class='sub'>+45 / β48</span></td><td class='num' data-sort='6.43'>6.43</td><td class='num' data-sort='61.2'>61.2</td><td class='num' data-sort='18.4'>18.4</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.519'>0.519</td><td class='num' data-sort='0.873'>0.873</td><td class='num' data-sort='6.3'>6.3</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='3'>3</td><td class='method' data-sort='CART'>CART</td><td class='' data-sort='Tree'><span class='pill cat-non-search'>Tree</span><span class='sub'>non-search</span></td><td class='num' data-sort='1355.0'>1355<span class='sub'>+50 / β48</span></td><td class='num' data-sort='6.79'>6.79</td><td class='num' data-sort='58.6'>58.6</td><td class='num' data-sort='19.1'>19.1</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.456'>0.456</td><td class='num' data-sort='0.638'>0.638</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='4'>4</td><td class='method' data-sort='MarkovBlanket'>MarkovBlanket</td><td class='' data-sort='Backward search'><span class='pill cat-search'>Backward search</span><span class='sub'>search</span></td><td class='num' data-sort='1340.0'>1340<span class='sub'>+45 / β48</span></td><td class='num' data-sort='7.03'>7.03</td><td class='num' data-sort='56.9'>56.9</td><td class='num' data-sort='20.8'>20.8</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.396'>0.396</td><td class='num' data-sort='0.467'>0.467</td><td class='num' data-sort='0.1'>0.1</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='5'>5</td><td class='method' data-sort='Lasso'>Lasso</td><td class='' data-sort='Regularization'><span class='pill cat-non-search'>Regularization</span><span class='sub'>non-search</span></td><td class='num' data-sort='1338.0'>1338<span class='sub'>+50 / β51</span></td><td class='num' data-sort='6.73'>6.73</td><td class='num' data-sort='59.1'>59.1</td><td class='num' data-sort='18.4'>18.4</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.422'>0.422</td><td class='num' data-sort='0.825'>0.825</td><td class='num' data-sort='2.1'>2.1</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='6'>6</td><td class='method' data-sort='MI'>MI</td><td class='' data-sort='Information theory'><span class='pill cat-non-search'>Information theory</span><span class='sub'>non-search</span></td><td class='num' data-sort='1325.0'>1325<span class='sub'>+40 / β35</span></td><td class='num' data-sort='7.01'>7.01</td><td class='num' data-sort='57.1'>57.1</td><td class='num' data-sort='19.5'>19.5</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.673'>0.673</td><td class='num' data-sort='0.863'>0.863</td><td class='num' data-sort='0.4'>0.4</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='7'>7</td><td class='method' data-sort='mRMR'>mRMR</td><td class='' data-sort='Information theory'><span class='pill cat-non-search'>Information theory</span><span class='sub'>non-search</span></td><td class='num' data-sort='1311.0'>1311<span class='sub'>+46 / β49</span></td><td class='num' data-sort='7.63'>7.63</td><td class='num' data-sort='52.6'>52.6</td><td class='num' data-sort='21.1'>21.1</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.578'>0.578</td><td class='num' data-sort='0.712'>0.712</td><td class='num' data-sort='0.3'>0.3</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='8'>8</td><td class='method' data-sort='SFS'>SFS</td><td class='' data-sort='Forward search'><span class='pill cat-search'>Forward search</span><span class='sub'>search</span></td><td class='num' data-sort='1294.0'>1294<span class='sub'>+49 / β46</span></td><td class='num' data-sort='7.71'>7.71</td><td class='num' data-sort='52.1'>52.1</td><td class='num' data-sort='22.7'>22.7</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.278'>0.278</td><td class='num' data-sort='0.664'>0.664</td><td class='num' data-sort='1770.9'>1,770.9</td><td class='num' data-sort='44.1'>44.1</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='9'>9</td><td class='method' data-sort='GainRatio'>GainRatio</td><td class='' data-sort='Information theory'><span class='pill cat-non-search'>Information theory</span><span class='sub'>non-search</span></td><td class='num' data-sort='1291.0'>1291<span class='sub'>+45 / β42</span></td><td class='num' data-sort='7.77'>7.77</td><td class='num' data-sort='51.7'>51.7</td><td class='num' data-sort='22.6'>22.6</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.681'>0.681</td><td class='num' data-sort='0.915'>0.915</td><td class='num' data-sort='0.1'>0.1</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='10'>10</td><td class='method' data-sort='F-Test'>F-Test</td><td class='' data-sort='Correlation'><span class='pill cat-non-search'>Correlation</span><span class='sub'>non-search</span></td><td class='num' data-sort='1289.0'>1289<span class='sub'>+50 / β47</span></td><td class='num' data-sort='7.73'>7.73</td><td class='num' data-sort='51.9'>51.9</td><td class='num' data-sort='22.0'>22.0</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.734'>0.734</td><td class='num' data-sort='0.932'>0.932</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='11'>11</td><td class='method' data-sort='(R)ReliefF'>(R)ReliefF</td><td class='' data-sort='Distance'><span class='pill cat-non-search'>Distance</span><span class='sub'>non-search</span></td><td class='num' data-sort='1275.0'>1275<span class='sub'>+50 / β52</span></td><td class='num' data-sort='8.04'>8.04</td><td class='num' data-sort='49.7'>49.7</td><td class='num' data-sort='22.5'>22.5</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.647'>0.647</td><td class='num' data-sort='0.808'>0.808</td><td class='num' data-sort='12.8'>12.8</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='12'>12</td><td class='method' data-sort='SBE'>SBE</td><td class='' data-sort='Backward search'><span class='pill cat-search'>Backward search</span><span class='sub'>search</span></td><td class='num' data-sort='1259.0'>1259<span class='sub'>+59 / β58</span></td><td class='num' data-sort='8.01'>8.01</td><td class='num' data-sort='50.0'>50.0</td><td class='num' data-sort='25.5'>25.5</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.175'>0.175</td><td class='num' data-sort='0.578'>0.578</td><td class='num' data-sort='3152.6'>3,152.6</td><td class='num' data-sort='48.8'>48.8</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='13'>13</td><td class='method' data-sort='LOCO'>LOCO</td><td class='' data-sort='Leave-one-out'><span class='pill cat-search'>Leave-one-out</span><span class='sub'>search</span></td><td class='num' data-sort='1241.0'>1241<span class='sub'>+59 / β57</span></td><td class='num' data-sort='8.54'>8.54</td><td class='num' data-sort='46.1'>46.1</td><td class='num' data-sort='25.5'>25.5</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.22'>0.220</td><td class='num' data-sort='0.613'>0.613</td><td class='num' data-sort='239.5'>239.5</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='14'>14</td><td class='method' data-sort='Random'>Random</td><td class='' data-sort='Baseline'><span class='pill cat-baseline'>Baseline</span></td><td class='num' data-sort='1000.0'>1000<span class='sub'>+48 / β55</span></td><td class='num' data-sort='11.82'>11.82</td><td class='num' data-sort='22.7'>22.7</td><td class='num' data-sort='38.0'>38.0</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.001'>0.001</td><td class='num' data-sort='0.499'>0.499</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='100.0'>100.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='15'>15</td><td class='method' data-sort='LaplacianScore'>LaplacianScore</td><td class='' data-sort='Distance'><span class='pill cat-non-search'>Distance</span><span class='sub'>non-search</span></td><td class='num' data-sort='995.0'>995<span class='sub'>+66 / β82</span></td><td class='num' data-sort='12.32'>12.32</td><td class='num' data-sort='19.1'>19.1</td><td class='num' data-sort='41.7'>41.7</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.644'>0.644</td><td class='num' data-sort='0.85'>0.850</td><td class='num' data-sort='0.2'>0.2</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr></tbody></table></div><script type='text/plain' id='sa-csv'>Rank,Method,Category,Subgroup,Elo,Elo CI low,Elo CI high,Avg. rank,Win rate (%),Improvability (%),Error vs. all features,Stability,Validity,Selection time (s),Fallback rate (%),#Datasets,#Comparisons
|
| 33 |
+
1,RFImportance,Non-search,Tree,1366.0,1314.0,1424.0,6.45,61.1,16.8,,0.653,0.741,0.2,0.0,71,213
|
| 34 |
+
2,ElasticNet,Non-search,Regularization,1366.0,1318.0,1411.0,6.43,61.2,18.4,,0.519,0.873,6.3,0.0,71,213
|
| 35 |
+
3,CART,Non-search,Tree,1355.0,1307.0,1405.0,6.79,58.6,19.1,,0.456,0.638,0.0,0.0,71,213
|
| 36 |
+
4,MarkovBlanket,Search,Backward search,1340.0,1292.0,1385.0,7.03,56.9,20.8,,0.396,0.467,0.1,0.0,71,213
|
| 37 |
+
5,Lasso,Non-search,Regularization,1338.0,1287.0,1388.0,6.73,59.1,18.4,,0.422,0.825,2.1,0.0,71,213
|
| 38 |
+
6,MI,Non-search,Information theory,1325.0,1290.0,1365.0,7.01,57.1,19.5,,0.673,0.863,0.4,0.0,71,213
|
| 39 |
+
7,mRMR,Non-search,Information theory,1311.0,1262.0,1357.0,7.63,52.6,21.1,,0.578,0.712,0.3,0.0,71,213
|
| 40 |
+
8,SFS,Search,Forward search,1294.0,1248.0,1343.0,7.71,52.1,22.7,,0.278,0.664,1770.9,44.1,71,213
|
| 41 |
+
9,GainRatio,Non-search,Information theory,1291.0,1249.0,1336.0,7.77,51.7,22.6,,0.681,0.915,0.1,0.0,71,213
|
| 42 |
+
10,F-Test,Non-search,Correlation,1289.0,1242.0,1339.0,7.73,51.9,22.0,,0.734,0.932,0.0,0.0,71,213
|
| 43 |
+
11,(R)ReliefF,Non-search,Distance,1275.0,1223.0,1325.0,8.04,49.7,22.5,,0.647,0.808,12.8,0.0,71,213
|
| 44 |
+
12,SBE,Search,Backward search,1259.0,1201.0,1318.0,8.01,50.0,25.5,,0.175,0.578,3152.6,48.8,71,213
|
| 45 |
+
13,LOCO,Search,Leave-one-out,1241.0,1184.0,1300.0,8.54,46.1,25.5,,0.22,0.613,239.5,0.0,71,213
|
| 46 |
+
14,Random,Baseline,Baseline,1000.0,945.0,1048.0,11.82,22.7,38.0,,0.001,0.499,0.0,100.0,71,213
|
| 47 |
+
15,LaplacianScore,Non-search,Distance,995.0,913.0,1061.0,12.32,19.1,41.7,,0.644,0.85,0.2,0.0,71,213
|
| 48 |
</script><script>
|
| 49 |
function saDownload(name, blob) {
|
| 50 |
var a = document.createElement('a');
|
data/subsets/tasks_all/models_lightgbm/budgets_1/website_leaderboard.csv
CHANGED
|
@@ -1,16 +1,16 @@
|
|
| 1 |
-
Rank,
|
| 2 |
-
1,
|
| 3 |
-
2,
|
| 4 |
-
3,
|
| 5 |
-
4,
|
| 6 |
-
5,
|
| 7 |
-
6,
|
| 8 |
-
7,
|
| 9 |
-
8,
|
| 10 |
-
9,
|
| 11 |
-
10,
|
| 12 |
-
11,
|
| 13 |
-
12,
|
| 14 |
-
13,
|
| 15 |
-
14,
|
| 16 |
-
15,
|
|
|
|
| 1 |
+
Rank,Method,Category,Subgroup,Elo,Elo CI low,Elo CI high,Avg. rank,Win rate (%),Improvability (%),Error vs. all features,Stability,Validity,Selection time (s),Fallback rate (%),#Datasets,#Comparisons
|
| 2 |
+
1,RFImportance,Non-search,Tree,1366.0,1314.0,1424.0,6.45,61.1,16.8,,0.653,0.741,0.2,0.0,71,213
|
| 3 |
+
2,ElasticNet,Non-search,Regularization,1366.0,1318.0,1411.0,6.43,61.2,18.4,,0.519,0.873,6.3,0.0,71,213
|
| 4 |
+
3,CART,Non-search,Tree,1355.0,1307.0,1405.0,6.79,58.6,19.1,,0.456,0.638,0.0,0.0,71,213
|
| 5 |
+
4,MarkovBlanket,Search,Backward search,1340.0,1292.0,1385.0,7.03,56.9,20.8,,0.396,0.467,0.1,0.0,71,213
|
| 6 |
+
5,Lasso,Non-search,Regularization,1338.0,1287.0,1388.0,6.73,59.1,18.4,,0.422,0.825,2.1,0.0,71,213
|
| 7 |
+
6,MI,Non-search,Information theory,1325.0,1290.0,1365.0,7.01,57.1,19.5,,0.673,0.863,0.4,0.0,71,213
|
| 8 |
+
7,mRMR,Non-search,Information theory,1311.0,1262.0,1357.0,7.63,52.6,21.1,,0.578,0.712,0.3,0.0,71,213
|
| 9 |
+
8,SFS,Search,Forward search,1294.0,1248.0,1343.0,7.71,52.1,22.7,,0.278,0.664,1770.9,44.1,71,213
|
| 10 |
+
9,GainRatio,Non-search,Information theory,1291.0,1249.0,1336.0,7.77,51.7,22.6,,0.681,0.915,0.1,0.0,71,213
|
| 11 |
+
10,F-Test,Non-search,Correlation,1289.0,1242.0,1339.0,7.73,51.9,22.0,,0.734,0.932,0.0,0.0,71,213
|
| 12 |
+
11,(R)ReliefF,Non-search,Distance,1275.0,1223.0,1325.0,8.04,49.7,22.5,,0.647,0.808,12.8,0.0,71,213
|
| 13 |
+
12,SBE,Search,Backward search,1259.0,1201.0,1318.0,8.01,50.0,25.5,,0.175,0.578,3152.6,48.8,71,213
|
| 14 |
+
13,LOCO,Search,Leave-one-out,1241.0,1184.0,1300.0,8.54,46.1,25.5,,0.22,0.613,239.5,0.0,71,213
|
| 15 |
+
14,Random,Baseline,Baseline,1000.0,945.0,1048.0,11.82,22.7,38.0,,0.001,0.499,0.0,100.0,71,213
|
| 16 |
+
15,LaplacianScore,Non-search,Distance,995.0,913.0,1061.0,12.32,19.1,41.7,,0.644,0.85,0.2,0.0,71,213
|
data/subsets/tasks_all/models_lightgbm/budgets_2/elo_explorer.html
CHANGED
|
@@ -19,67 +19,67 @@ svg { display:block; width:100%; height:auto; }
|
|
| 19 |
.sa-ink { stroke:var(--fg); }
|
| 20 |
.sa-hit { fill:transparent; }
|
| 21 |
.sa-hit:hover { fill:var(--fg); fill-opacity:0.06; }
|
| 22 |
-
</style><defs><pattern id='sa-na' width='6' height='6' patternUnits='userSpaceOnUse' patternTransform='rotate(45)'><rect width='6' height='6' fill='#e5e7eb'/><line x1='0' y1='0' x2='0' y2='6' stroke='#9ca3af' stroke-width='2'/></pattern><linearGradient id='sa-g-stab' x1='0' y1='1' x2='0' y2='0'><stop offset='0' stop-color='#ffffff'/><stop offset='1' stop-color='#0072B2'/></linearGradient><linearGradient id='sa-g-val' x1='0' y1='1' x2='0' y2='0'><stop offset='0' stop-color='#ffffff'/><stop offset='1' stop-color='#D55E00'/></linearGradient></defs><line class='sa-grid' x1='64' x2='1024' y1='287.6' y2='287.6' stroke-width='1'/><text class='sa-muted' x='56' y='291.6' text-anchor='end' font-size='12'>900</text><line class='sa-grid' x1='64' x2='1024' y1='233.7' y2='233.7' stroke-width='1'/><text class='sa-muted' x='56' y='237.7' text-anchor='end' font-size='12'>1000</text><line class='sa-grid' x1='64' x2='1024' y1='179.7' y2='179.7' stroke-width='1'/><text class='sa-muted' x='56' y='183.7' text-anchor='end' font-size='12'>1100</text><line class='sa-grid' x1='64' x2='1024' y1='125.8' y2='125.8' stroke-width='1'/><text class='sa-muted' x='56' y='129.8' text-anchor='end' font-size='12'>1200</text><line class='sa-grid' x1='64' x2='1024' y1='71.9' y2='71.9' stroke-width='1'/><text class='sa-muted' x='56' y='75.9' text-anchor='end' font-size='12'>1300</text><line class='sa-grid' x1='64' x2='1024' y1='18.0' y2='18.0' stroke-width='1'/><text class='sa-muted' x='56' y='22.0' text-anchor='end' font-size='12'>1400</text><text class='sa-text' transform='translate(16 173) rotate(-90)' text-anchor='middle' font-size='13'>Elo</text><line class='sa-ink' x1='64' x2='1024' y1='233.7' y2='233.7' stroke-width='1' stroke-dasharray='4 4' opacity='0.5'/><rect x='73.6' y='266.5' width='22.4' height='61.5' fill='#57a2cc' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='96.0' y='266.5' width='22.4' height='61.5' fill='#dc7a2c' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='96.0' x2='96.0' y1='228.8' y2='314.5'/><line x1='91.0' x2='101.0' y1='228.8' y2='228.8'/><line x1='91.0' x2='101.0' y1='314.5' y2='314.5'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(100.0 342.0) rotate(-35)'>LaplacianScore</text><rect class='sa-hit' x='64.0' y='18' width='64' height='414'><title>LaplacianScore (
|
| 23 |
Elo 939 [95% CI 850β1009]
|
| 24 |
Stability 0.660
|
| 25 |
-
Validity 0.827</title></rect><rect x='137.6' y='233.7' width='22.4' height='94.3' fill='#ffffff' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='160.0' y='233.7' width='22.4' height='94.3' fill='#e9ac7c' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='160.0' x2='160.0' y1='208.9' y2='261.7'/><line x1='155.0' x2='165.0' y1='208.9' y2='208.9'/><line x1='155.0' x2='165.0' y1='261.7' y2='261.7'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(164.0 342.0) rotate(-35)'>Random</text><rect class='sa-hit' x='128.0' y='18' width='64' height='414'><title>Random (Baseline)
|
| 26 |
Elo 1000 [95% CI 948β1046]
|
| 27 |
Stability -0.001
|
| 28 |
-
Validity 0.513</title></rect><rect x='201.6' y='139.3' width='22.4' height='188.7' fill='#d8eaf3' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='224.0' y='139.3' width='22.4' height='188.7' fill='#e7a26c' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='224.0' x2='224.0' y1='109.7' y2='166.3'/><line x1='219.0' x2='229.0' y1='109.7' y2='109.7'/><line x1='219.0' x2='229.0' y1='166.3' y2='166.3'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(228.0 342.0) rotate(-35)'>LOCO</text><rect class='sa-hit' x='192.0' y='18' width='64' height='414'><title>LOCO (
|
| 29 |
Elo 1175 [95% CI 1125β1230]
|
| 30 |
Stability 0.151
|
| 31 |
-
Validity 0.575</title></rect><rect x='265.6' y='129.6' width='22.4' height='198.4' fill='#c2dded' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='288.0' y='129.6' width='22.4' height='198.4' fill='#e59a5f' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='288.0' x2='288.0' y1='99.9' y2='155.5'/><line x1='283.0' x2='293.0' y1='99.9' y2='99.9'/><line x1='283.0' x2='293.0' y1='155.5' y2='155.5'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(292.0 342.0) rotate(-35)'>SFS</text><rect class='sa-hit' x='256.0' y='18' width='64' height='414'><title>SFS (
|
| 32 |
Elo 1193 [95% CI 1145β1248]
|
| 33 |
Stability 0.239
|
| 34 |
-
Validity 0.627</title></rect><rect x='329.6' y='127.4' width='22.4' height='200.6' fill='#62a8d0' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='352.0' y='127.4' width='22.4' height='200.6' fill='#e39557' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='352.0' x2='352.0' y1='103.2' y2='151.7'/><line x1='347.0' x2='357.0' y1='103.2' y2='103.2'/><line x1='347.0' x2='357.0' y1='151.7' y2='151.7'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(356.0 342.0) rotate(-35)'>mRMR</text><rect class='sa-hit' x='320.0' y='18' width='64' height='414'><title>mRMR (
|
| 35 |
Elo 1197 [95% CI 1152β1242]
|
| 36 |
Stability 0.614
|
| 37 |
-
Validity 0.660</title></rect><rect x='393.6' y='126.4' width='22.4' height='201.6' fill='#53a0cb' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='416.0' y='126.4' width='22.4' height='201.6' fill='#df853e' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='416.0' x2='416.0' y1='103.2' y2='152.8'/><line x1='411.0' x2='421.0' y1='103.2' y2='103.2'/><line x1='411.0' x2='421.0' y1='152.8' y2='152.8'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(420.0 342.0) rotate(-35)'>(R)ReliefF</text><rect class='sa-hit' x='384.0' y='18' width='64' height='414'><title>(R)ReliefF (
|
| 38 |
Elo 1199 [95% CI 1150β1242]
|
| 39 |
Stability 0.673
|
| 40 |
-
Validity 0.757</title></rect><rect x='457.6' y='123.1' width='22.4' height='204.9' fill='#54a1cb' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='480.0' y='123.1' width='22.4' height='204.9' fill='#db7525' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='480.0' x2='480.0' y1='98.9' y2='146.9'/><line x1='475.0' x2='485.0' y1='98.9' y2='98.9'/><line x1='475.0' x2='485.0' y1='146.9' y2='146.9'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(484.0 342.0) rotate(-35)'>GainRatio</text><rect class='sa-hit' x='448.0' y='18' width='64' height='414'><title>GainRatio (
|
| 41 |
Elo 1205 [95% CI 1161β1250]
|
| 42 |
Stability 0.670
|
| 43 |
-
Validity 0.856</title></rect><rect x='521.6' y='119.4' width='22.4' height='208.6' fill='#4b9bc9' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='544.0' y='119.4' width='22.4' height='208.6' fill='#da711f' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='544.0' x2='544.0' y1='94.0' y2='142.5'/><line x1='539.0' x2='549.0' y1='94.0' y2='94.0'/><line x1='539.0' x2='549.0' y1='142.5' y2='142.5'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(548.0 342.0) rotate(-35)'>F-Test</text><rect class='sa-hit' x='512.0' y='18' width='64' height='414'><title>F-Test (
|
| 44 |
Elo 1212 [95% CI 1169β1259]
|
| 45 |
Stability 0.706
|
| 46 |
-
Validity 0.879</title></rect><rect x='585.6' y='112.9' width='22.4' height='215.1' fill='#daebf4' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='608.0' y='112.9' width='22.4' height='215.1' fill='#e8a571' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='608.0' x2='608.0' y1='83.2' y2='138.2'/><line x1='603.0' x2='613.0' y1='83.2' y2='83.2'/><line x1='603.0' x2='613.0' y1='138.2' y2='138.2'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(612.0 342.0) rotate(-35)'>SBE</text><rect class='sa-hit' x='576.0' y='18' width='64' height='414'><title>SBE (
|
| 47 |
Elo 1224 [95% CI 1177β1279]
|
| 48 |
Stability 0.144
|
| 49 |
-
Validity 0.556</title></rect><rect x='649.6' y='99.9' width='22.4' height='228.1' fill='#96c5df' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='672.0' y='99.9' width='22.4' height='228.1' fill='#ecb78d' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='672.0' x2='672.0' y1='78.9' y2='119.9'/><line x1='667.0' x2='677.0' y1='78.9' y2='78.9'/><line x1='667.0' x2='677.0' y1='119.9' y2='119.9'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(676.0 342.0) rotate(-35)'>MarkovBlanket</text><rect class='sa-hit' x='640.0' y='18' width='64' height='414'><title>MarkovBlanket (
|
| 50 |
Elo 1248 [95% CI 1211β1287]
|
| 51 |
Stability 0.413
|
| 52 |
-
Validity 0.447</title></rect><rect x='713.6' y='97.8' width='22.4' height='230.2' fill='#84bbda' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='736.0' y='97.8' width='22.4' height='230.2' fill='#de8239' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='736.0' x2='736.0' y1='71.9' y2='122.6'/><line x1='731.0' x2='741.0' y1='71.9' y2='71.9'/><line x1='731.0' x2='741.0' y1='122.6' y2='122.6'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(740.0 342.0) rotate(-35)'>ElasticNet</text><rect class='sa-hit' x='704.0' y='18' width='64' height='414'><title>ElasticNet (
|
| 53 |
Elo 1252 [95% CI 1206β1300]
|
| 54 |
Stability 0.481
|
| 55 |
-
Validity 0.777</title></rect><rect x='777.6' y='89.2' width='22.4' height='238.8' fill='#9cc8e1' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='800.0' y='89.2' width='22.4' height='238.8' fill='#e08945' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='800.0' x2='800.0' y1='61.1' y2='116.1'/><line x1='795.0' x2='805.0' y1='61.1' y2='61.1'/><line x1='795.0' x2='805.0' y1='116.1' y2='116.1'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(804.0 342.0) rotate(-35)'>Lasso</text><rect class='sa-hit' x='768.0' y='18' width='64' height='414'><title>Lasso (
|
| 56 |
Elo 1268 [95% CI 1218β1320]
|
| 57 |
Stability 0.389
|
| 58 |
-
Validity 0.730</title></rect><rect x='841.6' y='81.6' width='22.4' height='246.4' fill='#519fca' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='864.0' y='81.6' width='22.4' height='246.4' fill='#dd7c2f' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='864.0' x2='864.0' y1='54.1' y2='106.4'/><line x1='859.0' x2='869.0' y1='54.1' y2='54.1'/><line x1='859.0' x2='869.0' y1='106.4' y2='106.4'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(868.0 342.0) rotate(-35)'>MI</text><rect class='sa-hit' x='832.0' y='18' width='64' height='414'><title>MI (
|
| 59 |
Elo 1282 [95% CI 1236β1333]
|
| 60 |
Stability 0.683
|
| 61 |
-
Validity 0.815</title></rect><rect x='905.6' y='81.1' width='22.4' height='246.9' fill='#8ec0dd' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='928.0' y='81.1' width='22.4' height='246.9' fill='#e7a36d' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='928.0' x2='928.0' y1='54.1' y2='108.0'/><line x1='923.0' x2='933.0' y1='54.1' y2='54.1'/><line x1='923.0' x2='933.0' y1='108.0' y2='108.0'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(932.0 342.0) rotate(-35)'>CART</text><rect class='sa-hit' x='896.0' y='18' width='64' height='414'><title>CART (
|
| 62 |
Elo 1283 [95% CI 1233β1333]
|
| 63 |
Stability 0.444
|
| 64 |
-
Validity 0.572</title></rect><rect x='969.6' y='60.6' width='22.4' height='267.4' fill='#53a0cb' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='992.0' y='60.6' width='22.4' height='267.4' fill='#e2904f' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='992.0' x2='992.0' y1='27.7' y2='89.7'/><line x1='987.0' x2='997.0' y1='27.7' y2='27.7'/><line x1='987.0' x2='997.0' y1='89.7' y2='89.7'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(996.0 342.0) rotate(-35)'>RFImportance</text><rect class='sa-hit' x='960.0' y='18' width='64' height='414'><title>RFImportance (
|
| 65 |
Elo 1321 [95% CI 1267β1382]
|
| 66 |
Stability 0.676
|
| 67 |
-
Validity 0.691</title></rect><line class='sa-ink' x1='64' x2='1024' y1='328.0' y2='328.0' stroke-width='1' opacity='0.4'/><rect x='1070' y='40.0' width='12' height='107.0' fill='url(#sa-g-stab)' stroke='#000' stroke-width='0.6'/><text class='sa-text' x='1076' y='30.0' text-anchor='middle' font-size='12'>stable</text><text class='sa-text' x='1076' y='165.0' text-anchor='middle' font-size='12'>unstable</text><text class='sa-muted' x='1088' y='151.0' font-size='11'>0.0</text><text class='sa-muted' x='1088' y='97.5' font-size='11'>0.5</text><text class='sa-muted' x='1088' y='44.0' font-size='11'>1.0</text><rect x='1070' y='207.0' width='12' height='107.0' fill='url(#sa-g-val)' stroke='#000' stroke-width='0.6'/><text class='sa-text' x='1076' y='197.0' text-anchor='middle' font-size='12'>valid</text><text class='sa-text' x='1076' y='332.0' text-anchor='middle' font-size='12'>invalid</text><text class='sa-muted' x='1088' y='318.0' font-size='11'>0.0</text><text class='sa-muted' x='1088' y='264.5' font-size='11'>0.5</text><text class='sa-muted' x='1088' y='211.0' font-size='11'>1.0</text><rect x='1070' y='352.0' width='12' height='12' fill='url(#sa-na)' stroke='#000' stroke-width='0.6'/><text class='sa-muted' x='1088' y='362.0' font-size='11'>no result</text></svg><script type='text/plain' id='sa-csv'>Method,
|
| 68 |
-
RFImportance,
|
| 69 |
-
CART,
|
| 70 |
-
MI,
|
| 71 |
-
Lasso,
|
| 72 |
-
ElasticNet,
|
| 73 |
-
MarkovBlanket,
|
| 74 |
-
SBE,
|
| 75 |
-
F-Test,
|
| 76 |
-
GainRatio,
|
| 77 |
-
(R)ReliefF,
|
| 78 |
-
mRMR,
|
| 79 |
-
SFS,
|
| 80 |
-
LOCO,
|
| 81 |
-
Random,Baseline,1000.0,948.0,1046.0,-0.001,0.513
|
| 82 |
-
LaplacianScore,
|
| 83 |
</script><script>
|
| 84 |
function saDownload(name, blob) {
|
| 85 |
var a = document.createElement('a');
|
|
|
|
| 19 |
.sa-ink { stroke:var(--fg); }
|
| 20 |
.sa-hit { fill:transparent; }
|
| 21 |
.sa-hit:hover { fill:var(--fg); fill-opacity:0.06; }
|
| 22 |
+
</style><defs><pattern id='sa-na' width='6' height='6' patternUnits='userSpaceOnUse' patternTransform='rotate(45)'><rect width='6' height='6' fill='#e5e7eb'/><line x1='0' y1='0' x2='0' y2='6' stroke='#9ca3af' stroke-width='2'/></pattern><linearGradient id='sa-g-stab' x1='0' y1='1' x2='0' y2='0'><stop offset='0' stop-color='#ffffff'/><stop offset='1' stop-color='#0072B2'/></linearGradient><linearGradient id='sa-g-val' x1='0' y1='1' x2='0' y2='0'><stop offset='0' stop-color='#ffffff'/><stop offset='1' stop-color='#D55E00'/></linearGradient></defs><line class='sa-grid' x1='64' x2='1024' y1='287.6' y2='287.6' stroke-width='1'/><text class='sa-muted' x='56' y='291.6' text-anchor='end' font-size='12'>900</text><line class='sa-grid' x1='64' x2='1024' y1='233.7' y2='233.7' stroke-width='1'/><text class='sa-muted' x='56' y='237.7' text-anchor='end' font-size='12'>1000</text><line class='sa-grid' x1='64' x2='1024' y1='179.7' y2='179.7' stroke-width='1'/><text class='sa-muted' x='56' y='183.7' text-anchor='end' font-size='12'>1100</text><line class='sa-grid' x1='64' x2='1024' y1='125.8' y2='125.8' stroke-width='1'/><text class='sa-muted' x='56' y='129.8' text-anchor='end' font-size='12'>1200</text><line class='sa-grid' x1='64' x2='1024' y1='71.9' y2='71.9' stroke-width='1'/><text class='sa-muted' x='56' y='75.9' text-anchor='end' font-size='12'>1300</text><line class='sa-grid' x1='64' x2='1024' y1='18.0' y2='18.0' stroke-width='1'/><text class='sa-muted' x='56' y='22.0' text-anchor='end' font-size='12'>1400</text><text class='sa-text' transform='translate(16 173) rotate(-90)' text-anchor='middle' font-size='13'>Elo</text><line class='sa-ink' x1='64' x2='1024' y1='233.7' y2='233.7' stroke-width='1' stroke-dasharray='4 4' opacity='0.5'/><rect x='73.6' y='266.5' width='22.4' height='61.5' fill='#57a2cc' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='96.0' y='266.5' width='22.4' height='61.5' fill='#dc7a2c' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='96.0' x2='96.0' y1='228.8' y2='314.5'/><line x1='91.0' x2='101.0' y1='228.8' y2='228.8'/><line x1='91.0' x2='101.0' y1='314.5' y2='314.5'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(100.0 342.0) rotate(-35)'>LaplacianScore</text><rect class='sa-hit' x='64.0' y='18' width='64' height='414'><title>LaplacianScore (Distance, non-search)
|
| 23 |
Elo 939 [95% CI 850β1009]
|
| 24 |
Stability 0.660
|
| 25 |
+
Validity 0.827</title></rect><rect x='137.6' y='233.7' width='22.4' height='94.3' fill='#ffffff' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='160.0' y='233.7' width='22.4' height='94.3' fill='#e9ac7c' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='160.0' x2='160.0' y1='208.9' y2='261.7'/><line x1='155.0' x2='165.0' y1='208.9' y2='208.9'/><line x1='155.0' x2='165.0' y1='261.7' y2='261.7'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(164.0 342.0) rotate(-35)'>Random</text><rect class='sa-hit' x='128.0' y='18' width='64' height='414'><title>Random (Baseline, baseline)
|
| 26 |
Elo 1000 [95% CI 948β1046]
|
| 27 |
Stability -0.001
|
| 28 |
+
Validity 0.513</title></rect><rect x='201.6' y='139.3' width='22.4' height='188.7' fill='#d8eaf3' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='224.0' y='139.3' width='22.4' height='188.7' fill='#e7a26c' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='224.0' x2='224.0' y1='109.7' y2='166.3'/><line x1='219.0' x2='229.0' y1='109.7' y2='109.7'/><line x1='219.0' x2='229.0' y1='166.3' y2='166.3'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(228.0 342.0) rotate(-35)'>LOCO</text><rect class='sa-hit' x='192.0' y='18' width='64' height='414'><title>LOCO (Leave-one-out, search)
|
| 29 |
Elo 1175 [95% CI 1125β1230]
|
| 30 |
Stability 0.151
|
| 31 |
+
Validity 0.575</title></rect><rect x='265.6' y='129.6' width='22.4' height='198.4' fill='#c2dded' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='288.0' y='129.6' width='22.4' height='198.4' fill='#e59a5f' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='288.0' x2='288.0' y1='99.9' y2='155.5'/><line x1='283.0' x2='293.0' y1='99.9' y2='99.9'/><line x1='283.0' x2='293.0' y1='155.5' y2='155.5'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(292.0 342.0) rotate(-35)'>SFS</text><rect class='sa-hit' x='256.0' y='18' width='64' height='414'><title>SFS (Forward search, search)
|
| 32 |
Elo 1193 [95% CI 1145β1248]
|
| 33 |
Stability 0.239
|
| 34 |
+
Validity 0.627</title></rect><rect x='329.6' y='127.4' width='22.4' height='200.6' fill='#62a8d0' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='352.0' y='127.4' width='22.4' height='200.6' fill='#e39557' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='352.0' x2='352.0' y1='103.2' y2='151.7'/><line x1='347.0' x2='357.0' y1='103.2' y2='103.2'/><line x1='347.0' x2='357.0' y1='151.7' y2='151.7'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(356.0 342.0) rotate(-35)'>mRMR</text><rect class='sa-hit' x='320.0' y='18' width='64' height='414'><title>mRMR (Information theory, non-search)
|
| 35 |
Elo 1197 [95% CI 1152β1242]
|
| 36 |
Stability 0.614
|
| 37 |
+
Validity 0.660</title></rect><rect x='393.6' y='126.4' width='22.4' height='201.6' fill='#53a0cb' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='416.0' y='126.4' width='22.4' height='201.6' fill='#df853e' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='416.0' x2='416.0' y1='103.2' y2='152.8'/><line x1='411.0' x2='421.0' y1='103.2' y2='103.2'/><line x1='411.0' x2='421.0' y1='152.8' y2='152.8'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(420.0 342.0) rotate(-35)'>(R)ReliefF</text><rect class='sa-hit' x='384.0' y='18' width='64' height='414'><title>(R)ReliefF (Distance, non-search)
|
| 38 |
Elo 1199 [95% CI 1150β1242]
|
| 39 |
Stability 0.673
|
| 40 |
+
Validity 0.757</title></rect><rect x='457.6' y='123.1' width='22.4' height='204.9' fill='#54a1cb' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='480.0' y='123.1' width='22.4' height='204.9' fill='#db7525' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='480.0' x2='480.0' y1='98.9' y2='146.9'/><line x1='475.0' x2='485.0' y1='98.9' y2='98.9'/><line x1='475.0' x2='485.0' y1='146.9' y2='146.9'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(484.0 342.0) rotate(-35)'>GainRatio</text><rect class='sa-hit' x='448.0' y='18' width='64' height='414'><title>GainRatio (Information theory, non-search)
|
| 41 |
Elo 1205 [95% CI 1161β1250]
|
| 42 |
Stability 0.670
|
| 43 |
+
Validity 0.856</title></rect><rect x='521.6' y='119.4' width='22.4' height='208.6' fill='#4b9bc9' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='544.0' y='119.4' width='22.4' height='208.6' fill='#da711f' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='544.0' x2='544.0' y1='94.0' y2='142.5'/><line x1='539.0' x2='549.0' y1='94.0' y2='94.0'/><line x1='539.0' x2='549.0' y1='142.5' y2='142.5'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(548.0 342.0) rotate(-35)'>F-Test</text><rect class='sa-hit' x='512.0' y='18' width='64' height='414'><title>F-Test (Correlation, non-search)
|
| 44 |
Elo 1212 [95% CI 1169β1259]
|
| 45 |
Stability 0.706
|
| 46 |
+
Validity 0.879</title></rect><rect x='585.6' y='112.9' width='22.4' height='215.1' fill='#daebf4' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='608.0' y='112.9' width='22.4' height='215.1' fill='#e8a571' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='608.0' x2='608.0' y1='83.2' y2='138.2'/><line x1='603.0' x2='613.0' y1='83.2' y2='83.2'/><line x1='603.0' x2='613.0' y1='138.2' y2='138.2'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(612.0 342.0) rotate(-35)'>SBE</text><rect class='sa-hit' x='576.0' y='18' width='64' height='414'><title>SBE (Backward search, search)
|
| 47 |
Elo 1224 [95% CI 1177β1279]
|
| 48 |
Stability 0.144
|
| 49 |
+
Validity 0.556</title></rect><rect x='649.6' y='99.9' width='22.4' height='228.1' fill='#96c5df' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='672.0' y='99.9' width='22.4' height='228.1' fill='#ecb78d' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='672.0' x2='672.0' y1='78.9' y2='119.9'/><line x1='667.0' x2='677.0' y1='78.9' y2='78.9'/><line x1='667.0' x2='677.0' y1='119.9' y2='119.9'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(676.0 342.0) rotate(-35)'>MarkovBlanket</text><rect class='sa-hit' x='640.0' y='18' width='64' height='414'><title>MarkovBlanket (Backward search, search)
|
| 50 |
Elo 1248 [95% CI 1211β1287]
|
| 51 |
Stability 0.413
|
| 52 |
+
Validity 0.447</title></rect><rect x='713.6' y='97.8' width='22.4' height='230.2' fill='#84bbda' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='736.0' y='97.8' width='22.4' height='230.2' fill='#de8239' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='736.0' x2='736.0' y1='71.9' y2='122.6'/><line x1='731.0' x2='741.0' y1='71.9' y2='71.9'/><line x1='731.0' x2='741.0' y1='122.6' y2='122.6'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(740.0 342.0) rotate(-35)'>ElasticNet</text><rect class='sa-hit' x='704.0' y='18' width='64' height='414'><title>ElasticNet (Regularization, non-search)
|
| 53 |
Elo 1252 [95% CI 1206β1300]
|
| 54 |
Stability 0.481
|
| 55 |
+
Validity 0.777</title></rect><rect x='777.6' y='89.2' width='22.4' height='238.8' fill='#9cc8e1' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='800.0' y='89.2' width='22.4' height='238.8' fill='#e08945' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='800.0' x2='800.0' y1='61.1' y2='116.1'/><line x1='795.0' x2='805.0' y1='61.1' y2='61.1'/><line x1='795.0' x2='805.0' y1='116.1' y2='116.1'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(804.0 342.0) rotate(-35)'>Lasso</text><rect class='sa-hit' x='768.0' y='18' width='64' height='414'><title>Lasso (Regularization, non-search)
|
| 56 |
Elo 1268 [95% CI 1218β1320]
|
| 57 |
Stability 0.389
|
| 58 |
+
Validity 0.730</title></rect><rect x='841.6' y='81.6' width='22.4' height='246.4' fill='#519fca' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='864.0' y='81.6' width='22.4' height='246.4' fill='#dd7c2f' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='864.0' x2='864.0' y1='54.1' y2='106.4'/><line x1='859.0' x2='869.0' y1='54.1' y2='54.1'/><line x1='859.0' x2='869.0' y1='106.4' y2='106.4'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(868.0 342.0) rotate(-35)'>MI</text><rect class='sa-hit' x='832.0' y='18' width='64' height='414'><title>MI (Information theory, non-search)
|
| 59 |
Elo 1282 [95% CI 1236β1333]
|
| 60 |
Stability 0.683
|
| 61 |
+
Validity 0.815</title></rect><rect x='905.6' y='81.1' width='22.4' height='246.9' fill='#8ec0dd' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='928.0' y='81.1' width='22.4' height='246.9' fill='#e7a36d' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='928.0' x2='928.0' y1='54.1' y2='108.0'/><line x1='923.0' x2='933.0' y1='54.1' y2='54.1'/><line x1='923.0' x2='933.0' y1='108.0' y2='108.0'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(932.0 342.0) rotate(-35)'>CART</text><rect class='sa-hit' x='896.0' y='18' width='64' height='414'><title>CART (Tree, non-search)
|
| 62 |
Elo 1283 [95% CI 1233β1333]
|
| 63 |
Stability 0.444
|
| 64 |
+
Validity 0.572</title></rect><rect x='969.6' y='60.6' width='22.4' height='267.4' fill='#53a0cb' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='992.0' y='60.6' width='22.4' height='267.4' fill='#e2904f' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='992.0' x2='992.0' y1='27.7' y2='89.7'/><line x1='987.0' x2='997.0' y1='27.7' y2='27.7'/><line x1='987.0' x2='997.0' y1='89.7' y2='89.7'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(996.0 342.0) rotate(-35)'>RFImportance</text><rect class='sa-hit' x='960.0' y='18' width='64' height='414'><title>RFImportance (Tree, non-search)
|
| 65 |
Elo 1321 [95% CI 1267β1382]
|
| 66 |
Stability 0.676
|
| 67 |
+
Validity 0.691</title></rect><line class='sa-ink' x1='64' x2='1024' y1='328.0' y2='328.0' stroke-width='1' opacity='0.4'/><rect x='1070' y='40.0' width='12' height='107.0' fill='url(#sa-g-stab)' stroke='#000' stroke-width='0.6'/><text class='sa-text' x='1076' y='30.0' text-anchor='middle' font-size='12'>stable</text><text class='sa-text' x='1076' y='165.0' text-anchor='middle' font-size='12'>unstable</text><text class='sa-muted' x='1088' y='151.0' font-size='11'>0.0</text><text class='sa-muted' x='1088' y='97.5' font-size='11'>0.5</text><text class='sa-muted' x='1088' y='44.0' font-size='11'>1.0</text><rect x='1070' y='207.0' width='12' height='107.0' fill='url(#sa-g-val)' stroke='#000' stroke-width='0.6'/><text class='sa-text' x='1076' y='197.0' text-anchor='middle' font-size='12'>valid</text><text class='sa-text' x='1076' y='332.0' text-anchor='middle' font-size='12'>invalid</text><text class='sa-muted' x='1088' y='318.0' font-size='11'>0.0</text><text class='sa-muted' x='1088' y='264.5' font-size='11'>0.5</text><text class='sa-muted' x='1088' y='211.0' font-size='11'>1.0</text><rect x='1070' y='352.0' width='12' height='12' fill='url(#sa-na)' stroke='#000' stroke-width='0.6'/><text class='sa-muted' x='1088' y='362.0' font-size='11'>no result</text></svg><script type='text/plain' id='sa-csv'>Method,Category,Subgroup,Elo,Elo CI low,Elo CI high,Stability,Validity
|
| 68 |
+
RFImportance,Non-search,Tree,1321.0,1267.0,1382.0,0.676,0.691
|
| 69 |
+
CART,Non-search,Tree,1283.0,1233.0,1333.0,0.444,0.572
|
| 70 |
+
MI,Non-search,Information theory,1282.0,1236.0,1333.0,0.683,0.815
|
| 71 |
+
Lasso,Non-search,Regularization,1268.0,1218.0,1320.0,0.389,0.73
|
| 72 |
+
ElasticNet,Non-search,Regularization,1252.0,1206.0,1300.0,0.481,0.777
|
| 73 |
+
MarkovBlanket,Search,Backward search,1248.0,1211.0,1287.0,0.413,0.447
|
| 74 |
+
SBE,Search,Backward search,1224.0,1177.0,1279.0,0.144,0.556
|
| 75 |
+
F-Test,Non-search,Correlation,1212.0,1169.0,1259.0,0.706,0.879
|
| 76 |
+
GainRatio,Non-search,Information theory,1205.0,1161.0,1250.0,0.67,0.856
|
| 77 |
+
(R)ReliefF,Non-search,Distance,1199.0,1150.0,1242.0,0.673,0.757
|
| 78 |
+
mRMR,Non-search,Information theory,1197.0,1152.0,1242.0,0.614,0.66
|
| 79 |
+
SFS,Search,Forward search,1193.0,1145.0,1248.0,0.239,0.627
|
| 80 |
+
LOCO,Search,Leave-one-out,1175.0,1125.0,1230.0,0.151,0.575
|
| 81 |
+
Random,Baseline,Baseline,1000.0,948.0,1046.0,-0.001,0.513
|
| 82 |
+
LaplacianScore,Non-search,Distance,939.0,850.0,1009.0,0.66,0.827
|
| 83 |
</script><script>
|
| 84 |
function saDownload(name, blob) {
|
| 85 |
var a = document.createElement('a');
|
data/subsets/tasks_all/models_lightgbm/budgets_2/leaderboard_table.html
CHANGED
|
@@ -26,23 +26,25 @@ td.method { font-weight:600; }
|
|
| 26 |
td .sub { color:var(--muted); font-size:12px; font-weight:400; margin-left:6px; }
|
| 27 |
td.na { color:var(--muted); }
|
| 28 |
.pill { display:inline-block; padding:0 8px; border:1px solid var(--line); border-radius:999px;
|
| 29 |
-
font-size:12px; color:var(--
|
| 30 |
-
</style></head><body><div class='toolbar'><span class='hint'>Click a column to sort. Hover a header for its definition.</span><button onclick="saCsv('selectarena_leaderboard.csv')">CSV</button></div><div class='wrap'><table id='sa-table'><thead><tr><th class='num'>#</th><th class=''>Method</th><th class='' title='Filter: scores features from the data alone. Wrapper: searches feature subsets by refitting a proxy model. Embedded: reads the selection off a fitted model.'>Type</th><th class='num' title='Bradley-Terry Elo from pairwise matches within each dataset, budget and downstream model. Random = 1000. +/β give the 95% bootstrap interval.'>Elo β</th><th class='num' title='Mean rank of the downstream error.'>Avg. rank β</th><th class='num' title='Share of pairwise comparisons won, in %. Ties count half.'>Win rate β</th><th class='num' title='Mean relative gap (error β best error) / error to the best method, in %.'>Improvability β</th><th class='num' title='Mean ratio of the error to the same model trained on all features. Below 1: the selection helped.'>Error vs. all β</th><th class='num' title='Agreement of the selected features across bootstrap resamples (Nogueira et al., 2018).'>Stability β</th><th class='num' title='Share of the budget filled with original features when as many noise features are added.'>Validity β</th><th class='num' title='Median wall-clock time of the selection.'>Time (s) β</th><th class='num' title='Share of runs where the method ran out of time and random features filled the budget.'>Fallback (%)</th><th class='num'>#Datasets</th></tr></thead><tbody><tr><td class='num' data-sort='1'>1</td><td class='method' data-sort='RFImportance'>RFImportance</td><td class='' data-sort='Embedded'><span class='pill'>π³ Embedded</span></td><td class='num' data-sort='1321.0'>1321<span class='sub'>+61 / β54</span></td><td class='num' data-sort='6.15'>6.15</td><td class='num' data-sort='63.2'>63.2</td><td class='num' data-sort='12.2'>12.2</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.676'>0.676</td><td class='num' data-sort='0.691'>0.691</td><td class='num' data-sort='0.2'>0.2</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='2'>2</td><td class='method' data-sort='CART'>CART</td><td class='' data-sort='Embedded'><span class='pill'>π³ Embedded</span></td><td class='num' data-sort='1283.0'>1283<span class='sub'>+50 / β50</span></td><td class='num' data-sort='6.72'>6.72</td><td class='num' data-sort='59.1'>59.1</td><td class='num' data-sort='16.5'>16.5</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.444'>0.444</td><td class='num' data-sort='0.572'>0.572</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='3'>3</td><td class='method' data-sort='MI'>MI</td><td class='' data-sort='Filter'><span class='pill'>π Filter</span></td><td class='num' data-sort='1282.0'>1282<span class='sub'>+51 / β46</span></td><td class='num' data-sort='6.86'>6.86</td><td class='num' data-sort='58.2'>58.2</td><td class='num' data-sort='15.7'>15.7</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.683'>0.683</td><td class='num' data-sort='0.815'>0.815</td><td class='num' data-sort='0.4'>0.4</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='4'>4</td><td class='method' data-sort='Lasso'>Lasso</td><td class='' data-sort='Embedded'><span class='pill'>π³ Embedded</span></td><td class='num' data-sort='1268.0'>1268<span class='sub'>+52 / β50</span></td><td class='num' data-sort='6.95'>6.95</td><td class='num' data-sort='57.5'>57.5</td><td class='num' data-sort='15.4'>15.4</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.389'>0.389</td><td class='num' data-sort='0.73'>0.730</td><td class='num' data-sort='1.8'>1.8</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='5'>5</td><td class='method' data-sort='ElasticNet'>ElasticNet</td><td class='' data-sort='Embedded'><span class='pill'>π³ Embedded</span></td><td class='num' data-sort='1252.0'>1252<span class='sub'>+48 / β46</span></td><td class='num' data-sort='7.01'>7.01</td><td class='num' data-sort='57.1'>57.1</td><td class='num' data-sort='16.3'>16.3</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.481'>0.481</td><td class='num' data-sort='0.777'>0.777</td><td class='num' data-sort='6.3'>6.3</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='6'>6</td><td class='method' data-sort='MarkovBlanket'>MarkovBlanket</td><td class='' data-sort='Filter'><span class='pill'>π Filter</span></td><td class='num' data-sort='1248.0'>1248<span class='sub'>+39 / β37</span></td><td class='num' data-sort='7.3'>7.30</td><td class='num' data-sort='55.0'>55.0</td><td class='num' data-sort='17.8'>17.8</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.413'>0.413</td><td class='num' data-sort='0.447'>0.447</td><td class='num' data-sort='0.1'>0.1</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='7'>7</td><td class='method' data-sort='SBE'>SBE</td><td class='' data-sort='Wrapper'><span class='pill'>π Wrapper</span></td><td class='num' data-sort='1224.0'>1224<span class='sub'>+55 / β47</span></td><td class='num' data-sort='7.76'>7.76</td><td class='num' data-sort='51.7'>51.7</td><td class='num' data-sort='21.9'>21.9</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.144'>0.144</td><td class='num' data-sort='0.556'>0.556</td><td class='num' data-sort='2887.2'>2,887.2</td><td class='num' data-sort='47.4'>47.4</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='8'>8</td><td class='method' data-sort='F-Test'>F-Test</td><td class='' data-sort='Filter'><span class='pill'>π Filter</span></td><td class='num' data-sort='1212.0'>1212<span class='sub'>+47 / β43</span></td><td class='num' data-sort='7.66'>7.66</td><td class='num' data-sort='52.4'>52.4</td><td class='num' data-sort='18.3'>18.3</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.706'>0.706</td><td class='num' data-sort='0.879'>0.879</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='9'>9</td><td class='method' data-sort='GainRatio'>GainRatio</td><td class='' data-sort='Filter'><span class='pill'>π Filter</span></td><td class='num' data-sort='1205.0'>1205<span class='sub'>+45 / β44</span></td><td class='num' data-sort='8.25'>8.25</td><td class='num' data-sort='48.2'>48.2</td><td class='num' data-sort='19.7'>19.7</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.67'>0.670</td><td class='num' data-sort='0.856'>0.856</td><td class='num' data-sort='0.2'>0.2</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='10'>10</td><td class='method' data-sort='(R)ReliefF'>(R)ReliefF</td><td class='' data-sort='Filter'><span class='pill'>π Filter</span></td><td class='num' data-sort='1199.0'>1199<span class='sub'>+43 / β49</span></td><td class='num' data-sort='7.77'>7.77</td><td class='num' data-sort='51.7'>51.7</td><td class='num' data-sort='17.9'>17.9</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.673'>0.673</td><td class='num' data-sort='0.757'>0.757</td><td class='num' data-sort='12.9'>12.9</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='11'>11</td><td class='method' data-sort='mRMR'>mRMR</td><td class='' data-sort='Filter'><span class='pill'>π Filter</span></td><td class='num' data-sort='1197.0'>1197<span class='sub'>+45 / β45</span></td><td class='num' data-sort='8.29'>8.29</td><td class='num' data-sort='47.9'>47.9</td><td class='num' data-sort='19.2'>19.2</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.614'>0.614</td><td class='num' data-sort='0.66'>0.660</td><td class='num' data-sort='0.4'>0.4</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='12'>12</td><td class='method' data-sort='SFS'>SFS</td><td class='' data-sort='Wrapper'><span class='pill'>π Wrapper</span></td><td class='num' data-sort='1193.0'>1193<span class='sub'>+55 / β48</span></td><td class='num' data-sort='8.11'>8.11</td><td class='num' data-sort='49.2'>49.2</td><td class='num' data-sort='20.8'>20.8</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.239'>0.239</td><td class='num' data-sort='0.627'>0.627</td><td class='num' data-sort='2274.8'>2,274.8</td><td class='num' data-sort='48.8'>48.8</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='13'>13</td><td class='method' data-sort='LOCO'>LOCO</td><td class='' data-sort='Wrapper'><span class='pill'>π Wrapper</span></td><td class='num' data-sort='1175.0'>1175<span class='sub'>+55 / β50</span></td><td class='num' data-sort='8.2'>8.20</td><td class='num' data-sort='48.6'>48.6</td><td class='num' data-sort='22.8'>22.8</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.151'>0.151</td><td class='num' data-sort='0.575'>0.575</td><td class='num' data-sort='251.9'>251.9</td><td class='num' data-sort='6.6'>6.6</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='14'>14</td><td class='method' data-sort='Random'>Random</td><td class='' data-sort='Baseline'><span class='pill'>βͺ Baseline</span></td><td class='num' data-sort='1000.0'>1000<span class='sub'>+46 / β52</span></td><td class='num' data-sort='10.87'>10.87</td><td class='num' data-sort='29.5'>29.5</td><td class='num' data-sort='32.0'>32.0</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='-0.001'>-0.001</td><td class='num' data-sort='0.513'>0.513</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='100.0'>100.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='15'>15</td><td class='method' data-sort='LaplacianScore'>LaplacianScore</td><td class='' data-sort='Filter'><span class='pill'>π Filter</span></td><td class='num' data-sort='939.0'>939<span class='sub'>+70 / β89</span></td><td class='num' data-sort='12.1'>12.10</td><td class='num' data-sort='20.7'>20.7</td><td class='num' data-sort='37.6'>37.6</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.66'>0.660</td><td class='num' data-sort='0.827'>0.827</td><td class='num' data-sort='0.2'>0.2</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr></tbody></table></div><script type='text/plain' id='sa-csv'>Rank,Type,TypeName,Method,Elo,Elo CI low,Elo CI high,Avg. rank,Win rate (%),Improvability (%),Error vs. all features,Stability,Validity,Selection time (s),Fallback rate (%),#Datasets,#Comparisons
|
| 31 |
-
|
| 32 |
-
2,π³,Embedded,CART,1283.0,1233.0,1333.0,6.72,59.1,16.5,,0.444,0.572,0.0,0.0,71,213
|
| 33 |
-
|
| 34 |
-
|
| 35 |
-
|
| 36 |
-
|
| 37 |
-
|
| 38 |
-
|
| 39 |
-
|
| 40 |
-
|
| 41 |
-
|
| 42 |
-
|
| 43 |
-
|
| 44 |
-
|
| 45 |
-
|
|
|
|
|
|
|
| 46 |
</script><script>
|
| 47 |
function saDownload(name, blob) {
|
| 48 |
var a = document.createElement('a');
|
|
|
|
| 26 |
td .sub { color:var(--muted); font-size:12px; font-weight:400; margin-left:6px; }
|
| 27 |
td.na { color:var(--muted); }
|
| 28 |
.pill { display:inline-block; padding:0 8px; border:1px solid var(--line); border-radius:999px;
|
| 29 |
+
font-size:12px; color:var(--fg); }
|
| 30 |
+
.pill.cat-search { border-color:var(--accent); }
|
| 31 |
+
.pill.cat-baseline { color:var(--muted); }
|
| 32 |
+
</style></head><body><div class='toolbar'><span class='hint'>Click a column to sort. Hover a header for its definition.</span><button onclick="saCsv('selectarena_leaderboard.csv')">CSV</button></div><div class='wrap'><table id='sa-table'><thead><tr><th class='num'>#</th><th class=''>Method</th><th class='' title='Taxonomy of the paper (Appendix A). Search methods evaluate many feature subsets with the model (forward, backward, leave-one-out); non-search methods use a proxy instead (correlation, distance, information theory, tree, regularization).'>Type</th><th class='num' title='Bradley-Terry Elo from pairwise matches within each dataset, budget and downstream model. Random = 1000. +/β give the 95% bootstrap interval.'>Elo β</th><th class='num' title='Mean rank of the downstream error.'>Avg. rank β</th><th class='num' title='Share of pairwise comparisons won, in %. Ties count half.'>Win rate β</th><th class='num' title='Mean relative gap (error β best error) / error to the best method, in %.'>Improvability β</th><th class='num' title='Mean ratio of the error to the same model trained on all features. Below 1: the selection helped.'>Error vs. all β</th><th class='num' title='Agreement of the selected features across bootstrap resamples (Nogueira et al., 2018).'>Stability β</th><th class='num' title='Share of the budget filled with original features when as many noise features are added.'>Validity β</th><th class='num' title='Median wall-clock time of the selection.'>Time (s) β</th><th class='num' title='Share of runs where the method ran out of time and random features filled the budget.'>Fallback (%)</th><th class='num'>#Datasets</th></tr></thead><tbody><tr><td class='num' data-sort='1'>1</td><td class='method' data-sort='RFImportance'>RFImportance</td><td class='' data-sort='Tree'><span class='pill cat-non-search'>Tree</span><span class='sub'>non-search</span></td><td class='num' data-sort='1321.0'>1321<span class='sub'>+61 / β54</span></td><td class='num' data-sort='6.15'>6.15</td><td class='num' data-sort='63.2'>63.2</td><td class='num' data-sort='12.2'>12.2</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.676'>0.676</td><td class='num' data-sort='0.691'>0.691</td><td class='num' data-sort='0.2'>0.2</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='2'>2</td><td class='method' data-sort='CART'>CART</td><td class='' data-sort='Tree'><span class='pill cat-non-search'>Tree</span><span class='sub'>non-search</span></td><td class='num' data-sort='1283.0'>1283<span class='sub'>+50 / β50</span></td><td class='num' data-sort='6.72'>6.72</td><td class='num' data-sort='59.1'>59.1</td><td class='num' data-sort='16.5'>16.5</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.444'>0.444</td><td class='num' data-sort='0.572'>0.572</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='3'>3</td><td class='method' data-sort='MI'>MI</td><td class='' data-sort='Information theory'><span class='pill cat-non-search'>Information theory</span><span class='sub'>non-search</span></td><td class='num' data-sort='1282.0'>1282<span class='sub'>+51 / β46</span></td><td class='num' data-sort='6.86'>6.86</td><td class='num' data-sort='58.2'>58.2</td><td class='num' data-sort='15.7'>15.7</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.683'>0.683</td><td class='num' data-sort='0.815'>0.815</td><td class='num' data-sort='0.4'>0.4</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='4'>4</td><td class='method' data-sort='Lasso'>Lasso</td><td class='' data-sort='Regularization'><span class='pill cat-non-search'>Regularization</span><span class='sub'>non-search</span></td><td class='num' data-sort='1268.0'>1268<span class='sub'>+52 / β50</span></td><td class='num' data-sort='6.95'>6.95</td><td class='num' data-sort='57.5'>57.5</td><td class='num' data-sort='15.4'>15.4</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.389'>0.389</td><td class='num' data-sort='0.73'>0.730</td><td class='num' data-sort='1.8'>1.8</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='5'>5</td><td class='method' data-sort='ElasticNet'>ElasticNet</td><td class='' data-sort='Regularization'><span class='pill cat-non-search'>Regularization</span><span class='sub'>non-search</span></td><td class='num' data-sort='1252.0'>1252<span class='sub'>+48 / β46</span></td><td class='num' data-sort='7.01'>7.01</td><td class='num' data-sort='57.1'>57.1</td><td class='num' data-sort='16.3'>16.3</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.481'>0.481</td><td class='num' data-sort='0.777'>0.777</td><td class='num' data-sort='6.3'>6.3</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='6'>6</td><td class='method' data-sort='MarkovBlanket'>MarkovBlanket</td><td class='' data-sort='Backward search'><span class='pill cat-search'>Backward search</span><span class='sub'>search</span></td><td class='num' data-sort='1248.0'>1248<span class='sub'>+39 / β37</span></td><td class='num' data-sort='7.3'>7.30</td><td class='num' data-sort='55.0'>55.0</td><td class='num' data-sort='17.8'>17.8</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.413'>0.413</td><td class='num' data-sort='0.447'>0.447</td><td class='num' data-sort='0.1'>0.1</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='7'>7</td><td class='method' data-sort='SBE'>SBE</td><td class='' data-sort='Backward search'><span class='pill cat-search'>Backward search</span><span class='sub'>search</span></td><td class='num' data-sort='1224.0'>1224<span class='sub'>+55 / β47</span></td><td class='num' data-sort='7.76'>7.76</td><td class='num' data-sort='51.7'>51.7</td><td class='num' data-sort='21.9'>21.9</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.144'>0.144</td><td class='num' data-sort='0.556'>0.556</td><td class='num' data-sort='2887.2'>2,887.2</td><td class='num' data-sort='47.4'>47.4</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='8'>8</td><td class='method' data-sort='F-Test'>F-Test</td><td class='' data-sort='Correlation'><span class='pill cat-non-search'>Correlation</span><span class='sub'>non-search</span></td><td class='num' data-sort='1212.0'>1212<span class='sub'>+47 / β43</span></td><td class='num' data-sort='7.66'>7.66</td><td class='num' data-sort='52.4'>52.4</td><td class='num' data-sort='18.3'>18.3</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.706'>0.706</td><td class='num' data-sort='0.879'>0.879</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='9'>9</td><td class='method' data-sort='GainRatio'>GainRatio</td><td class='' data-sort='Information theory'><span class='pill cat-non-search'>Information theory</span><span class='sub'>non-search</span></td><td class='num' data-sort='1205.0'>1205<span class='sub'>+45 / β44</span></td><td class='num' data-sort='8.25'>8.25</td><td class='num' data-sort='48.2'>48.2</td><td class='num' data-sort='19.7'>19.7</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.67'>0.670</td><td class='num' data-sort='0.856'>0.856</td><td class='num' data-sort='0.2'>0.2</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='10'>10</td><td class='method' data-sort='(R)ReliefF'>(R)ReliefF</td><td class='' data-sort='Distance'><span class='pill cat-non-search'>Distance</span><span class='sub'>non-search</span></td><td class='num' data-sort='1199.0'>1199<span class='sub'>+43 / β49</span></td><td class='num' data-sort='7.77'>7.77</td><td class='num' data-sort='51.7'>51.7</td><td class='num' data-sort='17.9'>17.9</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.673'>0.673</td><td class='num' data-sort='0.757'>0.757</td><td class='num' data-sort='12.9'>12.9</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='11'>11</td><td class='method' data-sort='mRMR'>mRMR</td><td class='' data-sort='Information theory'><span class='pill cat-non-search'>Information theory</span><span class='sub'>non-search</span></td><td class='num' data-sort='1197.0'>1197<span class='sub'>+45 / β45</span></td><td class='num' data-sort='8.29'>8.29</td><td class='num' data-sort='47.9'>47.9</td><td class='num' data-sort='19.2'>19.2</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.614'>0.614</td><td class='num' data-sort='0.66'>0.660</td><td class='num' data-sort='0.4'>0.4</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='12'>12</td><td class='method' data-sort='SFS'>SFS</td><td class='' data-sort='Forward search'><span class='pill cat-search'>Forward search</span><span class='sub'>search</span></td><td class='num' data-sort='1193.0'>1193<span class='sub'>+55 / β48</span></td><td class='num' data-sort='8.11'>8.11</td><td class='num' data-sort='49.2'>49.2</td><td class='num' data-sort='20.8'>20.8</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.239'>0.239</td><td class='num' data-sort='0.627'>0.627</td><td class='num' data-sort='2274.8'>2,274.8</td><td class='num' data-sort='48.8'>48.8</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='13'>13</td><td class='method' data-sort='LOCO'>LOCO</td><td class='' data-sort='Leave-one-out'><span class='pill cat-search'>Leave-one-out</span><span class='sub'>search</span></td><td class='num' data-sort='1175.0'>1175<span class='sub'>+55 / β50</span></td><td class='num' data-sort='8.2'>8.20</td><td class='num' data-sort='48.6'>48.6</td><td class='num' data-sort='22.8'>22.8</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.151'>0.151</td><td class='num' data-sort='0.575'>0.575</td><td class='num' data-sort='251.9'>251.9</td><td class='num' data-sort='6.6'>6.6</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='14'>14</td><td class='method' data-sort='Random'>Random</td><td class='' data-sort='Baseline'><span class='pill cat-baseline'>Baseline</span></td><td class='num' data-sort='1000.0'>1000<span class='sub'>+46 / β52</span></td><td class='num' data-sort='10.87'>10.87</td><td class='num' data-sort='29.5'>29.5</td><td class='num' data-sort='32.0'>32.0</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='-0.001'>-0.001</td><td class='num' data-sort='0.513'>0.513</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='100.0'>100.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='15'>15</td><td class='method' data-sort='LaplacianScore'>LaplacianScore</td><td class='' data-sort='Distance'><span class='pill cat-non-search'>Distance</span><span class='sub'>non-search</span></td><td class='num' data-sort='939.0'>939<span class='sub'>+70 / β89</span></td><td class='num' data-sort='12.1'>12.10</td><td class='num' data-sort='20.7'>20.7</td><td class='num' data-sort='37.6'>37.6</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.66'>0.660</td><td class='num' data-sort='0.827'>0.827</td><td class='num' data-sort='0.2'>0.2</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr></tbody></table></div><script type='text/plain' id='sa-csv'>Rank,Method,Category,Subgroup,Elo,Elo CI low,Elo CI high,Avg. rank,Win rate (%),Improvability (%),Error vs. all features,Stability,Validity,Selection time (s),Fallback rate (%),#Datasets,#Comparisons
|
| 33 |
+
1,RFImportance,Non-search,Tree,1321.0,1267.0,1382.0,6.15,63.2,12.2,,0.676,0.691,0.2,0.0,71,213
|
| 34 |
+
2,CART,Non-search,Tree,1283.0,1233.0,1333.0,6.72,59.1,16.5,,0.444,0.572,0.0,0.0,71,213
|
| 35 |
+
3,MI,Non-search,Information theory,1282.0,1236.0,1333.0,6.86,58.2,15.7,,0.683,0.815,0.4,0.0,71,213
|
| 36 |
+
4,Lasso,Non-search,Regularization,1268.0,1218.0,1320.0,6.95,57.5,15.4,,0.389,0.73,1.8,0.0,71,213
|
| 37 |
+
5,ElasticNet,Non-search,Regularization,1252.0,1206.0,1300.0,7.01,57.1,16.3,,0.481,0.777,6.3,0.0,71,213
|
| 38 |
+
6,MarkovBlanket,Search,Backward search,1248.0,1211.0,1287.0,7.3,55.0,17.8,,0.413,0.447,0.1,0.0,71,213
|
| 39 |
+
7,SBE,Search,Backward search,1224.0,1177.0,1279.0,7.76,51.7,21.9,,0.144,0.556,2887.2,47.4,71,213
|
| 40 |
+
8,F-Test,Non-search,Correlation,1212.0,1169.0,1259.0,7.66,52.4,18.3,,0.706,0.879,0.0,0.0,71,213
|
| 41 |
+
9,GainRatio,Non-search,Information theory,1205.0,1161.0,1250.0,8.25,48.2,19.7,,0.67,0.856,0.2,0.0,71,213
|
| 42 |
+
10,(R)ReliefF,Non-search,Distance,1199.0,1150.0,1242.0,7.77,51.7,17.9,,0.673,0.757,12.9,0.0,71,213
|
| 43 |
+
11,mRMR,Non-search,Information theory,1197.0,1152.0,1242.0,8.29,47.9,19.2,,0.614,0.66,0.4,0.0,71,213
|
| 44 |
+
12,SFS,Search,Forward search,1193.0,1145.0,1248.0,8.11,49.2,20.8,,0.239,0.627,2274.8,48.8,71,213
|
| 45 |
+
13,LOCO,Search,Leave-one-out,1175.0,1125.0,1230.0,8.2,48.6,22.8,,0.151,0.575,251.9,6.6,71,213
|
| 46 |
+
14,Random,Baseline,Baseline,1000.0,948.0,1046.0,10.87,29.5,32.0,,-0.001,0.513,0.0,100.0,71,213
|
| 47 |
+
15,LaplacianScore,Non-search,Distance,939.0,850.0,1009.0,12.1,20.7,37.6,,0.66,0.827,0.2,0.0,71,213
|
| 48 |
</script><script>
|
| 49 |
function saDownload(name, blob) {
|
| 50 |
var a = document.createElement('a');
|
data/subsets/tasks_all/models_lightgbm/budgets_2/website_leaderboard.csv
CHANGED
|
@@ -1,16 +1,16 @@
|
|
| 1 |
-
Rank,
|
| 2 |
-
1,
|
| 3 |
-
2,
|
| 4 |
-
3,
|
| 5 |
-
4,
|
| 6 |
-
5,
|
| 7 |
-
6,
|
| 8 |
-
7,
|
| 9 |
-
8,
|
| 10 |
-
9,
|
| 11 |
-
10,
|
| 12 |
-
11,
|
| 13 |
-
12,
|
| 14 |
-
13,
|
| 15 |
-
14,
|
| 16 |
-
15,
|
|
|
|
| 1 |
+
Rank,Method,Category,Subgroup,Elo,Elo CI low,Elo CI high,Avg. rank,Win rate (%),Improvability (%),Error vs. all features,Stability,Validity,Selection time (s),Fallback rate (%),#Datasets,#Comparisons
|
| 2 |
+
1,RFImportance,Non-search,Tree,1321.0,1267.0,1382.0,6.15,63.2,12.2,,0.676,0.691,0.2,0.0,71,213
|
| 3 |
+
2,CART,Non-search,Tree,1283.0,1233.0,1333.0,6.72,59.1,16.5,,0.444,0.572,0.0,0.0,71,213
|
| 4 |
+
3,MI,Non-search,Information theory,1282.0,1236.0,1333.0,6.86,58.2,15.7,,0.683,0.815,0.4,0.0,71,213
|
| 5 |
+
4,Lasso,Non-search,Regularization,1268.0,1218.0,1320.0,6.95,57.5,15.4,,0.389,0.73,1.8,0.0,71,213
|
| 6 |
+
5,ElasticNet,Non-search,Regularization,1252.0,1206.0,1300.0,7.01,57.1,16.3,,0.481,0.777,6.3,0.0,71,213
|
| 7 |
+
6,MarkovBlanket,Search,Backward search,1248.0,1211.0,1287.0,7.3,55.0,17.8,,0.413,0.447,0.1,0.0,71,213
|
| 8 |
+
7,SBE,Search,Backward search,1224.0,1177.0,1279.0,7.76,51.7,21.9,,0.144,0.556,2887.2,47.4,71,213
|
| 9 |
+
8,F-Test,Non-search,Correlation,1212.0,1169.0,1259.0,7.66,52.4,18.3,,0.706,0.879,0.0,0.0,71,213
|
| 10 |
+
9,GainRatio,Non-search,Information theory,1205.0,1161.0,1250.0,8.25,48.2,19.7,,0.67,0.856,0.2,0.0,71,213
|
| 11 |
+
10,(R)ReliefF,Non-search,Distance,1199.0,1150.0,1242.0,7.77,51.7,17.9,,0.673,0.757,12.9,0.0,71,213
|
| 12 |
+
11,mRMR,Non-search,Information theory,1197.0,1152.0,1242.0,8.29,47.9,19.2,,0.614,0.66,0.4,0.0,71,213
|
| 13 |
+
12,SFS,Search,Forward search,1193.0,1145.0,1248.0,8.11,49.2,20.8,,0.239,0.627,2274.8,48.8,71,213
|
| 14 |
+
13,LOCO,Search,Leave-one-out,1175.0,1125.0,1230.0,8.2,48.6,22.8,,0.151,0.575,251.9,6.6,71,213
|
| 15 |
+
14,Random,Baseline,Baseline,1000.0,948.0,1046.0,10.87,29.5,32.0,,-0.001,0.513,0.0,100.0,71,213
|
| 16 |
+
15,LaplacianScore,Non-search,Distance,939.0,850.0,1009.0,12.1,20.7,37.6,,0.66,0.827,0.2,0.0,71,213
|
data/subsets/tasks_all/models_lightgbm/budgets_3/elo_explorer.html
CHANGED
|
@@ -19,67 +19,67 @@ svg { display:block; width:100%; height:auto; }
|
|
| 19 |
.sa-ink { stroke:var(--fg); }
|
| 20 |
.sa-hit { fill:transparent; }
|
| 21 |
.sa-hit:hover { fill:var(--fg); fill-opacity:0.06; }
|
| 22 |
-
</style><defs><pattern id='sa-na' width='6' height='6' patternUnits='userSpaceOnUse' patternTransform='rotate(45)'><rect width='6' height='6' fill='#e5e7eb'/><line x1='0' y1='0' x2='0' y2='6' stroke='#9ca3af' stroke-width='2'/></pattern><linearGradient id='sa-g-stab' x1='0' y1='1' x2='0' y2='0'><stop offset='0' stop-color='#ffffff'/><stop offset='1' stop-color='#0072B2'/></linearGradient><linearGradient id='sa-g-val' x1='0' y1='1' x2='0' y2='0'><stop offset='0' stop-color='#ffffff'/><stop offset='1' stop-color='#D55E00'/></linearGradient></defs><line class='sa-grid' x1='64' x2='1024' y1='293.6' y2='293.6' stroke-width='1'/><text class='sa-muted' x='56' y='297.6' text-anchor='end' font-size='12'>800</text><line class='sa-grid' x1='64' x2='1024' y1='247.6' y2='247.6' stroke-width='1'/><text class='sa-muted' x='56' y='251.6' text-anchor='end' font-size='12'>900</text><line class='sa-grid' x1='64' x2='1024' y1='201.7' y2='201.7' stroke-width='1'/><text class='sa-muted' x='56' y='205.7' text-anchor='end' font-size='12'>1000</text><line class='sa-grid' x1='64' x2='1024' y1='155.8' y2='155.8' stroke-width='1'/><text class='sa-muted' x='56' y='159.8' text-anchor='end' font-size='12'>1100</text><line class='sa-grid' x1='64' x2='1024' y1='109.9' y2='109.9' stroke-width='1'/><text class='sa-muted' x='56' y='113.9' text-anchor='end' font-size='12'>1200</text><line class='sa-grid' x1='64' x2='1024' y1='63.9' y2='63.9' stroke-width='1'/><text class='sa-muted' x='56' y='67.9' text-anchor='end' font-size='12'>1300</text><line class='sa-grid' x1='64' x2='1024' y1='18.0' y2='18.0' stroke-width='1'/><text class='sa-muted' x='56' y='22.0' text-anchor='end' font-size='12'>1400</text><text class='sa-text' transform='translate(16 173) rotate(-90)' text-anchor='middle' font-size='13'>Elo</text><line class='sa-ink' x1='64' x2='1024' y1='201.7' y2='201.7' stroke-width='1' stroke-dasharray='4 4' opacity='0.5'/><rect x='73.6' y='257.7' width='22.4' height='70.3' fill='#4f9eca' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='96.0' y='257.7' width='22.4' height='70.3' fill='#df853e' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='96.0' x2='96.0' y1='226.5' y2='296.8'/><line x1='91.0' x2='101.0' y1='226.5' y2='226.5'/><line x1='91.0' x2='101.0' y1='296.8' y2='296.8'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(100.0 342.0) rotate(-35)'>LaplacianScore</text><rect class='sa-hit' x='64.0' y='18' width='64' height='414'><title>LaplacianScore (
|
| 23 |
Elo 878 [95% CI 793β946]
|
| 24 |
Stability 0.689
|
| 25 |
-
Validity 0.758</title></rect><rect x='137.6' y='201.7' width='22.4' height='126.3' fill='#feffff' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='160.0' y='201.7' width='22.4' height='126.3' fill='#eaaf81' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='160.0' x2='160.0' y1='177.4' y2='227.4'/><line x1='155.0' x2='165.0' y1='177.4' y2='177.4'/><line x1='155.0' x2='165.0' y1='227.4' y2='227.4'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(164.0 342.0) rotate(-35)'>Random</text><rect class='sa-hit' x='128.0' y='18' width='64' height='414'><title>Random (Baseline)
|
| 26 |
Elo 1000 [95% CI 944β1053]
|
| 27 |
Stability 0.002
|
| 28 |
-
Validity 0.494</title></rect><rect x='201.6' y='156.7' width='22.4' height='171.3' fill='#c6dfee' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='224.0' y='156.7' width='22.4' height='171.3' fill='#e7a46e' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='224.0' x2='224.0' y1='134.7' y2='177.8'/><line x1='219.0' x2='229.0' y1='134.7' y2='134.7'/><line x1='219.0' x2='229.0' y1='177.8' y2='177.8'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(228.0 342.0) rotate(-35)'>SFS</text><rect class='sa-hit' x='192.0' y='18' width='64' height='414'><title>SFS (
|
| 29 |
Elo 1098 [95% CI 1052β1146]
|
| 30 |
Stability 0.225
|
| 31 |
-
Validity 0.568</title></rect><rect x='265.6' y='148.9' width='22.4' height='179.1' fill='#5ca5ce' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='288.0' y='148.9' width='22.4' height='179.1' fill='#df863f' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='288.0' x2='288.0' y1='127.3' y2='170.0'/><line x1='283.0' x2='293.0' y1='127.3' y2='127.3'/><line x1='283.0' x2='293.0' y1='170.0' y2='170.0'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(292.0 342.0) rotate(-35)'>GainRatio</text><rect class='sa-hit' x='256.0' y='18' width='64' height='414'><title>GainRatio (
|
| 32 |
Elo 1115 [95% CI 1069β1162]
|
| 33 |
Stability 0.641
|
| 34 |
-
Validity 0.751</title></rect><rect x='329.6' y='147.5' width='22.4' height='180.5' fill='#e2eff6' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='352.0' y='147.5' width='22.4' height='180.5' fill='#e8a774' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='352.0' x2='352.0' y1='124.1' y2='171.4'/><line x1='347.0' x2='357.0' y1='124.1' y2='124.1'/><line x1='347.0' x2='357.0' y1='171.4' y2='171.4'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(356.0 342.0) rotate(-35)'>LOCO</text><rect class='sa-hit' x='320.0' y='18' width='64' height='414'><title>LOCO (
|
| 35 |
Elo 1118 [95% CI 1066β1169]
|
| 36 |
Stability 0.114
|
| 37 |
-
Validity 0.544</title></rect><rect x='393.6' y='145.7' width='22.4' height='182.3' fill='#e7f2f8' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='416.0' y='145.7' width='22.4' height='182.3' fill='#e9aa78' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='416.0' x2='416.0' y1='119.0' y2='170.9'/><line x1='411.0' x2='421.0' y1='119.0' y2='119.0'/><line x1='411.0' x2='421.0' y1='170.9' y2='170.9'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(420.0 342.0) rotate(-35)'>SBE</text><rect class='sa-hit' x='384.0' y='18' width='64' height='414'><title>SBE (
|
| 38 |
Elo 1122 [95% CI 1067β1180]
|
| 39 |
Stability 0.095
|
| 40 |
-
Validity 0.530</title></rect><rect x='457.6' y='142.9' width='22.4' height='185.1' fill='#90c2de' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='480.0' y='142.9' width='22.4' height='185.1' fill='#ebb386' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='480.0' x2='480.0' y1='123.6' y2='163.6'/><line x1='475.0' x2='485.0' y1='123.6' y2='123.6'/><line x1='475.0' x2='485.0' y1='163.6' y2='163.6'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(484.0 342.0) rotate(-35)'>MarkovBlanket</text><rect class='sa-hit' x='448.0' y='18' width='64' height='414'><title>MarkovBlanket (
|
| 41 |
Elo 1128 [95% CI 1083β1170]
|
| 42 |
Stability 0.434
|
| 43 |
-
Validity 0.473</title></rect><rect x='521.6' y='135.1' width='22.4' height='192.9' fill='#58a3cc' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='544.0' y='135.1' width='22.4' height='192.9' fill='#e39455' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='544.0' x2='544.0' y1='112.1' y2='158.1'/><line x1='539.0' x2='549.0' y1='112.1' y2='112.1'/><line x1='539.0' x2='549.0' y1='158.1' y2='158.1'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(548.0 342.0) rotate(-35)'>(R)ReliefF</text><rect class='sa-hit' x='512.0' y='18' width='64' height='414'><title>(R)ReliefF (
|
| 44 |
Elo 1145 [95% CI 1095β1195]
|
| 45 |
Stability 0.656
|
| 46 |
-
Validity 0.667</title></rect><rect x='585.6' y='131.4' width='22.4' height='196.6' fill='#58a3cc' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='608.0' y='131.4' width='22.4' height='196.6' fill='#e6a068' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='608.0' x2='608.0' y1='112.1' y2='151.2'/><line x1='603.0' x2='613.0' y1='112.1' y2='112.1'/><line x1='603.0' x2='613.0' y1='151.2' y2='151.2'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(612.0 342.0) rotate(-35)'>mRMR</text><rect class='sa-hit' x='576.0' y='18' width='64' height='414'><title>mRMR (
|
| 47 |
Elo 1153 [95% CI 1110β1195]
|
| 48 |
Stability 0.656
|
| 49 |
-
Validity 0.593</title></rect><rect x='649.6' y='117.7' width='22.4' height='210.3' fill='#55a1cc' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='672.0' y='117.7' width='22.4' height='210.3' fill='#de8138' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='672.0' x2='672.0' y1='94.7' y2='139.2'/><line x1='667.0' x2='677.0' y1='94.7' y2='94.7'/><line x1='667.0' x2='677.0' y1='139.2' y2='139.2'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(676.0 342.0) rotate(-35)'>F-Test</text><rect class='sa-hit' x='640.0' y='18' width='64' height='414'><title>F-Test (
|
| 50 |
Elo 1183 [95% CI 1136β1233]
|
| 51 |
Stability 0.668
|
| 52 |
-
Validity 0.781</title></rect><rect x='713.6' y='111.2' width='22.4' height='216.8' fill='#5aa3cd' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='736.0' y='111.2' width='22.4' height='216.8' fill='#e18b47' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='736.0' x2='736.0' y1='89.6' y2='132.4'/><line x1='731.0' x2='741.0' y1='89.6' y2='89.6'/><line x1='731.0' x2='741.0' y1='132.4' y2='132.4'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(740.0 342.0) rotate(-35)'>MI</text><rect class='sa-hit' x='704.0' y='18' width='64' height='414'><title>MI (
|
| 53 |
Elo 1197 [95% CI 1151β1244]
|
| 54 |
Stability 0.649
|
| 55 |
-
Validity 0.723</title></rect><rect x='777.6' y='106.6' width='22.4' height='221.4' fill='#aad0e5' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='800.0' y='106.6' width='22.4' height='221.4' fill='#e4975b' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='800.0' x2='800.0' y1='82.8' y2='128.2'/><line x1='795.0' x2='805.0' y1='82.8' y2='82.8'/><line x1='795.0' x2='805.0' y1='128.2' y2='128.2'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(804.0 342.0) rotate(-35)'>Lasso</text><rect class='sa-hit' x='768.0' y='18' width='64' height='414'><title>Lasso (
|
| 56 |
Elo 1207 [95% CI 1160β1259]
|
| 57 |
Stability 0.334
|
| 58 |
-
Validity 0.643</title></rect><rect x='841.6' y='106.6' width='22.4' height='221.4' fill='#94c4df' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='864.0' y='106.6' width='22.4' height='221.4' fill='#e39455' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='864.0' x2='864.0' y1='81.4' y2='131.0'/><line x1='859.0' x2='869.0' y1='81.4' y2='81.4'/><line x1='859.0' x2='869.0' y1='131.0' y2='131.0'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(868.0 342.0) rotate(-35)'>ElasticNet</text><rect class='sa-hit' x='832.0' y='18' width='64' height='414'><title>ElasticNet (
|
| 59 |
Elo 1207 [95% CI 1154β1262]
|
| 60 |
Stability 0.419
|
| 61 |
-
Validity 0.665</title></rect><rect x='905.6' y='105.3' width='22.4' height='222.7' fill='#98c6e0' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='928.0' y='105.3' width='22.4' height='222.7' fill='#eaad7e' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='928.0' x2='928.0' y1='85.1' y2='127.3'/><line x1='923.0' x2='933.0' y1='85.1' y2='85.1'/><line x1='923.0' x2='933.0' y1='127.3' y2='127.3'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(932.0 342.0) rotate(-35)'>CART</text><rect class='sa-hit' x='896.0' y='18' width='64' height='414'><title>CART (
|
| 62 |
Elo 1210 [95% CI 1162β1254]
|
| 63 |
Stability 0.402
|
| 64 |
-
Validity 0.507</title></rect><rect x='969.6' y='76.3' width='22.4' height='251.7' fill='#56a2cc' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='992.0' y='76.3' width='22.4' height='251.7' fill='#e69e66' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='992.0' x2='992.0' y1='48.3' y2='100.2'/><line x1='987.0' x2='997.0' y1='48.3' y2='48.3'/><line x1='987.0' x2='997.0' y1='100.2' y2='100.2'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(996.0 342.0) rotate(-35)'>RFImportance</text><rect class='sa-hit' x='960.0' y='18' width='64' height='414'><title>RFImportance (
|
| 65 |
Elo 1273 [95% CI 1221β1334]
|
| 66 |
Stability 0.661
|
| 67 |
-
Validity 0.600</title></rect><line class='sa-ink' x1='64' x2='1024' y1='328.0' y2='328.0' stroke-width='1' opacity='0.4'/><rect x='1070' y='40.0' width='12' height='107.0' fill='url(#sa-g-stab)' stroke='#000' stroke-width='0.6'/><text class='sa-text' x='1076' y='30.0' text-anchor='middle' font-size='12'>stable</text><text class='sa-text' x='1076' y='165.0' text-anchor='middle' font-size='12'>unstable</text><text class='sa-muted' x='1088' y='151.0' font-size='11'>0.0</text><text class='sa-muted' x='1088' y='97.5' font-size='11'>0.5</text><text class='sa-muted' x='1088' y='44.0' font-size='11'>1.0</text><rect x='1070' y='207.0' width='12' height='107.0' fill='url(#sa-g-val)' stroke='#000' stroke-width='0.6'/><text class='sa-text' x='1076' y='197.0' text-anchor='middle' font-size='12'>valid</text><text class='sa-text' x='1076' y='332.0' text-anchor='middle' font-size='12'>invalid</text><text class='sa-muted' x='1088' y='318.0' font-size='11'>0.0</text><text class='sa-muted' x='1088' y='264.5' font-size='11'>0.5</text><text class='sa-muted' x='1088' y='211.0' font-size='11'>1.0</text><rect x='1070' y='352.0' width='12' height='12' fill='url(#sa-na)' stroke='#000' stroke-width='0.6'/><text class='sa-muted' x='1088' y='362.0' font-size='11'>no result</text></svg><script type='text/plain' id='sa-csv'>Method,
|
| 68 |
-
RFImportance,
|
| 69 |
-
CART,
|
| 70 |
-
Lasso,
|
| 71 |
-
ElasticNet,
|
| 72 |
-
MI,
|
| 73 |
-
F-Test,
|
| 74 |
-
mRMR,
|
| 75 |
-
(R)ReliefF,
|
| 76 |
-
MarkovBlanket,
|
| 77 |
-
SBE,
|
| 78 |
-
LOCO,
|
| 79 |
-
GainRatio,
|
| 80 |
-
SFS,
|
| 81 |
-
Random,Baseline,1000.0,944.0,1053.0,0.002,0.494
|
| 82 |
-
LaplacianScore,
|
| 83 |
</script><script>
|
| 84 |
function saDownload(name, blob) {
|
| 85 |
var a = document.createElement('a');
|
|
|
|
| 19 |
.sa-ink { stroke:var(--fg); }
|
| 20 |
.sa-hit { fill:transparent; }
|
| 21 |
.sa-hit:hover { fill:var(--fg); fill-opacity:0.06; }
|
| 22 |
+
</style><defs><pattern id='sa-na' width='6' height='6' patternUnits='userSpaceOnUse' patternTransform='rotate(45)'><rect width='6' height='6' fill='#e5e7eb'/><line x1='0' y1='0' x2='0' y2='6' stroke='#9ca3af' stroke-width='2'/></pattern><linearGradient id='sa-g-stab' x1='0' y1='1' x2='0' y2='0'><stop offset='0' stop-color='#ffffff'/><stop offset='1' stop-color='#0072B2'/></linearGradient><linearGradient id='sa-g-val' x1='0' y1='1' x2='0' y2='0'><stop offset='0' stop-color='#ffffff'/><stop offset='1' stop-color='#D55E00'/></linearGradient></defs><line class='sa-grid' x1='64' x2='1024' y1='293.6' y2='293.6' stroke-width='1'/><text class='sa-muted' x='56' y='297.6' text-anchor='end' font-size='12'>800</text><line class='sa-grid' x1='64' x2='1024' y1='247.6' y2='247.6' stroke-width='1'/><text class='sa-muted' x='56' y='251.6' text-anchor='end' font-size='12'>900</text><line class='sa-grid' x1='64' x2='1024' y1='201.7' y2='201.7' stroke-width='1'/><text class='sa-muted' x='56' y='205.7' text-anchor='end' font-size='12'>1000</text><line class='sa-grid' x1='64' x2='1024' y1='155.8' y2='155.8' stroke-width='1'/><text class='sa-muted' x='56' y='159.8' text-anchor='end' font-size='12'>1100</text><line class='sa-grid' x1='64' x2='1024' y1='109.9' y2='109.9' stroke-width='1'/><text class='sa-muted' x='56' y='113.9' text-anchor='end' font-size='12'>1200</text><line class='sa-grid' x1='64' x2='1024' y1='63.9' y2='63.9' stroke-width='1'/><text class='sa-muted' x='56' y='67.9' text-anchor='end' font-size='12'>1300</text><line class='sa-grid' x1='64' x2='1024' y1='18.0' y2='18.0' stroke-width='1'/><text class='sa-muted' x='56' y='22.0' text-anchor='end' font-size='12'>1400</text><text class='sa-text' transform='translate(16 173) rotate(-90)' text-anchor='middle' font-size='13'>Elo</text><line class='sa-ink' x1='64' x2='1024' y1='201.7' y2='201.7' stroke-width='1' stroke-dasharray='4 4' opacity='0.5'/><rect x='73.6' y='257.7' width='22.4' height='70.3' fill='#4f9eca' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='96.0' y='257.7' width='22.4' height='70.3' fill='#df853e' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='96.0' x2='96.0' y1='226.5' y2='296.8'/><line x1='91.0' x2='101.0' y1='226.5' y2='226.5'/><line x1='91.0' x2='101.0' y1='296.8' y2='296.8'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(100.0 342.0) rotate(-35)'>LaplacianScore</text><rect class='sa-hit' x='64.0' y='18' width='64' height='414'><title>LaplacianScore (Distance, non-search)
|
| 23 |
Elo 878 [95% CI 793β946]
|
| 24 |
Stability 0.689
|
| 25 |
+
Validity 0.758</title></rect><rect x='137.6' y='201.7' width='22.4' height='126.3' fill='#feffff' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='160.0' y='201.7' width='22.4' height='126.3' fill='#eaaf81' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='160.0' x2='160.0' y1='177.4' y2='227.4'/><line x1='155.0' x2='165.0' y1='177.4' y2='177.4'/><line x1='155.0' x2='165.0' y1='227.4' y2='227.4'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(164.0 342.0) rotate(-35)'>Random</text><rect class='sa-hit' x='128.0' y='18' width='64' height='414'><title>Random (Baseline, baseline)
|
| 26 |
Elo 1000 [95% CI 944β1053]
|
| 27 |
Stability 0.002
|
| 28 |
+
Validity 0.494</title></rect><rect x='201.6' y='156.7' width='22.4' height='171.3' fill='#c6dfee' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='224.0' y='156.7' width='22.4' height='171.3' fill='#e7a46e' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='224.0' x2='224.0' y1='134.7' y2='177.8'/><line x1='219.0' x2='229.0' y1='134.7' y2='134.7'/><line x1='219.0' x2='229.0' y1='177.8' y2='177.8'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(228.0 342.0) rotate(-35)'>SFS</text><rect class='sa-hit' x='192.0' y='18' width='64' height='414'><title>SFS (Forward search, search)
|
| 29 |
Elo 1098 [95% CI 1052β1146]
|
| 30 |
Stability 0.225
|
| 31 |
+
Validity 0.568</title></rect><rect x='265.6' y='148.9' width='22.4' height='179.1' fill='#5ca5ce' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='288.0' y='148.9' width='22.4' height='179.1' fill='#df863f' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='288.0' x2='288.0' y1='127.3' y2='170.0'/><line x1='283.0' x2='293.0' y1='127.3' y2='127.3'/><line x1='283.0' x2='293.0' y1='170.0' y2='170.0'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(292.0 342.0) rotate(-35)'>GainRatio</text><rect class='sa-hit' x='256.0' y='18' width='64' height='414'><title>GainRatio (Information theory, non-search)
|
| 32 |
Elo 1115 [95% CI 1069β1162]
|
| 33 |
Stability 0.641
|
| 34 |
+
Validity 0.751</title></rect><rect x='329.6' y='147.5' width='22.4' height='180.5' fill='#e2eff6' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='352.0' y='147.5' width='22.4' height='180.5' fill='#e8a774' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='352.0' x2='352.0' y1='124.1' y2='171.4'/><line x1='347.0' x2='357.0' y1='124.1' y2='124.1'/><line x1='347.0' x2='357.0' y1='171.4' y2='171.4'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(356.0 342.0) rotate(-35)'>LOCO</text><rect class='sa-hit' x='320.0' y='18' width='64' height='414'><title>LOCO (Leave-one-out, search)
|
| 35 |
Elo 1118 [95% CI 1066β1169]
|
| 36 |
Stability 0.114
|
| 37 |
+
Validity 0.544</title></rect><rect x='393.6' y='145.7' width='22.4' height='182.3' fill='#e7f2f8' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='416.0' y='145.7' width='22.4' height='182.3' fill='#e9aa78' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='416.0' x2='416.0' y1='119.0' y2='170.9'/><line x1='411.0' x2='421.0' y1='119.0' y2='119.0'/><line x1='411.0' x2='421.0' y1='170.9' y2='170.9'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(420.0 342.0) rotate(-35)'>SBE</text><rect class='sa-hit' x='384.0' y='18' width='64' height='414'><title>SBE (Backward search, search)
|
| 38 |
Elo 1122 [95% CI 1067β1180]
|
| 39 |
Stability 0.095
|
| 40 |
+
Validity 0.530</title></rect><rect x='457.6' y='142.9' width='22.4' height='185.1' fill='#90c2de' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='480.0' y='142.9' width='22.4' height='185.1' fill='#ebb386' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='480.0' x2='480.0' y1='123.6' y2='163.6'/><line x1='475.0' x2='485.0' y1='123.6' y2='123.6'/><line x1='475.0' x2='485.0' y1='163.6' y2='163.6'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(484.0 342.0) rotate(-35)'>MarkovBlanket</text><rect class='sa-hit' x='448.0' y='18' width='64' height='414'><title>MarkovBlanket (Backward search, search)
|
| 41 |
Elo 1128 [95% CI 1083β1170]
|
| 42 |
Stability 0.434
|
| 43 |
+
Validity 0.473</title></rect><rect x='521.6' y='135.1' width='22.4' height='192.9' fill='#58a3cc' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='544.0' y='135.1' width='22.4' height='192.9' fill='#e39455' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='544.0' x2='544.0' y1='112.1' y2='158.1'/><line x1='539.0' x2='549.0' y1='112.1' y2='112.1'/><line x1='539.0' x2='549.0' y1='158.1' y2='158.1'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(548.0 342.0) rotate(-35)'>(R)ReliefF</text><rect class='sa-hit' x='512.0' y='18' width='64' height='414'><title>(R)ReliefF (Distance, non-search)
|
| 44 |
Elo 1145 [95% CI 1095β1195]
|
| 45 |
Stability 0.656
|
| 46 |
+
Validity 0.667</title></rect><rect x='585.6' y='131.4' width='22.4' height='196.6' fill='#58a3cc' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='608.0' y='131.4' width='22.4' height='196.6' fill='#e6a068' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='608.0' x2='608.0' y1='112.1' y2='151.2'/><line x1='603.0' x2='613.0' y1='112.1' y2='112.1'/><line x1='603.0' x2='613.0' y1='151.2' y2='151.2'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(612.0 342.0) rotate(-35)'>mRMR</text><rect class='sa-hit' x='576.0' y='18' width='64' height='414'><title>mRMR (Information theory, non-search)
|
| 47 |
Elo 1153 [95% CI 1110β1195]
|
| 48 |
Stability 0.656
|
| 49 |
+
Validity 0.593</title></rect><rect x='649.6' y='117.7' width='22.4' height='210.3' fill='#55a1cc' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='672.0' y='117.7' width='22.4' height='210.3' fill='#de8138' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='672.0' x2='672.0' y1='94.7' y2='139.2'/><line x1='667.0' x2='677.0' y1='94.7' y2='94.7'/><line x1='667.0' x2='677.0' y1='139.2' y2='139.2'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(676.0 342.0) rotate(-35)'>F-Test</text><rect class='sa-hit' x='640.0' y='18' width='64' height='414'><title>F-Test (Correlation, non-search)
|
| 50 |
Elo 1183 [95% CI 1136β1233]
|
| 51 |
Stability 0.668
|
| 52 |
+
Validity 0.781</title></rect><rect x='713.6' y='111.2' width='22.4' height='216.8' fill='#5aa3cd' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='736.0' y='111.2' width='22.4' height='216.8' fill='#e18b47' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='736.0' x2='736.0' y1='89.6' y2='132.4'/><line x1='731.0' x2='741.0' y1='89.6' y2='89.6'/><line x1='731.0' x2='741.0' y1='132.4' y2='132.4'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(740.0 342.0) rotate(-35)'>MI</text><rect class='sa-hit' x='704.0' y='18' width='64' height='414'><title>MI (Information theory, non-search)
|
| 53 |
Elo 1197 [95% CI 1151β1244]
|
| 54 |
Stability 0.649
|
| 55 |
+
Validity 0.723</title></rect><rect x='777.6' y='106.6' width='22.4' height='221.4' fill='#aad0e5' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='800.0' y='106.6' width='22.4' height='221.4' fill='#e4975b' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='800.0' x2='800.0' y1='82.8' y2='128.2'/><line x1='795.0' x2='805.0' y1='82.8' y2='82.8'/><line x1='795.0' x2='805.0' y1='128.2' y2='128.2'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(804.0 342.0) rotate(-35)'>Lasso</text><rect class='sa-hit' x='768.0' y='18' width='64' height='414'><title>Lasso (Regularization, non-search)
|
| 56 |
Elo 1207 [95% CI 1160β1259]
|
| 57 |
Stability 0.334
|
| 58 |
+
Validity 0.643</title></rect><rect x='841.6' y='106.6' width='22.4' height='221.4' fill='#94c4df' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='864.0' y='106.6' width='22.4' height='221.4' fill='#e39455' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='864.0' x2='864.0' y1='81.4' y2='131.0'/><line x1='859.0' x2='869.0' y1='81.4' y2='81.4'/><line x1='859.0' x2='869.0' y1='131.0' y2='131.0'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(868.0 342.0) rotate(-35)'>ElasticNet</text><rect class='sa-hit' x='832.0' y='18' width='64' height='414'><title>ElasticNet (Regularization, non-search)
|
| 59 |
Elo 1207 [95% CI 1154β1262]
|
| 60 |
Stability 0.419
|
| 61 |
+
Validity 0.665</title></rect><rect x='905.6' y='105.3' width='22.4' height='222.7' fill='#98c6e0' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='928.0' y='105.3' width='22.4' height='222.7' fill='#eaad7e' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='928.0' x2='928.0' y1='85.1' y2='127.3'/><line x1='923.0' x2='933.0' y1='85.1' y2='85.1'/><line x1='923.0' x2='933.0' y1='127.3' y2='127.3'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(932.0 342.0) rotate(-35)'>CART</text><rect class='sa-hit' x='896.0' y='18' width='64' height='414'><title>CART (Tree, non-search)
|
| 62 |
Elo 1210 [95% CI 1162β1254]
|
| 63 |
Stability 0.402
|
| 64 |
+
Validity 0.507</title></rect><rect x='969.6' y='76.3' width='22.4' height='251.7' fill='#56a2cc' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='992.0' y='76.3' width='22.4' height='251.7' fill='#e69e66' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='992.0' x2='992.0' y1='48.3' y2='100.2'/><line x1='987.0' x2='997.0' y1='48.3' y2='48.3'/><line x1='987.0' x2='997.0' y1='100.2' y2='100.2'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(996.0 342.0) rotate(-35)'>RFImportance</text><rect class='sa-hit' x='960.0' y='18' width='64' height='414'><title>RFImportance (Tree, non-search)
|
| 65 |
Elo 1273 [95% CI 1221β1334]
|
| 66 |
Stability 0.661
|
| 67 |
+
Validity 0.600</title></rect><line class='sa-ink' x1='64' x2='1024' y1='328.0' y2='328.0' stroke-width='1' opacity='0.4'/><rect x='1070' y='40.0' width='12' height='107.0' fill='url(#sa-g-stab)' stroke='#000' stroke-width='0.6'/><text class='sa-text' x='1076' y='30.0' text-anchor='middle' font-size='12'>stable</text><text class='sa-text' x='1076' y='165.0' text-anchor='middle' font-size='12'>unstable</text><text class='sa-muted' x='1088' y='151.0' font-size='11'>0.0</text><text class='sa-muted' x='1088' y='97.5' font-size='11'>0.5</text><text class='sa-muted' x='1088' y='44.0' font-size='11'>1.0</text><rect x='1070' y='207.0' width='12' height='107.0' fill='url(#sa-g-val)' stroke='#000' stroke-width='0.6'/><text class='sa-text' x='1076' y='197.0' text-anchor='middle' font-size='12'>valid</text><text class='sa-text' x='1076' y='332.0' text-anchor='middle' font-size='12'>invalid</text><text class='sa-muted' x='1088' y='318.0' font-size='11'>0.0</text><text class='sa-muted' x='1088' y='264.5' font-size='11'>0.5</text><text class='sa-muted' x='1088' y='211.0' font-size='11'>1.0</text><rect x='1070' y='352.0' width='12' height='12' fill='url(#sa-na)' stroke='#000' stroke-width='0.6'/><text class='sa-muted' x='1088' y='362.0' font-size='11'>no result</text></svg><script type='text/plain' id='sa-csv'>Method,Category,Subgroup,Elo,Elo CI low,Elo CI high,Stability,Validity
|
| 68 |
+
RFImportance,Non-search,Tree,1273.0,1221.0,1334.0,0.661,0.6
|
| 69 |
+
CART,Non-search,Tree,1210.0,1162.0,1254.0,0.402,0.507
|
| 70 |
+
Lasso,Non-search,Regularization,1207.0,1160.0,1259.0,0.334,0.643
|
| 71 |
+
ElasticNet,Non-search,Regularization,1207.0,1154.0,1262.0,0.419,0.665
|
| 72 |
+
MI,Non-search,Information theory,1197.0,1151.0,1244.0,0.649,0.723
|
| 73 |
+
F-Test,Non-search,Correlation,1183.0,1136.0,1233.0,0.668,0.781
|
| 74 |
+
mRMR,Non-search,Information theory,1153.0,1110.0,1195.0,0.656,0.593
|
| 75 |
+
(R)ReliefF,Non-search,Distance,1145.0,1095.0,1195.0,0.656,0.667
|
| 76 |
+
MarkovBlanket,Search,Backward search,1128.0,1083.0,1170.0,0.434,0.473
|
| 77 |
+
SBE,Search,Backward search,1122.0,1067.0,1180.0,0.095,0.53
|
| 78 |
+
LOCO,Search,Leave-one-out,1118.0,1066.0,1169.0,0.114,0.544
|
| 79 |
+
GainRatio,Non-search,Information theory,1115.0,1069.0,1162.0,0.641,0.751
|
| 80 |
+
SFS,Search,Forward search,1098.0,1052.0,1146.0,0.225,0.568
|
| 81 |
+
Random,Baseline,Baseline,1000.0,944.0,1053.0,0.002,0.494
|
| 82 |
+
LaplacianScore,Non-search,Distance,878.0,793.0,946.0,0.689,0.758
|
| 83 |
</script><script>
|
| 84 |
function saDownload(name, blob) {
|
| 85 |
var a = document.createElement('a');
|
data/subsets/tasks_all/models_lightgbm/budgets_3/leaderboard_table.html
CHANGED
|
@@ -26,23 +26,25 @@ td.method { font-weight:600; }
|
|
| 26 |
td .sub { color:var(--muted); font-size:12px; font-weight:400; margin-left:6px; }
|
| 27 |
td.na { color:var(--muted); }
|
| 28 |
.pill { display:inline-block; padding:0 8px; border:1px solid var(--line); border-radius:999px;
|
| 29 |
-
font-size:12px; color:var(--
|
| 30 |
-
</style></head><body><div class='toolbar'><span class='hint'>Click a column to sort. Hover a header for its definition.</span><button onclick="saCsv('selectarena_leaderboard.csv')">CSV</button></div><div class='wrap'><table id='sa-table'><thead><tr><th class='num'>#</th><th class=''>Method</th><th class='' title='Filter: scores features from the data alone. Wrapper: searches feature subsets by refitting a proxy model. Embedded: reads the selection off a fitted model.'>Type</th><th class='num' title='Bradley-Terry Elo from pairwise matches within each dataset, budget and downstream model. Random = 1000. +/β give the 95% bootstrap interval.'>Elo β</th><th class='num' title='Mean rank of the downstream error.'>Avg. rank β</th><th class='num' title='Share of pairwise comparisons won, in %. Ties count half.'>Win rate β</th><th class='num' title='Mean relative gap (error β best error) / error to the best method, in %.'>Improvability β</th><th class='num' title='Mean ratio of the error to the same model trained on all features. Below 1: the selection helped.'>Error vs. all β</th><th class='num' title='Agreement of the selected features across bootstrap resamples (Nogueira et al., 2018).'>Stability β</th><th class='num' title='Share of the budget filled with original features when as many noise features are added.'>Validity β</th><th class='num' title='Median wall-clock time of the selection.'>Time (s) β</th><th class='num' title='Share of runs where the method ran out of time and random features filled the budget.'>Fallback (%)</th><th class='num'>#Datasets</th></tr></thead><tbody><tr><td class='num' data-sort='1'>1</td><td class='method' data-sort='RFImportance'>RFImportance</td><td class='' data-sort='Embedded'><span class='pill'>π³ Embedded</span></td><td class='num' data-sort='1273.0'>1273<span class='sub'>+61 / β52</span></td><td class='num' data-sort='5.82'>5.82</td><td class='num' data-sort='65.6'>65.6</td><td class='num' data-sort='11.0'>11.0</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.661'>0.661</td><td class='num' data-sort='0.6'>0.600</td><td class='num' data-sort='0.2'>0.2</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='2'>2</td><td class='method' data-sort='CART'>CART</td><td class='' data-sort='Embedded'><span class='pill'>π³ Embedded</span></td><td class='num' data-sort='1210.0'>1210<span class='sub'>+44 / β48</span></td><td class='num' data-sort='6.66'>6.66</td><td class='num' data-sort='59.6'>59.6</td><td class='num' data-sort='14.0'>14.0</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.402'>0.402</td><td class='num' data-sort='0.507'>0.507</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='3'>3</td><td class='method' data-sort='Lasso'>Lasso</td><td class='' data-sort='Embedded'><span class='pill'>π³ Embedded</span></td><td class='num' data-sort='1207.0'>1207<span class='sub'>+52 / β47</span></td><td class='num' data-sort='6.9'>6.90</td><td class='num' data-sort='57.9'>57.9</td><td class='num' data-sort='13.4'>13.4</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.334'>0.334</td><td class='num' data-sort='0.643'>0.643</td><td class='num' data-sort='1.9'>1.9</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='4'>4</td><td class='method' data-sort='ElasticNet'>ElasticNet</td><td class='' data-sort='Embedded'><span class='pill'>π³ Embedded</span></td><td class='num' data-sort='1207.0'>1207<span class='sub'>+55 / β53</span></td><td class='num' data-sort='6.79'>6.79</td><td class='num' data-sort='58.6'>58.6</td><td class='num' data-sort='14.1'>14.1</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.419'>0.419</td><td class='num' data-sort='0.665'>0.665</td><td class='num' data-sort='6.1'>6.1</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='5'>5</td><td class='method' data-sort='MI'>MI</td><td class='' data-sort='Filter'><span class='pill'>π Filter</span></td><td class='num' data-sort='1197.0'>1197<span class='sub'>+47 / β46</span></td><td class='num' data-sort='7.19'>7.19</td><td class='num' data-sort='55.8'>55.8</td><td class='num' data-sort='14.4'>14.4</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.649'>0.649</td><td class='num' data-sort='0.723'>0.723</td><td class='num' data-sort='0.4'>0.4</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='6'>6</td><td class='method' data-sort='F-Test'>F-Test</td><td class='' data-sort='Filter'><span class='pill'>π Filter</span></td><td class='num' data-sort='1183.0'>1183<span class='sub'>+50 / β47</span></td><td class='num' data-sort='7.42'>7.42</td><td class='num' data-sort='54.1'>54.1</td><td class='num' data-sort='13.5'>13.5</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.668'>0.668</td><td class='num' data-sort='0.781'>0.781</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='7'>7</td><td class='method' data-sort='mRMR'>mRMR</td><td class='' data-sort='Filter'><span class='pill'>π Filter</span></td><td class='num' data-sort='1153.0'>1153<span class='sub'>+42 / β43</span></td><td class='num' data-sort='8.12'>8.12</td><td class='num' data-sort='49.1'>49.1</td><td class='num' data-sort='15.4'>15.4</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.656'>0.656</td><td class='num' data-sort='0.593'>0.593</td><td class='num' data-sort='0.5'>0.5</td><td class='num' data-sort='7.0'>7.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='8'>8</td><td class='method' data-sort='(R)ReliefF'>(R)ReliefF</td><td class='' data-sort='Filter'><span class='pill'>π Filter</span></td><td class='num' data-sort='1145.0'>1145<span class='sub'>+50 / β50</span></td><td class='num' data-sort='7.9'>7.90</td><td class='num' data-sort='50.7'>50.7</td><td class='num' data-sort='14.4'>14.4</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.656'>0.656</td><td class='num' data-sort='0.667'>0.667</td><td class='num' data-sort='13.1'>13.1</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='9'>9</td><td class='method' data-sort='MarkovBlanket'>MarkovBlanket</td><td class='' data-sort='Filter'><span class='pill'>π Filter</span></td><td class='num' data-sort='1128.0'>1128<span class='sub'>+42 / β45</span></td><td class='num' data-sort='8.14'>8.14</td><td class='num' data-sort='49.0'>49.0</td><td class='num' data-sort='16.5'>16.5</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.434'>0.434</td><td class='num' data-sort='0.473'>0.473</td><td class='num' data-sort='0.1'>0.1</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='10'>10</td><td class='method' data-sort='SBE'>SBE</td><td class='' data-sort='Wrapper'><span class='pill'>π Wrapper</span></td><td class='num' data-sort='1122.0'>1122<span class='sub'>+58 / β55</span></td><td class='num' data-sort='7.98'>7.98</td><td class='num' data-sort='50.1'>50.1</td><td class='num' data-sort='18.4'>18.4</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.095'>0.095</td><td class='num' data-sort='0.53'>0.530</td><td class='num' data-sort='1923.4'>1,923.4</td><td class='num' data-sort='44.6'>44.6</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='11'>11</td><td class='method' data-sort='LOCO'>LOCO</td><td class='' data-sort='Wrapper'><span class='pill'>π Wrapper</span></td><td class='num' data-sort='1118.0'>1118<span class='sub'>+51 / β52</span></td><td class='num' data-sort='8.21'>8.21</td><td class='num' data-sort='48.5'>48.5</td><td class='num' data-sort='19.1'>19.1</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.114'>0.114</td><td class='num' data-sort='0.544'>0.544</td><td class='num' data-sort='236.6'>236.6</td><td class='num' data-sort='11.3'>11.3</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='12'>12</td><td class='method' data-sort='GainRatio'>GainRatio</td><td class='' data-sort='Filter'><span class='pill'>π Filter</span></td><td class='num' data-sort='1115.0'>1115<span class='sub'>+47 / β46</span></td><td class='num' data-sort='8.49'>8.49</td><td class='num' data-sort='46.5'>46.5</td><td class='num' data-sort='16.2'>16.2</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.641'>0.641</td><td class='num' data-sort='0.751'>0.751</td><td class='num' data-sort='0.2'>0.2</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='13'>13</td><td class='method' data-sort='SFS'>SFS</td><td class='' data-sort='Wrapper'><span class='pill'>π Wrapper</span></td><td class='num' data-sort='1098.0'>1098<span class='sub'>+48 / β46</span></td><td class='num' data-sort='8.54'>8.54</td><td class='num' data-sort='46.1'>46.1</td><td class='num' data-sort='19.2'>19.2</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.225'>0.225</td><td class='num' data-sort='0.568'>0.568</td><td class='num' data-sort='2771.3'>2,771.3</td><td class='num' data-sort='48.4'>48.4</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='14'>14</td><td class='method' data-sort='Random'>Random</td><td class='' data-sort='Baseline'><span class='pill'>βͺ Baseline</span></td><td class='num' data-sort='1000.0'>1000<span class='sub'>+53 / β56</span></td><td class='num' data-sort='9.97'>9.97</td><td class='num' data-sort='35.9'>35.9</td><td class='num' data-sort='24.9'>24.9</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.002'>0.002</td><td class='num' data-sort='0.494'>0.494</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='100.0'>100.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='15'>15</td><td class='method' data-sort='LaplacianScore'>LaplacianScore</td><td class='' data-sort='Filter'><span class='pill'>π Filter</span></td><td class='num' data-sort='878.0'>878<span class='sub'>+68 / β85</span></td><td class='num' data-sort='11.86'>11.86</td><td class='num' data-sort='22.4'>22.4</td><td class='num' data-sort='31.8'>31.8</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.689'>0.689</td><td class='num' data-sort='0.758'>0.758</td><td class='num' data-sort='0.2'>0.2</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr></tbody></table></div><script type='text/plain' id='sa-csv'>Rank,Type,TypeName,Method,Elo,Elo CI low,Elo CI high,Avg. rank,Win rate (%),Improvability (%),Error vs. all features,Stability,Validity,Selection time (s),Fallback rate (%),#Datasets,#Comparisons
|
| 31 |
-
|
| 32 |
-
2,π³,Embedded,CART,1210.0,1162.0,1254.0,6.66,59.6,14.0,,0.402,0.507,0.0,0.0,71,213
|
| 33 |
-
|
| 34 |
-
|
| 35 |
-
|
| 36 |
-
|
| 37 |
-
|
| 38 |
-
|
| 39 |
-
|
| 40 |
-
|
| 41 |
-
|
| 42 |
-
|
| 43 |
-
|
| 44 |
-
|
| 45 |
-
|
|
|
|
|
|
|
| 46 |
</script><script>
|
| 47 |
function saDownload(name, blob) {
|
| 48 |
var a = document.createElement('a');
|
|
|
|
| 26 |
td .sub { color:var(--muted); font-size:12px; font-weight:400; margin-left:6px; }
|
| 27 |
td.na { color:var(--muted); }
|
| 28 |
.pill { display:inline-block; padding:0 8px; border:1px solid var(--line); border-radius:999px;
|
| 29 |
+
font-size:12px; color:var(--fg); }
|
| 30 |
+
.pill.cat-search { border-color:var(--accent); }
|
| 31 |
+
.pill.cat-baseline { color:var(--muted); }
|
| 32 |
+
</style></head><body><div class='toolbar'><span class='hint'>Click a column to sort. Hover a header for its definition.</span><button onclick="saCsv('selectarena_leaderboard.csv')">CSV</button></div><div class='wrap'><table id='sa-table'><thead><tr><th class='num'>#</th><th class=''>Method</th><th class='' title='Taxonomy of the paper (Appendix A). Search methods evaluate many feature subsets with the model (forward, backward, leave-one-out); non-search methods use a proxy instead (correlation, distance, information theory, tree, regularization).'>Type</th><th class='num' title='Bradley-Terry Elo from pairwise matches within each dataset, budget and downstream model. Random = 1000. +/β give the 95% bootstrap interval.'>Elo β</th><th class='num' title='Mean rank of the downstream error.'>Avg. rank β</th><th class='num' title='Share of pairwise comparisons won, in %. Ties count half.'>Win rate β</th><th class='num' title='Mean relative gap (error β best error) / error to the best method, in %.'>Improvability β</th><th class='num' title='Mean ratio of the error to the same model trained on all features. Below 1: the selection helped.'>Error vs. all β</th><th class='num' title='Agreement of the selected features across bootstrap resamples (Nogueira et al., 2018).'>Stability β</th><th class='num' title='Share of the budget filled with original features when as many noise features are added.'>Validity β</th><th class='num' title='Median wall-clock time of the selection.'>Time (s) β</th><th class='num' title='Share of runs where the method ran out of time and random features filled the budget.'>Fallback (%)</th><th class='num'>#Datasets</th></tr></thead><tbody><tr><td class='num' data-sort='1'>1</td><td class='method' data-sort='RFImportance'>RFImportance</td><td class='' data-sort='Tree'><span class='pill cat-non-search'>Tree</span><span class='sub'>non-search</span></td><td class='num' data-sort='1273.0'>1273<span class='sub'>+61 / β52</span></td><td class='num' data-sort='5.82'>5.82</td><td class='num' data-sort='65.6'>65.6</td><td class='num' data-sort='11.0'>11.0</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.661'>0.661</td><td class='num' data-sort='0.6'>0.600</td><td class='num' data-sort='0.2'>0.2</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='2'>2</td><td class='method' data-sort='CART'>CART</td><td class='' data-sort='Tree'><span class='pill cat-non-search'>Tree</span><span class='sub'>non-search</span></td><td class='num' data-sort='1210.0'>1210<span class='sub'>+44 / β48</span></td><td class='num' data-sort='6.66'>6.66</td><td class='num' data-sort='59.6'>59.6</td><td class='num' data-sort='14.0'>14.0</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.402'>0.402</td><td class='num' data-sort='0.507'>0.507</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='3'>3</td><td class='method' data-sort='Lasso'>Lasso</td><td class='' data-sort='Regularization'><span class='pill cat-non-search'>Regularization</span><span class='sub'>non-search</span></td><td class='num' data-sort='1207.0'>1207<span class='sub'>+52 / β47</span></td><td class='num' data-sort='6.9'>6.90</td><td class='num' data-sort='57.9'>57.9</td><td class='num' data-sort='13.4'>13.4</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.334'>0.334</td><td class='num' data-sort='0.643'>0.643</td><td class='num' data-sort='1.9'>1.9</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='4'>4</td><td class='method' data-sort='ElasticNet'>ElasticNet</td><td class='' data-sort='Regularization'><span class='pill cat-non-search'>Regularization</span><span class='sub'>non-search</span></td><td class='num' data-sort='1207.0'>1207<span class='sub'>+55 / β53</span></td><td class='num' data-sort='6.79'>6.79</td><td class='num' data-sort='58.6'>58.6</td><td class='num' data-sort='14.1'>14.1</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.419'>0.419</td><td class='num' data-sort='0.665'>0.665</td><td class='num' data-sort='6.1'>6.1</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='5'>5</td><td class='method' data-sort='MI'>MI</td><td class='' data-sort='Information theory'><span class='pill cat-non-search'>Information theory</span><span class='sub'>non-search</span></td><td class='num' data-sort='1197.0'>1197<span class='sub'>+47 / β46</span></td><td class='num' data-sort='7.19'>7.19</td><td class='num' data-sort='55.8'>55.8</td><td class='num' data-sort='14.4'>14.4</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.649'>0.649</td><td class='num' data-sort='0.723'>0.723</td><td class='num' data-sort='0.4'>0.4</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='6'>6</td><td class='method' data-sort='F-Test'>F-Test</td><td class='' data-sort='Correlation'><span class='pill cat-non-search'>Correlation</span><span class='sub'>non-search</span></td><td class='num' data-sort='1183.0'>1183<span class='sub'>+50 / β47</span></td><td class='num' data-sort='7.42'>7.42</td><td class='num' data-sort='54.1'>54.1</td><td class='num' data-sort='13.5'>13.5</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.668'>0.668</td><td class='num' data-sort='0.781'>0.781</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='7'>7</td><td class='method' data-sort='mRMR'>mRMR</td><td class='' data-sort='Information theory'><span class='pill cat-non-search'>Information theory</span><span class='sub'>non-search</span></td><td class='num' data-sort='1153.0'>1153<span class='sub'>+42 / β43</span></td><td class='num' data-sort='8.12'>8.12</td><td class='num' data-sort='49.1'>49.1</td><td class='num' data-sort='15.4'>15.4</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.656'>0.656</td><td class='num' data-sort='0.593'>0.593</td><td class='num' data-sort='0.5'>0.5</td><td class='num' data-sort='7.0'>7.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='8'>8</td><td class='method' data-sort='(R)ReliefF'>(R)ReliefF</td><td class='' data-sort='Distance'><span class='pill cat-non-search'>Distance</span><span class='sub'>non-search</span></td><td class='num' data-sort='1145.0'>1145<span class='sub'>+50 / β50</span></td><td class='num' data-sort='7.9'>7.90</td><td class='num' data-sort='50.7'>50.7</td><td class='num' data-sort='14.4'>14.4</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.656'>0.656</td><td class='num' data-sort='0.667'>0.667</td><td class='num' data-sort='13.1'>13.1</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='9'>9</td><td class='method' data-sort='MarkovBlanket'>MarkovBlanket</td><td class='' data-sort='Backward search'><span class='pill cat-search'>Backward search</span><span class='sub'>search</span></td><td class='num' data-sort='1128.0'>1128<span class='sub'>+42 / β45</span></td><td class='num' data-sort='8.14'>8.14</td><td class='num' data-sort='49.0'>49.0</td><td class='num' data-sort='16.5'>16.5</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.434'>0.434</td><td class='num' data-sort='0.473'>0.473</td><td class='num' data-sort='0.1'>0.1</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='10'>10</td><td class='method' data-sort='SBE'>SBE</td><td class='' data-sort='Backward search'><span class='pill cat-search'>Backward search</span><span class='sub'>search</span></td><td class='num' data-sort='1122.0'>1122<span class='sub'>+58 / β55</span></td><td class='num' data-sort='7.98'>7.98</td><td class='num' data-sort='50.1'>50.1</td><td class='num' data-sort='18.4'>18.4</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.095'>0.095</td><td class='num' data-sort='0.53'>0.530</td><td class='num' data-sort='1923.4'>1,923.4</td><td class='num' data-sort='44.6'>44.6</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='11'>11</td><td class='method' data-sort='LOCO'>LOCO</td><td class='' data-sort='Leave-one-out'><span class='pill cat-search'>Leave-one-out</span><span class='sub'>search</span></td><td class='num' data-sort='1118.0'>1118<span class='sub'>+51 / β52</span></td><td class='num' data-sort='8.21'>8.21</td><td class='num' data-sort='48.5'>48.5</td><td class='num' data-sort='19.1'>19.1</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.114'>0.114</td><td class='num' data-sort='0.544'>0.544</td><td class='num' data-sort='236.6'>236.6</td><td class='num' data-sort='11.3'>11.3</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='12'>12</td><td class='method' data-sort='GainRatio'>GainRatio</td><td class='' data-sort='Information theory'><span class='pill cat-non-search'>Information theory</span><span class='sub'>non-search</span></td><td class='num' data-sort='1115.0'>1115<span class='sub'>+47 / β46</span></td><td class='num' data-sort='8.49'>8.49</td><td class='num' data-sort='46.5'>46.5</td><td class='num' data-sort='16.2'>16.2</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.641'>0.641</td><td class='num' data-sort='0.751'>0.751</td><td class='num' data-sort='0.2'>0.2</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='13'>13</td><td class='method' data-sort='SFS'>SFS</td><td class='' data-sort='Forward search'><span class='pill cat-search'>Forward search</span><span class='sub'>search</span></td><td class='num' data-sort='1098.0'>1098<span class='sub'>+48 / β46</span></td><td class='num' data-sort='8.54'>8.54</td><td class='num' data-sort='46.1'>46.1</td><td class='num' data-sort='19.2'>19.2</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.225'>0.225</td><td class='num' data-sort='0.568'>0.568</td><td class='num' data-sort='2771.3'>2,771.3</td><td class='num' data-sort='48.4'>48.4</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='14'>14</td><td class='method' data-sort='Random'>Random</td><td class='' data-sort='Baseline'><span class='pill cat-baseline'>Baseline</span></td><td class='num' data-sort='1000.0'>1000<span class='sub'>+53 / β56</span></td><td class='num' data-sort='9.97'>9.97</td><td class='num' data-sort='35.9'>35.9</td><td class='num' data-sort='24.9'>24.9</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.002'>0.002</td><td class='num' data-sort='0.494'>0.494</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='100.0'>100.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='15'>15</td><td class='method' data-sort='LaplacianScore'>LaplacianScore</td><td class='' data-sort='Distance'><span class='pill cat-non-search'>Distance</span><span class='sub'>non-search</span></td><td class='num' data-sort='878.0'>878<span class='sub'>+68 / β85</span></td><td class='num' data-sort='11.86'>11.86</td><td class='num' data-sort='22.4'>22.4</td><td class='num' data-sort='31.8'>31.8</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.689'>0.689</td><td class='num' data-sort='0.758'>0.758</td><td class='num' data-sort='0.2'>0.2</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr></tbody></table></div><script type='text/plain' id='sa-csv'>Rank,Method,Category,Subgroup,Elo,Elo CI low,Elo CI high,Avg. rank,Win rate (%),Improvability (%),Error vs. all features,Stability,Validity,Selection time (s),Fallback rate (%),#Datasets,#Comparisons
|
| 33 |
+
1,RFImportance,Non-search,Tree,1273.0,1221.0,1334.0,5.82,65.6,11.0,,0.661,0.6,0.2,0.0,71,213
|
| 34 |
+
2,CART,Non-search,Tree,1210.0,1162.0,1254.0,6.66,59.6,14.0,,0.402,0.507,0.0,0.0,71,213
|
| 35 |
+
3,Lasso,Non-search,Regularization,1207.0,1160.0,1259.0,6.9,57.9,13.4,,0.334,0.643,1.9,0.0,71,213
|
| 36 |
+
4,ElasticNet,Non-search,Regularization,1207.0,1154.0,1262.0,6.79,58.6,14.1,,0.419,0.665,6.1,0.0,71,213
|
| 37 |
+
5,MI,Non-search,Information theory,1197.0,1151.0,1244.0,7.19,55.8,14.4,,0.649,0.723,0.4,0.0,71,213
|
| 38 |
+
6,F-Test,Non-search,Correlation,1183.0,1136.0,1233.0,7.42,54.1,13.5,,0.668,0.781,0.0,0.0,71,213
|
| 39 |
+
7,mRMR,Non-search,Information theory,1153.0,1110.0,1195.0,8.12,49.1,15.4,,0.656,0.593,0.5,7.0,71,213
|
| 40 |
+
8,(R)ReliefF,Non-search,Distance,1145.0,1095.0,1195.0,7.9,50.7,14.4,,0.656,0.667,13.1,0.0,71,213
|
| 41 |
+
9,MarkovBlanket,Search,Backward search,1128.0,1083.0,1170.0,8.14,49.0,16.5,,0.434,0.473,0.1,0.0,71,213
|
| 42 |
+
10,SBE,Search,Backward search,1122.0,1067.0,1180.0,7.98,50.1,18.4,,0.095,0.53,1923.4,44.6,71,213
|
| 43 |
+
11,LOCO,Search,Leave-one-out,1118.0,1066.0,1169.0,8.21,48.5,19.1,,0.114,0.544,236.6,11.3,71,213
|
| 44 |
+
12,GainRatio,Non-search,Information theory,1115.0,1069.0,1162.0,8.49,46.5,16.2,,0.641,0.751,0.2,0.0,71,213
|
| 45 |
+
13,SFS,Search,Forward search,1098.0,1052.0,1146.0,8.54,46.1,19.2,,0.225,0.568,2771.3,48.4,71,213
|
| 46 |
+
14,Random,Baseline,Baseline,1000.0,944.0,1053.0,9.97,35.9,24.9,,0.002,0.494,0.0,100.0,71,213
|
| 47 |
+
15,LaplacianScore,Non-search,Distance,878.0,793.0,946.0,11.86,22.4,31.8,,0.689,0.758,0.2,0.0,71,213
|
| 48 |
</script><script>
|
| 49 |
function saDownload(name, blob) {
|
| 50 |
var a = document.createElement('a');
|
data/subsets/tasks_all/models_lightgbm/budgets_3/website_leaderboard.csv
CHANGED
|
@@ -1,16 +1,16 @@
|
|
| 1 |
-
Rank,
|
| 2 |
-
1,
|
| 3 |
-
2,
|
| 4 |
-
3,
|
| 5 |
-
4,
|
| 6 |
-
5,
|
| 7 |
-
6,
|
| 8 |
-
7,
|
| 9 |
-
8,
|
| 10 |
-
9,
|
| 11 |
-
10,
|
| 12 |
-
11,
|
| 13 |
-
12,
|
| 14 |
-
13,
|
| 15 |
-
14,
|
| 16 |
-
15,
|
|
|
|
| 1 |
+
Rank,Method,Category,Subgroup,Elo,Elo CI low,Elo CI high,Avg. rank,Win rate (%),Improvability (%),Error vs. all features,Stability,Validity,Selection time (s),Fallback rate (%),#Datasets,#Comparisons
|
| 2 |
+
1,RFImportance,Non-search,Tree,1273.0,1221.0,1334.0,5.82,65.6,11.0,,0.661,0.6,0.2,0.0,71,213
|
| 3 |
+
2,CART,Non-search,Tree,1210.0,1162.0,1254.0,6.66,59.6,14.0,,0.402,0.507,0.0,0.0,71,213
|
| 4 |
+
3,Lasso,Non-search,Regularization,1207.0,1160.0,1259.0,6.9,57.9,13.4,,0.334,0.643,1.9,0.0,71,213
|
| 5 |
+
4,ElasticNet,Non-search,Regularization,1207.0,1154.0,1262.0,6.79,58.6,14.1,,0.419,0.665,6.1,0.0,71,213
|
| 6 |
+
5,MI,Non-search,Information theory,1197.0,1151.0,1244.0,7.19,55.8,14.4,,0.649,0.723,0.4,0.0,71,213
|
| 7 |
+
6,F-Test,Non-search,Correlation,1183.0,1136.0,1233.0,7.42,54.1,13.5,,0.668,0.781,0.0,0.0,71,213
|
| 8 |
+
7,mRMR,Non-search,Information theory,1153.0,1110.0,1195.0,8.12,49.1,15.4,,0.656,0.593,0.5,7.0,71,213
|
| 9 |
+
8,(R)ReliefF,Non-search,Distance,1145.0,1095.0,1195.0,7.9,50.7,14.4,,0.656,0.667,13.1,0.0,71,213
|
| 10 |
+
9,MarkovBlanket,Search,Backward search,1128.0,1083.0,1170.0,8.14,49.0,16.5,,0.434,0.473,0.1,0.0,71,213
|
| 11 |
+
10,SBE,Search,Backward search,1122.0,1067.0,1180.0,7.98,50.1,18.4,,0.095,0.53,1923.4,44.6,71,213
|
| 12 |
+
11,LOCO,Search,Leave-one-out,1118.0,1066.0,1169.0,8.21,48.5,19.1,,0.114,0.544,236.6,11.3,71,213
|
| 13 |
+
12,GainRatio,Non-search,Information theory,1115.0,1069.0,1162.0,8.49,46.5,16.2,,0.641,0.751,0.2,0.0,71,213
|
| 14 |
+
13,SFS,Search,Forward search,1098.0,1052.0,1146.0,8.54,46.1,19.2,,0.225,0.568,2771.3,48.4,71,213
|
| 15 |
+
14,Random,Baseline,Baseline,1000.0,944.0,1053.0,9.97,35.9,24.9,,0.002,0.494,0.0,100.0,71,213
|
| 16 |
+
15,LaplacianScore,Non-search,Distance,878.0,793.0,946.0,11.86,22.4,31.8,,0.689,0.758,0.2,0.0,71,213
|
data/subsets/tasks_all/models_lightgbm/budgets_4/elo_explorer.html
CHANGED
|
@@ -19,67 +19,67 @@ svg { display:block; width:100%; height:auto; }
|
|
| 19 |
.sa-ink { stroke:var(--fg); }
|
| 20 |
.sa-hit { fill:transparent; }
|
| 21 |
.sa-hit:hover { fill:var(--fg); fill-opacity:0.06; }
|
| 22 |
-
</style><defs><pattern id='sa-na' width='6' height='6' patternUnits='userSpaceOnUse' patternTransform='rotate(45)'><rect width='6' height='6' fill='#e5e7eb'/><line x1='0' y1='0' x2='0' y2='6' stroke='#9ca3af' stroke-width='2'/></pattern><linearGradient id='sa-g-stab' x1='0' y1='1' x2='0' y2='0'><stop offset='0' stop-color='#ffffff'/><stop offset='1' stop-color='#0072B2'/></linearGradient><linearGradient id='sa-g-val' x1='0' y1='1' x2='0' y2='0'><stop offset='0' stop-color='#ffffff'/><stop offset='1' stop-color='#D55E00'/></linearGradient></defs><line class='sa-grid' x1='64' x2='1024' y1='313.2' y2='313.2' stroke-width='1'/><text class='sa-muted' x='56' y='317.2' text-anchor='end' font-size='12'>900</text><line class='sa-grid' x1='64' x2='1024' y1='254.2' y2='254.2' stroke-width='1'/><text class='sa-muted' x='56' y='258.2' text-anchor='end' font-size='12'>1000</text><line class='sa-grid' x1='64' x2='1024' y1='195.1' y2='195.1' stroke-width='1'/><text class='sa-muted' x='56' y='199.1' text-anchor='end' font-size='12'>1100</text><line class='sa-grid' x1='64' x2='1024' y1='136.1' y2='136.1' stroke-width='1'/><text class='sa-muted' x='56' y='140.1' text-anchor='end' font-size='12'>1200</text><line class='sa-grid' x1='64' x2='1024' y1='77.0' y2='77.0' stroke-width='1'/><text class='sa-muted' x='56' y='81.0' text-anchor='end' font-size='12'>1300</text><line class='sa-grid' x1='64' x2='1024' y1='18.0' y2='18.0' stroke-width='1'/><text class='sa-muted' x='56' y='22.0' text-anchor='end' font-size='12'>1400</text><text class='sa-text' transform='translate(16 173) rotate(-90)' text-anchor='middle' font-size='13'>Elo</text><line class='sa-ink' x1='64' x2='1024' y1='254.2' y2='254.2' stroke-width='1' stroke-dasharray='4 4' opacity='0.5'/><rect x='73.6' y='273.1' width='22.4' height='54.9' fill='#529fcb' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='96.0' y='273.1' width='22.4' height='54.9' fill='#e28e4c' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='96.0' x2='96.0' y1='235.9' y2='312.6'/><line x1='91.0' x2='101.0' y1='235.9' y2='235.9'/><line x1='91.0' x2='101.0' y1='312.6' y2='312.6'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(100.0 342.0) rotate(-35)'>LaplacianScore</text><rect class='sa-hit' x='64.0' y='18' width='64' height='414'><title>LaplacianScore (
|
| 23 |
Elo 968 [95% CI 901β1031]
|
| 24 |
Stability 0.678
|
| 25 |
-
Validity 0.701</title></rect><rect x='137.6' y='254.2' width='22.4' height='73.8' fill='#feffff' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='160.0' y='254.2' width='22.4' height='73.8' fill='#eaae80' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='160.0' x2='160.0' y1='222.9' y2='289.6'/><line x1='155.0' x2='165.0' y1='222.9' y2='222.9'/><line x1='155.0' x2='165.0' y1='289.6' y2='289.6'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(164.0 342.0) rotate(-35)'>Random</text><rect class='sa-hit' x='128.0' y='18' width='64' height='414'><title>Random (Baseline)
|
| 26 |
Elo 1000 [95% CI 940β1053]
|
| 27 |
Stability 0.002
|
| 28 |
-
Validity 0.500</title></rect><rect x='201.6' y='200.5' width='22.4' height='127.5' fill='#73b1d5' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='224.0' y='200.5' width='22.4' height='127.5' fill='#e39557' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='224.0' x2='224.0' y1='175.1' y2='229.4'/><line x1='219.0' x2='229.0' y1='175.1' y2='175.1'/><line x1='219.0' x2='229.0' y1='229.4' y2='229.4'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(228.0 342.0) rotate(-35)'>GainRatio</text><rect class='sa-hit' x='192.0' y='18' width='64' height='414'><title>GainRatio (
|
| 29 |
Elo 1091 [95% CI 1042β1134]
|
| 30 |
Stability 0.550
|
| 31 |
-
Validity 0.659</title></rect><rect x='265.6' y='199.9' width='22.4' height='128.1' fill='#ebf4f9' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='288.0' y='199.9' width='22.4' height='128.1' fill='#e9ab7a' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='288.0' x2='288.0' y1='165.6' y2='231.2'/><line x1='283.0' x2='293.0' y1='165.6' y2='165.6'/><line x1='283.0' x2='293.0' y1='231.2' y2='231.2'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(292.0 342.0) rotate(-35)'>LOCO</text><rect class='sa-hit' x='256.0' y='18' width='64' height='414'><title>LOCO (
|
| 32 |
Elo 1092 [95% CI 1039β1150]
|
| 33 |
Stability 0.080
|
| 34 |
-
Validity 0.520</title></rect><rect x='329.6' y='196.9' width='22.4' height='131.1' fill='#d2e6f1' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='352.0' y='196.9' width='22.4' height='131.1' fill='#e9aa79' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='352.0' x2='352.0' y1='162.1' y2='227.6'/><line x1='347.0' x2='357.0' y1='162.1' y2='162.1'/><line x1='347.0' x2='357.0' y1='227.6' y2='227.6'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(356.0 342.0) rotate(-35)'>SFS</text><rect class='sa-hit' x='320.0' y='18' width='64' height='414'><title>SFS (
|
| 35 |
Elo 1097 [95% CI 1045β1156]
|
| 36 |
Stability 0.176
|
| 37 |
-
Validity 0.525</title></rect><rect x='393.6' y='188.1' width='22.4' height='139.9' fill='#f0f7fa' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='416.0' y='188.1' width='22.4' height='139.9' fill='#e9ac7b' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='416.0' x2='416.0' y1='156.8' y2='218.2'/><line x1='411.0' x2='421.0' y1='156.8' y2='156.8'/><line x1='411.0' x2='421.0' y1='218.2' y2='218.2'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(420.0 342.0) rotate(-35)'>SBE</text><rect class='sa-hit' x='384.0' y='18' width='64' height='414'><title>SBE (
|
| 38 |
Elo 1112 [95% CI 1061β1165]
|
| 39 |
Stability 0.059
|
| 40 |
-
Validity 0.517</title></rect><rect x='457.6' y='185.7' width='22.4' height='142.3' fill='#98c6e0' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='480.0' y='185.7' width='22.4' height='142.3' fill='#ecb489' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='480.0' x2='480.0' y1='158.5' y2='215.2'/><line x1='475.0' x2='485.0' y1='158.5' y2='158.5'/><line x1='475.0' x2='485.0' y1='215.2' y2='215.2'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(484.0 342.0) rotate(-35)'>MarkovBlanket</text><rect class='sa-hit' x='448.0' y='18' width='64' height='414'><title>MarkovBlanket (
|
| 41 |
Elo 1116 [95% CI 1066β1162]
|
| 42 |
Stability 0.404
|
| 43 |
-
Validity 0.463</title></rect><rect x='521.6' y='180.4' width='22.4' height='147.6' fill='#59a3cd' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='544.0' y='180.4' width='22.4' height='147.6' fill='#e9aa78' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='544.0' x2='544.0' y1='151.4' y2='205.8'/><line x1='539.0' x2='549.0' y1='151.4' y2='151.4'/><line x1='539.0' x2='549.0' y1='205.8' y2='205.8'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(548.0 342.0) rotate(-35)'>mRMR</text><rect class='sa-hit' x='512.0' y='18' width='64' height='414'><title>mRMR (
|
| 44 |
Elo 1125 [95% CI 1082β1174]
|
| 45 |
Stability 0.651
|
| 46 |
-
Validity 0.530</title></rect><rect x='585.6' y='166.8' width='22.4' height='161.2' fill='#63a9d0' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='608.0' y='166.8' width='22.4' height='161.2' fill='#e69f68' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='608.0' x2='608.0' y1='140.2' y2='192.2'/><line x1='603.0' x2='613.0' y1='140.2' y2='140.2'/><line x1='603.0' x2='613.0' y1='192.2' y2='192.2'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(612.0 342.0) rotate(-35)'>(R)ReliefF</text><rect class='sa-hit' x='576.0' y='18' width='64' height='414'><title>(R)ReliefF (
|
| 47 |
Elo 1148 [95% CI 1105β1193]
|
| 48 |
Stability 0.612
|
| 49 |
-
Validity 0.594</title></rect><rect x='649.6' y='163.3' width='22.4' height='164.7' fill='#70b0d4' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='672.0' y='163.3' width='22.4' height='164.7' fill='#e2904f' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='672.0' x2='672.0' y1='134.3' y2='192.2'/><line x1='667.0' x2='677.0' y1='134.3' y2='134.3'/><line x1='667.0' x2='677.0' y1='192.2' y2='192.2'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(676.0 342.0) rotate(-35)'>F-Test</text><rect class='sa-hit' x='640.0' y='18' width='64' height='414'><title>F-Test (
|
| 50 |
Elo 1154 [95% CI 1105β1203]
|
| 51 |
Stability 0.562
|
| 52 |
-
Validity 0.689</title></rect><rect x='713.6' y='159.7' width='22.4' height='168.3' fill='#69acd2' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='736.0' y='159.7' width='22.4' height='168.3' fill='#e59a5f' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='736.0' x2='736.0' y1='134.9' y2='182.7'/><line x1='731.0' x2='741.0' y1='134.9' y2='134.9'/><line x1='731.0' x2='741.0' y1='182.7' y2='182.7'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(740.0 342.0) rotate(-35)'>MI</text><rect class='sa-hit' x='704.0' y='18' width='64' height='414'><title>MI (
|
| 53 |
Elo 1160 [95% CI 1121β1202]
|
| 54 |
Stability 0.587
|
| 55 |
-
Validity 0.627</title></rect><rect x='777.6' y='154.4' width='22.4' height='173.6' fill='#c0dcec' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='800.0' y='154.4' width='22.4' height='173.6' fill='#e7a46f' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='800.0' x2='800.0' y1='126.6' y2='179.8'/><line x1='795.0' x2='805.0' y1='126.6' y2='126.6'/><line x1='795.0' x2='805.0' y1='179.8' y2='179.8'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(804.0 342.0) rotate(-35)'>Lasso</text><rect class='sa-hit' x='768.0' y='18' width='64' height='414'><title>Lasso (
|
| 56 |
Elo 1169 [95% CI 1126β1216]
|
| 57 |
Stability 0.246
|
| 58 |
-
Validity 0.563</title></rect><rect x='841.6' y='153.2' width='22.4' height='174.8' fill='#b0d4e7' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='864.0' y='153.2' width='22.4' height='174.8' fill='#e7a46e' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='864.0' x2='864.0' y1='124.9' y2='181.6'/><line x1='859.0' x2='869.0' y1='124.9' y2='124.9'/><line x1='859.0' x2='869.0' y1='181.6' y2='181.6'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(868.0 342.0) rotate(-35)'>ElasticNet</text><rect class='sa-hit' x='832.0' y='18' width='64' height='414'><title>ElasticNet (
|
| 59 |
Elo 1171 [95% CI 1123β1219]
|
| 60 |
Stability 0.308
|
| 61 |
-
Validity 0.568</title></rect><rect x='905.6' y='149.7' width='22.4' height='178.3' fill='#acd1e6' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='928.0' y='149.7' width='22.4' height='178.3' fill='#ebb184' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='928.0' x2='928.0' y1='120.2' y2='178.6'/><line x1='923.0' x2='933.0' y1='120.2' y2='120.2'/><line x1='923.0' x2='933.0' y1='178.6' y2='178.6'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(932.0 342.0) rotate(-35)'>CART</text><rect class='sa-hit' x='896.0' y='18' width='64' height='414'><title>CART (
|
| 62 |
Elo 1177 [95% CI 1128β1227]
|
| 63 |
Stability 0.324
|
| 64 |
-
Validity 0.482</title></rect><rect x='969.6' y='92.4' width='22.4' height='235.6' fill='#67abd1' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='992.0' y='92.4' width='22.4' height='235.6' fill='#eaad7e' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='992.0' x2='992.0' y1='59.3' y2='121.3'/><line x1='987.0' x2='997.0' y1='59.3' y2='59.3'/><line x1='987.0' x2='997.0' y1='121.3' y2='121.3'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(996.0 342.0) rotate(-35)'>RFImportance</text><rect class='sa-hit' x='960.0' y='18' width='64' height='414'><title>RFImportance (
|
| 65 |
Elo 1274 [95% CI 1225β1330]
|
| 66 |
Stability 0.597
|
| 67 |
-
Validity 0.507</title></rect><line class='sa-ink' x1='64' x2='1024' y1='328.0' y2='328.0' stroke-width='1' opacity='0.4'/><rect x='1070' y='40.0' width='12' height='107.0' fill='url(#sa-g-stab)' stroke='#000' stroke-width='0.6'/><text class='sa-text' x='1076' y='30.0' text-anchor='middle' font-size='12'>stable</text><text class='sa-text' x='1076' y='165.0' text-anchor='middle' font-size='12'>unstable</text><text class='sa-muted' x='1088' y='151.0' font-size='11'>0.0</text><text class='sa-muted' x='1088' y='97.5' font-size='11'>0.5</text><text class='sa-muted' x='1088' y='44.0' font-size='11'>1.0</text><rect x='1070' y='207.0' width='12' height='107.0' fill='url(#sa-g-val)' stroke='#000' stroke-width='0.6'/><text class='sa-text' x='1076' y='197.0' text-anchor='middle' font-size='12'>valid</text><text class='sa-text' x='1076' y='332.0' text-anchor='middle' font-size='12'>invalid</text><text class='sa-muted' x='1088' y='318.0' font-size='11'>0.0</text><text class='sa-muted' x='1088' y='264.5' font-size='11'>0.5</text><text class='sa-muted' x='1088' y='211.0' font-size='11'>1.0</text><rect x='1070' y='352.0' width='12' height='12' fill='url(#sa-na)' stroke='#000' stroke-width='0.6'/><text class='sa-muted' x='1088' y='362.0' font-size='11'>no result</text></svg><script type='text/plain' id='sa-csv'>Method,
|
| 68 |
-
RFImportance,
|
| 69 |
-
CART,
|
| 70 |
-
ElasticNet,
|
| 71 |
-
Lasso,
|
| 72 |
-
MI,
|
| 73 |
-
F-Test,
|
| 74 |
-
(R)ReliefF,
|
| 75 |
-
mRMR,
|
| 76 |
-
MarkovBlanket,
|
| 77 |
-
SBE,
|
| 78 |
-
SFS,
|
| 79 |
-
LOCO,
|
| 80 |
-
GainRatio,
|
| 81 |
-
Random,Baseline,1000.0,940.0,1053.0,0.002,0.5
|
| 82 |
-
LaplacianScore,
|
| 83 |
</script><script>
|
| 84 |
function saDownload(name, blob) {
|
| 85 |
var a = document.createElement('a');
|
|
|
|
| 19 |
.sa-ink { stroke:var(--fg); }
|
| 20 |
.sa-hit { fill:transparent; }
|
| 21 |
.sa-hit:hover { fill:var(--fg); fill-opacity:0.06; }
|
| 22 |
+
</style><defs><pattern id='sa-na' width='6' height='6' patternUnits='userSpaceOnUse' patternTransform='rotate(45)'><rect width='6' height='6' fill='#e5e7eb'/><line x1='0' y1='0' x2='0' y2='6' stroke='#9ca3af' stroke-width='2'/></pattern><linearGradient id='sa-g-stab' x1='0' y1='1' x2='0' y2='0'><stop offset='0' stop-color='#ffffff'/><stop offset='1' stop-color='#0072B2'/></linearGradient><linearGradient id='sa-g-val' x1='0' y1='1' x2='0' y2='0'><stop offset='0' stop-color='#ffffff'/><stop offset='1' stop-color='#D55E00'/></linearGradient></defs><line class='sa-grid' x1='64' x2='1024' y1='313.2' y2='313.2' stroke-width='1'/><text class='sa-muted' x='56' y='317.2' text-anchor='end' font-size='12'>900</text><line class='sa-grid' x1='64' x2='1024' y1='254.2' y2='254.2' stroke-width='1'/><text class='sa-muted' x='56' y='258.2' text-anchor='end' font-size='12'>1000</text><line class='sa-grid' x1='64' x2='1024' y1='195.1' y2='195.1' stroke-width='1'/><text class='sa-muted' x='56' y='199.1' text-anchor='end' font-size='12'>1100</text><line class='sa-grid' x1='64' x2='1024' y1='136.1' y2='136.1' stroke-width='1'/><text class='sa-muted' x='56' y='140.1' text-anchor='end' font-size='12'>1200</text><line class='sa-grid' x1='64' x2='1024' y1='77.0' y2='77.0' stroke-width='1'/><text class='sa-muted' x='56' y='81.0' text-anchor='end' font-size='12'>1300</text><line class='sa-grid' x1='64' x2='1024' y1='18.0' y2='18.0' stroke-width='1'/><text class='sa-muted' x='56' y='22.0' text-anchor='end' font-size='12'>1400</text><text class='sa-text' transform='translate(16 173) rotate(-90)' text-anchor='middle' font-size='13'>Elo</text><line class='sa-ink' x1='64' x2='1024' y1='254.2' y2='254.2' stroke-width='1' stroke-dasharray='4 4' opacity='0.5'/><rect x='73.6' y='273.1' width='22.4' height='54.9' fill='#529fcb' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='96.0' y='273.1' width='22.4' height='54.9' fill='#e28e4c' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='96.0' x2='96.0' y1='235.9' y2='312.6'/><line x1='91.0' x2='101.0' y1='235.9' y2='235.9'/><line x1='91.0' x2='101.0' y1='312.6' y2='312.6'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(100.0 342.0) rotate(-35)'>LaplacianScore</text><rect class='sa-hit' x='64.0' y='18' width='64' height='414'><title>LaplacianScore (Distance, non-search)
|
| 23 |
Elo 968 [95% CI 901β1031]
|
| 24 |
Stability 0.678
|
| 25 |
+
Validity 0.701</title></rect><rect x='137.6' y='254.2' width='22.4' height='73.8' fill='#feffff' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='160.0' y='254.2' width='22.4' height='73.8' fill='#eaae80' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='160.0' x2='160.0' y1='222.9' y2='289.6'/><line x1='155.0' x2='165.0' y1='222.9' y2='222.9'/><line x1='155.0' x2='165.0' y1='289.6' y2='289.6'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(164.0 342.0) rotate(-35)'>Random</text><rect class='sa-hit' x='128.0' y='18' width='64' height='414'><title>Random (Baseline, baseline)
|
| 26 |
Elo 1000 [95% CI 940β1053]
|
| 27 |
Stability 0.002
|
| 28 |
+
Validity 0.500</title></rect><rect x='201.6' y='200.5' width='22.4' height='127.5' fill='#73b1d5' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='224.0' y='200.5' width='22.4' height='127.5' fill='#e39557' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='224.0' x2='224.0' y1='175.1' y2='229.4'/><line x1='219.0' x2='229.0' y1='175.1' y2='175.1'/><line x1='219.0' x2='229.0' y1='229.4' y2='229.4'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(228.0 342.0) rotate(-35)'>GainRatio</text><rect class='sa-hit' x='192.0' y='18' width='64' height='414'><title>GainRatio (Information theory, non-search)
|
| 29 |
Elo 1091 [95% CI 1042β1134]
|
| 30 |
Stability 0.550
|
| 31 |
+
Validity 0.659</title></rect><rect x='265.6' y='199.9' width='22.4' height='128.1' fill='#ebf4f9' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='288.0' y='199.9' width='22.4' height='128.1' fill='#e9ab7a' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='288.0' x2='288.0' y1='165.6' y2='231.2'/><line x1='283.0' x2='293.0' y1='165.6' y2='165.6'/><line x1='283.0' x2='293.0' y1='231.2' y2='231.2'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(292.0 342.0) rotate(-35)'>LOCO</text><rect class='sa-hit' x='256.0' y='18' width='64' height='414'><title>LOCO (Leave-one-out, search)
|
| 32 |
Elo 1092 [95% CI 1039β1150]
|
| 33 |
Stability 0.080
|
| 34 |
+
Validity 0.520</title></rect><rect x='329.6' y='196.9' width='22.4' height='131.1' fill='#d2e6f1' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='352.0' y='196.9' width='22.4' height='131.1' fill='#e9aa79' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='352.0' x2='352.0' y1='162.1' y2='227.6'/><line x1='347.0' x2='357.0' y1='162.1' y2='162.1'/><line x1='347.0' x2='357.0' y1='227.6' y2='227.6'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(356.0 342.0) rotate(-35)'>SFS</text><rect class='sa-hit' x='320.0' y='18' width='64' height='414'><title>SFS (Forward search, search)
|
| 35 |
Elo 1097 [95% CI 1045β1156]
|
| 36 |
Stability 0.176
|
| 37 |
+
Validity 0.525</title></rect><rect x='393.6' y='188.1' width='22.4' height='139.9' fill='#f0f7fa' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='416.0' y='188.1' width='22.4' height='139.9' fill='#e9ac7b' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='416.0' x2='416.0' y1='156.8' y2='218.2'/><line x1='411.0' x2='421.0' y1='156.8' y2='156.8'/><line x1='411.0' x2='421.0' y1='218.2' y2='218.2'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(420.0 342.0) rotate(-35)'>SBE</text><rect class='sa-hit' x='384.0' y='18' width='64' height='414'><title>SBE (Backward search, search)
|
| 38 |
Elo 1112 [95% CI 1061β1165]
|
| 39 |
Stability 0.059
|
| 40 |
+
Validity 0.517</title></rect><rect x='457.6' y='185.7' width='22.4' height='142.3' fill='#98c6e0' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='480.0' y='185.7' width='22.4' height='142.3' fill='#ecb489' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='480.0' x2='480.0' y1='158.5' y2='215.2'/><line x1='475.0' x2='485.0' y1='158.5' y2='158.5'/><line x1='475.0' x2='485.0' y1='215.2' y2='215.2'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(484.0 342.0) rotate(-35)'>MarkovBlanket</text><rect class='sa-hit' x='448.0' y='18' width='64' height='414'><title>MarkovBlanket (Backward search, search)
|
| 41 |
Elo 1116 [95% CI 1066β1162]
|
| 42 |
Stability 0.404
|
| 43 |
+
Validity 0.463</title></rect><rect x='521.6' y='180.4' width='22.4' height='147.6' fill='#59a3cd' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='544.0' y='180.4' width='22.4' height='147.6' fill='#e9aa78' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='544.0' x2='544.0' y1='151.4' y2='205.8'/><line x1='539.0' x2='549.0' y1='151.4' y2='151.4'/><line x1='539.0' x2='549.0' y1='205.8' y2='205.8'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(548.0 342.0) rotate(-35)'>mRMR</text><rect class='sa-hit' x='512.0' y='18' width='64' height='414'><title>mRMR (Information theory, non-search)
|
| 44 |
Elo 1125 [95% CI 1082β1174]
|
| 45 |
Stability 0.651
|
| 46 |
+
Validity 0.530</title></rect><rect x='585.6' y='166.8' width='22.4' height='161.2' fill='#63a9d0' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='608.0' y='166.8' width='22.4' height='161.2' fill='#e69f68' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='608.0' x2='608.0' y1='140.2' y2='192.2'/><line x1='603.0' x2='613.0' y1='140.2' y2='140.2'/><line x1='603.0' x2='613.0' y1='192.2' y2='192.2'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(612.0 342.0) rotate(-35)'>(R)ReliefF</text><rect class='sa-hit' x='576.0' y='18' width='64' height='414'><title>(R)ReliefF (Distance, non-search)
|
| 47 |
Elo 1148 [95% CI 1105β1193]
|
| 48 |
Stability 0.612
|
| 49 |
+
Validity 0.594</title></rect><rect x='649.6' y='163.3' width='22.4' height='164.7' fill='#70b0d4' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='672.0' y='163.3' width='22.4' height='164.7' fill='#e2904f' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='672.0' x2='672.0' y1='134.3' y2='192.2'/><line x1='667.0' x2='677.0' y1='134.3' y2='134.3'/><line x1='667.0' x2='677.0' y1='192.2' y2='192.2'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(676.0 342.0) rotate(-35)'>F-Test</text><rect class='sa-hit' x='640.0' y='18' width='64' height='414'><title>F-Test (Correlation, non-search)
|
| 50 |
Elo 1154 [95% CI 1105β1203]
|
| 51 |
Stability 0.562
|
| 52 |
+
Validity 0.689</title></rect><rect x='713.6' y='159.7' width='22.4' height='168.3' fill='#69acd2' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='736.0' y='159.7' width='22.4' height='168.3' fill='#e59a5f' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='736.0' x2='736.0' y1='134.9' y2='182.7'/><line x1='731.0' x2='741.0' y1='134.9' y2='134.9'/><line x1='731.0' x2='741.0' y1='182.7' y2='182.7'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(740.0 342.0) rotate(-35)'>MI</text><rect class='sa-hit' x='704.0' y='18' width='64' height='414'><title>MI (Information theory, non-search)
|
| 53 |
Elo 1160 [95% CI 1121β1202]
|
| 54 |
Stability 0.587
|
| 55 |
+
Validity 0.627</title></rect><rect x='777.6' y='154.4' width='22.4' height='173.6' fill='#c0dcec' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='800.0' y='154.4' width='22.4' height='173.6' fill='#e7a46f' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='800.0' x2='800.0' y1='126.6' y2='179.8'/><line x1='795.0' x2='805.0' y1='126.6' y2='126.6'/><line x1='795.0' x2='805.0' y1='179.8' y2='179.8'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(804.0 342.0) rotate(-35)'>Lasso</text><rect class='sa-hit' x='768.0' y='18' width='64' height='414'><title>Lasso (Regularization, non-search)
|
| 56 |
Elo 1169 [95% CI 1126β1216]
|
| 57 |
Stability 0.246
|
| 58 |
+
Validity 0.563</title></rect><rect x='841.6' y='153.2' width='22.4' height='174.8' fill='#b0d4e7' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='864.0' y='153.2' width='22.4' height='174.8' fill='#e7a46e' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='864.0' x2='864.0' y1='124.9' y2='181.6'/><line x1='859.0' x2='869.0' y1='124.9' y2='124.9'/><line x1='859.0' x2='869.0' y1='181.6' y2='181.6'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(868.0 342.0) rotate(-35)'>ElasticNet</text><rect class='sa-hit' x='832.0' y='18' width='64' height='414'><title>ElasticNet (Regularization, non-search)
|
| 59 |
Elo 1171 [95% CI 1123β1219]
|
| 60 |
Stability 0.308
|
| 61 |
+
Validity 0.568</title></rect><rect x='905.6' y='149.7' width='22.4' height='178.3' fill='#acd1e6' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='928.0' y='149.7' width='22.4' height='178.3' fill='#ebb184' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='928.0' x2='928.0' y1='120.2' y2='178.6'/><line x1='923.0' x2='933.0' y1='120.2' y2='120.2'/><line x1='923.0' x2='933.0' y1='178.6' y2='178.6'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(932.0 342.0) rotate(-35)'>CART</text><rect class='sa-hit' x='896.0' y='18' width='64' height='414'><title>CART (Tree, non-search)
|
| 62 |
Elo 1177 [95% CI 1128β1227]
|
| 63 |
Stability 0.324
|
| 64 |
+
Validity 0.482</title></rect><rect x='969.6' y='92.4' width='22.4' height='235.6' fill='#67abd1' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='992.0' y='92.4' width='22.4' height='235.6' fill='#eaad7e' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='992.0' x2='992.0' y1='59.3' y2='121.3'/><line x1='987.0' x2='997.0' y1='59.3' y2='59.3'/><line x1='987.0' x2='997.0' y1='121.3' y2='121.3'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(996.0 342.0) rotate(-35)'>RFImportance</text><rect class='sa-hit' x='960.0' y='18' width='64' height='414'><title>RFImportance (Tree, non-search)
|
| 65 |
Elo 1274 [95% CI 1225β1330]
|
| 66 |
Stability 0.597
|
| 67 |
+
Validity 0.507</title></rect><line class='sa-ink' x1='64' x2='1024' y1='328.0' y2='328.0' stroke-width='1' opacity='0.4'/><rect x='1070' y='40.0' width='12' height='107.0' fill='url(#sa-g-stab)' stroke='#000' stroke-width='0.6'/><text class='sa-text' x='1076' y='30.0' text-anchor='middle' font-size='12'>stable</text><text class='sa-text' x='1076' y='165.0' text-anchor='middle' font-size='12'>unstable</text><text class='sa-muted' x='1088' y='151.0' font-size='11'>0.0</text><text class='sa-muted' x='1088' y='97.5' font-size='11'>0.5</text><text class='sa-muted' x='1088' y='44.0' font-size='11'>1.0</text><rect x='1070' y='207.0' width='12' height='107.0' fill='url(#sa-g-val)' stroke='#000' stroke-width='0.6'/><text class='sa-text' x='1076' y='197.0' text-anchor='middle' font-size='12'>valid</text><text class='sa-text' x='1076' y='332.0' text-anchor='middle' font-size='12'>invalid</text><text class='sa-muted' x='1088' y='318.0' font-size='11'>0.0</text><text class='sa-muted' x='1088' y='264.5' font-size='11'>0.5</text><text class='sa-muted' x='1088' y='211.0' font-size='11'>1.0</text><rect x='1070' y='352.0' width='12' height='12' fill='url(#sa-na)' stroke='#000' stroke-width='0.6'/><text class='sa-muted' x='1088' y='362.0' font-size='11'>no result</text></svg><script type='text/plain' id='sa-csv'>Method,Category,Subgroup,Elo,Elo CI low,Elo CI high,Stability,Validity
|
| 68 |
+
RFImportance,Non-search,Tree,1274.0,1225.0,1330.0,0.597,0.507
|
| 69 |
+
CART,Non-search,Tree,1177.0,1128.0,1227.0,0.324,0.482
|
| 70 |
+
ElasticNet,Non-search,Regularization,1171.0,1123.0,1219.0,0.308,0.568
|
| 71 |
+
Lasso,Non-search,Regularization,1169.0,1126.0,1216.0,0.246,0.563
|
| 72 |
+
MI,Non-search,Information theory,1160.0,1121.0,1202.0,0.587,0.627
|
| 73 |
+
F-Test,Non-search,Correlation,1154.0,1105.0,1203.0,0.562,0.689
|
| 74 |
+
(R)ReliefF,Non-search,Distance,1148.0,1105.0,1193.0,0.612,0.594
|
| 75 |
+
mRMR,Non-search,Information theory,1125.0,1082.0,1174.0,0.651,0.53
|
| 76 |
+
MarkovBlanket,Search,Backward search,1116.0,1066.0,1162.0,0.404,0.463
|
| 77 |
+
SBE,Search,Backward search,1112.0,1061.0,1165.0,0.059,0.517
|
| 78 |
+
SFS,Search,Forward search,1097.0,1045.0,1156.0,0.176,0.525
|
| 79 |
+
LOCO,Search,Leave-one-out,1092.0,1039.0,1150.0,0.08,0.52
|
| 80 |
+
GainRatio,Non-search,Information theory,1091.0,1042.0,1134.0,0.55,0.659
|
| 81 |
+
Random,Baseline,Baseline,1000.0,940.0,1053.0,0.002,0.5
|
| 82 |
+
LaplacianScore,Non-search,Distance,968.0,901.0,1031.0,0.678,0.701
|
| 83 |
</script><script>
|
| 84 |
function saDownload(name, blob) {
|
| 85 |
var a = document.createElement('a');
|
data/subsets/tasks_all/models_lightgbm/budgets_4/leaderboard_table.html
CHANGED
|
@@ -26,23 +26,25 @@ td.method { font-weight:600; }
|
|
| 26 |
td .sub { color:var(--muted); font-size:12px; font-weight:400; margin-left:6px; }
|
| 27 |
td.na { color:var(--muted); }
|
| 28 |
.pill { display:inline-block; padding:0 8px; border:1px solid var(--line); border-radius:999px;
|
| 29 |
-
font-size:12px; color:var(--
|
| 30 |
-
</style></head><body><div class='toolbar'><span class='hint'>Click a column to sort. Hover a header for its definition.</span><button onclick="saCsv('selectarena_leaderboard.csv')">CSV</button></div><div class='wrap'><table id='sa-table'><thead><tr><th class='num'>#</th><th class=''>Method</th><th class='' title='Filter: scores features from the data alone. Wrapper: searches feature subsets by refitting a proxy model. Embedded: reads the selection off a fitted model.'>Type</th><th class='num' title='Bradley-Terry Elo from pairwise matches within each dataset, budget and downstream model. Random = 1000. +/β give the 95% bootstrap interval.'>Elo β</th><th class='num' title='Mean rank of the downstream error.'>Avg. rank β</th><th class='num' title='Share of pairwise comparisons won, in %. Ties count half.'>Win rate β</th><th class='num' title='Mean relative gap (error β best error) / error to the best method, in %.'>Improvability β</th><th class='num' title='Mean ratio of the error to the same model trained on all features. Below 1: the selection helped.'>Error vs. all β</th><th class='num' title='Agreement of the selected features across bootstrap resamples (Nogueira et al., 2018).'>Stability β</th><th class='num' title='Share of the budget filled with original features when as many noise features are added.'>Validity β</th><th class='num' title='Median wall-clock time of the selection.'>Time (s) β</th><th class='num' title='Share of runs where the method ran out of time and random features filled the budget.'>Fallback (%)</th><th class='num'>#Datasets</th></tr></thead><tbody><tr><td class='num' data-sort='1'>1</td><td class='method' data-sort='RFImportance'>RFImportance</td><td class='' data-sort='Embedded'><span class='pill'>π³ Embedded</span></td><td class='num' data-sort='1274.0'>1274<span class='sub'>+56 / β49</span></td><td class='num' data-sort='5.9'>5.90</td><td class='num' data-sort='65.0'>65.0</td><td class='num' data-sort='8.7'>8.7</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.597'>0.597</td><td class='num' data-sort='0.507'>0.507</td><td class='num' data-sort='0.2'>0.2</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='2'>2</td><td class='method' data-sort='CART'>CART</td><td class='' data-sort='Embedded'><span class='pill'>π³ Embedded</span></td><td class='num' data-sort='1177.0'>1177<span class='sub'>+50 / β49</span></td><td class='num' data-sort='6.9'>6.90</td><td class='num' data-sort='57.9'>57.9</td><td class='num' data-sort='11.0'>11.0</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.324'>0.324</td><td class='num' data-sort='0.482'>0.482</td><td class='num' data-sort='0.1'>0.1</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='3'>3</td><td class='method' data-sort='ElasticNet'>ElasticNet</td><td class='' data-sort='Embedded'><span class='pill'>π³ Embedded</span></td><td class='num' data-sort='1171.0'>1171<span class='sub'>+48 / β48</span></td><td class='num' data-sort='7.05'>7.05</td><td class='num' data-sort='56.8'>56.8</td><td class='num' data-sort='10.8'>10.8</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.308'>0.308</td><td class='num' data-sort='0.568'>0.568</td><td class='num' data-sort='6.1'>6.1</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='4'>4</td><td class='method' data-sort='Lasso'>Lasso</td><td class='' data-sort='Embedded'><span class='pill'>π³ Embedded</span></td><td class='num' data-sort='1169.0'>1169<span class='sub'>+47 / β43</span></td><td class='num' data-sort='7.22'>7.22</td><td class='num' data-sort='55.6'>55.6</td><td class='num' data-sort='10.5'>10.5</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.246'>0.246</td><td class='num' data-sort='0.563'>0.563</td><td class='num' data-sort='1.8'>1.8</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='5'>5</td><td class='method' data-sort='MI'>MI</td><td class='' data-sort='Filter'><span class='pill'>π Filter</span></td><td class='num' data-sort='1160.0'>1160<span class='sub'>+42 / β39</span></td><td class='num' data-sort='7.29'>7.29</td><td class='num' data-sort='55.1'>55.1</td><td class='num' data-sort='11.4'>11.4</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.587'>0.587</td><td class='num' data-sort='0.627'>0.627</td><td class='num' data-sort='0.4'>0.4</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='6'>6</td><td class='method' data-sort='F-Test'>F-Test</td><td class='' data-sort='Filter'><span class='pill'>π Filter</span></td><td class='num' data-sort='1154.0'>1154<span class='sub'>+49 / β49</span></td><td class='num' data-sort='7.5'>7.50</td><td class='num' data-sort='53.6'>53.6</td><td class='num' data-sort='10.2'>10.2</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.562'>0.562</td><td class='num' data-sort='0.689'>0.689</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='7'>7</td><td class='method' data-sort='(R)ReliefF'>(R)ReliefF</td><td class='' data-sort='Filter'><span class='pill'>π Filter</span></td><td class='num' data-sort='1148.0'>1148<span class='sub'>+45 / β43</span></td><td class='num' data-sort='7.86'>7.86</td><td class='num' data-sort='51.0'>51.0</td><td class='num' data-sort='11.2'>11.2</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.612'>0.612</td><td class='num' data-sort='0.594'>0.594</td><td class='num' data-sort='13.2'>13.2</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='8'>8</td><td class='method' data-sort='mRMR'>mRMR</td><td class='' data-sort='Filter'><span class='pill'>π Filter</span></td><td class='num' data-sort='1125.0'>1125<span class='sub'>+49 / β43</span></td><td class='num' data-sort='7.75'>7.75</td><td class='num' data-sort='51.8'>51.8</td><td class='num' data-sort='10.2'>10.2</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.651'>0.651</td><td class='num' data-sort='0.53'>0.530</td><td class='num' data-sort='0.5'>0.5</td><td class='num' data-sort='9.9'>9.9</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='9'>9</td><td class='method' data-sort='MarkovBlanket'>MarkovBlanket</td><td class='' data-sort='Filter'><span class='pill'>π Filter</span></td><td class='num' data-sort='1116.0'>1116<span class='sub'>+46 / β50</span></td><td class='num' data-sort='8.39'>8.39</td><td class='num' data-sort='47.2'>47.2</td><td class='num' data-sort='11.3'>11.3</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.404'>0.404</td><td class='num' data-sort='0.463'>0.463</td><td class='num' data-sort='0.1'>0.1</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='10'>10</td><td class='method' data-sort='SBE'>SBE</td><td class='' data-sort='Wrapper'><span class='pill'>π Wrapper</span></td><td class='num' data-sort='1112.0'>1112<span class='sub'>+53 / β51</span></td><td class='num' data-sort='8.4'>8.40</td><td class='num' data-sort='47.1'>47.1</td><td class='num' data-sort='13.1'>13.1</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.059'>0.059</td><td class='num' data-sort='0.517'>0.517</td><td class='num' data-sort='1001.8'>1,001.8</td><td class='num' data-sort='40.8'>40.8</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='11'>11</td><td class='method' data-sort='SFS'>SFS</td><td class='' data-sort='Wrapper'><span class='pill'>π Wrapper</span></td><td class='num' data-sort='1097.0'>1097<span class='sub'>+59 / β52</span></td><td class='num' data-sort='8.7'>8.70</td><td class='num' data-sort='45.0'>45.0</td><td class='num' data-sort='13.1'>13.1</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.176'>0.176</td><td class='num' data-sort='0.525'>0.525</td><td class='num' data-sort='3227.2'>3,227.2</td><td class='num' data-sort='48.4'>48.4</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='12'>12</td><td class='method' data-sort='LOCO'>LOCO</td><td class='' data-sort='Wrapper'><span class='pill'>π Wrapper</span></td><td class='num' data-sort='1092.0'>1092<span class='sub'>+58 / β53</span></td><td class='num' data-sort='8.36'>8.36</td><td class='num' data-sort='47.4'>47.4</td><td class='num' data-sort='13.8'>13.8</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.08'>0.080</td><td class='num' data-sort='0.52'>0.520</td><td class='num' data-sort='249.4'>249.4</td><td class='num' data-sort='18.3'>18.3</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='13'>13</td><td class='method' data-sort='GainRatio'>GainRatio</td><td class='' data-sort='Filter'><span class='pill'>π Filter</span></td><td class='num' data-sort='1091.0'>1091<span class='sub'>+43 / β49</span></td><td class='num' data-sort='8.61'>8.61</td><td class='num' data-sort='45.7'>45.7</td><td class='num' data-sort='12.7'>12.7</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.55'>0.550</td><td class='num' data-sort='0.659'>0.659</td><td class='num' data-sort='0.2'>0.2</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='14'>14</td><td class='method' data-sort='Random'>Random</td><td class='' data-sort='Baseline'><span class='pill'>βͺ Baseline</span></td><td class='num' data-sort='1000.0'>1000<span class='sub'>+53 / β60</span></td><td class='num' data-sort='9.8'>9.80</td><td class='num' data-sort='37.1'>37.1</td><td class='num' data-sort='16.9'>16.9</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.002'>0.002</td><td class='num' data-sort='0.5'>0.500</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='100.0'>100.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='15'>15</td><td class='method' data-sort='LaplacianScore'>LaplacianScore</td><td class='' data-sort='Filter'><span class='pill'>π Filter</span></td><td class='num' data-sort='968.0'>968<span class='sub'>+63 / β67</span></td><td class='num' data-sort='10.28'>10.28</td><td class='num' data-sort='33.7'>33.7</td><td class='num' data-sort='20.2'>20.2</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.678'>0.678</td><td class='num' data-sort='0.701'>0.701</td><td class='num' data-sort='0.2'>0.2</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr></tbody></table></div><script type='text/plain' id='sa-csv'>Rank,Type,TypeName,Method,Elo,Elo CI low,Elo CI high,Avg. rank,Win rate (%),Improvability (%),Error vs. all features,Stability,Validity,Selection time (s),Fallback rate (%),#Datasets,#Comparisons
|
| 31 |
-
|
| 32 |
-
2,π³,Embedded,CART,1177.0,1128.0,1227.0,6.9,57.9,11.0,,0.324,0.482,0.1,0.0,71,213
|
| 33 |
-
|
| 34 |
-
|
| 35 |
-
|
| 36 |
-
|
| 37 |
-
|
| 38 |
-
|
| 39 |
-
|
| 40 |
-
|
| 41 |
-
|
| 42 |
-
|
| 43 |
-
|
| 44 |
-
|
| 45 |
-
|
|
|
|
|
|
|
| 46 |
</script><script>
|
| 47 |
function saDownload(name, blob) {
|
| 48 |
var a = document.createElement('a');
|
|
|
|
| 26 |
td .sub { color:var(--muted); font-size:12px; font-weight:400; margin-left:6px; }
|
| 27 |
td.na { color:var(--muted); }
|
| 28 |
.pill { display:inline-block; padding:0 8px; border:1px solid var(--line); border-radius:999px;
|
| 29 |
+
font-size:12px; color:var(--fg); }
|
| 30 |
+
.pill.cat-search { border-color:var(--accent); }
|
| 31 |
+
.pill.cat-baseline { color:var(--muted); }
|
| 32 |
+
</style></head><body><div class='toolbar'><span class='hint'>Click a column to sort. Hover a header for its definition.</span><button onclick="saCsv('selectarena_leaderboard.csv')">CSV</button></div><div class='wrap'><table id='sa-table'><thead><tr><th class='num'>#</th><th class=''>Method</th><th class='' title='Taxonomy of the paper (Appendix A). Search methods evaluate many feature subsets with the model (forward, backward, leave-one-out); non-search methods use a proxy instead (correlation, distance, information theory, tree, regularization).'>Type</th><th class='num' title='Bradley-Terry Elo from pairwise matches within each dataset, budget and downstream model. Random = 1000. +/β give the 95% bootstrap interval.'>Elo β</th><th class='num' title='Mean rank of the downstream error.'>Avg. rank β</th><th class='num' title='Share of pairwise comparisons won, in %. Ties count half.'>Win rate β</th><th class='num' title='Mean relative gap (error β best error) / error to the best method, in %.'>Improvability β</th><th class='num' title='Mean ratio of the error to the same model trained on all features. Below 1: the selection helped.'>Error vs. all β</th><th class='num' title='Agreement of the selected features across bootstrap resamples (Nogueira et al., 2018).'>Stability β</th><th class='num' title='Share of the budget filled with original features when as many noise features are added.'>Validity β</th><th class='num' title='Median wall-clock time of the selection.'>Time (s) β</th><th class='num' title='Share of runs where the method ran out of time and random features filled the budget.'>Fallback (%)</th><th class='num'>#Datasets</th></tr></thead><tbody><tr><td class='num' data-sort='1'>1</td><td class='method' data-sort='RFImportance'>RFImportance</td><td class='' data-sort='Tree'><span class='pill cat-non-search'>Tree</span><span class='sub'>non-search</span></td><td class='num' data-sort='1274.0'>1274<span class='sub'>+56 / β49</span></td><td class='num' data-sort='5.9'>5.90</td><td class='num' data-sort='65.0'>65.0</td><td class='num' data-sort='8.7'>8.7</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.597'>0.597</td><td class='num' data-sort='0.507'>0.507</td><td class='num' data-sort='0.2'>0.2</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='2'>2</td><td class='method' data-sort='CART'>CART</td><td class='' data-sort='Tree'><span class='pill cat-non-search'>Tree</span><span class='sub'>non-search</span></td><td class='num' data-sort='1177.0'>1177<span class='sub'>+50 / β49</span></td><td class='num' data-sort='6.9'>6.90</td><td class='num' data-sort='57.9'>57.9</td><td class='num' data-sort='11.0'>11.0</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.324'>0.324</td><td class='num' data-sort='0.482'>0.482</td><td class='num' data-sort='0.1'>0.1</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='3'>3</td><td class='method' data-sort='ElasticNet'>ElasticNet</td><td class='' data-sort='Regularization'><span class='pill cat-non-search'>Regularization</span><span class='sub'>non-search</span></td><td class='num' data-sort='1171.0'>1171<span class='sub'>+48 / β48</span></td><td class='num' data-sort='7.05'>7.05</td><td class='num' data-sort='56.8'>56.8</td><td class='num' data-sort='10.8'>10.8</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.308'>0.308</td><td class='num' data-sort='0.568'>0.568</td><td class='num' data-sort='6.1'>6.1</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='4'>4</td><td class='method' data-sort='Lasso'>Lasso</td><td class='' data-sort='Regularization'><span class='pill cat-non-search'>Regularization</span><span class='sub'>non-search</span></td><td class='num' data-sort='1169.0'>1169<span class='sub'>+47 / β43</span></td><td class='num' data-sort='7.22'>7.22</td><td class='num' data-sort='55.6'>55.6</td><td class='num' data-sort='10.5'>10.5</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.246'>0.246</td><td class='num' data-sort='0.563'>0.563</td><td class='num' data-sort='1.8'>1.8</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='5'>5</td><td class='method' data-sort='MI'>MI</td><td class='' data-sort='Information theory'><span class='pill cat-non-search'>Information theory</span><span class='sub'>non-search</span></td><td class='num' data-sort='1160.0'>1160<span class='sub'>+42 / β39</span></td><td class='num' data-sort='7.29'>7.29</td><td class='num' data-sort='55.1'>55.1</td><td class='num' data-sort='11.4'>11.4</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.587'>0.587</td><td class='num' data-sort='0.627'>0.627</td><td class='num' data-sort='0.4'>0.4</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='6'>6</td><td class='method' data-sort='F-Test'>F-Test</td><td class='' data-sort='Correlation'><span class='pill cat-non-search'>Correlation</span><span class='sub'>non-search</span></td><td class='num' data-sort='1154.0'>1154<span class='sub'>+49 / β49</span></td><td class='num' data-sort='7.5'>7.50</td><td class='num' data-sort='53.6'>53.6</td><td class='num' data-sort='10.2'>10.2</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.562'>0.562</td><td class='num' data-sort='0.689'>0.689</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='7'>7</td><td class='method' data-sort='(R)ReliefF'>(R)ReliefF</td><td class='' data-sort='Distance'><span class='pill cat-non-search'>Distance</span><span class='sub'>non-search</span></td><td class='num' data-sort='1148.0'>1148<span class='sub'>+45 / β43</span></td><td class='num' data-sort='7.86'>7.86</td><td class='num' data-sort='51.0'>51.0</td><td class='num' data-sort='11.2'>11.2</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.612'>0.612</td><td class='num' data-sort='0.594'>0.594</td><td class='num' data-sort='13.2'>13.2</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='8'>8</td><td class='method' data-sort='mRMR'>mRMR</td><td class='' data-sort='Information theory'><span class='pill cat-non-search'>Information theory</span><span class='sub'>non-search</span></td><td class='num' data-sort='1125.0'>1125<span class='sub'>+49 / β43</span></td><td class='num' data-sort='7.75'>7.75</td><td class='num' data-sort='51.8'>51.8</td><td class='num' data-sort='10.2'>10.2</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.651'>0.651</td><td class='num' data-sort='0.53'>0.530</td><td class='num' data-sort='0.5'>0.5</td><td class='num' data-sort='9.9'>9.9</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='9'>9</td><td class='method' data-sort='MarkovBlanket'>MarkovBlanket</td><td class='' data-sort='Backward search'><span class='pill cat-search'>Backward search</span><span class='sub'>search</span></td><td class='num' data-sort='1116.0'>1116<span class='sub'>+46 / β50</span></td><td class='num' data-sort='8.39'>8.39</td><td class='num' data-sort='47.2'>47.2</td><td class='num' data-sort='11.3'>11.3</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.404'>0.404</td><td class='num' data-sort='0.463'>0.463</td><td class='num' data-sort='0.1'>0.1</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='10'>10</td><td class='method' data-sort='SBE'>SBE</td><td class='' data-sort='Backward search'><span class='pill cat-search'>Backward search</span><span class='sub'>search</span></td><td class='num' data-sort='1112.0'>1112<span class='sub'>+53 / β51</span></td><td class='num' data-sort='8.4'>8.40</td><td class='num' data-sort='47.1'>47.1</td><td class='num' data-sort='13.1'>13.1</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.059'>0.059</td><td class='num' data-sort='0.517'>0.517</td><td class='num' data-sort='1001.8'>1,001.8</td><td class='num' data-sort='40.8'>40.8</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='11'>11</td><td class='method' data-sort='SFS'>SFS</td><td class='' data-sort='Forward search'><span class='pill cat-search'>Forward search</span><span class='sub'>search</span></td><td class='num' data-sort='1097.0'>1097<span class='sub'>+59 / β52</span></td><td class='num' data-sort='8.7'>8.70</td><td class='num' data-sort='45.0'>45.0</td><td class='num' data-sort='13.1'>13.1</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.176'>0.176</td><td class='num' data-sort='0.525'>0.525</td><td class='num' data-sort='3227.2'>3,227.2</td><td class='num' data-sort='48.4'>48.4</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='12'>12</td><td class='method' data-sort='LOCO'>LOCO</td><td class='' data-sort='Leave-one-out'><span class='pill cat-search'>Leave-one-out</span><span class='sub'>search</span></td><td class='num' data-sort='1092.0'>1092<span class='sub'>+58 / β53</span></td><td class='num' data-sort='8.36'>8.36</td><td class='num' data-sort='47.4'>47.4</td><td class='num' data-sort='13.8'>13.8</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.08'>0.080</td><td class='num' data-sort='0.52'>0.520</td><td class='num' data-sort='249.4'>249.4</td><td class='num' data-sort='18.3'>18.3</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='13'>13</td><td class='method' data-sort='GainRatio'>GainRatio</td><td class='' data-sort='Information theory'><span class='pill cat-non-search'>Information theory</span><span class='sub'>non-search</span></td><td class='num' data-sort='1091.0'>1091<span class='sub'>+43 / β49</span></td><td class='num' data-sort='8.61'>8.61</td><td class='num' data-sort='45.7'>45.7</td><td class='num' data-sort='12.7'>12.7</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.55'>0.550</td><td class='num' data-sort='0.659'>0.659</td><td class='num' data-sort='0.2'>0.2</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='14'>14</td><td class='method' data-sort='Random'>Random</td><td class='' data-sort='Baseline'><span class='pill cat-baseline'>Baseline</span></td><td class='num' data-sort='1000.0'>1000<span class='sub'>+53 / β60</span></td><td class='num' data-sort='9.8'>9.80</td><td class='num' data-sort='37.1'>37.1</td><td class='num' data-sort='16.9'>16.9</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.002'>0.002</td><td class='num' data-sort='0.5'>0.500</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='100.0'>100.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='15'>15</td><td class='method' data-sort='LaplacianScore'>LaplacianScore</td><td class='' data-sort='Distance'><span class='pill cat-non-search'>Distance</span><span class='sub'>non-search</span></td><td class='num' data-sort='968.0'>968<span class='sub'>+63 / β67</span></td><td class='num' data-sort='10.28'>10.28</td><td class='num' data-sort='33.7'>33.7</td><td class='num' data-sort='20.2'>20.2</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.678'>0.678</td><td class='num' data-sort='0.701'>0.701</td><td class='num' data-sort='0.2'>0.2</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr></tbody></table></div><script type='text/plain' id='sa-csv'>Rank,Method,Category,Subgroup,Elo,Elo CI low,Elo CI high,Avg. rank,Win rate (%),Improvability (%),Error vs. all features,Stability,Validity,Selection time (s),Fallback rate (%),#Datasets,#Comparisons
|
| 33 |
+
1,RFImportance,Non-search,Tree,1274.0,1225.0,1330.0,5.9,65.0,8.7,,0.597,0.507,0.2,0.0,71,213
|
| 34 |
+
2,CART,Non-search,Tree,1177.0,1128.0,1227.0,6.9,57.9,11.0,,0.324,0.482,0.1,0.0,71,213
|
| 35 |
+
3,ElasticNet,Non-search,Regularization,1171.0,1123.0,1219.0,7.05,56.8,10.8,,0.308,0.568,6.1,0.0,71,213
|
| 36 |
+
4,Lasso,Non-search,Regularization,1169.0,1126.0,1216.0,7.22,55.6,10.5,,0.246,0.563,1.8,0.0,71,213
|
| 37 |
+
5,MI,Non-search,Information theory,1160.0,1121.0,1202.0,7.29,55.1,11.4,,0.587,0.627,0.4,0.0,71,213
|
| 38 |
+
6,F-Test,Non-search,Correlation,1154.0,1105.0,1203.0,7.5,53.6,10.2,,0.562,0.689,0.0,0.0,71,213
|
| 39 |
+
7,(R)ReliefF,Non-search,Distance,1148.0,1105.0,1193.0,7.86,51.0,11.2,,0.612,0.594,13.2,0.0,71,213
|
| 40 |
+
8,mRMR,Non-search,Information theory,1125.0,1082.0,1174.0,7.75,51.8,10.2,,0.651,0.53,0.5,9.9,71,213
|
| 41 |
+
9,MarkovBlanket,Search,Backward search,1116.0,1066.0,1162.0,8.39,47.2,11.3,,0.404,0.463,0.1,0.0,71,213
|
| 42 |
+
10,SBE,Search,Backward search,1112.0,1061.0,1165.0,8.4,47.1,13.1,,0.059,0.517,1001.8,40.8,71,213
|
| 43 |
+
11,SFS,Search,Forward search,1097.0,1045.0,1156.0,8.7,45.0,13.1,,0.176,0.525,3227.2,48.4,71,213
|
| 44 |
+
12,LOCO,Search,Leave-one-out,1092.0,1039.0,1150.0,8.36,47.4,13.8,,0.08,0.52,249.4,18.3,71,213
|
| 45 |
+
13,GainRatio,Non-search,Information theory,1091.0,1042.0,1134.0,8.61,45.7,12.7,,0.55,0.659,0.2,0.0,71,213
|
| 46 |
+
14,Random,Baseline,Baseline,1000.0,940.0,1053.0,9.8,37.1,16.9,,0.002,0.5,0.0,100.0,71,213
|
| 47 |
+
15,LaplacianScore,Non-search,Distance,968.0,901.0,1031.0,10.28,33.7,20.2,,0.678,0.701,0.2,0.0,71,213
|
| 48 |
</script><script>
|
| 49 |
function saDownload(name, blob) {
|
| 50 |
var a = document.createElement('a');
|
data/subsets/tasks_all/models_lightgbm/budgets_4/website_leaderboard.csv
CHANGED
|
@@ -1,16 +1,16 @@
|
|
| 1 |
-
Rank,
|
| 2 |
-
1,
|
| 3 |
-
2,
|
| 4 |
-
3,
|
| 5 |
-
4,
|
| 6 |
-
5,
|
| 7 |
-
6,
|
| 8 |
-
7,
|
| 9 |
-
8,
|
| 10 |
-
9,
|
| 11 |
-
10,
|
| 12 |
-
11,
|
| 13 |
-
12,
|
| 14 |
-
13,
|
| 15 |
-
14,
|
| 16 |
-
15,
|
|
|
|
| 1 |
+
Rank,Method,Category,Subgroup,Elo,Elo CI low,Elo CI high,Avg. rank,Win rate (%),Improvability (%),Error vs. all features,Stability,Validity,Selection time (s),Fallback rate (%),#Datasets,#Comparisons
|
| 2 |
+
1,RFImportance,Non-search,Tree,1274.0,1225.0,1330.0,5.9,65.0,8.7,,0.597,0.507,0.2,0.0,71,213
|
| 3 |
+
2,CART,Non-search,Tree,1177.0,1128.0,1227.0,6.9,57.9,11.0,,0.324,0.482,0.1,0.0,71,213
|
| 4 |
+
3,ElasticNet,Non-search,Regularization,1171.0,1123.0,1219.0,7.05,56.8,10.8,,0.308,0.568,6.1,0.0,71,213
|
| 5 |
+
4,Lasso,Non-search,Regularization,1169.0,1126.0,1216.0,7.22,55.6,10.5,,0.246,0.563,1.8,0.0,71,213
|
| 6 |
+
5,MI,Non-search,Information theory,1160.0,1121.0,1202.0,7.29,55.1,11.4,,0.587,0.627,0.4,0.0,71,213
|
| 7 |
+
6,F-Test,Non-search,Correlation,1154.0,1105.0,1203.0,7.5,53.6,10.2,,0.562,0.689,0.0,0.0,71,213
|
| 8 |
+
7,(R)ReliefF,Non-search,Distance,1148.0,1105.0,1193.0,7.86,51.0,11.2,,0.612,0.594,13.2,0.0,71,213
|
| 9 |
+
8,mRMR,Non-search,Information theory,1125.0,1082.0,1174.0,7.75,51.8,10.2,,0.651,0.53,0.5,9.9,71,213
|
| 10 |
+
9,MarkovBlanket,Search,Backward search,1116.0,1066.0,1162.0,8.39,47.2,11.3,,0.404,0.463,0.1,0.0,71,213
|
| 11 |
+
10,SBE,Search,Backward search,1112.0,1061.0,1165.0,8.4,47.1,13.1,,0.059,0.517,1001.8,40.8,71,213
|
| 12 |
+
11,SFS,Search,Forward search,1097.0,1045.0,1156.0,8.7,45.0,13.1,,0.176,0.525,3227.2,48.4,71,213
|
| 13 |
+
12,LOCO,Search,Leave-one-out,1092.0,1039.0,1150.0,8.36,47.4,13.8,,0.08,0.52,249.4,18.3,71,213
|
| 14 |
+
13,GainRatio,Non-search,Information theory,1091.0,1042.0,1134.0,8.61,45.7,12.7,,0.55,0.659,0.2,0.0,71,213
|
| 15 |
+
14,Random,Baseline,Baseline,1000.0,940.0,1053.0,9.8,37.1,16.9,,0.002,0.5,0.0,100.0,71,213
|
| 16 |
+
15,LaplacianScore,Non-search,Distance,968.0,901.0,1031.0,10.28,33.7,20.2,,0.678,0.701,0.2,0.0,71,213
|
data/subsets/tasks_all/models_lightgbm/budgets_all/elo_explorer.html
CHANGED
|
@@ -19,67 +19,67 @@ svg { display:block; width:100%; height:auto; }
|
|
| 19 |
.sa-ink { stroke:var(--fg); }
|
| 20 |
.sa-hit { fill:transparent; }
|
| 21 |
.sa-hit:hover { fill:var(--fg); fill-opacity:0.06; }
|
| 22 |
-
</style><defs><pattern id='sa-na' width='6' height='6' patternUnits='userSpaceOnUse' patternTransform='rotate(45)'><rect width='6' height='6' fill='#e5e7eb'/><line x1='0' y1='0' x2='0' y2='6' stroke='#9ca3af' stroke-width='2'/></pattern><linearGradient id='sa-g-stab' x1='0' y1='1' x2='0' y2='0'><stop offset='0' stop-color='#ffffff'/><stop offset='1' stop-color='#0072B2'/></linearGradient><linearGradient id='sa-g-val' x1='0' y1='1' x2='0' y2='0'><stop offset='0' stop-color='#ffffff'/><stop offset='1' stop-color='#D55E00'/></linearGradient></defs><line class='sa-grid' x1='64' x2='1024' y1='287.6' y2='287.6' stroke-width='1'/><text class='sa-muted' x='56' y='291.6' text-anchor='end' font-size='12'>900</text><line class='sa-grid' x1='64' x2='1024' y1='233.7' y2='233.7' stroke-width='1'/><text class='sa-muted' x='56' y='237.7' text-anchor='end' font-size='12'>1000</text><line class='sa-grid' x1='64' x2='1024' y1='179.7' y2='179.7' stroke-width='1'/><text class='sa-muted' x='56' y='183.7' text-anchor='end' font-size='12'>1100</text><line class='sa-grid' x1='64' x2='1024' y1='125.8' y2='125.8' stroke-width='1'/><text class='sa-muted' x='56' y='129.8' text-anchor='end' font-size='12'>1200</text><line class='sa-grid' x1='64' x2='1024' y1='71.9' y2='71.9' stroke-width='1'/><text class='sa-muted' x='56' y='75.9' text-anchor='end' font-size='12'>1300</text><line class='sa-grid' x1='64' x2='1024' y1='18.0' y2='18.0' stroke-width='1'/><text class='sa-muted' x='56' y='22.0' text-anchor='end' font-size='12'>1400</text><text class='sa-text' transform='translate(16 173) rotate(-90)' text-anchor='middle' font-size='13'>Elo</text><line class='sa-ink' x1='64' x2='1024' y1='233.7' y2='233.7' stroke-width='1' stroke-dasharray='4 4' opacity='0.5'/><rect x='73.6' y='264.4' width='22.4' height='63.6' fill='#59a3cd' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='96.0' y='264.4' width='22.4' height='63.6' fill='#de7f34' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='96.0' x2='96.0' y1='233.7' y2='299.4'/><line x1='91.0' x2='101.0' y1='233.7' y2='233.7'/><line x1='91.0' x2='101.0' y1='299.4' y2='299.4'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(100.0 342.0) rotate(-35)'>LaplacianScore</text><rect class='sa-hit' x='64.0' y='18' width='64' height='414'><title>LaplacianScore (
|
| 23 |
Elo 943 [95% CI 878β1000]
|
| 24 |
Stability 0.652
|
| 25 |
-
Validity 0.797</title></rect><rect x='137.6' y='233.7' width='22.4' height='94.3' fill='#ffffff' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='160.0' y='233.7' width='22.4' height='94.3' fill='#eaae80' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='160.0' x2='160.0' y1='215.3' y2='255.2'/><line x1='155.0' x2='165.0' y1='215.3' y2='215.3'/><line x1='155.0' x2='165.0' y1='255.2' y2='255.2'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(164.0 342.0) rotate(-35)'>Random</text><rect class='sa-hit' x='128.0' y='18' width='64' height='414'><title>Random (Baseline)
|
| 26 |
Elo 1000 [95% CI 960β1034]
|
| 27 |
Stability 0.001
|
| 28 |
-
Validity 0.500</title></rect><rect x='201.6' y='148.5' width='22.4' height='179.5' fill='#d3e7f2' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='224.0' y='148.5' width='22.4' height='179.5' fill='#e7a16a' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='224.0' x2='224.0' y1='125.3' y2='170.6'/><line x1='219.0' x2='229.0' y1='125.3' y2='125.3'/><line x1='219.0' x2='229.0' y1='170.6' y2='170.6'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(228.0 342.0) rotate(-35)'>LOCO</text><rect class='sa-hit' x='192.0' y='18' width='64' height='414'><title>LOCO (
|
| 29 |
Elo 1158 [95% CI 1117β1201]
|
| 30 |
Stability 0.173
|
| 31 |
-
Validity 0.583</title></rect><rect x='265.6' y='136.1' width='22.4' height='191.9' fill='#5ba4cd' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='288.0' y='136.1' width='22.4' height='191.9' fill='#dc7a2d' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='288.0' x2='288.0' y1='118.8' y2='153.9'/><line x1='283.0' x2='293.0' y1='118.8' y2='118.8'/><line x1='283.0' x2='293.0' y1='153.9' y2='153.9'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(292.0 342.0) rotate(-35)'>GainRatio</text><rect class='sa-hit' x='256.0' y='18' width='64' height='414'><title>GainRatio (
|
| 32 |
Elo 1181 [95% CI 1148β1213]
|
| 33 |
Stability 0.644
|
| 34 |
-
Validity 0.825</title></rect><rect x='329.6' y='134.5' width='22.4' height='193.5' fill='#ddecf5' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='352.0' y='134.5' width='22.4' height='193.5' fill='#e7a570' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='352.0' x2='352.0' y1='109.7' y2='157.1'/><line x1='347.0' x2='357.0' y1='109.7' y2='109.7'/><line x1='347.0' x2='357.0' y1='157.1' y2='157.1'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(356.0 342.0) rotate(-35)'>SBE</text><rect class='sa-hit' x='320.0' y='18' width='64' height='414'><title>SBE (
|
| 35 |
Elo 1184 [95% CI 1142β1230]
|
| 36 |
Stability 0.132
|
| 37 |
-
Validity 0.560</title></rect><rect x='393.6' y='133.9' width='22.4' height='194.1' fill='#5aa4cd' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='416.0' y='133.9' width='22.4' height='194.1' fill='#e08842' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='416.0' x2='416.0' y1='115.0' y2='153.9'/><line x1='411.0' x2='421.0' y1='115.0' y2='115.0'/><line x1='411.0' x2='421.0' y1='153.9' y2='153.9'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(420.0 342.0) rotate(-35)'>(R)ReliefF</text><rect class='sa-hit' x='384.0' y='18' width='64' height='414'><title>(R)ReliefF (
|
| 38 |
Elo 1185 [95% CI 1148β1220]
|
| 39 |
Stability 0.646
|
| 40 |
-
Validity 0.740</title></rect><rect x='457.6' y='130.1' width='22.4' height='197.9' fill='#bbd9ea' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='480.0' y='130.1' width='22.4' height='197.9' fill='#e59a5f' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='480.0' x2='480.0' y1='109.1' y2='148.5'/><line x1='475.0' x2='485.0' y1='109.1' y2='109.1'/><line x1='475.0' x2='485.0' y1='148.5' y2='148.5'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(484.0 342.0) rotate(-35)'>SFS</text><rect class='sa-hit' x='448.0' y='18' width='64' height='414'><title>SFS (
|
| 41 |
Elo 1192 [95% CI 1158β1231]
|
| 42 |
Stability 0.267
|
| 43 |
-
Validity 0.627</title></rect><rect x='521.6' y='125.3' width='22.4' height='202.7' fill='#65aad0' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='544.0' y='125.3' width='22.4' height='202.7' fill='#e39658' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='544.0' x2='544.0' y1='108.6' y2='143.6'/><line x1='539.0' x2='549.0' y1='108.6' y2='108.6'/><line x1='539.0' x2='549.0' y1='143.6' y2='143.6'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(548.0 342.0) rotate(-35)'>mRMR</text><rect class='sa-hit' x='512.0' y='18' width='64' height='414'><title>mRMR (
|
| 44 |
Elo 1201 [95% CI 1167β1232]
|
| 45 |
Stability 0.604
|
| 46 |
-
Validity 0.655</title></rect><rect x='585.6' y='116.7' width='22.4' height='211.3' fill='#529fcb' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='608.0' y='116.7' width='22.4' height='211.3' fill='#db7727' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='608.0' x2='608.0' y1='97.8' y2='136.6'/><line x1='603.0' x2='613.0' y1='97.8' y2='97.8'/><line x1='603.0' x2='613.0' y1='136.6' y2='136.6'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(612.0 342.0) rotate(-35)'>F-Test</text><rect class='sa-hit' x='576.0' y='18' width='64' height='414'><title>F-Test (
|
| 47 |
Elo 1217 [95% CI 1180β1252]
|
| 48 |
Stability 0.679
|
| 49 |
-
Validity 0.846</title></rect><rect x='649.6' y='112.9' width='22.4' height='215.1' fill='#96c5df' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='672.0' y='112.9' width='22.4' height='215.1' fill='#ecb489' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='672.0' x2='672.0' y1='97.3' y2='130.7'/><line x1='667.0' x2='677.0' y1='97.3' y2='97.3'/><line x1='667.0' x2='677.0' y1='130.7' y2='130.7'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(676.0 342.0) rotate(-35)'>MarkovBlanket</text><rect class='sa-hit' x='640.0' y='18' width='64' height='414'><title>MarkovBlanket (
|
| 50 |
Elo 1224 [95% CI 1191β1253]
|
| 51 |
Stability 0.413
|
| 52 |
-
Validity 0.463</title></rect><rect x='713.6' y='102.1' width='22.4' height='225.9' fill='#5aa4cd' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='736.0' y='102.1' width='22.4' height='225.9' fill='#de8137' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='736.0' x2='736.0' y1='83.2' y2='117.2'/><line x1='731.0' x2='741.0' y1='83.2' y2='83.2'/><line x1='731.0' x2='741.0' y1='117.2' y2='117.2'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(740.0 342.0) rotate(-35)'>MI</text><rect class='sa-hit' x='704.0' y='18' width='64' height='414'><title>MI (
|
| 53 |
Elo 1244 [95% CI 1216β1279]
|
| 54 |
Stability 0.647
|
| 55 |
-
Validity 0.785</title></rect><rect x='777.6' y='102.1' width='22.4' height='225.9' fill='#a3cce3' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='800.0' y='102.1' width='22.4' height='225.9' fill='#e08945' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='800.0' x2='800.0' y1='82.2' y2='121.0'/><line x1='795.0' x2='805.0' y1='82.2' y2='82.2'/><line x1='795.0' x2='805.0' y1='121.0' y2='121.0'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(804.0 342.0) rotate(-35)'>Lasso</text><rect class='sa-hit' x='768.0' y='18' width='64' height='414'><title>Lasso (
|
| 56 |
Elo 1244 [95% CI 1209β1281]
|
| 57 |
Stability 0.362
|
| 58 |
-
Validity 0.730</title></rect><rect x='841.6' y='97.8' width='22.4' height='230.2' fill='#8dc0dd' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='864.0' y='97.8' width='22.4' height='230.2' fill='#df853d' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='864.0' x2='864.0' y1='78.4' y2='116.7'/><line x1='859.0' x2='869.0' y1='78.4' y2='78.4'/><line x1='859.0' x2='869.0' y1='116.7' y2='116.7'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(868.0 342.0) rotate(-35)'>ElasticNet</text><rect class='sa-hit' x='832.0' y='18' width='64' height='414'><title>ElasticNet (
|
| 59 |
Elo 1252 [95% CI 1217β1288]
|
| 60 |
Stability 0.447
|
| 61 |
-
Validity 0.760</title></rect><rect x='905.6' y='89.7' width='22.4' height='238.3' fill='#93c3df' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='928.0' y='89.7' width='22.4' height='238.3' fill='#e6a16a' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='928.0' x2='928.0' y1='70.3' y2='109.1'/><line x1='923.0' x2='933.0' y1='70.3' y2='70.3'/><line x1='923.0' x2='933.0' y1='109.1' y2='109.1'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(932.0 342.0) rotate(-35)'>CART</text><rect class='sa-hit' x='896.0' y='18' width='64' height='414'><title>CART (
|
| 62 |
Elo 1267 [95% CI 1231β1303]
|
| 63 |
Stability 0.422
|
| 64 |
-
Validity 0.586</title></rect><rect x='969.6' y='64.9' width='22.4' height='263.1' fill='#5ba4cd' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='992.0' y='64.9' width='22.4' height='263.1' fill='#e39354' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='992.0' x2='992.0' y1='42.3' y2='86.5'/><line x1='987.0' x2='997.0' y1='42.3' y2='42.3'/><line x1='987.0' x2='997.0' y1='86.5' y2='86.5'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(996.0 342.0) rotate(-35)'>RFImportance</text><rect class='sa-hit' x='960.0' y='18' width='64' height='414'><title>RFImportance (
|
| 65 |
Elo 1313 [95% CI 1273β1355]
|
| 66 |
Stability 0.643
|
| 67 |
-
Validity 0.669</title></rect><line class='sa-ink' x1='64' x2='1024' y1='328.0' y2='328.0' stroke-width='1' opacity='0.4'/><rect x='1070' y='40.0' width='12' height='107.0' fill='url(#sa-g-stab)' stroke='#000' stroke-width='0.6'/><text class='sa-text' x='1076' y='30.0' text-anchor='middle' font-size='12'>stable</text><text class='sa-text' x='1076' y='165.0' text-anchor='middle' font-size='12'>unstable</text><text class='sa-muted' x='1088' y='151.0' font-size='11'>0.0</text><text class='sa-muted' x='1088' y='97.5' font-size='11'>0.5</text><text class='sa-muted' x='1088' y='44.0' font-size='11'>1.0</text><rect x='1070' y='207.0' width='12' height='107.0' fill='url(#sa-g-val)' stroke='#000' stroke-width='0.6'/><text class='sa-text' x='1076' y='197.0' text-anchor='middle' font-size='12'>valid</text><text class='sa-text' x='1076' y='332.0' text-anchor='middle' font-size='12'>invalid</text><text class='sa-muted' x='1088' y='318.0' font-size='11'>0.0</text><text class='sa-muted' x='1088' y='264.5' font-size='11'>0.5</text><text class='sa-muted' x='1088' y='211.0' font-size='11'>1.0</text><rect x='1070' y='352.0' width='12' height='12' fill='url(#sa-na)' stroke='#000' stroke-width='0.6'/><text class='sa-muted' x='1088' y='362.0' font-size='11'>no result</text></svg><script type='text/plain' id='sa-csv'>Method,
|
| 68 |
-
RFImportance,
|
| 69 |
-
CART,
|
| 70 |
-
ElasticNet,
|
| 71 |
-
MI,
|
| 72 |
-
Lasso,
|
| 73 |
-
MarkovBlanket,
|
| 74 |
-
F-Test,
|
| 75 |
-
mRMR,
|
| 76 |
-
SFS,
|
| 77 |
-
(R)ReliefF,
|
| 78 |
-
SBE,
|
| 79 |
-
GainRatio,
|
| 80 |
-
LOCO,
|
| 81 |
-
Random,Baseline,1000.0,960.0,1034.0,0.001,0.5
|
| 82 |
-
LaplacianScore,
|
| 83 |
</script><script>
|
| 84 |
function saDownload(name, blob) {
|
| 85 |
var a = document.createElement('a');
|
|
|
|
| 19 |
.sa-ink { stroke:var(--fg); }
|
| 20 |
.sa-hit { fill:transparent; }
|
| 21 |
.sa-hit:hover { fill:var(--fg); fill-opacity:0.06; }
|
| 22 |
+
</style><defs><pattern id='sa-na' width='6' height='6' patternUnits='userSpaceOnUse' patternTransform='rotate(45)'><rect width='6' height='6' fill='#e5e7eb'/><line x1='0' y1='0' x2='0' y2='6' stroke='#9ca3af' stroke-width='2'/></pattern><linearGradient id='sa-g-stab' x1='0' y1='1' x2='0' y2='0'><stop offset='0' stop-color='#ffffff'/><stop offset='1' stop-color='#0072B2'/></linearGradient><linearGradient id='sa-g-val' x1='0' y1='1' x2='0' y2='0'><stop offset='0' stop-color='#ffffff'/><stop offset='1' stop-color='#D55E00'/></linearGradient></defs><line class='sa-grid' x1='64' x2='1024' y1='287.6' y2='287.6' stroke-width='1'/><text class='sa-muted' x='56' y='291.6' text-anchor='end' font-size='12'>900</text><line class='sa-grid' x1='64' x2='1024' y1='233.7' y2='233.7' stroke-width='1'/><text class='sa-muted' x='56' y='237.7' text-anchor='end' font-size='12'>1000</text><line class='sa-grid' x1='64' x2='1024' y1='179.7' y2='179.7' stroke-width='1'/><text class='sa-muted' x='56' y='183.7' text-anchor='end' font-size='12'>1100</text><line class='sa-grid' x1='64' x2='1024' y1='125.8' y2='125.8' stroke-width='1'/><text class='sa-muted' x='56' y='129.8' text-anchor='end' font-size='12'>1200</text><line class='sa-grid' x1='64' x2='1024' y1='71.9' y2='71.9' stroke-width='1'/><text class='sa-muted' x='56' y='75.9' text-anchor='end' font-size='12'>1300</text><line class='sa-grid' x1='64' x2='1024' y1='18.0' y2='18.0' stroke-width='1'/><text class='sa-muted' x='56' y='22.0' text-anchor='end' font-size='12'>1400</text><text class='sa-text' transform='translate(16 173) rotate(-90)' text-anchor='middle' font-size='13'>Elo</text><line class='sa-ink' x1='64' x2='1024' y1='233.7' y2='233.7' stroke-width='1' stroke-dasharray='4 4' opacity='0.5'/><rect x='73.6' y='264.4' width='22.4' height='63.6' fill='#59a3cd' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='96.0' y='264.4' width='22.4' height='63.6' fill='#de7f34' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='96.0' x2='96.0' y1='233.7' y2='299.4'/><line x1='91.0' x2='101.0' y1='233.7' y2='233.7'/><line x1='91.0' x2='101.0' y1='299.4' y2='299.4'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(100.0 342.0) rotate(-35)'>LaplacianScore</text><rect class='sa-hit' x='64.0' y='18' width='64' height='414'><title>LaplacianScore (Distance, non-search)
|
| 23 |
Elo 943 [95% CI 878β1000]
|
| 24 |
Stability 0.652
|
| 25 |
+
Validity 0.797</title></rect><rect x='137.6' y='233.7' width='22.4' height='94.3' fill='#ffffff' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='160.0' y='233.7' width='22.4' height='94.3' fill='#eaae80' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='160.0' x2='160.0' y1='215.3' y2='255.2'/><line x1='155.0' x2='165.0' y1='215.3' y2='215.3'/><line x1='155.0' x2='165.0' y1='255.2' y2='255.2'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(164.0 342.0) rotate(-35)'>Random</text><rect class='sa-hit' x='128.0' y='18' width='64' height='414'><title>Random (Baseline, baseline)
|
| 26 |
Elo 1000 [95% CI 960β1034]
|
| 27 |
Stability 0.001
|
| 28 |
+
Validity 0.500</title></rect><rect x='201.6' y='148.5' width='22.4' height='179.5' fill='#d3e7f2' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='224.0' y='148.5' width='22.4' height='179.5' fill='#e7a16a' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='224.0' x2='224.0' y1='125.3' y2='170.6'/><line x1='219.0' x2='229.0' y1='125.3' y2='125.3'/><line x1='219.0' x2='229.0' y1='170.6' y2='170.6'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(228.0 342.0) rotate(-35)'>LOCO</text><rect class='sa-hit' x='192.0' y='18' width='64' height='414'><title>LOCO (Leave-one-out, search)
|
| 29 |
Elo 1158 [95% CI 1117β1201]
|
| 30 |
Stability 0.173
|
| 31 |
+
Validity 0.583</title></rect><rect x='265.6' y='136.1' width='22.4' height='191.9' fill='#5ba4cd' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='288.0' y='136.1' width='22.4' height='191.9' fill='#dc7a2d' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='288.0' x2='288.0' y1='118.8' y2='153.9'/><line x1='283.0' x2='293.0' y1='118.8' y2='118.8'/><line x1='283.0' x2='293.0' y1='153.9' y2='153.9'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(292.0 342.0) rotate(-35)'>GainRatio</text><rect class='sa-hit' x='256.0' y='18' width='64' height='414'><title>GainRatio (Information theory, non-search)
|
| 32 |
Elo 1181 [95% CI 1148β1213]
|
| 33 |
Stability 0.644
|
| 34 |
+
Validity 0.825</title></rect><rect x='329.6' y='134.5' width='22.4' height='193.5' fill='#ddecf5' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='352.0' y='134.5' width='22.4' height='193.5' fill='#e7a570' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='352.0' x2='352.0' y1='109.7' y2='157.1'/><line x1='347.0' x2='357.0' y1='109.7' y2='109.7'/><line x1='347.0' x2='357.0' y1='157.1' y2='157.1'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(356.0 342.0) rotate(-35)'>SBE</text><rect class='sa-hit' x='320.0' y='18' width='64' height='414'><title>SBE (Backward search, search)
|
| 35 |
Elo 1184 [95% CI 1142β1230]
|
| 36 |
Stability 0.132
|
| 37 |
+
Validity 0.560</title></rect><rect x='393.6' y='133.9' width='22.4' height='194.1' fill='#5aa4cd' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='416.0' y='133.9' width='22.4' height='194.1' fill='#e08842' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='416.0' x2='416.0' y1='115.0' y2='153.9'/><line x1='411.0' x2='421.0' y1='115.0' y2='115.0'/><line x1='411.0' x2='421.0' y1='153.9' y2='153.9'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(420.0 342.0) rotate(-35)'>(R)ReliefF</text><rect class='sa-hit' x='384.0' y='18' width='64' height='414'><title>(R)ReliefF (Distance, non-search)
|
| 38 |
Elo 1185 [95% CI 1148β1220]
|
| 39 |
Stability 0.646
|
| 40 |
+
Validity 0.740</title></rect><rect x='457.6' y='130.1' width='22.4' height='197.9' fill='#bbd9ea' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='480.0' y='130.1' width='22.4' height='197.9' fill='#e59a5f' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='480.0' x2='480.0' y1='109.1' y2='148.5'/><line x1='475.0' x2='485.0' y1='109.1' y2='109.1'/><line x1='475.0' x2='485.0' y1='148.5' y2='148.5'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(484.0 342.0) rotate(-35)'>SFS</text><rect class='sa-hit' x='448.0' y='18' width='64' height='414'><title>SFS (Forward search, search)
|
| 41 |
Elo 1192 [95% CI 1158β1231]
|
| 42 |
Stability 0.267
|
| 43 |
+
Validity 0.627</title></rect><rect x='521.6' y='125.3' width='22.4' height='202.7' fill='#65aad0' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='544.0' y='125.3' width='22.4' height='202.7' fill='#e39658' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='544.0' x2='544.0' y1='108.6' y2='143.6'/><line x1='539.0' x2='549.0' y1='108.6' y2='108.6'/><line x1='539.0' x2='549.0' y1='143.6' y2='143.6'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(548.0 342.0) rotate(-35)'>mRMR</text><rect class='sa-hit' x='512.0' y='18' width='64' height='414'><title>mRMR (Information theory, non-search)
|
| 44 |
Elo 1201 [95% CI 1167β1232]
|
| 45 |
Stability 0.604
|
| 46 |
+
Validity 0.655</title></rect><rect x='585.6' y='116.7' width='22.4' height='211.3' fill='#529fcb' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='608.0' y='116.7' width='22.4' height='211.3' fill='#db7727' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='608.0' x2='608.0' y1='97.8' y2='136.6'/><line x1='603.0' x2='613.0' y1='97.8' y2='97.8'/><line x1='603.0' x2='613.0' y1='136.6' y2='136.6'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(612.0 342.0) rotate(-35)'>F-Test</text><rect class='sa-hit' x='576.0' y='18' width='64' height='414'><title>F-Test (Correlation, non-search)
|
| 47 |
Elo 1217 [95% CI 1180β1252]
|
| 48 |
Stability 0.679
|
| 49 |
+
Validity 0.846</title></rect><rect x='649.6' y='112.9' width='22.4' height='215.1' fill='#96c5df' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='672.0' y='112.9' width='22.4' height='215.1' fill='#ecb489' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='672.0' x2='672.0' y1='97.3' y2='130.7'/><line x1='667.0' x2='677.0' y1='97.3' y2='97.3'/><line x1='667.0' x2='677.0' y1='130.7' y2='130.7'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(676.0 342.0) rotate(-35)'>MarkovBlanket</text><rect class='sa-hit' x='640.0' y='18' width='64' height='414'><title>MarkovBlanket (Backward search, search)
|
| 50 |
Elo 1224 [95% CI 1191β1253]
|
| 51 |
Stability 0.413
|
| 52 |
+
Validity 0.463</title></rect><rect x='713.6' y='102.1' width='22.4' height='225.9' fill='#5aa4cd' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='736.0' y='102.1' width='22.4' height='225.9' fill='#de8137' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='736.0' x2='736.0' y1='83.2' y2='117.2'/><line x1='731.0' x2='741.0' y1='83.2' y2='83.2'/><line x1='731.0' x2='741.0' y1='117.2' y2='117.2'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(740.0 342.0) rotate(-35)'>MI</text><rect class='sa-hit' x='704.0' y='18' width='64' height='414'><title>MI (Information theory, non-search)
|
| 53 |
Elo 1244 [95% CI 1216β1279]
|
| 54 |
Stability 0.647
|
| 55 |
+
Validity 0.785</title></rect><rect x='777.6' y='102.1' width='22.4' height='225.9' fill='#a3cce3' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='800.0' y='102.1' width='22.4' height='225.9' fill='#e08945' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='800.0' x2='800.0' y1='82.2' y2='121.0'/><line x1='795.0' x2='805.0' y1='82.2' y2='82.2'/><line x1='795.0' x2='805.0' y1='121.0' y2='121.0'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(804.0 342.0) rotate(-35)'>Lasso</text><rect class='sa-hit' x='768.0' y='18' width='64' height='414'><title>Lasso (Regularization, non-search)
|
| 56 |
Elo 1244 [95% CI 1209β1281]
|
| 57 |
Stability 0.362
|
| 58 |
+
Validity 0.730</title></rect><rect x='841.6' y='97.8' width='22.4' height='230.2' fill='#8dc0dd' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='864.0' y='97.8' width='22.4' height='230.2' fill='#df853d' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='864.0' x2='864.0' y1='78.4' y2='116.7'/><line x1='859.0' x2='869.0' y1='78.4' y2='78.4'/><line x1='859.0' x2='869.0' y1='116.7' y2='116.7'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(868.0 342.0) rotate(-35)'>ElasticNet</text><rect class='sa-hit' x='832.0' y='18' width='64' height='414'><title>ElasticNet (Regularization, non-search)
|
| 59 |
Elo 1252 [95% CI 1217β1288]
|
| 60 |
Stability 0.447
|
| 61 |
+
Validity 0.760</title></rect><rect x='905.6' y='89.7' width='22.4' height='238.3' fill='#93c3df' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='928.0' y='89.7' width='22.4' height='238.3' fill='#e6a16a' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='928.0' x2='928.0' y1='70.3' y2='109.1'/><line x1='923.0' x2='933.0' y1='70.3' y2='70.3'/><line x1='923.0' x2='933.0' y1='109.1' y2='109.1'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(932.0 342.0) rotate(-35)'>CART</text><rect class='sa-hit' x='896.0' y='18' width='64' height='414'><title>CART (Tree, non-search)
|
| 62 |
Elo 1267 [95% CI 1231β1303]
|
| 63 |
Stability 0.422
|
| 64 |
+
Validity 0.586</title></rect><rect x='969.6' y='64.9' width='22.4' height='263.1' fill='#5ba4cd' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='992.0' y='64.9' width='22.4' height='263.1' fill='#e39354' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='992.0' x2='992.0' y1='42.3' y2='86.5'/><line x1='987.0' x2='997.0' y1='42.3' y2='42.3'/><line x1='987.0' x2='997.0' y1='86.5' y2='86.5'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(996.0 342.0) rotate(-35)'>RFImportance</text><rect class='sa-hit' x='960.0' y='18' width='64' height='414'><title>RFImportance (Tree, non-search)
|
| 65 |
Elo 1313 [95% CI 1273β1355]
|
| 66 |
Stability 0.643
|
| 67 |
+
Validity 0.669</title></rect><line class='sa-ink' x1='64' x2='1024' y1='328.0' y2='328.0' stroke-width='1' opacity='0.4'/><rect x='1070' y='40.0' width='12' height='107.0' fill='url(#sa-g-stab)' stroke='#000' stroke-width='0.6'/><text class='sa-text' x='1076' y='30.0' text-anchor='middle' font-size='12'>stable</text><text class='sa-text' x='1076' y='165.0' text-anchor='middle' font-size='12'>unstable</text><text class='sa-muted' x='1088' y='151.0' font-size='11'>0.0</text><text class='sa-muted' x='1088' y='97.5' font-size='11'>0.5</text><text class='sa-muted' x='1088' y='44.0' font-size='11'>1.0</text><rect x='1070' y='207.0' width='12' height='107.0' fill='url(#sa-g-val)' stroke='#000' stroke-width='0.6'/><text class='sa-text' x='1076' y='197.0' text-anchor='middle' font-size='12'>valid</text><text class='sa-text' x='1076' y='332.0' text-anchor='middle' font-size='12'>invalid</text><text class='sa-muted' x='1088' y='318.0' font-size='11'>0.0</text><text class='sa-muted' x='1088' y='264.5' font-size='11'>0.5</text><text class='sa-muted' x='1088' y='211.0' font-size='11'>1.0</text><rect x='1070' y='352.0' width='12' height='12' fill='url(#sa-na)' stroke='#000' stroke-width='0.6'/><text class='sa-muted' x='1088' y='362.0' font-size='11'>no result</text></svg><script type='text/plain' id='sa-csv'>Method,Category,Subgroup,Elo,Elo CI low,Elo CI high,Stability,Validity
|
| 68 |
+
RFImportance,Non-search,Tree,1313.0,1273.0,1355.0,0.643,0.669
|
| 69 |
+
CART,Non-search,Tree,1267.0,1231.0,1303.0,0.422,0.586
|
| 70 |
+
ElasticNet,Non-search,Regularization,1252.0,1217.0,1288.0,0.447,0.76
|
| 71 |
+
MI,Non-search,Information theory,1244.0,1216.0,1279.0,0.647,0.785
|
| 72 |
+
Lasso,Non-search,Regularization,1244.0,1209.0,1281.0,0.362,0.73
|
| 73 |
+
MarkovBlanket,Search,Backward search,1224.0,1191.0,1253.0,0.413,0.463
|
| 74 |
+
F-Test,Non-search,Correlation,1217.0,1180.0,1252.0,0.679,0.846
|
| 75 |
+
mRMR,Non-search,Information theory,1201.0,1167.0,1232.0,0.604,0.655
|
| 76 |
+
SFS,Search,Forward search,1192.0,1158.0,1231.0,0.267,0.627
|
| 77 |
+
(R)ReliefF,Non-search,Distance,1185.0,1148.0,1220.0,0.646,0.74
|
| 78 |
+
SBE,Search,Backward search,1184.0,1142.0,1230.0,0.132,0.56
|
| 79 |
+
GainRatio,Non-search,Information theory,1181.0,1148.0,1213.0,0.644,0.825
|
| 80 |
+
LOCO,Search,Leave-one-out,1158.0,1117.0,1201.0,0.173,0.583
|
| 81 |
+
Random,Baseline,Baseline,1000.0,960.0,1034.0,0.001,0.5
|
| 82 |
+
LaplacianScore,Non-search,Distance,943.0,878.0,1000.0,0.652,0.797
|
| 83 |
</script><script>
|
| 84 |
function saDownload(name, blob) {
|
| 85 |
var a = document.createElement('a');
|
data/subsets/tasks_all/models_lightgbm/budgets_all/leaderboard_table.html
CHANGED
|
@@ -26,23 +26,25 @@ td.method { font-weight:600; }
|
|
| 26 |
td .sub { color:var(--muted); font-size:12px; font-weight:400; margin-left:6px; }
|
| 27 |
td.na { color:var(--muted); }
|
| 28 |
.pill { display:inline-block; padding:0 8px; border:1px solid var(--line); border-radius:999px;
|
| 29 |
-
font-size:12px; color:var(--
|
| 30 |
-
</style></head><body><div class='toolbar'><span class='hint'>Click a column to sort. Hover a header for its definition.</span><button onclick="saCsv('selectarena_leaderboard.csv')">CSV</button></div><div class='wrap'><table id='sa-table'><thead><tr><th class='num'>#</th><th class=''>Method</th><th class='' title='Filter: scores features from the data alone. Wrapper: searches feature subsets by refitting a proxy model. Embedded: reads the selection off a fitted model.'>Type</th><th class='num' title='Bradley-Terry Elo from pairwise matches within each dataset, budget and downstream model. Random = 1000. +/β give the 95% bootstrap interval.'>Elo β</th><th class='num' title='Mean rank of the downstream error.'>Avg. rank β</th><th class='num' title='Share of pairwise comparisons won, in %. Ties count half.'>Win rate β</th><th class='num' title='Mean relative gap (error β best error) / error to the best method, in %.'>Improvability β</th><th class='num' title='Mean ratio of the error to the same model trained on all features. Below 1: the selection helped.'>Error vs. all β</th><th class='num' title='Agreement of the selected features across bootstrap resamples (Nogueira et al., 2018).'>Stability β</th><th class='num' title='Share of the budget filled with original features when as many noise features are added.'>Validity β</th><th class='num' title='Median wall-clock time of the selection.'>Time (s) β</th><th class='num' title='Share of runs where the method ran out of time and random features filled the budget.'>Fallback (%)</th><th class='num'>#Datasets</th></tr></thead><tbody><tr><td class='num' data-sort='1'>1</td><td class='method' data-sort='RFImportance'>RFImportance</td><td class='' data-sort='Embedded'><span class='pill'>π³ Embedded</span></td><td class='num' data-sort='1313.0'>1313<span class='sub'>+42 / β40</span></td><td class='num' data-sort='6.08'>6.08</td><td class='num' data-sort='63.7'>63.7</td><td class='num' data-sort='13.4'>13.4</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.643'>0.643</td><td class='num' data-sort='0.669'>0.669</td><td class='num' data-sort='0.2'>0.2</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='2'>2</td><td class='method' data-sort='CART'>CART</td><td class='' data-sort='Embedded'><span class='pill'>π³ Embedded</span></td><td class='num' data-sort='1267.0'>1267<span class='sub'>+36 / β36</span></td><td class='num' data-sort='6.67'>6.67</td><td class='num' data-sort='59.5'>59.5</td><td class='num' data-sort='16.1'>16.1</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.422'>0.422</td><td class='num' data-sort='0.586'>0.586</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='3'>3</td><td class='method' data-sort='ElasticNet'>ElasticNet</td><td class='' data-sort='Embedded'><span class='pill'>π³ Embedded</span></td><td class='num' data-sort='1252.0'>1252<span class='sub'>+36 / β35</span></td><td class='num' data-sort='6.86'>6.86</td><td class='num' data-sort='58.2'>58.2</td><td class='num' data-sort='16.4'>16.4</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.447'>0.447</td><td class='num' data-sort='0.76'>0.760</td><td class='num' data-sort='6.1'>6.1</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='4'>4</td><td class='method' data-sort='MI'>MI</td><td class='' data-sort='Filter'><span class='pill'>π Filter</span></td><td class='num' data-sort='1244.0'>1244<span class='sub'>+35 / β28</span></td><td class='num' data-sort='7.09'>7.09</td><td class='num' data-sort='56.5'>56.5</td><td class='num' data-sort='16.6'>16.6</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.647'>0.647</td><td class='num' data-sort='0.785'>0.785</td><td class='num' data-sort='0.4'>0.4</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='5'>5</td><td class='method' data-sort='Lasso'>Lasso</td><td class='' data-sort='Embedded'><span class='pill'>π³ Embedded</span></td><td class='num' data-sort='1244.0'>1244<span class='sub'>+37 / β35</span></td><td class='num' data-sort='7.06'>7.06</td><td class='num' data-sort='56.7'>56.7</td><td class='num' data-sort='16.4'>16.4</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.362'>0.362</td><td class='num' data-sort='0.73'>0.730</td><td class='num' data-sort='1.9'>1.9</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='6'>6</td><td class='method' data-sort='MarkovBlanket'>MarkovBlanket</td><td class='' data-sort='Filter'><span class='pill'>π Filter</span></td><td class='num' data-sort='1224.0'>1224<span class='sub'>+29 / β33</span></td><td class='num' data-sort='7.48'>7.48</td><td class='num' data-sort='53.7'>53.7</td><td class='num' data-sort='17.5'>17.5</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.413'>0.413</td><td class='num' data-sort='0.463'>0.463</td><td class='num' data-sort='0.1'>0.1</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='7'>7</td><td class='method' data-sort='F-Test'>F-Test</td><td class='' data-sort='Filter'><span class='pill'>π Filter</span></td><td class='num' data-sort='1217.0'>1217<span class='sub'>+35 / β37</span></td><td class='num' data-sort='7.56'>7.56</td><td class='num' data-sort='53.2'>53.2</td><td class='num' data-sort='17.3'>17.3</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.679'>0.679</td><td class='num' data-sort='0.846'>0.846</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='8'>8</td><td class='method' data-sort='mRMR'>mRMR</td><td class='' data-sort='Filter'><span class='pill'>π Filter</span></td><td class='num' data-sort='1201.0'>1201<span class='sub'>+31 / β34</span></td><td class='num' data-sort='7.9'>7.90</td><td class='num' data-sort='50.7'>50.7</td><td class='num' data-sort='17.8'>17.8</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.604'>0.604</td><td class='num' data-sort='0.655'>0.655</td><td class='num' data-sort='0.4'>0.4</td><td class='num' data-sort='3.4'>3.4</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='9'>9</td><td class='method' data-sort='SFS'>SFS</td><td class='' data-sort='Wrapper'><span class='pill'>π Wrapper</span></td><td class='num' data-sort='1192.0'>1192<span class='sub'>+39 / β34</span></td><td class='num' data-sort='7.96'>7.96</td><td class='num' data-sort='50.3'>50.3</td><td class='num' data-sort='19.5'>19.5</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.267'>0.267</td><td class='num' data-sort='0.627'>0.627</td><td class='num' data-sort='2166.9'>2,166.9</td><td class='num' data-sort='43.8'>43.8</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='10'>10</td><td class='method' data-sort='(R)ReliefF'>(R)ReliefF</td><td class='' data-sort='Filter'><span class='pill'>π Filter</span></td><td class='num' data-sort='1185.0'>1185<span class='sub'>+35 / β37</span></td><td class='num' data-sort='8.07'>8.07</td><td class='num' data-sort='49.5'>49.5</td><td class='num' data-sort='18.9'>18.9</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.646'>0.646</td><td class='num' data-sort='0.74'>0.740</td><td class='num' data-sort='13.1'>13.1</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='11'>11</td><td class='method' data-sort='SBE'>SBE</td><td class='' data-sort='Wrapper'><span class='pill'>π Wrapper</span></td><td class='num' data-sort='1184.0'>1184<span class='sub'>+46 / β42</span></td><td class='num' data-sort='8.09'>8.09</td><td class='num' data-sort='49.4'>49.4</td><td class='num' data-sort='21.2'>21.2</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.132'>0.132</td><td class='num' data-sort='0.56'>0.560</td><td class='num' data-sort='2611.6'>2,611.6</td><td class='num' data-sort='46.2'>46.2</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='12'>12</td><td class='method' data-sort='GainRatio'>GainRatio</td><td class='' data-sort='Filter'><span class='pill'>π Filter</span></td><td class='num' data-sort='1181.0'>1181<span class='sub'>+32 / β33</span></td><td class='num' data-sort='8.25'>8.25</td><td class='num' data-sort='48.2'>48.2</td><td class='num' data-sort='19.3'>19.3</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.644'>0.644</td><td class='num' data-sort='0.825'>0.825</td><td class='num' data-sort='0.2'>0.2</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='13'>13</td><td class='method' data-sort='LOCO'>LOCO</td><td class='' data-sort='Wrapper'><span class='pill'>π Wrapper</span></td><td class='num' data-sort='1158.0'>1158<span class='sub'>+43 / β41</span></td><td class='num' data-sort='8.35'>8.35</td><td class='num' data-sort='47.5'>47.5</td><td class='num' data-sort='21.6'>21.6</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.173'>0.173</td><td class='num' data-sort='0.583'>0.583</td><td class='num' data-sort='240.9'>240.9</td><td class='num' data-sort='7.2'>7.2</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='14'>14</td><td class='method' data-sort='Random'>Random</td><td class='' data-sort='Baseline'><span class='pill'>βͺ Baseline</span></td><td class='num' data-sort='1000.0'>1000<span class='sub'>+34 / β40</span></td><td class='num' data-sort='10.78'>10.78</td><td class='num' data-sort='30.1'>30.1</td><td class='num' data-sort='30.3'>30.3</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.001'>0.001</td><td class='num' data-sort='0.5'>0.500</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='100.0'>100.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='15'>15</td><td class='method' data-sort='LaplacianScore'>LaplacianScore</td><td class='' data-sort='Filter'><span class='pill'>π Filter</span></td><td class='num' data-sort='943.0'>943<span class='sub'>+57 / β65</span></td><td class='num' data-sort='11.8'>11.80</td><td class='num' data-sort='22.8'>22.8</td><td class='num' data-sort='35.1'>35.1</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.652'>0.652</td><td class='num' data-sort='0.797'>0.797</td><td class='num' data-sort='0.2'>0.2</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr></tbody></table></div><script type='text/plain' id='sa-csv'>Rank,Type,TypeName,Method,Elo,Elo CI low,Elo CI high,Avg. rank,Win rate (%),Improvability (%),Error vs. all features,Stability,Validity,Selection time (s),Fallback rate (%),#Datasets,#Comparisons
|
| 31 |
-
|
| 32 |
-
2,π³,Embedded,CART,1267.0,1231.0,1303.0,6.67,59.5,16.1,,0.422,0.586,0.0,0.0,71,1065
|
| 33 |
-
|
| 34 |
-
|
| 35 |
-
|
| 36 |
-
|
| 37 |
-
|
| 38 |
-
|
| 39 |
-
|
| 40 |
-
|
| 41 |
-
|
| 42 |
-
|
| 43 |
-
|
| 44 |
-
|
| 45 |
-
|
|
|
|
|
|
|
| 46 |
</script><script>
|
| 47 |
function saDownload(name, blob) {
|
| 48 |
var a = document.createElement('a');
|
|
|
|
| 26 |
td .sub { color:var(--muted); font-size:12px; font-weight:400; margin-left:6px; }
|
| 27 |
td.na { color:var(--muted); }
|
| 28 |
.pill { display:inline-block; padding:0 8px; border:1px solid var(--line); border-radius:999px;
|
| 29 |
+
font-size:12px; color:var(--fg); }
|
| 30 |
+
.pill.cat-search { border-color:var(--accent); }
|
| 31 |
+
.pill.cat-baseline { color:var(--muted); }
|
| 32 |
+
</style></head><body><div class='toolbar'><span class='hint'>Click a column to sort. Hover a header for its definition.</span><button onclick="saCsv('selectarena_leaderboard.csv')">CSV</button></div><div class='wrap'><table id='sa-table'><thead><tr><th class='num'>#</th><th class=''>Method</th><th class='' title='Taxonomy of the paper (Appendix A). Search methods evaluate many feature subsets with the model (forward, backward, leave-one-out); non-search methods use a proxy instead (correlation, distance, information theory, tree, regularization).'>Type</th><th class='num' title='Bradley-Terry Elo from pairwise matches within each dataset, budget and downstream model. Random = 1000. +/β give the 95% bootstrap interval.'>Elo β</th><th class='num' title='Mean rank of the downstream error.'>Avg. rank β</th><th class='num' title='Share of pairwise comparisons won, in %. Ties count half.'>Win rate β</th><th class='num' title='Mean relative gap (error β best error) / error to the best method, in %.'>Improvability β</th><th class='num' title='Mean ratio of the error to the same model trained on all features. Below 1: the selection helped.'>Error vs. all β</th><th class='num' title='Agreement of the selected features across bootstrap resamples (Nogueira et al., 2018).'>Stability β</th><th class='num' title='Share of the budget filled with original features when as many noise features are added.'>Validity β</th><th class='num' title='Median wall-clock time of the selection.'>Time (s) β</th><th class='num' title='Share of runs where the method ran out of time and random features filled the budget.'>Fallback (%)</th><th class='num'>#Datasets</th></tr></thead><tbody><tr><td class='num' data-sort='1'>1</td><td class='method' data-sort='RFImportance'>RFImportance</td><td class='' data-sort='Tree'><span class='pill cat-non-search'>Tree</span><span class='sub'>non-search</span></td><td class='num' data-sort='1313.0'>1313<span class='sub'>+42 / β40</span></td><td class='num' data-sort='6.08'>6.08</td><td class='num' data-sort='63.7'>63.7</td><td class='num' data-sort='13.4'>13.4</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.643'>0.643</td><td class='num' data-sort='0.669'>0.669</td><td class='num' data-sort='0.2'>0.2</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='2'>2</td><td class='method' data-sort='CART'>CART</td><td class='' data-sort='Tree'><span class='pill cat-non-search'>Tree</span><span class='sub'>non-search</span></td><td class='num' data-sort='1267.0'>1267<span class='sub'>+36 / β36</span></td><td class='num' data-sort='6.67'>6.67</td><td class='num' data-sort='59.5'>59.5</td><td class='num' data-sort='16.1'>16.1</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.422'>0.422</td><td class='num' data-sort='0.586'>0.586</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='3'>3</td><td class='method' data-sort='ElasticNet'>ElasticNet</td><td class='' data-sort='Regularization'><span class='pill cat-non-search'>Regularization</span><span class='sub'>non-search</span></td><td class='num' data-sort='1252.0'>1252<span class='sub'>+36 / β35</span></td><td class='num' data-sort='6.86'>6.86</td><td class='num' data-sort='58.2'>58.2</td><td class='num' data-sort='16.4'>16.4</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.447'>0.447</td><td class='num' data-sort='0.76'>0.760</td><td class='num' data-sort='6.1'>6.1</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='4'>4</td><td class='method' data-sort='MI'>MI</td><td class='' data-sort='Information theory'><span class='pill cat-non-search'>Information theory</span><span class='sub'>non-search</span></td><td class='num' data-sort='1244.0'>1244<span class='sub'>+35 / β28</span></td><td class='num' data-sort='7.09'>7.09</td><td class='num' data-sort='56.5'>56.5</td><td class='num' data-sort='16.6'>16.6</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.647'>0.647</td><td class='num' data-sort='0.785'>0.785</td><td class='num' data-sort='0.4'>0.4</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='5'>5</td><td class='method' data-sort='Lasso'>Lasso</td><td class='' data-sort='Regularization'><span class='pill cat-non-search'>Regularization</span><span class='sub'>non-search</span></td><td class='num' data-sort='1244.0'>1244<span class='sub'>+37 / β35</span></td><td class='num' data-sort='7.06'>7.06</td><td class='num' data-sort='56.7'>56.7</td><td class='num' data-sort='16.4'>16.4</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.362'>0.362</td><td class='num' data-sort='0.73'>0.730</td><td class='num' data-sort='1.9'>1.9</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='6'>6</td><td class='method' data-sort='MarkovBlanket'>MarkovBlanket</td><td class='' data-sort='Backward search'><span class='pill cat-search'>Backward search</span><span class='sub'>search</span></td><td class='num' data-sort='1224.0'>1224<span class='sub'>+29 / β33</span></td><td class='num' data-sort='7.48'>7.48</td><td class='num' data-sort='53.7'>53.7</td><td class='num' data-sort='17.5'>17.5</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.413'>0.413</td><td class='num' data-sort='0.463'>0.463</td><td class='num' data-sort='0.1'>0.1</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='7'>7</td><td class='method' data-sort='F-Test'>F-Test</td><td class='' data-sort='Correlation'><span class='pill cat-non-search'>Correlation</span><span class='sub'>non-search</span></td><td class='num' data-sort='1217.0'>1217<span class='sub'>+35 / β37</span></td><td class='num' data-sort='7.56'>7.56</td><td class='num' data-sort='53.2'>53.2</td><td class='num' data-sort='17.3'>17.3</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.679'>0.679</td><td class='num' data-sort='0.846'>0.846</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='8'>8</td><td class='method' data-sort='mRMR'>mRMR</td><td class='' data-sort='Information theory'><span class='pill cat-non-search'>Information theory</span><span class='sub'>non-search</span></td><td class='num' data-sort='1201.0'>1201<span class='sub'>+31 / β34</span></td><td class='num' data-sort='7.9'>7.90</td><td class='num' data-sort='50.7'>50.7</td><td class='num' data-sort='17.8'>17.8</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.604'>0.604</td><td class='num' data-sort='0.655'>0.655</td><td class='num' data-sort='0.4'>0.4</td><td class='num' data-sort='3.4'>3.4</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='9'>9</td><td class='method' data-sort='SFS'>SFS</td><td class='' data-sort='Forward search'><span class='pill cat-search'>Forward search</span><span class='sub'>search</span></td><td class='num' data-sort='1192.0'>1192<span class='sub'>+39 / β34</span></td><td class='num' data-sort='7.96'>7.96</td><td class='num' data-sort='50.3'>50.3</td><td class='num' data-sort='19.5'>19.5</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.267'>0.267</td><td class='num' data-sort='0.627'>0.627</td><td class='num' data-sort='2166.9'>2,166.9</td><td class='num' data-sort='43.8'>43.8</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='10'>10</td><td class='method' data-sort='(R)ReliefF'>(R)ReliefF</td><td class='' data-sort='Distance'><span class='pill cat-non-search'>Distance</span><span class='sub'>non-search</span></td><td class='num' data-sort='1185.0'>1185<span class='sub'>+35 / β37</span></td><td class='num' data-sort='8.07'>8.07</td><td class='num' data-sort='49.5'>49.5</td><td class='num' data-sort='18.9'>18.9</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.646'>0.646</td><td class='num' data-sort='0.74'>0.740</td><td class='num' data-sort='13.1'>13.1</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='11'>11</td><td class='method' data-sort='SBE'>SBE</td><td class='' data-sort='Backward search'><span class='pill cat-search'>Backward search</span><span class='sub'>search</span></td><td class='num' data-sort='1184.0'>1184<span class='sub'>+46 / β42</span></td><td class='num' data-sort='8.09'>8.09</td><td class='num' data-sort='49.4'>49.4</td><td class='num' data-sort='21.2'>21.2</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.132'>0.132</td><td class='num' data-sort='0.56'>0.560</td><td class='num' data-sort='2611.6'>2,611.6</td><td class='num' data-sort='46.2'>46.2</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='12'>12</td><td class='method' data-sort='GainRatio'>GainRatio</td><td class='' data-sort='Information theory'><span class='pill cat-non-search'>Information theory</span><span class='sub'>non-search</span></td><td class='num' data-sort='1181.0'>1181<span class='sub'>+32 / β33</span></td><td class='num' data-sort='8.25'>8.25</td><td class='num' data-sort='48.2'>48.2</td><td class='num' data-sort='19.3'>19.3</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.644'>0.644</td><td class='num' data-sort='0.825'>0.825</td><td class='num' data-sort='0.2'>0.2</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='13'>13</td><td class='method' data-sort='LOCO'>LOCO</td><td class='' data-sort='Leave-one-out'><span class='pill cat-search'>Leave-one-out</span><span class='sub'>search</span></td><td class='num' data-sort='1158.0'>1158<span class='sub'>+43 / β41</span></td><td class='num' data-sort='8.35'>8.35</td><td class='num' data-sort='47.5'>47.5</td><td class='num' data-sort='21.6'>21.6</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.173'>0.173</td><td class='num' data-sort='0.583'>0.583</td><td class='num' data-sort='240.9'>240.9</td><td class='num' data-sort='7.2'>7.2</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='14'>14</td><td class='method' data-sort='Random'>Random</td><td class='' data-sort='Baseline'><span class='pill cat-baseline'>Baseline</span></td><td class='num' data-sort='1000.0'>1000<span class='sub'>+34 / β40</span></td><td class='num' data-sort='10.78'>10.78</td><td class='num' data-sort='30.1'>30.1</td><td class='num' data-sort='30.3'>30.3</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.001'>0.001</td><td class='num' data-sort='0.5'>0.500</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='100.0'>100.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='15'>15</td><td class='method' data-sort='LaplacianScore'>LaplacianScore</td><td class='' data-sort='Distance'><span class='pill cat-non-search'>Distance</span><span class='sub'>non-search</span></td><td class='num' data-sort='943.0'>943<span class='sub'>+57 / β65</span></td><td class='num' data-sort='11.8'>11.80</td><td class='num' data-sort='22.8'>22.8</td><td class='num' data-sort='35.1'>35.1</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.652'>0.652</td><td class='num' data-sort='0.797'>0.797</td><td class='num' data-sort='0.2'>0.2</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr></tbody></table></div><script type='text/plain' id='sa-csv'>Rank,Method,Category,Subgroup,Elo,Elo CI low,Elo CI high,Avg. rank,Win rate (%),Improvability (%),Error vs. all features,Stability,Validity,Selection time (s),Fallback rate (%),#Datasets,#Comparisons
|
| 33 |
+
1,RFImportance,Non-search,Tree,1313.0,1273.0,1355.0,6.08,63.7,13.4,,0.643,0.669,0.2,0.0,71,1065
|
| 34 |
+
2,CART,Non-search,Tree,1267.0,1231.0,1303.0,6.67,59.5,16.1,,0.422,0.586,0.0,0.0,71,1065
|
| 35 |
+
3,ElasticNet,Non-search,Regularization,1252.0,1217.0,1288.0,6.86,58.2,16.4,,0.447,0.76,6.1,0.0,71,1065
|
| 36 |
+
4,MI,Non-search,Information theory,1244.0,1216.0,1279.0,7.09,56.5,16.6,,0.647,0.785,0.4,0.0,71,1065
|
| 37 |
+
5,Lasso,Non-search,Regularization,1244.0,1209.0,1281.0,7.06,56.7,16.4,,0.362,0.73,1.9,0.0,71,1065
|
| 38 |
+
6,MarkovBlanket,Search,Backward search,1224.0,1191.0,1253.0,7.48,53.7,17.5,,0.413,0.463,0.1,0.0,71,1065
|
| 39 |
+
7,F-Test,Non-search,Correlation,1217.0,1180.0,1252.0,7.56,53.2,17.3,,0.679,0.846,0.0,0.0,71,1065
|
| 40 |
+
8,mRMR,Non-search,Information theory,1201.0,1167.0,1232.0,7.9,50.7,17.8,,0.604,0.655,0.4,3.4,71,1065
|
| 41 |
+
9,SFS,Search,Forward search,1192.0,1158.0,1231.0,7.96,50.3,19.5,,0.267,0.627,2166.9,43.8,71,1065
|
| 42 |
+
10,(R)ReliefF,Non-search,Distance,1185.0,1148.0,1220.0,8.07,49.5,18.9,,0.646,0.74,13.1,0.0,71,1065
|
| 43 |
+
11,SBE,Search,Backward search,1184.0,1142.0,1230.0,8.09,49.4,21.2,,0.132,0.56,2611.6,46.2,71,1065
|
| 44 |
+
12,GainRatio,Non-search,Information theory,1181.0,1148.0,1213.0,8.25,48.2,19.3,,0.644,0.825,0.2,0.0,71,1065
|
| 45 |
+
13,LOCO,Search,Leave-one-out,1158.0,1117.0,1201.0,8.35,47.5,21.6,,0.173,0.583,240.9,7.2,71,1065
|
| 46 |
+
14,Random,Baseline,Baseline,1000.0,960.0,1034.0,10.78,30.1,30.3,,0.001,0.5,0.0,100.0,71,1065
|
| 47 |
+
15,LaplacianScore,Non-search,Distance,943.0,878.0,1000.0,11.8,22.8,35.1,,0.652,0.797,0.2,0.0,71,1065
|
| 48 |
</script><script>
|
| 49 |
function saDownload(name, blob) {
|
| 50 |
var a = document.createElement('a');
|
data/subsets/tasks_all/models_lightgbm/budgets_all/website_leaderboard.csv
CHANGED
|
@@ -1,16 +1,16 @@
|
|
| 1 |
-
Rank,
|
| 2 |
-
1,
|
| 3 |
-
2,
|
| 4 |
-
3,
|
| 5 |
-
4,
|
| 6 |
-
5,
|
| 7 |
-
6,
|
| 8 |
-
7,
|
| 9 |
-
8,
|
| 10 |
-
9,
|
| 11 |
-
10,
|
| 12 |
-
11,
|
| 13 |
-
12,
|
| 14 |
-
13,
|
| 15 |
-
14,
|
| 16 |
-
15,
|
|
|
|
| 1 |
+
Rank,Method,Category,Subgroup,Elo,Elo CI low,Elo CI high,Avg. rank,Win rate (%),Improvability (%),Error vs. all features,Stability,Validity,Selection time (s),Fallback rate (%),#Datasets,#Comparisons
|
| 2 |
+
1,RFImportance,Non-search,Tree,1313.0,1273.0,1355.0,6.08,63.7,13.4,,0.643,0.669,0.2,0.0,71,1065
|
| 3 |
+
2,CART,Non-search,Tree,1267.0,1231.0,1303.0,6.67,59.5,16.1,,0.422,0.586,0.0,0.0,71,1065
|
| 4 |
+
3,ElasticNet,Non-search,Regularization,1252.0,1217.0,1288.0,6.86,58.2,16.4,,0.447,0.76,6.1,0.0,71,1065
|
| 5 |
+
4,MI,Non-search,Information theory,1244.0,1216.0,1279.0,7.09,56.5,16.6,,0.647,0.785,0.4,0.0,71,1065
|
| 6 |
+
5,Lasso,Non-search,Regularization,1244.0,1209.0,1281.0,7.06,56.7,16.4,,0.362,0.73,1.9,0.0,71,1065
|
| 7 |
+
6,MarkovBlanket,Search,Backward search,1224.0,1191.0,1253.0,7.48,53.7,17.5,,0.413,0.463,0.1,0.0,71,1065
|
| 8 |
+
7,F-Test,Non-search,Correlation,1217.0,1180.0,1252.0,7.56,53.2,17.3,,0.679,0.846,0.0,0.0,71,1065
|
| 9 |
+
8,mRMR,Non-search,Information theory,1201.0,1167.0,1232.0,7.9,50.7,17.8,,0.604,0.655,0.4,3.4,71,1065
|
| 10 |
+
9,SFS,Search,Forward search,1192.0,1158.0,1231.0,7.96,50.3,19.5,,0.267,0.627,2166.9,43.8,71,1065
|
| 11 |
+
10,(R)ReliefF,Non-search,Distance,1185.0,1148.0,1220.0,8.07,49.5,18.9,,0.646,0.74,13.1,0.0,71,1065
|
| 12 |
+
11,SBE,Search,Backward search,1184.0,1142.0,1230.0,8.09,49.4,21.2,,0.132,0.56,2611.6,46.2,71,1065
|
| 13 |
+
12,GainRatio,Non-search,Information theory,1181.0,1148.0,1213.0,8.25,48.2,19.3,,0.644,0.825,0.2,0.0,71,1065
|
| 14 |
+
13,LOCO,Search,Leave-one-out,1158.0,1117.0,1201.0,8.35,47.5,21.6,,0.173,0.583,240.9,7.2,71,1065
|
| 15 |
+
14,Random,Baseline,Baseline,1000.0,960.0,1034.0,10.78,30.1,30.3,,0.001,0.5,0.0,100.0,71,1065
|
| 16 |
+
15,LaplacianScore,Non-search,Distance,943.0,878.0,1000.0,11.8,22.8,35.1,,0.652,0.797,0.2,0.0,71,1065
|
data/subsets/tasks_all/models_linearmodel/budgets_0/elo_explorer.html
CHANGED
|
@@ -19,67 +19,67 @@ svg { display:block; width:100%; height:auto; }
|
|
| 19 |
.sa-ink { stroke:var(--fg); }
|
| 20 |
.sa-hit { fill:transparent; }
|
| 21 |
.sa-hit:hover { fill:var(--fg); fill-opacity:0.06; }
|
| 22 |
-
</style><defs><pattern id='sa-na' width='6' height='6' patternUnits='userSpaceOnUse' patternTransform='rotate(45)'><rect width='6' height='6' fill='#e5e7eb'/><line x1='0' y1='0' x2='0' y2='6' stroke='#9ca3af' stroke-width='2'/></pattern><linearGradient id='sa-g-stab' x1='0' y1='1' x2='0' y2='0'><stop offset='0' stop-color='#ffffff'/><stop offset='1' stop-color='#0072B2'/></linearGradient><linearGradient id='sa-g-val' x1='0' y1='1' x2='0' y2='0'><stop offset='0' stop-color='#ffffff'/><stop offset='1' stop-color='#D55E00'/></linearGradient></defs><line class='sa-grid' x1='64' x2='1024' y1='293.6' y2='293.6' stroke-width='1'/><text class='sa-muted' x='56' y='297.6' text-anchor='end' font-size='12'>900</text><line class='sa-grid' x1='64' x2='1024' y1='247.6' y2='247.6' stroke-width='1'/><text class='sa-muted' x='56' y='251.6' text-anchor='end' font-size='12'>1000</text><line class='sa-grid' x1='64' x2='1024' y1='201.7' y2='201.7' stroke-width='1'/><text class='sa-muted' x='56' y='205.7' text-anchor='end' font-size='12'>1100</text><line class='sa-grid' x1='64' x2='1024' y1='155.8' y2='155.8' stroke-width='1'/><text class='sa-muted' x='56' y='159.8' text-anchor='end' font-size='12'>1200</text><line class='sa-grid' x1='64' x2='1024' y1='109.9' y2='109.9' stroke-width='1'/><text class='sa-muted' x='56' y='113.9' text-anchor='end' font-size='12'>1300</text><line class='sa-grid' x1='64' x2='1024' y1='63.9' y2='63.9' stroke-width='1'/><text class='sa-muted' x='56' y='67.9' text-anchor='end' font-size='12'>1400</text><line class='sa-grid' x1='64' x2='1024' y1='18.0' y2='18.0' stroke-width='1'/><text class='sa-muted' x='56' y='22.0' text-anchor='end' font-size='12'>1500</text><text class='sa-text' transform='translate(16 173) rotate(-90)' text-anchor='middle' font-size='13'>Elo</text><line class='sa-ink' x1='64' x2='1024' y1='247.6' y2='247.6' stroke-width='1' stroke-dasharray='4 4' opacity='0.5'/><rect x='73.6' y='260.0' width='22.4' height='68.0' fill='#69acd2' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='96.0' y='260.0' width='22.4' height='68.0' fill='#db7727' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='96.0' x2='96.0' y1='230.2' y2='302.3'/><line x1='91.0' x2='101.0' y1='230.2' y2='230.2'/><line x1='91.0' x2='101.0' y1='302.3' y2='302.3'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(100.0 342.0) rotate(-35)'>LaplacianScore</text><rect class='sa-hit' x='64.0' y='18' width='64' height='414'><title>LaplacianScore (
|
| 23 |
Elo 973 [95% CI 881β1038]
|
| 24 |
Stability 0.588
|
| 25 |
-
Validity 0.847</title></rect><rect x='137.6' y='247.6' width='22.4' height='80.4' fill='#ffffff' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='160.0' y='247.6' width='22.4' height='80.4' fill='#eab081' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='160.0' x2='160.0' y1='223.3' y2='274.7'/><line x1='155.0' x2='165.0' y1='223.3' y2='223.3'/><line x1='155.0' x2='165.0' y1='274.7' y2='274.7'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(164.0 342.0) rotate(-35)'>Random</text><rect class='sa-hit' x='128.0' y='18' width='64' height='414'><title>Random (Baseline)
|
| 26 |
Elo 1000 [95% CI 941β1053]
|
| 27 |
Stability -0.001
|
| 28 |
-
Validity 0.493</title></rect><rect x='201.6' y='170.5' width='22.4' height='157.5' fill='#cde4f0' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='224.0' y='170.5' width='22.4' height='157.5' fill='#e59b61' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='224.0' x2='224.0' y1='147.1' y2='195.3'/><line x1='219.0' x2='229.0' y1='147.1' y2='147.1'/><line x1='219.0' x2='229.0' y1='195.3' y2='195.3'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(228.0 342.0) rotate(-35)'>SBE</text><rect class='sa-hit' x='192.0' y='18' width='64' height='414'><title>SBE (
|
| 29 |
Elo 1168 [95% CI 1114β1219]
|
| 30 |
Stability 0.195
|
| 31 |
-
Validity 0.621</title></rect><rect x='265.6' y='157.2' width='22.4' height='170.8' fill='#b3d5e8' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='288.0' y='157.2' width='22.4' height='170.8' fill='#e39455' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='288.0' x2='288.0' y1='135.6' y2='178.3'/><line x1='283.0' x2='293.0' y1='135.6' y2='135.6'/><line x1='283.0' x2='293.0' y1='178.3' y2='178.3'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(292.0 342.0) rotate(-35)'>LOCO</text><rect class='sa-hit' x='256.0' y='18' width='64' height='414'><title>LOCO (
|
| 32 |
Elo 1197 [95% CI 1151β1244]
|
| 33 |
Stability 0.299
|
| 34 |
-
Validity 0.665</title></rect><rect x='329.6' y='143.8' width='22.4' height='184.2' fill='#5ba4ce' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='352.0' y='143.8' width='22.4' height='184.2' fill='#da7220' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='352.0' x2='352.0' y1='118.6' y2='168.2'/><line x1='347.0' x2='357.0' y1='118.6' y2='118.6'/><line x1='347.0' x2='357.0' y1='168.2' y2='168.2'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(356.0 342.0) rotate(-35)'>(R)ReliefF</text><rect class='sa-hit' x='320.0' y='18' width='64' height='414'><title>(R)ReliefF (
|
| 35 |
Elo 1226 [95% CI 1173β1281]
|
| 36 |
Stability 0.642
|
| 37 |
-
Validity 0.873</title></rect><rect x='393.6' y='121.8' width='22.4' height='206.2' fill='#529fcb' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='416.0' y='121.8' width='22.4' height='206.2' fill='#d7670f' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='416.0' x2='416.0' y1='101.6' y2='142.9'/><line x1='411.0' x2='421.0' y1='101.6' y2='101.6'/><line x1='411.0' x2='421.0' y1='142.9' y2='142.9'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(420.0 342.0) rotate(-35)'>GainRatio</text><rect class='sa-hit' x='384.0' y='18' width='64' height='414'><title>GainRatio (
|
| 38 |
Elo 1274 [95% CI 1228β1318]
|
| 39 |
Stability 0.679
|
| 40 |
-
Validity 0.942</title></rect><rect x='457.6' y='114.0' width='22.4' height='214.0' fill='#94c4df' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='480.0' y='114.0' width='22.4' height='214.0' fill='#da701c' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='480.0' x2='480.0' y1='92.4' y2='134.7'/><line x1='475.0' x2='485.0' y1='92.4' y2='92.4'/><line x1='475.0' x2='485.0' y1='134.7' y2='134.7'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(484.0 342.0) rotate(-35)'>Lasso</text><rect class='sa-hit' x='448.0' y='18' width='64' height='414'><title>Lasso (
|
| 41 |
Elo 1291 [95% CI 1246β1338]
|
| 42 |
Stability 0.421
|
| 43 |
-
Validity 0.890</title></rect><rect x='521.6' y='112.1' width='22.4' height='215.9' fill='#a1cbe3' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='544.0' y='112.1' width='22.4' height='215.9' fill='#e08640' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='544.0' x2='544.0' y1='88.3' y2='132.8'/><line x1='539.0' x2='549.0' y1='88.3' y2='88.3'/><line x1='539.0' x2='549.0' y1='132.8' y2='132.8'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(548.0 342.0) rotate(-35)'>SFS</text><rect class='sa-hit' x='512.0' y='18' width='64' height='414'><title>SFS (
|
| 44 |
Elo 1295 [95% CI 1250β1347]
|
| 45 |
Stability 0.368
|
| 46 |
-
Validity 0.750</title></rect><rect x='585.6' y='109.4' width='22.4' height='218.6' fill='#78b5d6' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='608.0' y='109.4' width='22.4' height='218.6' fill='#de8238' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='608.0' x2='608.0' y1='86.4' y2='131.4'/><line x1='603.0' x2='613.0' y1='86.4' y2='86.4'/><line x1='603.0' x2='613.0' y1='131.4' y2='131.4'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(612.0 342.0) rotate(-35)'>mRMR</text><rect class='sa-hit' x='576.0' y='18' width='64' height='414'><title>mRMR (
|
| 47 |
Elo 1301 [95% CI 1253β1351]
|
| 48 |
Stability 0.528
|
| 49 |
-
Validity 0.779</title></rect><rect x='649.6' y='99.3' width='22.4' height='228.7' fill='#94c4df' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='672.0' y='99.3' width='22.4' height='228.7' fill='#ebb488' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='672.0' x2='672.0' y1='79.1' y2='120.0'/><line x1='667.0' x2='677.0' y1='79.1' y2='79.1'/><line x1='667.0' x2='677.0' y1='120.0' y2='120.0'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(676.0 342.0) rotate(-35)'>MarkovBlanket</text><rect class='sa-hit' x='640.0' y='18' width='64' height='414'><title>MarkovBlanket (
|
| 50 |
Elo 1323 [95% CI 1278β1367]
|
| 51 |
Stability 0.418
|
| 52 |
-
Validity 0.465</title></rect><rect x='713.6' y='98.8' width='22.4' height='229.2' fill='#5ba4cd' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='736.0' y='98.8' width='22.4' height='229.2' fill='#d96e1a' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='736.0' x2='736.0' y1='79.1' y2='118.1'/><line x1='731.0' x2='741.0' y1='79.1' y2='79.1'/><line x1='731.0' x2='741.0' y1='118.1' y2='118.1'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(740.0 342.0) rotate(-35)'>MI</text><rect class='sa-hit' x='704.0' y='18' width='64' height='414'><title>MI (
|
| 53 |
Elo 1324 [95% CI 1282β1367]
|
| 54 |
Stability 0.645
|
| 55 |
-
Validity 0.898</title></rect><rect x='777.6' y='97.5' width='22.4' height='230.5' fill='#7db7d8' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='800.0' y='97.5' width='22.4' height='230.5' fill='#d96c16' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='800.0' x2='800.0' y1='71.3' y2='119.5'/><line x1='795.0' x2='805.0' y1='71.3' y2='71.3'/><line x1='795.0' x2='805.0' y1='119.5' y2='119.5'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(804.0 342.0) rotate(-35)'>ElasticNet</text><rect class='sa-hit' x='768.0' y='18' width='64' height='414'><title>ElasticNet (
|
| 56 |
Elo 1327 [95% CI 1279β1384]
|
| 57 |
Stability 0.509
|
| 58 |
-
Validity 0.915</title></rect><rect x='841.6' y='89.6' width='22.4' height='238.4' fill='#84bbda' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='864.0' y='89.6' width='22.4' height='238.4' fill='#e08945' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='864.0' x2='864.0' y1='63.5' y2='115.8'/><line x1='859.0' x2='869.0' y1='63.5' y2='63.5'/><line x1='859.0' x2='869.0' y1='115.8' y2='115.8'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(868.0 342.0) rotate(-35)'>CART</text><rect class='sa-hit' x='832.0' y='18' width='64' height='414'><title>CART (
|
| 59 |
Elo 1344 [95% CI 1287β1401]
|
| 60 |
Stability 0.484
|
| 61 |
-
Validity 0.731</title></rect><rect x='905.6' y='86.9' width='22.4' height='241.1' fill='#4699c7' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='928.0' y='86.9' width='22.4' height='241.1' fill='#d7660c' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='928.0' x2='928.0' y1='65.8' y2='106.6'/><line x1='923.0' x2='933.0' y1='65.8' y2='65.8'/><line x1='923.0' x2='933.0' y1='106.6' y2='106.6'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(932.0 342.0) rotate(-35)'>F-Test</text><rect class='sa-hit' x='896.0' y='18' width='64' height='414'><title>F-Test (
|
| 62 |
Elo 1350 [95% CI 1307β1396]
|
| 63 |
Stability 0.725
|
| 64 |
-
Validity 0.952</title></rect><rect x='969.6' y='86.0' width='22.4' height='242.0' fill='#5fa6cf' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='992.0' y='86.0' width='22.4' height='242.0' fill='#dd7d32' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='992.0' x2='992.0' y1='62.1' y2='109.4'/><line x1='987.0' x2='997.0' y1='62.1' y2='62.1'/><line x1='987.0' x2='997.0' y1='109.4' y2='109.4'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(996.0 342.0) rotate(-35)'>RFImportance</text><rect class='sa-hit' x='960.0' y='18' width='64' height='414'><title>RFImportance (
|
| 65 |
Elo 1352 [95% CI 1301β1404]
|
| 66 |
Stability 0.628
|
| 67 |
-
Validity 0.805</title></rect><line class='sa-ink' x1='64' x2='1024' y1='328.0' y2='328.0' stroke-width='1' opacity='0.4'/><rect x='1070' y='40.0' width='12' height='107.0' fill='url(#sa-g-stab)' stroke='#000' stroke-width='0.6'/><text class='sa-text' x='1076' y='30.0' text-anchor='middle' font-size='12'>stable</text><text class='sa-text' x='1076' y='165.0' text-anchor='middle' font-size='12'>unstable</text><text class='sa-muted' x='1088' y='151.0' font-size='11'>0.0</text><text class='sa-muted' x='1088' y='97.5' font-size='11'>0.5</text><text class='sa-muted' x='1088' y='44.0' font-size='11'>1.0</text><rect x='1070' y='207.0' width='12' height='107.0' fill='url(#sa-g-val)' stroke='#000' stroke-width='0.6'/><text class='sa-text' x='1076' y='197.0' text-anchor='middle' font-size='12'>valid</text><text class='sa-text' x='1076' y='332.0' text-anchor='middle' font-size='12'>invalid</text><text class='sa-muted' x='1088' y='318.0' font-size='11'>0.0</text><text class='sa-muted' x='1088' y='264.5' font-size='11'>0.5</text><text class='sa-muted' x='1088' y='211.0' font-size='11'>1.0</text><rect x='1070' y='352.0' width='12' height='12' fill='url(#sa-na)' stroke='#000' stroke-width='0.6'/><text class='sa-muted' x='1088' y='362.0' font-size='11'>no result</text></svg><script type='text/plain' id='sa-csv'>Method,
|
| 68 |
-
RFImportance,
|
| 69 |
-
F-Test,
|
| 70 |
-
CART,
|
| 71 |
-
ElasticNet,
|
| 72 |
-
MI,
|
| 73 |
-
MarkovBlanket,
|
| 74 |
-
mRMR,
|
| 75 |
-
SFS,
|
| 76 |
-
Lasso,
|
| 77 |
-
GainRatio,
|
| 78 |
-
(R)ReliefF,
|
| 79 |
-
LOCO,
|
| 80 |
-
SBE,
|
| 81 |
-
Random,Baseline,1000.0,941.0,1053.0,-0.001,0.493
|
| 82 |
-
LaplacianScore,
|
| 83 |
</script><script>
|
| 84 |
function saDownload(name, blob) {
|
| 85 |
var a = document.createElement('a');
|
|
|
|
| 19 |
.sa-ink { stroke:var(--fg); }
|
| 20 |
.sa-hit { fill:transparent; }
|
| 21 |
.sa-hit:hover { fill:var(--fg); fill-opacity:0.06; }
|
| 22 |
+
</style><defs><pattern id='sa-na' width='6' height='6' patternUnits='userSpaceOnUse' patternTransform='rotate(45)'><rect width='6' height='6' fill='#e5e7eb'/><line x1='0' y1='0' x2='0' y2='6' stroke='#9ca3af' stroke-width='2'/></pattern><linearGradient id='sa-g-stab' x1='0' y1='1' x2='0' y2='0'><stop offset='0' stop-color='#ffffff'/><stop offset='1' stop-color='#0072B2'/></linearGradient><linearGradient id='sa-g-val' x1='0' y1='1' x2='0' y2='0'><stop offset='0' stop-color='#ffffff'/><stop offset='1' stop-color='#D55E00'/></linearGradient></defs><line class='sa-grid' x1='64' x2='1024' y1='293.6' y2='293.6' stroke-width='1'/><text class='sa-muted' x='56' y='297.6' text-anchor='end' font-size='12'>900</text><line class='sa-grid' x1='64' x2='1024' y1='247.6' y2='247.6' stroke-width='1'/><text class='sa-muted' x='56' y='251.6' text-anchor='end' font-size='12'>1000</text><line class='sa-grid' x1='64' x2='1024' y1='201.7' y2='201.7' stroke-width='1'/><text class='sa-muted' x='56' y='205.7' text-anchor='end' font-size='12'>1100</text><line class='sa-grid' x1='64' x2='1024' y1='155.8' y2='155.8' stroke-width='1'/><text class='sa-muted' x='56' y='159.8' text-anchor='end' font-size='12'>1200</text><line class='sa-grid' x1='64' x2='1024' y1='109.9' y2='109.9' stroke-width='1'/><text class='sa-muted' x='56' y='113.9' text-anchor='end' font-size='12'>1300</text><line class='sa-grid' x1='64' x2='1024' y1='63.9' y2='63.9' stroke-width='1'/><text class='sa-muted' x='56' y='67.9' text-anchor='end' font-size='12'>1400</text><line class='sa-grid' x1='64' x2='1024' y1='18.0' y2='18.0' stroke-width='1'/><text class='sa-muted' x='56' y='22.0' text-anchor='end' font-size='12'>1500</text><text class='sa-text' transform='translate(16 173) rotate(-90)' text-anchor='middle' font-size='13'>Elo</text><line class='sa-ink' x1='64' x2='1024' y1='247.6' y2='247.6' stroke-width='1' stroke-dasharray='4 4' opacity='0.5'/><rect x='73.6' y='260.0' width='22.4' height='68.0' fill='#69acd2' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='96.0' y='260.0' width='22.4' height='68.0' fill='#db7727' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='96.0' x2='96.0' y1='230.2' y2='302.3'/><line x1='91.0' x2='101.0' y1='230.2' y2='230.2'/><line x1='91.0' x2='101.0' y1='302.3' y2='302.3'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(100.0 342.0) rotate(-35)'>LaplacianScore</text><rect class='sa-hit' x='64.0' y='18' width='64' height='414'><title>LaplacianScore (Distance, non-search)
|
| 23 |
Elo 973 [95% CI 881β1038]
|
| 24 |
Stability 0.588
|
| 25 |
+
Validity 0.847</title></rect><rect x='137.6' y='247.6' width='22.4' height='80.4' fill='#ffffff' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='160.0' y='247.6' width='22.4' height='80.4' fill='#eab081' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='160.0' x2='160.0' y1='223.3' y2='274.7'/><line x1='155.0' x2='165.0' y1='223.3' y2='223.3'/><line x1='155.0' x2='165.0' y1='274.7' y2='274.7'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(164.0 342.0) rotate(-35)'>Random</text><rect class='sa-hit' x='128.0' y='18' width='64' height='414'><title>Random (Baseline, baseline)
|
| 26 |
Elo 1000 [95% CI 941β1053]
|
| 27 |
Stability -0.001
|
| 28 |
+
Validity 0.493</title></rect><rect x='201.6' y='170.5' width='22.4' height='157.5' fill='#cde4f0' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='224.0' y='170.5' width='22.4' height='157.5' fill='#e59b61' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='224.0' x2='224.0' y1='147.1' y2='195.3'/><line x1='219.0' x2='229.0' y1='147.1' y2='147.1'/><line x1='219.0' x2='229.0' y1='195.3' y2='195.3'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(228.0 342.0) rotate(-35)'>SBE</text><rect class='sa-hit' x='192.0' y='18' width='64' height='414'><title>SBE (Backward search, search)
|
| 29 |
Elo 1168 [95% CI 1114β1219]
|
| 30 |
Stability 0.195
|
| 31 |
+
Validity 0.621</title></rect><rect x='265.6' y='157.2' width='22.4' height='170.8' fill='#b3d5e8' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='288.0' y='157.2' width='22.4' height='170.8' fill='#e39455' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='288.0' x2='288.0' y1='135.6' y2='178.3'/><line x1='283.0' x2='293.0' y1='135.6' y2='135.6'/><line x1='283.0' x2='293.0' y1='178.3' y2='178.3'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(292.0 342.0) rotate(-35)'>LOCO</text><rect class='sa-hit' x='256.0' y='18' width='64' height='414'><title>LOCO (Leave-one-out, search)
|
| 32 |
Elo 1197 [95% CI 1151β1244]
|
| 33 |
Stability 0.299
|
| 34 |
+
Validity 0.665</title></rect><rect x='329.6' y='143.8' width='22.4' height='184.2' fill='#5ba4ce' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='352.0' y='143.8' width='22.4' height='184.2' fill='#da7220' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='352.0' x2='352.0' y1='118.6' y2='168.2'/><line x1='347.0' x2='357.0' y1='118.6' y2='118.6'/><line x1='347.0' x2='357.0' y1='168.2' y2='168.2'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(356.0 342.0) rotate(-35)'>(R)ReliefF</text><rect class='sa-hit' x='320.0' y='18' width='64' height='414'><title>(R)ReliefF (Distance, non-search)
|
| 35 |
Elo 1226 [95% CI 1173β1281]
|
| 36 |
Stability 0.642
|
| 37 |
+
Validity 0.873</title></rect><rect x='393.6' y='121.8' width='22.4' height='206.2' fill='#529fcb' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='416.0' y='121.8' width='22.4' height='206.2' fill='#d7670f' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='416.0' x2='416.0' y1='101.6' y2='142.9'/><line x1='411.0' x2='421.0' y1='101.6' y2='101.6'/><line x1='411.0' x2='421.0' y1='142.9' y2='142.9'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(420.0 342.0) rotate(-35)'>GainRatio</text><rect class='sa-hit' x='384.0' y='18' width='64' height='414'><title>GainRatio (Information theory, non-search)
|
| 38 |
Elo 1274 [95% CI 1228β1318]
|
| 39 |
Stability 0.679
|
| 40 |
+
Validity 0.942</title></rect><rect x='457.6' y='114.0' width='22.4' height='214.0' fill='#94c4df' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='480.0' y='114.0' width='22.4' height='214.0' fill='#da701c' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='480.0' x2='480.0' y1='92.4' y2='134.7'/><line x1='475.0' x2='485.0' y1='92.4' y2='92.4'/><line x1='475.0' x2='485.0' y1='134.7' y2='134.7'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(484.0 342.0) rotate(-35)'>Lasso</text><rect class='sa-hit' x='448.0' y='18' width='64' height='414'><title>Lasso (Regularization, non-search)
|
| 41 |
Elo 1291 [95% CI 1246β1338]
|
| 42 |
Stability 0.421
|
| 43 |
+
Validity 0.890</title></rect><rect x='521.6' y='112.1' width='22.4' height='215.9' fill='#a1cbe3' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='544.0' y='112.1' width='22.4' height='215.9' fill='#e08640' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='544.0' x2='544.0' y1='88.3' y2='132.8'/><line x1='539.0' x2='549.0' y1='88.3' y2='88.3'/><line x1='539.0' x2='549.0' y1='132.8' y2='132.8'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(548.0 342.0) rotate(-35)'>SFS</text><rect class='sa-hit' x='512.0' y='18' width='64' height='414'><title>SFS (Forward search, search)
|
| 44 |
Elo 1295 [95% CI 1250β1347]
|
| 45 |
Stability 0.368
|
| 46 |
+
Validity 0.750</title></rect><rect x='585.6' y='109.4' width='22.4' height='218.6' fill='#78b5d6' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='608.0' y='109.4' width='22.4' height='218.6' fill='#de8238' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='608.0' x2='608.0' y1='86.4' y2='131.4'/><line x1='603.0' x2='613.0' y1='86.4' y2='86.4'/><line x1='603.0' x2='613.0' y1='131.4' y2='131.4'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(612.0 342.0) rotate(-35)'>mRMR</text><rect class='sa-hit' x='576.0' y='18' width='64' height='414'><title>mRMR (Information theory, non-search)
|
| 47 |
Elo 1301 [95% CI 1253β1351]
|
| 48 |
Stability 0.528
|
| 49 |
+
Validity 0.779</title></rect><rect x='649.6' y='99.3' width='22.4' height='228.7' fill='#94c4df' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='672.0' y='99.3' width='22.4' height='228.7' fill='#ebb488' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='672.0' x2='672.0' y1='79.1' y2='120.0'/><line x1='667.0' x2='677.0' y1='79.1' y2='79.1'/><line x1='667.0' x2='677.0' y1='120.0' y2='120.0'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(676.0 342.0) rotate(-35)'>MarkovBlanket</text><rect class='sa-hit' x='640.0' y='18' width='64' height='414'><title>MarkovBlanket (Backward search, search)
|
| 50 |
Elo 1323 [95% CI 1278β1367]
|
| 51 |
Stability 0.418
|
| 52 |
+
Validity 0.465</title></rect><rect x='713.6' y='98.8' width='22.4' height='229.2' fill='#5ba4cd' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='736.0' y='98.8' width='22.4' height='229.2' fill='#d96e1a' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='736.0' x2='736.0' y1='79.1' y2='118.1'/><line x1='731.0' x2='741.0' y1='79.1' y2='79.1'/><line x1='731.0' x2='741.0' y1='118.1' y2='118.1'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(740.0 342.0) rotate(-35)'>MI</text><rect class='sa-hit' x='704.0' y='18' width='64' height='414'><title>MI (Information theory, non-search)
|
| 53 |
Elo 1324 [95% CI 1282β1367]
|
| 54 |
Stability 0.645
|
| 55 |
+
Validity 0.898</title></rect><rect x='777.6' y='97.5' width='22.4' height='230.5' fill='#7db7d8' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='800.0' y='97.5' width='22.4' height='230.5' fill='#d96c16' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='800.0' x2='800.0' y1='71.3' y2='119.5'/><line x1='795.0' x2='805.0' y1='71.3' y2='71.3'/><line x1='795.0' x2='805.0' y1='119.5' y2='119.5'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(804.0 342.0) rotate(-35)'>ElasticNet</text><rect class='sa-hit' x='768.0' y='18' width='64' height='414'><title>ElasticNet (Regularization, non-search)
|
| 56 |
Elo 1327 [95% CI 1279β1384]
|
| 57 |
Stability 0.509
|
| 58 |
+
Validity 0.915</title></rect><rect x='841.6' y='89.6' width='22.4' height='238.4' fill='#84bbda' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='864.0' y='89.6' width='22.4' height='238.4' fill='#e08945' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='864.0' x2='864.0' y1='63.5' y2='115.8'/><line x1='859.0' x2='869.0' y1='63.5' y2='63.5'/><line x1='859.0' x2='869.0' y1='115.8' y2='115.8'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(868.0 342.0) rotate(-35)'>CART</text><rect class='sa-hit' x='832.0' y='18' width='64' height='414'><title>CART (Tree, non-search)
|
| 59 |
Elo 1344 [95% CI 1287β1401]
|
| 60 |
Stability 0.484
|
| 61 |
+
Validity 0.731</title></rect><rect x='905.6' y='86.9' width='22.4' height='241.1' fill='#4699c7' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='928.0' y='86.9' width='22.4' height='241.1' fill='#d7660c' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='928.0' x2='928.0' y1='65.8' y2='106.6'/><line x1='923.0' x2='933.0' y1='65.8' y2='65.8'/><line x1='923.0' x2='933.0' y1='106.6' y2='106.6'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(932.0 342.0) rotate(-35)'>F-Test</text><rect class='sa-hit' x='896.0' y='18' width='64' height='414'><title>F-Test (Correlation, non-search)
|
| 62 |
Elo 1350 [95% CI 1307β1396]
|
| 63 |
Stability 0.725
|
| 64 |
+
Validity 0.952</title></rect><rect x='969.6' y='86.0' width='22.4' height='242.0' fill='#5fa6cf' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='992.0' y='86.0' width='22.4' height='242.0' fill='#dd7d32' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='992.0' x2='992.0' y1='62.1' y2='109.4'/><line x1='987.0' x2='997.0' y1='62.1' y2='62.1'/><line x1='987.0' x2='997.0' y1='109.4' y2='109.4'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(996.0 342.0) rotate(-35)'>RFImportance</text><rect class='sa-hit' x='960.0' y='18' width='64' height='414'><title>RFImportance (Tree, non-search)
|
| 65 |
Elo 1352 [95% CI 1301β1404]
|
| 66 |
Stability 0.628
|
| 67 |
+
Validity 0.805</title></rect><line class='sa-ink' x1='64' x2='1024' y1='328.0' y2='328.0' stroke-width='1' opacity='0.4'/><rect x='1070' y='40.0' width='12' height='107.0' fill='url(#sa-g-stab)' stroke='#000' stroke-width='0.6'/><text class='sa-text' x='1076' y='30.0' text-anchor='middle' font-size='12'>stable</text><text class='sa-text' x='1076' y='165.0' text-anchor='middle' font-size='12'>unstable</text><text class='sa-muted' x='1088' y='151.0' font-size='11'>0.0</text><text class='sa-muted' x='1088' y='97.5' font-size='11'>0.5</text><text class='sa-muted' x='1088' y='44.0' font-size='11'>1.0</text><rect x='1070' y='207.0' width='12' height='107.0' fill='url(#sa-g-val)' stroke='#000' stroke-width='0.6'/><text class='sa-text' x='1076' y='197.0' text-anchor='middle' font-size='12'>valid</text><text class='sa-text' x='1076' y='332.0' text-anchor='middle' font-size='12'>invalid</text><text class='sa-muted' x='1088' y='318.0' font-size='11'>0.0</text><text class='sa-muted' x='1088' y='264.5' font-size='11'>0.5</text><text class='sa-muted' x='1088' y='211.0' font-size='11'>1.0</text><rect x='1070' y='352.0' width='12' height='12' fill='url(#sa-na)' stroke='#000' stroke-width='0.6'/><text class='sa-muted' x='1088' y='362.0' font-size='11'>no result</text></svg><script type='text/plain' id='sa-csv'>Method,Category,Subgroup,Elo,Elo CI low,Elo CI high,Stability,Validity
|
| 68 |
+
RFImportance,Non-search,Tree,1352.0,1301.0,1404.0,0.628,0.805
|
| 69 |
+
F-Test,Non-search,Correlation,1350.0,1307.0,1396.0,0.725,0.952
|
| 70 |
+
CART,Non-search,Tree,1344.0,1287.0,1401.0,0.484,0.731
|
| 71 |
+
ElasticNet,Non-search,Regularization,1327.0,1279.0,1384.0,0.509,0.915
|
| 72 |
+
MI,Non-search,Information theory,1324.0,1282.0,1367.0,0.645,0.898
|
| 73 |
+
MarkovBlanket,Search,Backward search,1323.0,1278.0,1367.0,0.418,0.465
|
| 74 |
+
mRMR,Non-search,Information theory,1301.0,1253.0,1351.0,0.528,0.779
|
| 75 |
+
SFS,Search,Forward search,1295.0,1250.0,1347.0,0.368,0.75
|
| 76 |
+
Lasso,Non-search,Regularization,1291.0,1246.0,1338.0,0.421,0.89
|
| 77 |
+
GainRatio,Non-search,Information theory,1274.0,1228.0,1318.0,0.679,0.942
|
| 78 |
+
(R)ReliefF,Non-search,Distance,1226.0,1173.0,1281.0,0.642,0.873
|
| 79 |
+
LOCO,Search,Leave-one-out,1197.0,1151.0,1244.0,0.299,0.665
|
| 80 |
+
SBE,Search,Backward search,1168.0,1114.0,1219.0,0.195,0.621
|
| 81 |
+
Random,Baseline,Baseline,1000.0,941.0,1053.0,-0.001,0.493
|
| 82 |
+
LaplacianScore,Non-search,Distance,973.0,881.0,1038.0,0.588,0.847
|
| 83 |
</script><script>
|
| 84 |
function saDownload(name, blob) {
|
| 85 |
var a = document.createElement('a');
|
data/subsets/tasks_all/models_linearmodel/budgets_0/leaderboard_table.html
CHANGED
|
@@ -26,23 +26,25 @@ td.method { font-weight:600; }
|
|
| 26 |
td .sub { color:var(--muted); font-size:12px; font-weight:400; margin-left:6px; }
|
| 27 |
td.na { color:var(--muted); }
|
| 28 |
.pill { display:inline-block; padding:0 8px; border:1px solid var(--line); border-radius:999px;
|
| 29 |
-
font-size:12px; color:var(--
|
| 30 |
-
</style></head><body><div class='toolbar'><span class='hint'>Click a column to sort. Hover a header for its definition.</span><button onclick="saCsv('selectarena_leaderboard.csv')">CSV</button></div><div class='wrap'><table id='sa-table'><thead><tr><th class='num'>#</th><th class=''>Method</th><th class='' title='Filter: scores features from the data alone. Wrapper: searches feature subsets by refitting a proxy model. Embedded: reads the selection off a fitted model.'>Type</th><th class='num' title='Bradley-Terry Elo from pairwise matches within each dataset, budget and downstream model. Random = 1000. +/β give the 95% bootstrap interval.'>Elo β</th><th class='num' title='Mean rank of the downstream error.'>Avg. rank β</th><th class='num' title='Share of pairwise comparisons won, in %. Ties count half.'>Win rate β</th><th class='num' title='Mean relative gap (error β best error) / error to the best method, in %.'>Improvability β</th><th class='num' title='Mean ratio of the error to the same model trained on all features. Below 1: the selection helped.'>Error vs. all β</th><th class='num' title='Agreement of the selected features across bootstrap resamples (Nogueira et al., 2018).'>Stability β</th><th class='num' title='Share of the budget filled with original features when as many noise features are added.'>Validity β</th><th class='num' title='Median wall-clock time of the selection.'>Time (s) β</th><th class='num' title='Share of runs where the method ran out of time and random features filled the budget.'>Fallback (%)</th><th class='num'>#Datasets</th></tr></thead><tbody><tr><td class='num' data-sort='1'>1</td><td class='method' data-sort='RFImportance'>RFImportance</td><td class='' data-sort='Embedded'><span class='pill'>π³ Embedded</span></td><td class='num' data-sort='1352.0'>1352<span class='sub'>+52 / β51</span></td><td class='num' data-sort='6.54'>6.54</td><td class='num' data-sort='60.4'>60.4</td><td class='num' data-sort='17.7'>17.7</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.628'>0.628</td><td class='num' data-sort='0.805'>0.805</td><td class='num' data-sort='0.2'>0.2</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='2'>2</td><td class='method' data-sort='F-Test'>F-Test</td><td class='' data-sort='Filter'><span class='pill'>π Filter</span></td><td class='num' data-sort='1350.0'>1350<span class='sub'>+46 / β43</span></td><td class='num' data-sort='6.6'>6.60</td><td class='num' data-sort='60.0'>60.0</td><td class='num' data-sort='19.5'>19.5</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.725'>0.725</td><td class='num' data-sort='0.952'>0.952</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='3'>3</td><td class='method' data-sort='CART'>CART</td><td class='' data-sort='Embedded'><span class='pill'>π³ Embedded</span></td><td class='num' data-sort='1344.0'>1344<span class='sub'>+57 / β57</span></td><td class='num' data-sort='6.66'>6.66</td><td class='num' data-sort='59.6'>59.6</td><td class='num' data-sort='19.8'>19.8</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.484'>0.484</td><td class='num' data-sort='0.731'>0.731</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='4'>4</td><td class='method' data-sort='ElasticNet'>ElasticNet</td><td class='' data-sort='Embedded'><span class='pill'>π³ Embedded</span></td><td class='num' data-sort='1327.0'>1327<span class='sub'>+57 / β48</span></td><td class='num' data-sort='6.35'>6.35</td><td class='num' data-sort='61.8'>61.8</td><td class='num' data-sort='20.5'>20.5</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.509'>0.509</td><td class='num' data-sort='0.915'>0.915</td><td class='num' data-sort='6.0'>6.0</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='5'>5</td><td class='method' data-sort='MI'>MI</td><td class='' data-sort='Filter'><span class='pill'>π Filter</span></td><td class='num' data-sort='1324.0'>1324<span class='sub'>+43 / β42</span></td><td class='num' data-sort='6.75'>6.75</td><td class='num' data-sort='58.9'>58.9</td><td class='num' data-sort='20.7'>20.7</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.645'>0.645</td><td class='num' data-sort='0.898'>0.898</td><td class='num' data-sort='0.4'>0.4</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='6'>6</td><td class='method' data-sort='MarkovBlanket'>MarkovBlanket</td><td class='' data-sort='Filter'><span class='pill'>π Filter</span></td><td class='num' data-sort='1323.0'>1323<span class='sub'>+44 / β45</span></td><td class='num' data-sort='6.9'>6.90</td><td class='num' data-sort='57.8'>57.8</td><td class='num' data-sort='20.1'>20.1</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.418'>0.418</td><td class='num' data-sort='0.465'>0.465</td><td class='num' data-sort='0.1'>0.1</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='7'>7</td><td class='method' data-sort='mRMR'>mRMR</td><td class='' data-sort='Filter'><span class='pill'>π Filter</span></td><td class='num' data-sort='1301.0'>1301<span class='sub'>+50 / β48</span></td><td class='num' data-sort='7.44'>7.44</td><td class='num' data-sort='54.0'>54.0</td><td class='num' data-sort='20.4'>20.4</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.528'>0.528</td><td class='num' data-sort='0.779'>0.779</td><td class='num' data-sort='0.3'>0.3</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='8'>8</td><td class='method' data-sort='SFS'>SFS</td><td class='' data-sort='Wrapper'><span class='pill'>π Wrapper</span></td><td class='num' data-sort='1295.0'>1295<span class='sub'>+52 / β45</span></td><td class='num' data-sort='7.39'>7.39</td><td class='num' data-sort='54.4'>54.4</td><td class='num' data-sort='22.7'>22.7</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.368'>0.368</td><td class='num' data-sort='0.75'>0.750</td><td class='num' data-sort='970.7'>970.7</td><td class='num' data-sort='29.1'>29.1</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='9'>9</td><td class='method' data-sort='Lasso'>Lasso</td><td class='' data-sort='Embedded'><span class='pill'>π³ Embedded</span></td><td class='num' data-sort='1291.0'>1291<span class='sub'>+47 / β45</span></td><td class='num' data-sort='7.02'>7.02</td><td class='num' data-sort='57.0'>57.0</td><td class='num' data-sort='22.1'>22.1</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.421'>0.421</td><td class='num' data-sort='0.89'>0.890</td><td class='num' data-sort='1.9'>1.9</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='10'>10</td><td class='method' data-sort='GainRatio'>GainRatio</td><td class='' data-sort='Filter'><span class='pill'>π Filter</span></td><td class='num' data-sort='1274.0'>1274<span class='sub'>+44 / β46</span></td><td class='num' data-sort='7.81'>7.81</td><td class='num' data-sort='51.3'>51.3</td><td class='num' data-sort='24.0'>24.0</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.679'>0.679</td><td class='num' data-sort='0.942'>0.942</td><td class='num' data-sort='0.2'>0.2</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='11'>11</td><td class='method' data-sort='(R)ReliefF'>(R)ReliefF</td><td class='' data-sort='Filter'><span class='pill'>π Filter</span></td><td class='num' data-sort='1226.0'>1226<span class='sub'>+55 / β53</span></td><td class='num' data-sort='8.6'>8.60</td><td class='num' data-sort='45.7'>45.7</td><td class='num' data-sort='25.1'>25.1</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.642'>0.642</td><td class='num' data-sort='0.873'>0.873</td><td class='num' data-sort='13.7'>13.7</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='12'>12</td><td class='method' data-sort='LOCO'>LOCO</td><td class='' data-sort='Wrapper'><span class='pill'>π Wrapper</span></td><td class='num' data-sort='1197.0'>1197<span class='sub'>+47 / β46</span></td><td class='num' data-sort='8.73'>8.73</td><td class='num' data-sort='44.8'>44.8</td><td class='num' data-sort='27.7'>27.7</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.299'>0.299</td><td class='num' data-sort='0.665'>0.665</td><td class='num' data-sort='245.6'>245.6</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='13'>13</td><td class='method' data-sort='SBE'>SBE</td><td class='' data-sort='Wrapper'><span class='pill'>π Wrapper</span></td><td class='num' data-sort='1168.0'>1168<span class='sub'>+51 / β54</span></td><td class='num' data-sort='9.36'>9.36</td><td class='num' data-sort='40.3'>40.3</td><td class='num' data-sort='30.0'>30.0</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.195'>0.195</td><td class='num' data-sort='0.621'>0.621</td><td class='num' data-sort='3600.1'>3,600.1</td><td class='num' data-sort='49.3'>49.3</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='14'>14</td><td class='method' data-sort='Random'>Random</td><td class='' data-sort='Baseline'><span class='pill'>βͺ Baseline</span></td><td class='num' data-sort='1000.0'>1000<span class='sub'>+53 / β59</span></td><td class='num' data-sort='11.53'>11.53</td><td class='num' data-sort='24.8'>24.8</td><td class='num' data-sort='40.2'>40.2</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='-0.001'>-0.001</td><td class='num' data-sort='0.493'>0.493</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='100.0'>100.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='15'>15</td><td class='method' data-sort='LaplacianScore'>LaplacianScore</td><td class='' data-sort='Filter'><span class='pill'>π Filter</span></td><td class='num' data-sort='973.0'>973<span class='sub'>+65 / β92</span></td><td class='num' data-sort='12.31'>12.31</td><td class='num' data-sort='19.2'>19.2</td><td class='num' data-sort='42.8'>42.8</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.588'>0.588</td><td class='num' data-sort='0.847'>0.847</td><td class='num' data-sort='0.2'>0.2</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr></tbody></table></div><script type='text/plain' id='sa-csv'>Rank,Type,TypeName,Method,Elo,Elo CI low,Elo CI high,Avg. rank,Win rate (%),Improvability (%),Error vs. all features,Stability,Validity,Selection time (s),Fallback rate (%),#Datasets,#Comparisons
|
| 31 |
-
|
| 32 |
-
2,π,Filter,F-Test,1350.0,1307.0,1396.0,6.6,60.0,19.5,,0.725,0.952,0.0,0.0,71,213
|
| 33 |
-
|
| 34 |
-
|
| 35 |
-
|
| 36 |
-
|
| 37 |
-
|
| 38 |
-
|
| 39 |
-
|
| 40 |
-
|
| 41 |
-
|
| 42 |
-
|
| 43 |
-
|
| 44 |
-
|
| 45 |
-
|
|
|
|
|
|
|
| 46 |
</script><script>
|
| 47 |
function saDownload(name, blob) {
|
| 48 |
var a = document.createElement('a');
|
|
|
|
| 26 |
td .sub { color:var(--muted); font-size:12px; font-weight:400; margin-left:6px; }
|
| 27 |
td.na { color:var(--muted); }
|
| 28 |
.pill { display:inline-block; padding:0 8px; border:1px solid var(--line); border-radius:999px;
|
| 29 |
+
font-size:12px; color:var(--fg); }
|
| 30 |
+
.pill.cat-search { border-color:var(--accent); }
|
| 31 |
+
.pill.cat-baseline { color:var(--muted); }
|
| 32 |
+
</style></head><body><div class='toolbar'><span class='hint'>Click a column to sort. Hover a header for its definition.</span><button onclick="saCsv('selectarena_leaderboard.csv')">CSV</button></div><div class='wrap'><table id='sa-table'><thead><tr><th class='num'>#</th><th class=''>Method</th><th class='' title='Taxonomy of the paper (Appendix A). Search methods evaluate many feature subsets with the model (forward, backward, leave-one-out); non-search methods use a proxy instead (correlation, distance, information theory, tree, regularization).'>Type</th><th class='num' title='Bradley-Terry Elo from pairwise matches within each dataset, budget and downstream model. Random = 1000. +/β give the 95% bootstrap interval.'>Elo β</th><th class='num' title='Mean rank of the downstream error.'>Avg. rank β</th><th class='num' title='Share of pairwise comparisons won, in %. Ties count half.'>Win rate β</th><th class='num' title='Mean relative gap (error β best error) / error to the best method, in %.'>Improvability β</th><th class='num' title='Mean ratio of the error to the same model trained on all features. Below 1: the selection helped.'>Error vs. all β</th><th class='num' title='Agreement of the selected features across bootstrap resamples (Nogueira et al., 2018).'>Stability β</th><th class='num' title='Share of the budget filled with original features when as many noise features are added.'>Validity β</th><th class='num' title='Median wall-clock time of the selection.'>Time (s) β</th><th class='num' title='Share of runs where the method ran out of time and random features filled the budget.'>Fallback (%)</th><th class='num'>#Datasets</th></tr></thead><tbody><tr><td class='num' data-sort='1'>1</td><td class='method' data-sort='RFImportance'>RFImportance</td><td class='' data-sort='Tree'><span class='pill cat-non-search'>Tree</span><span class='sub'>non-search</span></td><td class='num' data-sort='1352.0'>1352<span class='sub'>+52 / β51</span></td><td class='num' data-sort='6.54'>6.54</td><td class='num' data-sort='60.4'>60.4</td><td class='num' data-sort='17.7'>17.7</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.628'>0.628</td><td class='num' data-sort='0.805'>0.805</td><td class='num' data-sort='0.2'>0.2</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='2'>2</td><td class='method' data-sort='F-Test'>F-Test</td><td class='' data-sort='Correlation'><span class='pill cat-non-search'>Correlation</span><span class='sub'>non-search</span></td><td class='num' data-sort='1350.0'>1350<span class='sub'>+46 / β43</span></td><td class='num' data-sort='6.6'>6.60</td><td class='num' data-sort='60.0'>60.0</td><td class='num' data-sort='19.5'>19.5</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.725'>0.725</td><td class='num' data-sort='0.952'>0.952</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='3'>3</td><td class='method' data-sort='CART'>CART</td><td class='' data-sort='Tree'><span class='pill cat-non-search'>Tree</span><span class='sub'>non-search</span></td><td class='num' data-sort='1344.0'>1344<span class='sub'>+57 / β57</span></td><td class='num' data-sort='6.66'>6.66</td><td class='num' data-sort='59.6'>59.6</td><td class='num' data-sort='19.8'>19.8</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.484'>0.484</td><td class='num' data-sort='0.731'>0.731</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='4'>4</td><td class='method' data-sort='ElasticNet'>ElasticNet</td><td class='' data-sort='Regularization'><span class='pill cat-non-search'>Regularization</span><span class='sub'>non-search</span></td><td class='num' data-sort='1327.0'>1327<span class='sub'>+57 / β48</span></td><td class='num' data-sort='6.35'>6.35</td><td class='num' data-sort='61.8'>61.8</td><td class='num' data-sort='20.5'>20.5</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.509'>0.509</td><td class='num' data-sort='0.915'>0.915</td><td class='num' data-sort='6.0'>6.0</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='5'>5</td><td class='method' data-sort='MI'>MI</td><td class='' data-sort='Information theory'><span class='pill cat-non-search'>Information theory</span><span class='sub'>non-search</span></td><td class='num' data-sort='1324.0'>1324<span class='sub'>+43 / β42</span></td><td class='num' data-sort='6.75'>6.75</td><td class='num' data-sort='58.9'>58.9</td><td class='num' data-sort='20.7'>20.7</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.645'>0.645</td><td class='num' data-sort='0.898'>0.898</td><td class='num' data-sort='0.4'>0.4</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='6'>6</td><td class='method' data-sort='MarkovBlanket'>MarkovBlanket</td><td class='' data-sort='Backward search'><span class='pill cat-search'>Backward search</span><span class='sub'>search</span></td><td class='num' data-sort='1323.0'>1323<span class='sub'>+44 / β45</span></td><td class='num' data-sort='6.9'>6.90</td><td class='num' data-sort='57.8'>57.8</td><td class='num' data-sort='20.1'>20.1</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.418'>0.418</td><td class='num' data-sort='0.465'>0.465</td><td class='num' data-sort='0.1'>0.1</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='7'>7</td><td class='method' data-sort='mRMR'>mRMR</td><td class='' data-sort='Information theory'><span class='pill cat-non-search'>Information theory</span><span class='sub'>non-search</span></td><td class='num' data-sort='1301.0'>1301<span class='sub'>+50 / β48</span></td><td class='num' data-sort='7.44'>7.44</td><td class='num' data-sort='54.0'>54.0</td><td class='num' data-sort='20.4'>20.4</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.528'>0.528</td><td class='num' data-sort='0.779'>0.779</td><td class='num' data-sort='0.3'>0.3</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='8'>8</td><td class='method' data-sort='SFS'>SFS</td><td class='' data-sort='Forward search'><span class='pill cat-search'>Forward search</span><span class='sub'>search</span></td><td class='num' data-sort='1295.0'>1295<span class='sub'>+52 / β45</span></td><td class='num' data-sort='7.39'>7.39</td><td class='num' data-sort='54.4'>54.4</td><td class='num' data-sort='22.7'>22.7</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.368'>0.368</td><td class='num' data-sort='0.75'>0.750</td><td class='num' data-sort='970.7'>970.7</td><td class='num' data-sort='29.1'>29.1</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='9'>9</td><td class='method' data-sort='Lasso'>Lasso</td><td class='' data-sort='Regularization'><span class='pill cat-non-search'>Regularization</span><span class='sub'>non-search</span></td><td class='num' data-sort='1291.0'>1291<span class='sub'>+47 / β45</span></td><td class='num' data-sort='7.02'>7.02</td><td class='num' data-sort='57.0'>57.0</td><td class='num' data-sort='22.1'>22.1</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.421'>0.421</td><td class='num' data-sort='0.89'>0.890</td><td class='num' data-sort='1.9'>1.9</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='10'>10</td><td class='method' data-sort='GainRatio'>GainRatio</td><td class='' data-sort='Information theory'><span class='pill cat-non-search'>Information theory</span><span class='sub'>non-search</span></td><td class='num' data-sort='1274.0'>1274<span class='sub'>+44 / β46</span></td><td class='num' data-sort='7.81'>7.81</td><td class='num' data-sort='51.3'>51.3</td><td class='num' data-sort='24.0'>24.0</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.679'>0.679</td><td class='num' data-sort='0.942'>0.942</td><td class='num' data-sort='0.2'>0.2</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='11'>11</td><td class='method' data-sort='(R)ReliefF'>(R)ReliefF</td><td class='' data-sort='Distance'><span class='pill cat-non-search'>Distance</span><span class='sub'>non-search</span></td><td class='num' data-sort='1226.0'>1226<span class='sub'>+55 / β53</span></td><td class='num' data-sort='8.6'>8.60</td><td class='num' data-sort='45.7'>45.7</td><td class='num' data-sort='25.1'>25.1</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.642'>0.642</td><td class='num' data-sort='0.873'>0.873</td><td class='num' data-sort='13.7'>13.7</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='12'>12</td><td class='method' data-sort='LOCO'>LOCO</td><td class='' data-sort='Leave-one-out'><span class='pill cat-search'>Leave-one-out</span><span class='sub'>search</span></td><td class='num' data-sort='1197.0'>1197<span class='sub'>+47 / β46</span></td><td class='num' data-sort='8.73'>8.73</td><td class='num' data-sort='44.8'>44.8</td><td class='num' data-sort='27.7'>27.7</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.299'>0.299</td><td class='num' data-sort='0.665'>0.665</td><td class='num' data-sort='245.6'>245.6</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='13'>13</td><td class='method' data-sort='SBE'>SBE</td><td class='' data-sort='Backward search'><span class='pill cat-search'>Backward search</span><span class='sub'>search</span></td><td class='num' data-sort='1168.0'>1168<span class='sub'>+51 / β54</span></td><td class='num' data-sort='9.36'>9.36</td><td class='num' data-sort='40.3'>40.3</td><td class='num' data-sort='30.0'>30.0</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.195'>0.195</td><td class='num' data-sort='0.621'>0.621</td><td class='num' data-sort='3600.1'>3,600.1</td><td class='num' data-sort='49.3'>49.3</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='14'>14</td><td class='method' data-sort='Random'>Random</td><td class='' data-sort='Baseline'><span class='pill cat-baseline'>Baseline</span></td><td class='num' data-sort='1000.0'>1000<span class='sub'>+53 / β59</span></td><td class='num' data-sort='11.53'>11.53</td><td class='num' data-sort='24.8'>24.8</td><td class='num' data-sort='40.2'>40.2</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='-0.001'>-0.001</td><td class='num' data-sort='0.493'>0.493</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='100.0'>100.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='15'>15</td><td class='method' data-sort='LaplacianScore'>LaplacianScore</td><td class='' data-sort='Distance'><span class='pill cat-non-search'>Distance</span><span class='sub'>non-search</span></td><td class='num' data-sort='973.0'>973<span class='sub'>+65 / β92</span></td><td class='num' data-sort='12.31'>12.31</td><td class='num' data-sort='19.2'>19.2</td><td class='num' data-sort='42.8'>42.8</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.588'>0.588</td><td class='num' data-sort='0.847'>0.847</td><td class='num' data-sort='0.2'>0.2</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr></tbody></table></div><script type='text/plain' id='sa-csv'>Rank,Method,Category,Subgroup,Elo,Elo CI low,Elo CI high,Avg. rank,Win rate (%),Improvability (%),Error vs. all features,Stability,Validity,Selection time (s),Fallback rate (%),#Datasets,#Comparisons
|
| 33 |
+
1,RFImportance,Non-search,Tree,1352.0,1301.0,1404.0,6.54,60.4,17.7,,0.628,0.805,0.2,0.0,71,213
|
| 34 |
+
2,F-Test,Non-search,Correlation,1350.0,1307.0,1396.0,6.6,60.0,19.5,,0.725,0.952,0.0,0.0,71,213
|
| 35 |
+
3,CART,Non-search,Tree,1344.0,1287.0,1401.0,6.66,59.6,19.8,,0.484,0.731,0.0,0.0,71,213
|
| 36 |
+
4,ElasticNet,Non-search,Regularization,1327.0,1279.0,1384.0,6.35,61.8,20.5,,0.509,0.915,6.0,0.0,71,213
|
| 37 |
+
5,MI,Non-search,Information theory,1324.0,1282.0,1367.0,6.75,58.9,20.7,,0.645,0.898,0.4,0.0,71,213
|
| 38 |
+
6,MarkovBlanket,Search,Backward search,1323.0,1278.0,1367.0,6.9,57.8,20.1,,0.418,0.465,0.1,0.0,71,213
|
| 39 |
+
7,mRMR,Non-search,Information theory,1301.0,1253.0,1351.0,7.44,54.0,20.4,,0.528,0.779,0.3,0.0,71,213
|
| 40 |
+
8,SFS,Search,Forward search,1295.0,1250.0,1347.0,7.39,54.4,22.7,,0.368,0.75,970.7,29.1,71,213
|
| 41 |
+
9,Lasso,Non-search,Regularization,1291.0,1246.0,1338.0,7.02,57.0,22.1,,0.421,0.89,1.9,0.0,71,213
|
| 42 |
+
10,GainRatio,Non-search,Information theory,1274.0,1228.0,1318.0,7.81,51.3,24.0,,0.679,0.942,0.2,0.0,71,213
|
| 43 |
+
11,(R)ReliefF,Non-search,Distance,1226.0,1173.0,1281.0,8.6,45.7,25.1,,0.642,0.873,13.7,0.0,71,213
|
| 44 |
+
12,LOCO,Search,Leave-one-out,1197.0,1151.0,1244.0,8.73,44.8,27.7,,0.299,0.665,245.6,0.0,71,213
|
| 45 |
+
13,SBE,Search,Backward search,1168.0,1114.0,1219.0,9.36,40.3,30.0,,0.195,0.621,3600.1,49.3,71,213
|
| 46 |
+
14,Random,Baseline,Baseline,1000.0,941.0,1053.0,11.53,24.8,40.2,,-0.001,0.493,0.0,100.0,71,213
|
| 47 |
+
15,LaplacianScore,Non-search,Distance,973.0,881.0,1038.0,12.31,19.2,42.8,,0.588,0.847,0.2,0.0,71,213
|
| 48 |
</script><script>
|
| 49 |
function saDownload(name, blob) {
|
| 50 |
var a = document.createElement('a');
|
data/subsets/tasks_all/models_linearmodel/budgets_0/website_leaderboard.csv
CHANGED
|
@@ -1,16 +1,16 @@
|
|
| 1 |
-
Rank,
|
| 2 |
-
1,
|
| 3 |
-
2,
|
| 4 |
-
3,
|
| 5 |
-
4,
|
| 6 |
-
5,
|
| 7 |
-
6,
|
| 8 |
-
7,
|
| 9 |
-
8,
|
| 10 |
-
9,
|
| 11 |
-
10,
|
| 12 |
-
11,
|
| 13 |
-
12,
|
| 14 |
-
13,
|
| 15 |
-
14,
|
| 16 |
-
15,
|
|
|
|
| 1 |
+
Rank,Method,Category,Subgroup,Elo,Elo CI low,Elo CI high,Avg. rank,Win rate (%),Improvability (%),Error vs. all features,Stability,Validity,Selection time (s),Fallback rate (%),#Datasets,#Comparisons
|
| 2 |
+
1,RFImportance,Non-search,Tree,1352.0,1301.0,1404.0,6.54,60.4,17.7,,0.628,0.805,0.2,0.0,71,213
|
| 3 |
+
2,F-Test,Non-search,Correlation,1350.0,1307.0,1396.0,6.6,60.0,19.5,,0.725,0.952,0.0,0.0,71,213
|
| 4 |
+
3,CART,Non-search,Tree,1344.0,1287.0,1401.0,6.66,59.6,19.8,,0.484,0.731,0.0,0.0,71,213
|
| 5 |
+
4,ElasticNet,Non-search,Regularization,1327.0,1279.0,1384.0,6.35,61.8,20.5,,0.509,0.915,6.0,0.0,71,213
|
| 6 |
+
5,MI,Non-search,Information theory,1324.0,1282.0,1367.0,6.75,58.9,20.7,,0.645,0.898,0.4,0.0,71,213
|
| 7 |
+
6,MarkovBlanket,Search,Backward search,1323.0,1278.0,1367.0,6.9,57.8,20.1,,0.418,0.465,0.1,0.0,71,213
|
| 8 |
+
7,mRMR,Non-search,Information theory,1301.0,1253.0,1351.0,7.44,54.0,20.4,,0.528,0.779,0.3,0.0,71,213
|
| 9 |
+
8,SFS,Search,Forward search,1295.0,1250.0,1347.0,7.39,54.4,22.7,,0.368,0.75,970.7,29.1,71,213
|
| 10 |
+
9,Lasso,Non-search,Regularization,1291.0,1246.0,1338.0,7.02,57.0,22.1,,0.421,0.89,1.9,0.0,71,213
|
| 11 |
+
10,GainRatio,Non-search,Information theory,1274.0,1228.0,1318.0,7.81,51.3,24.0,,0.679,0.942,0.2,0.0,71,213
|
| 12 |
+
11,(R)ReliefF,Non-search,Distance,1226.0,1173.0,1281.0,8.6,45.7,25.1,,0.642,0.873,13.7,0.0,71,213
|
| 13 |
+
12,LOCO,Search,Leave-one-out,1197.0,1151.0,1244.0,8.73,44.8,27.7,,0.299,0.665,245.6,0.0,71,213
|
| 14 |
+
13,SBE,Search,Backward search,1168.0,1114.0,1219.0,9.36,40.3,30.0,,0.195,0.621,3600.1,49.3,71,213
|
| 15 |
+
14,Random,Baseline,Baseline,1000.0,941.0,1053.0,11.53,24.8,40.2,,-0.001,0.493,0.0,100.0,71,213
|
| 16 |
+
15,LaplacianScore,Non-search,Distance,973.0,881.0,1038.0,12.31,19.2,42.8,,0.588,0.847,0.2,0.0,71,213
|
data/subsets/tasks_all/models_linearmodel/budgets_1/elo_explorer.html
CHANGED
|
@@ -19,67 +19,67 @@ svg { display:block; width:100%; height:auto; }
|
|
| 19 |
.sa-ink { stroke:var(--fg); }
|
| 20 |
.sa-hit { fill:transparent; }
|
| 21 |
.sa-hit:hover { fill:var(--fg); fill-opacity:0.06; }
|
| 22 |
-
</style><defs><pattern id='sa-na' width='6' height='6' patternUnits='userSpaceOnUse' patternTransform='rotate(45)'><rect width='6' height='6' fill='#e5e7eb'/><line x1='0' y1='0' x2='0' y2='6' stroke='#9ca3af' stroke-width='2'/></pattern><linearGradient id='sa-g-stab' x1='0' y1='1' x2='0' y2='0'><stop offset='0' stop-color='#ffffff'/><stop offset='1' stop-color='#0072B2'/></linearGradient><linearGradient id='sa-g-val' x1='0' y1='1' x2='0' y2='0'><stop offset='0' stop-color='#ffffff'/><stop offset='1' stop-color='#D55E00'/></linearGradient></defs><line class='sa-grid' x1='64' x2='1024' y1='315.6' y2='315.6' stroke-width='1'/><text class='sa-muted' x='56' y='319.6' text-anchor='end' font-size='12'>900</text><line class='sa-grid' x1='64' x2='1024' y1='266.0' y2='266.0' stroke-width='1'/><text class='sa-muted' x='56' y='270.0' text-anchor='end' font-size='12'>1000</text><line class='sa-grid' x1='64' x2='1024' y1='216.4' y2='216.4' stroke-width='1'/><text class='sa-muted' x='56' y='220.4' text-anchor='end' font-size='12'>1100</text><line class='sa-grid' x1='64' x2='1024' y1='166.8' y2='166.8' stroke-width='1'/><text class='sa-muted' x='56' y='170.8' text-anchor='end' font-size='12'>1200</text><line class='sa-grid' x1='64' x2='1024' y1='117.2' y2='117.2' stroke-width='1'/><text class='sa-muted' x='56' y='121.2' text-anchor='end' font-size='12'>1300</text><line class='sa-grid' x1='64' x2='1024' y1='67.6' y2='67.6' stroke-width='1'/><text class='sa-muted' x='56' y='71.6' text-anchor='end' font-size='12'>1400</text><line class='sa-grid' x1='64' x2='1024' y1='18.0' y2='18.0' stroke-width='1'/><text class='sa-muted' x='56' y='22.0' text-anchor='end' font-size='12'>1500</text><text class='sa-text' transform='translate(16 173) rotate(-90)' text-anchor='middle' font-size='13'>Elo</text><line class='sa-ink' x1='64' x2='1024' y1='266.0' y2='266.0' stroke-width='1' stroke-dasharray='4 4' opacity='0.5'/><rect x='73.6' y='266.0' width='22.4' height='62.0' fill='#ffffff' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='96.0' y='266.0' width='22.4' height='62.0' fill='#eaaf80' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='96.0' x2='96.0' y1='240.2' y2='297.7'/><line x1='91.0' x2='101.0' y1='240.2' y2='240.2'/><line x1='91.0' x2='101.0' y1='297.7' y2='297.7'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(100.0 342.0) rotate(-35)'>Random</text><rect class='sa-hit' x='64.0' y='18' width='64' height='414'><title>Random (Baseline)
|
| 23 |
Elo 1000 [95% CI 936β1052]
|
| 24 |
Stability 0.001
|
| 25 |
-
Validity 0.499</title></rect><rect x='137.6' y='253.1' width='22.4' height='74.9' fill='#5ba4cd' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='160.0' y='253.1' width='22.4' height='74.9' fill='#db7626' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='160.0' x2='160.0' y1='223.8' y2='288.3'/><line x1='155.0' x2='165.0' y1='223.8' y2='223.8'/><line x1='155.0' x2='165.0' y1='288.3' y2='288.3'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(164.0 342.0) rotate(-35)'>LaplacianScore</text><rect class='sa-hit' x='128.0' y='18' width='64' height='414'><title>LaplacianScore (
|
| 26 |
Elo 1026 [95% CI 955β1085]
|
| 27 |
Stability 0.644
|
| 28 |
-
Validity 0.850</title></rect><rect x='201.6' y='174.7' width='22.4' height='153.3' fill='#d2e6f2' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='224.0' y='174.7' width='22.4' height='153.3' fill='#e7a26c' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='224.0' x2='224.0' y1='149.9' y2='200.0'/><line x1='219.0' x2='229.0' y1='149.9' y2='149.9'/><line x1='219.0' x2='229.0' y1='200.0' y2='200.0'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(228.0 342.0) rotate(-35)'>SBE</text><rect class='sa-hit' x='192.0' y='18' width='64' height='414'><title>SBE (
|
| 29 |
Elo 1184 [95% CI 1133β1234]
|
| 30 |
Stability 0.175
|
| 31 |
-
Validity 0.578</title></rect><rect x='265.6' y='167.3' width='22.4' height='160.7' fill='#c7e0ee' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='288.0' y='167.3' width='22.4' height='160.7' fill='#e59c63' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='288.0' x2='288.0' y1='141.0' y2='192.1'/><line x1='283.0' x2='293.0' y1='141.0' y2='141.0'/><line x1='283.0' x2='293.0' y1='192.1' y2='192.1'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(292.0 342.0) rotate(-35)'>LOCO</text><rect class='sa-hit' x='256.0' y='18' width='64' height='414'><title>LOCO (
|
| 32 |
Elo 1199 [95% CI 1149β1252]
|
| 33 |
Stability 0.220
|
| 34 |
-
Validity 0.613</title></rect><rect x='329.6' y='158.9' width='22.4' height='169.1' fill='#5aa4cd' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='352.0' y='158.9' width='22.4' height='169.1' fill='#dd7d31' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='352.0' x2='352.0' y1='136.5' y2='182.2'/><line x1='347.0' x2='357.0' y1='136.5' y2='136.5'/><line x1='347.0' x2='357.0' y1='182.2' y2='182.2'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(356.0 342.0) rotate(-35)'>(R)ReliefF</text><rect class='sa-hit' x='320.0' y='18' width='64' height='414'><title>(R)ReliefF (
|
| 35 |
Elo 1216 [95% CI 1169β1261]
|
| 36 |
Stability 0.647
|
| 37 |
-
Validity 0.808</title></rect><rect x='393.6' y='133.6' width='22.4' height='194.4' fill='#b8d8ea' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='416.0' y='133.6' width='22.4' height='194.4' fill='#e39456' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='416.0' x2='416.0' y1='107.3' y2='156.4'/><line x1='411.0' x2='421.0' y1='107.3' y2='107.3'/><line x1='411.0' x2='421.0' y1='156.4' y2='156.4'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(420.0 342.0) rotate(-35)'>SFS</text><rect class='sa-hit' x='384.0' y='18' width='64' height='414'><title>SFS (
|
| 38 |
Elo 1267 [95% CI 1221β1320]
|
| 39 |
Stability 0.278
|
| 40 |
-
Validity 0.664</title></rect><rect x='457.6' y='127.1' width='22.4' height='200.9' fill='#519fcb' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='480.0' y='127.1' width='22.4' height='200.9' fill='#d96c16' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='480.0' x2='480.0' y1='104.3' y2='146.5'/><line x1='475.0' x2='485.0' y1='104.3' y2='104.3'/><line x1='475.0' x2='485.0' y1='146.5' y2='146.5'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(484.0 342.0) rotate(-35)'>GainRatio</text><rect class='sa-hit' x='448.0' y='18' width='64' height='414'><title>GainRatio (
|
| 41 |
Elo 1280 [95% CI 1241β1326]
|
| 42 |
Stability 0.681
|
| 43 |
-
Validity 0.915</title></rect><rect x='521.6' y='119.2' width='22.4' height='208.8' fill='#6caed2' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='544.0' y='119.2' width='22.4' height='208.8' fill='#e18c49' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='544.0' x2='544.0' y1='93.4' y2='141.5'/><line x1='539.0' x2='549.0' y1='93.4' y2='93.4'/><line x1='539.0' x2='549.0' y1='141.5' y2='141.5'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(548.0 342.0) rotate(-35)'>mRMR</text><rect class='sa-hit' x='512.0' y='18' width='64' height='414'><title>mRMR (
|
| 44 |
Elo 1296 [95% CI 1251β1348]
|
| 45 |
Stability 0.578
|
| 46 |
-
Validity 0.712</title></rect><rect x='585.6' y='116.7' width='22.4' height='211.3' fill='#9ac7e1' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='608.0' y='116.7' width='22.4' height='211.3' fill='#ebb488' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='608.0' x2='608.0' y1='95.4' y2='138.5'/><line x1='603.0' x2='613.0' y1='95.4' y2='95.4'/><line x1='603.0' x2='613.0' y1='138.5' y2='138.5'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(612.0 342.0) rotate(-35)'>MarkovBlanket</text><rect class='sa-hit' x='576.0' y='18' width='64' height='414'><title>MarkovBlanket (
|
| 47 |
Elo 1301 [95% CI 1257β1344]
|
| 48 |
Stability 0.396
|
| 49 |
-
Validity 0.467</title></rect><rect x='649.6' y='108.3' width='22.4' height='219.7' fill='#8bbfdc' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='672.0' y='108.3' width='22.4' height='219.7' fill='#e4985c' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='672.0' x2='672.0' y1='81.5' y2='132.6'/><line x1='667.0' x2='677.0' y1='81.5' y2='81.5'/><line x1='667.0' x2='677.0' y1='132.6' y2='132.6'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(676.0 342.0) rotate(-35)'>CART</text><rect class='sa-hit' x='640.0' y='18' width='64' height='414'><title>CART (
|
| 50 |
Elo 1318 [95% CI 1269β1372]
|
| 51 |
Stability 0.456
|
| 52 |
-
Validity 0.638</title></rect><rect x='713.6' y='104.3' width='22.4' height='223.7' fill='#4498c6' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='736.0' y='104.3' width='22.4' height='223.7' fill='#d86911' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='736.0' x2='736.0' y1='81.5' y2='125.1'/><line x1='731.0' x2='741.0' y1='81.5' y2='81.5'/><line x1='731.0' x2='741.0' y1='125.1' y2='125.1'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(740.0 342.0) rotate(-35)'>F-Test</text><rect class='sa-hit' x='704.0' y='18' width='64' height='414'><title>F-Test (
|
| 53 |
Elo 1326 [95% CI 1284β1372]
|
| 54 |
Stability 0.734
|
| 55 |
-
Validity 0.932</title></rect><rect x='777.6' y='96.4' width='22.4' height='231.6' fill='#53a0cb' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='800.0' y='96.4' width='22.4' height='231.6' fill='#db7423' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='800.0' x2='800.0' y1='71.1' y2='118.2'/><line x1='795.0' x2='805.0' y1='71.1' y2='71.1'/><line x1='795.0' x2='805.0' y1='118.2' y2='118.2'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(804.0 342.0) rotate(-35)'>MI</text><rect class='sa-hit' x='768.0' y='18' width='64' height='414'><title>MI (
|
| 56 |
Elo 1342 [95% CI 1298β1393]
|
| 57 |
Stability 0.673
|
| 58 |
-
Validity 0.863</title></rect><rect x='841.6' y='85.0' width='22.4' height='243.0' fill='#58a3cd' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='864.0' y='85.0' width='22.4' height='243.0' fill='#e08842' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='864.0' x2='864.0' y1='58.2' y2='110.3'/><line x1='859.0' x2='869.0' y1='58.2' y2='58.2'/><line x1='859.0' x2='869.0' y1='110.3' y2='110.3'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(868.0 342.0) rotate(-35)'>RFImportance</text><rect class='sa-hit' x='832.0' y='18' width='64' height='414'><title>RFImportance (
|
| 59 |
Elo 1365 [95% CI 1314β1419]
|
| 60 |
Stability 0.653
|
| 61 |
-
Validity 0.741</title></rect><rect x='905.6' y='79.0' width='22.4' height='249.0' fill='#93c3df' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='928.0' y='79.0' width='22.4' height='249.0' fill='#dc7a2d' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='928.0' x2='928.0' y1='50.7' y2='104.8'/><line x1='923.0' x2='933.0' y1='50.7' y2='50.7'/><line x1='923.0' x2='933.0' y1='104.8' y2='104.8'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(932.0 342.0) rotate(-35)'>Lasso</text><rect class='sa-hit' x='896.0' y='18' width='64' height='414'><title>Lasso (
|
| 62 |
Elo 1377 [95% CI 1325β1434]
|
| 63 |
Stability 0.422
|
| 64 |
-
Validity 0.825</title></rect><rect x='969.6' y='65.1' width='22.4' height='262.9' fill='#7bb6d7' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='992.0' y='65.1' width='22.4' height='262.9' fill='#da7220' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='992.0' x2='992.0' y1='37.3' y2='93.4'/><line x1='987.0' x2='997.0' y1='37.3' y2='37.3'/><line x1='987.0' x2='997.0' y1='93.4' y2='93.4'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(996.0 342.0) rotate(-35)'>ElasticNet</text><rect class='sa-hit' x='960.0' y='18' width='64' height='414'><title>ElasticNet (
|
| 65 |
Elo 1405 [95% CI 1348β1461]
|
| 66 |
Stability 0.519
|
| 67 |
-
Validity 0.873</title></rect><line class='sa-ink' x1='64' x2='1024' y1='328.0' y2='328.0' stroke-width='1' opacity='0.4'/><rect x='1070' y='40.0' width='12' height='107.0' fill='url(#sa-g-stab)' stroke='#000' stroke-width='0.6'/><text class='sa-text' x='1076' y='30.0' text-anchor='middle' font-size='12'>stable</text><text class='sa-text' x='1076' y='165.0' text-anchor='middle' font-size='12'>unstable</text><text class='sa-muted' x='1088' y='151.0' font-size='11'>0.0</text><text class='sa-muted' x='1088' y='97.5' font-size='11'>0.5</text><text class='sa-muted' x='1088' y='44.0' font-size='11'>1.0</text><rect x='1070' y='207.0' width='12' height='107.0' fill='url(#sa-g-val)' stroke='#000' stroke-width='0.6'/><text class='sa-text' x='1076' y='197.0' text-anchor='middle' font-size='12'>valid</text><text class='sa-text' x='1076' y='332.0' text-anchor='middle' font-size='12'>invalid</text><text class='sa-muted' x='1088' y='318.0' font-size='11'>0.0</text><text class='sa-muted' x='1088' y='264.5' font-size='11'>0.5</text><text class='sa-muted' x='1088' y='211.0' font-size='11'>1.0</text><rect x='1070' y='352.0' width='12' height='12' fill='url(#sa-na)' stroke='#000' stroke-width='0.6'/><text class='sa-muted' x='1088' y='362.0' font-size='11'>no result</text></svg><script type='text/plain' id='sa-csv'>Method,
|
| 68 |
-
ElasticNet,
|
| 69 |
-
Lasso,
|
| 70 |
-
RFImportance,
|
| 71 |
-
MI,
|
| 72 |
-
F-Test,
|
| 73 |
-
CART,
|
| 74 |
-
MarkovBlanket,
|
| 75 |
-
mRMR,
|
| 76 |
-
GainRatio,
|
| 77 |
-
SFS,
|
| 78 |
-
(R)ReliefF,
|
| 79 |
-
LOCO,
|
| 80 |
-
SBE,
|
| 81 |
-
LaplacianScore,
|
| 82 |
-
Random,Baseline,1000.0,936.0,1052.0,0.001,0.499
|
| 83 |
</script><script>
|
| 84 |
function saDownload(name, blob) {
|
| 85 |
var a = document.createElement('a');
|
|
|
|
| 19 |
.sa-ink { stroke:var(--fg); }
|
| 20 |
.sa-hit { fill:transparent; }
|
| 21 |
.sa-hit:hover { fill:var(--fg); fill-opacity:0.06; }
|
| 22 |
+
</style><defs><pattern id='sa-na' width='6' height='6' patternUnits='userSpaceOnUse' patternTransform='rotate(45)'><rect width='6' height='6' fill='#e5e7eb'/><line x1='0' y1='0' x2='0' y2='6' stroke='#9ca3af' stroke-width='2'/></pattern><linearGradient id='sa-g-stab' x1='0' y1='1' x2='0' y2='0'><stop offset='0' stop-color='#ffffff'/><stop offset='1' stop-color='#0072B2'/></linearGradient><linearGradient id='sa-g-val' x1='0' y1='1' x2='0' y2='0'><stop offset='0' stop-color='#ffffff'/><stop offset='1' stop-color='#D55E00'/></linearGradient></defs><line class='sa-grid' x1='64' x2='1024' y1='315.6' y2='315.6' stroke-width='1'/><text class='sa-muted' x='56' y='319.6' text-anchor='end' font-size='12'>900</text><line class='sa-grid' x1='64' x2='1024' y1='266.0' y2='266.0' stroke-width='1'/><text class='sa-muted' x='56' y='270.0' text-anchor='end' font-size='12'>1000</text><line class='sa-grid' x1='64' x2='1024' y1='216.4' y2='216.4' stroke-width='1'/><text class='sa-muted' x='56' y='220.4' text-anchor='end' font-size='12'>1100</text><line class='sa-grid' x1='64' x2='1024' y1='166.8' y2='166.8' stroke-width='1'/><text class='sa-muted' x='56' y='170.8' text-anchor='end' font-size='12'>1200</text><line class='sa-grid' x1='64' x2='1024' y1='117.2' y2='117.2' stroke-width='1'/><text class='sa-muted' x='56' y='121.2' text-anchor='end' font-size='12'>1300</text><line class='sa-grid' x1='64' x2='1024' y1='67.6' y2='67.6' stroke-width='1'/><text class='sa-muted' x='56' y='71.6' text-anchor='end' font-size='12'>1400</text><line class='sa-grid' x1='64' x2='1024' y1='18.0' y2='18.0' stroke-width='1'/><text class='sa-muted' x='56' y='22.0' text-anchor='end' font-size='12'>1500</text><text class='sa-text' transform='translate(16 173) rotate(-90)' text-anchor='middle' font-size='13'>Elo</text><line class='sa-ink' x1='64' x2='1024' y1='266.0' y2='266.0' stroke-width='1' stroke-dasharray='4 4' opacity='0.5'/><rect x='73.6' y='266.0' width='22.4' height='62.0' fill='#ffffff' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='96.0' y='266.0' width='22.4' height='62.0' fill='#eaaf80' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='96.0' x2='96.0' y1='240.2' y2='297.7'/><line x1='91.0' x2='101.0' y1='240.2' y2='240.2'/><line x1='91.0' x2='101.0' y1='297.7' y2='297.7'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(100.0 342.0) rotate(-35)'>Random</text><rect class='sa-hit' x='64.0' y='18' width='64' height='414'><title>Random (Baseline, baseline)
|
| 23 |
Elo 1000 [95% CI 936β1052]
|
| 24 |
Stability 0.001
|
| 25 |
+
Validity 0.499</title></rect><rect x='137.6' y='253.1' width='22.4' height='74.9' fill='#5ba4cd' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='160.0' y='253.1' width='22.4' height='74.9' fill='#db7626' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='160.0' x2='160.0' y1='223.8' y2='288.3'/><line x1='155.0' x2='165.0' y1='223.8' y2='223.8'/><line x1='155.0' x2='165.0' y1='288.3' y2='288.3'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(164.0 342.0) rotate(-35)'>LaplacianScore</text><rect class='sa-hit' x='128.0' y='18' width='64' height='414'><title>LaplacianScore (Distance, non-search)
|
| 26 |
Elo 1026 [95% CI 955β1085]
|
| 27 |
Stability 0.644
|
| 28 |
+
Validity 0.850</title></rect><rect x='201.6' y='174.7' width='22.4' height='153.3' fill='#d2e6f2' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='224.0' y='174.7' width='22.4' height='153.3' fill='#e7a26c' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='224.0' x2='224.0' y1='149.9' y2='200.0'/><line x1='219.0' x2='229.0' y1='149.9' y2='149.9'/><line x1='219.0' x2='229.0' y1='200.0' y2='200.0'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(228.0 342.0) rotate(-35)'>SBE</text><rect class='sa-hit' x='192.0' y='18' width='64' height='414'><title>SBE (Backward search, search)
|
| 29 |
Elo 1184 [95% CI 1133β1234]
|
| 30 |
Stability 0.175
|
| 31 |
+
Validity 0.578</title></rect><rect x='265.6' y='167.3' width='22.4' height='160.7' fill='#c7e0ee' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='288.0' y='167.3' width='22.4' height='160.7' fill='#e59c63' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='288.0' x2='288.0' y1='141.0' y2='192.1'/><line x1='283.0' x2='293.0' y1='141.0' y2='141.0'/><line x1='283.0' x2='293.0' y1='192.1' y2='192.1'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(292.0 342.0) rotate(-35)'>LOCO</text><rect class='sa-hit' x='256.0' y='18' width='64' height='414'><title>LOCO (Leave-one-out, search)
|
| 32 |
Elo 1199 [95% CI 1149β1252]
|
| 33 |
Stability 0.220
|
| 34 |
+
Validity 0.613</title></rect><rect x='329.6' y='158.9' width='22.4' height='169.1' fill='#5aa4cd' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='352.0' y='158.9' width='22.4' height='169.1' fill='#dd7d31' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='352.0' x2='352.0' y1='136.5' y2='182.2'/><line x1='347.0' x2='357.0' y1='136.5' y2='136.5'/><line x1='347.0' x2='357.0' y1='182.2' y2='182.2'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(356.0 342.0) rotate(-35)'>(R)ReliefF</text><rect class='sa-hit' x='320.0' y='18' width='64' height='414'><title>(R)ReliefF (Distance, non-search)
|
| 35 |
Elo 1216 [95% CI 1169β1261]
|
| 36 |
Stability 0.647
|
| 37 |
+
Validity 0.808</title></rect><rect x='393.6' y='133.6' width='22.4' height='194.4' fill='#b8d8ea' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='416.0' y='133.6' width='22.4' height='194.4' fill='#e39456' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='416.0' x2='416.0' y1='107.3' y2='156.4'/><line x1='411.0' x2='421.0' y1='107.3' y2='107.3'/><line x1='411.0' x2='421.0' y1='156.4' y2='156.4'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(420.0 342.0) rotate(-35)'>SFS</text><rect class='sa-hit' x='384.0' y='18' width='64' height='414'><title>SFS (Forward search, search)
|
| 38 |
Elo 1267 [95% CI 1221β1320]
|
| 39 |
Stability 0.278
|
| 40 |
+
Validity 0.664</title></rect><rect x='457.6' y='127.1' width='22.4' height='200.9' fill='#519fcb' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='480.0' y='127.1' width='22.4' height='200.9' fill='#d96c16' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='480.0' x2='480.0' y1='104.3' y2='146.5'/><line x1='475.0' x2='485.0' y1='104.3' y2='104.3'/><line x1='475.0' x2='485.0' y1='146.5' y2='146.5'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(484.0 342.0) rotate(-35)'>GainRatio</text><rect class='sa-hit' x='448.0' y='18' width='64' height='414'><title>GainRatio (Information theory, non-search)
|
| 41 |
Elo 1280 [95% CI 1241β1326]
|
| 42 |
Stability 0.681
|
| 43 |
+
Validity 0.915</title></rect><rect x='521.6' y='119.2' width='22.4' height='208.8' fill='#6caed2' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='544.0' y='119.2' width='22.4' height='208.8' fill='#e18c49' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='544.0' x2='544.0' y1='93.4' y2='141.5'/><line x1='539.0' x2='549.0' y1='93.4' y2='93.4'/><line x1='539.0' x2='549.0' y1='141.5' y2='141.5'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(548.0 342.0) rotate(-35)'>mRMR</text><rect class='sa-hit' x='512.0' y='18' width='64' height='414'><title>mRMR (Information theory, non-search)
|
| 44 |
Elo 1296 [95% CI 1251β1348]
|
| 45 |
Stability 0.578
|
| 46 |
+
Validity 0.712</title></rect><rect x='585.6' y='116.7' width='22.4' height='211.3' fill='#9ac7e1' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='608.0' y='116.7' width='22.4' height='211.3' fill='#ebb488' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='608.0' x2='608.0' y1='95.4' y2='138.5'/><line x1='603.0' x2='613.0' y1='95.4' y2='95.4'/><line x1='603.0' x2='613.0' y1='138.5' y2='138.5'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(612.0 342.0) rotate(-35)'>MarkovBlanket</text><rect class='sa-hit' x='576.0' y='18' width='64' height='414'><title>MarkovBlanket (Backward search, search)
|
| 47 |
Elo 1301 [95% CI 1257β1344]
|
| 48 |
Stability 0.396
|
| 49 |
+
Validity 0.467</title></rect><rect x='649.6' y='108.3' width='22.4' height='219.7' fill='#8bbfdc' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='672.0' y='108.3' width='22.4' height='219.7' fill='#e4985c' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='672.0' x2='672.0' y1='81.5' y2='132.6'/><line x1='667.0' x2='677.0' y1='81.5' y2='81.5'/><line x1='667.0' x2='677.0' y1='132.6' y2='132.6'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(676.0 342.0) rotate(-35)'>CART</text><rect class='sa-hit' x='640.0' y='18' width='64' height='414'><title>CART (Tree, non-search)
|
| 50 |
Elo 1318 [95% CI 1269β1372]
|
| 51 |
Stability 0.456
|
| 52 |
+
Validity 0.638</title></rect><rect x='713.6' y='104.3' width='22.4' height='223.7' fill='#4498c6' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='736.0' y='104.3' width='22.4' height='223.7' fill='#d86911' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='736.0' x2='736.0' y1='81.5' y2='125.1'/><line x1='731.0' x2='741.0' y1='81.5' y2='81.5'/><line x1='731.0' x2='741.0' y1='125.1' y2='125.1'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(740.0 342.0) rotate(-35)'>F-Test</text><rect class='sa-hit' x='704.0' y='18' width='64' height='414'><title>F-Test (Correlation, non-search)
|
| 53 |
Elo 1326 [95% CI 1284β1372]
|
| 54 |
Stability 0.734
|
| 55 |
+
Validity 0.932</title></rect><rect x='777.6' y='96.4' width='22.4' height='231.6' fill='#53a0cb' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='800.0' y='96.4' width='22.4' height='231.6' fill='#db7423' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='800.0' x2='800.0' y1='71.1' y2='118.2'/><line x1='795.0' x2='805.0' y1='71.1' y2='71.1'/><line x1='795.0' x2='805.0' y1='118.2' y2='118.2'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(804.0 342.0) rotate(-35)'>MI</text><rect class='sa-hit' x='768.0' y='18' width='64' height='414'><title>MI (Information theory, non-search)
|
| 56 |
Elo 1342 [95% CI 1298β1393]
|
| 57 |
Stability 0.673
|
| 58 |
+
Validity 0.863</title></rect><rect x='841.6' y='85.0' width='22.4' height='243.0' fill='#58a3cd' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='864.0' y='85.0' width='22.4' height='243.0' fill='#e08842' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='864.0' x2='864.0' y1='58.2' y2='110.3'/><line x1='859.0' x2='869.0' y1='58.2' y2='58.2'/><line x1='859.0' x2='869.0' y1='110.3' y2='110.3'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(868.0 342.0) rotate(-35)'>RFImportance</text><rect class='sa-hit' x='832.0' y='18' width='64' height='414'><title>RFImportance (Tree, non-search)
|
| 59 |
Elo 1365 [95% CI 1314β1419]
|
| 60 |
Stability 0.653
|
| 61 |
+
Validity 0.741</title></rect><rect x='905.6' y='79.0' width='22.4' height='249.0' fill='#93c3df' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='928.0' y='79.0' width='22.4' height='249.0' fill='#dc7a2d' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='928.0' x2='928.0' y1='50.7' y2='104.8'/><line x1='923.0' x2='933.0' y1='50.7' y2='50.7'/><line x1='923.0' x2='933.0' y1='104.8' y2='104.8'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(932.0 342.0) rotate(-35)'>Lasso</text><rect class='sa-hit' x='896.0' y='18' width='64' height='414'><title>Lasso (Regularization, non-search)
|
| 62 |
Elo 1377 [95% CI 1325β1434]
|
| 63 |
Stability 0.422
|
| 64 |
+
Validity 0.825</title></rect><rect x='969.6' y='65.1' width='22.4' height='262.9' fill='#7bb6d7' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><rect x='992.0' y='65.1' width='22.4' height='262.9' fill='#da7220' stroke='#000' stroke-width='0.8' fill-opacity='0.9'/><g class='sa-ink' stroke-width='1.5'><line x1='992.0' x2='992.0' y1='37.3' y2='93.4'/><line x1='987.0' x2='997.0' y1='37.3' y2='37.3'/><line x1='987.0' x2='997.0' y1='93.4' y2='93.4'/></g><text class='sa-text' font-size='12' text-anchor='end' transform='translate(996.0 342.0) rotate(-35)'>ElasticNet</text><rect class='sa-hit' x='960.0' y='18' width='64' height='414'><title>ElasticNet (Regularization, non-search)
|
| 65 |
Elo 1405 [95% CI 1348β1461]
|
| 66 |
Stability 0.519
|
| 67 |
+
Validity 0.873</title></rect><line class='sa-ink' x1='64' x2='1024' y1='328.0' y2='328.0' stroke-width='1' opacity='0.4'/><rect x='1070' y='40.0' width='12' height='107.0' fill='url(#sa-g-stab)' stroke='#000' stroke-width='0.6'/><text class='sa-text' x='1076' y='30.0' text-anchor='middle' font-size='12'>stable</text><text class='sa-text' x='1076' y='165.0' text-anchor='middle' font-size='12'>unstable</text><text class='sa-muted' x='1088' y='151.0' font-size='11'>0.0</text><text class='sa-muted' x='1088' y='97.5' font-size='11'>0.5</text><text class='sa-muted' x='1088' y='44.0' font-size='11'>1.0</text><rect x='1070' y='207.0' width='12' height='107.0' fill='url(#sa-g-val)' stroke='#000' stroke-width='0.6'/><text class='sa-text' x='1076' y='197.0' text-anchor='middle' font-size='12'>valid</text><text class='sa-text' x='1076' y='332.0' text-anchor='middle' font-size='12'>invalid</text><text class='sa-muted' x='1088' y='318.0' font-size='11'>0.0</text><text class='sa-muted' x='1088' y='264.5' font-size='11'>0.5</text><text class='sa-muted' x='1088' y='211.0' font-size='11'>1.0</text><rect x='1070' y='352.0' width='12' height='12' fill='url(#sa-na)' stroke='#000' stroke-width='0.6'/><text class='sa-muted' x='1088' y='362.0' font-size='11'>no result</text></svg><script type='text/plain' id='sa-csv'>Method,Category,Subgroup,Elo,Elo CI low,Elo CI high,Stability,Validity
|
| 68 |
+
ElasticNet,Non-search,Regularization,1405.0,1348.0,1461.0,0.519,0.873
|
| 69 |
+
Lasso,Non-search,Regularization,1377.0,1325.0,1434.0,0.422,0.825
|
| 70 |
+
RFImportance,Non-search,Tree,1365.0,1314.0,1419.0,0.653,0.741
|
| 71 |
+
MI,Non-search,Information theory,1342.0,1298.0,1393.0,0.673,0.863
|
| 72 |
+
F-Test,Non-search,Correlation,1326.0,1284.0,1372.0,0.734,0.932
|
| 73 |
+
CART,Non-search,Tree,1318.0,1269.0,1372.0,0.456,0.638
|
| 74 |
+
MarkovBlanket,Search,Backward search,1301.0,1257.0,1344.0,0.396,0.467
|
| 75 |
+
mRMR,Non-search,Information theory,1296.0,1251.0,1348.0,0.578,0.712
|
| 76 |
+
GainRatio,Non-search,Information theory,1280.0,1241.0,1326.0,0.681,0.915
|
| 77 |
+
SFS,Search,Forward search,1267.0,1221.0,1320.0,0.278,0.664
|
| 78 |
+
(R)ReliefF,Non-search,Distance,1216.0,1169.0,1261.0,0.647,0.808
|
| 79 |
+
LOCO,Search,Leave-one-out,1199.0,1149.0,1252.0,0.22,0.613
|
| 80 |
+
SBE,Search,Backward search,1184.0,1133.0,1234.0,0.175,0.578
|
| 81 |
+
LaplacianScore,Non-search,Distance,1026.0,955.0,1085.0,0.644,0.85
|
| 82 |
+
Random,Baseline,Baseline,1000.0,936.0,1052.0,0.001,0.499
|
| 83 |
</script><script>
|
| 84 |
function saDownload(name, blob) {
|
| 85 |
var a = document.createElement('a');
|
data/subsets/tasks_all/models_linearmodel/budgets_1/leaderboard_table.html
CHANGED
|
@@ -26,23 +26,25 @@ td.method { font-weight:600; }
|
|
| 26 |
td .sub { color:var(--muted); font-size:12px; font-weight:400; margin-left:6px; }
|
| 27 |
td.na { color:var(--muted); }
|
| 28 |
.pill { display:inline-block; padding:0 8px; border:1px solid var(--line); border-radius:999px;
|
| 29 |
-
font-size:12px; color:var(--
|
| 30 |
-
</style></head><body><div class='toolbar'><span class='hint'>Click a column to sort. Hover a header for its definition.</span><button onclick="saCsv('selectarena_leaderboard.csv')">CSV</button></div><div class='wrap'><table id='sa-table'><thead><tr><th class='num'>#</th><th class=''>Method</th><th class='' title='Filter: scores features from the data alone. Wrapper: searches feature subsets by refitting a proxy model. Embedded: reads the selection off a fitted model.'>Type</th><th class='num' title='Bradley-Terry Elo from pairwise matches within each dataset, budget and downstream model. Random = 1000. +/β give the 95% bootstrap interval.'>Elo β</th><th class='num' title='Mean rank of the downstream error.'>Avg. rank β</th><th class='num' title='Share of pairwise comparisons won, in %. Ties count half.'>Win rate β</th><th class='num' title='Mean relative gap (error β best error) / error to the best method, in %.'>Improvability β</th><th class='num' title='Mean ratio of the error to the same model trained on all features. Below 1: the selection helped.'>Error vs. all β</th><th class='num' title='Agreement of the selected features across bootstrap resamples (Nogueira et al., 2018).'>Stability β</th><th class='num' title='Share of the budget filled with original features when as many noise features are added.'>Validity β</th><th class='num' title='Median wall-clock time of the selection.'>Time (s) β</th><th class='num' title='Share of runs where the method ran out of time and random features filled the budget.'>Fallback (%)</th><th class='num'>#Datasets</th></tr></thead><tbody><tr><td class='num' data-sort='1'>1</td><td class='method' data-sort='ElasticNet'>ElasticNet</td><td class='' data-sort='Embedded'><span class='pill'>π³ Embedded</span></td><td class='num' data-sort='1405.0'>1405<span class='sub'>+56 / β57</span></td><td class='num' data-sort='5.71'>5.71</td><td class='num' data-sort='66.4'>66.4</td><td class='num' data-sort='15.8'>15.8</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.519'>0.519</td><td class='num' data-sort='0.873'>0.873</td><td class='num' data-sort='6.3'>6.3</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='2'>2</td><td class='method' data-sort='Lasso'>Lasso</td><td class='' data-sort='Embedded'><span class='pill'>π³ Embedded</span></td><td class='num' data-sort='1377.0'>1377<span class='sub'>+57 / β52</span></td><td class='num' data-sort='5.96'>5.96</td><td class='num' data-sort='64.6'>64.6</td><td class='num' data-sort='15.2'>15.2</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.422'>0.422</td><td class='num' data-sort='0.825'>0.825</td><td class='num' data-sort='2.0'>2.0</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='3'>3</td><td class='method' data-sort='RFImportance'>RFImportance</td><td class='' data-sort='Embedded'><span class='pill'>π³ Embedded</span></td><td class='num' data-sort='1365.0'>1365<span class='sub'>+54 / β51</span></td><td class='num' data-sort='6.6'>6.60</td><td class='num' data-sort='60.0'>60.0</td><td class='num' data-sort='16.0'>16.0</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.653'>0.653</td><td class='num' data-sort='0.741'>0.741</td><td class='num' data-sort='0.2'>0.2</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='4'>4</td><td class='method' data-sort='MI'>MI</td><td class='' data-sort='Filter'><span class='pill'>π Filter</span></td><td class='num' data-sort='1342.0'>1342<span class='sub'>+51 / β44</span></td><td class='num' data-sort='6.59'>6.59</td><td class='num' data-sort='60.1'>60.1</td><td class='num' data-sort='17.7'>17.7</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.673'>0.673</td><td class='num' data-sort='0.863'>0.863</td><td class='num' data-sort='0.4'>0.4</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='5'>5</td><td class='method' data-sort='F-Test'>F-Test</td><td class='' data-sort='Filter'><span class='pill'>π Filter</span></td><td class='num' data-sort='1326.0'>1326<span class='sub'>+46 / β42</span></td><td class='num' data-sort='6.86'>6.86</td><td class='num' data-sort='58.1'>58.1</td><td class='num' data-sort='19.4'>19.4</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.734'>0.734</td><td class='num' data-sort='0.932'>0.932</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='6'>6</td><td class='method' data-sort='CART'>CART</td><td class='' data-sort='Embedded'><span class='pill'>π³ Embedded</span></td><td class='num' data-sort='1318.0'>1318<span class='sub'>+54 / β49</span></td><td class='num' data-sort='7.01'>7.01</td><td class='num' data-sort='57.1'>57.1</td><td class='num' data-sort='20.0'>20.0</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.456'>0.456</td><td class='num' data-sort='0.638'>0.638</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='7'>7</td><td class='method' data-sort='MarkovBlanket'>MarkovBlanket</td><td class='' data-sort='Filter'><span class='pill'>π Filter</span></td><td class='num' data-sort='1301.0'>1301<span class='sub'>+43 / β44</span></td><td class='num' data-sort='7.49'>7.49</td><td class='num' data-sort='53.7'>53.7</td><td class='num' data-sort='21.9'>21.9</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.396'>0.396</td><td class='num' data-sort='0.467'>0.467</td><td class='num' data-sort='0.1'>0.1</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='8'>8</td><td class='method' data-sort='mRMR'>mRMR</td><td class='' data-sort='Filter'><span class='pill'>π Filter</span></td><td class='num' data-sort='1296.0'>1296<span class='sub'>+52 / β45</span></td><td class='num' data-sort='7.46'>7.46</td><td class='num' data-sort='53.9'>53.9</td><td class='num' data-sort='20.6'>20.6</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.578'>0.578</td><td class='num' data-sort='0.712'>0.712</td><td class='num' data-sort='0.3'>0.3</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='9'>9</td><td class='method' data-sort='GainRatio'>GainRatio</td><td class='' data-sort='Filter'><span class='pill'>π Filter</span></td><td class='num' data-sort='1280.0'>1280<span class='sub'>+46 / β39</span></td><td class='num' data-sort='7.86'>7.86</td><td class='num' data-sort='51.0'>51.0</td><td class='num' data-sort='23.5'>23.5</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.681'>0.681</td><td class='num' data-sort='0.915'>0.915</td><td class='num' data-sort='0.2'>0.2</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='10'>10</td><td class='method' data-sort='SFS'>SFS</td><td class='' data-sort='Wrapper'><span class='pill'>π Wrapper</span></td><td class='num' data-sort='1267.0'>1267<span class='sub'>+53 / β46</span></td><td class='num' data-sort='7.86'>7.86</td><td class='num' data-sort='51.0'>51.0</td><td class='num' data-sort='22.4'>22.4</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.278'>0.278</td><td class='num' data-sort='0.664'>0.664</td><td class='num' data-sort='1791.6'>1,791.6</td><td class='num' data-sort='43.7'>43.7</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='11'>11</td><td class='method' data-sort='(R)ReliefF'>(R)ReliefF</td><td class='' data-sort='Filter'><span class='pill'>π Filter</span></td><td class='num' data-sort='1216.0'>1216<span class='sub'>+45 / β47</span></td><td class='num' data-sort='9.07'>9.07</td><td class='num' data-sort='42.4'>42.4</td><td class='num' data-sort='23.2'>23.2</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.647'>0.647</td><td class='num' data-sort='0.808'>0.808</td><td class='num' data-sort='13.0'>13.0</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='12'>12</td><td class='method' data-sort='LOCO'>LOCO</td><td class='' data-sort='Wrapper'><span class='pill'>π Wrapper</span></td><td class='num' data-sort='1199.0'>1199<span class='sub'>+53 / β50</span></td><td class='num' data-sort='8.97'>8.97</td><td class='num' data-sort='43.1'>43.1</td><td class='num' data-sort='27.4'>27.4</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.22'>0.220</td><td class='num' data-sort='0.613'>0.613</td><td class='num' data-sort='235.0'>235.0</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='13'>13</td><td class='method' data-sort='SBE'>SBE</td><td class='' data-sort='Wrapper'><span class='pill'>π Wrapper</span></td><td class='num' data-sort='1184.0'>1184<span class='sub'>+50 / β51</span></td><td class='num' data-sort='9.08'>9.08</td><td class='num' data-sort='42.2'>42.2</td><td class='num' data-sort='29.2'>29.2</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.175'>0.175</td><td class='num' data-sort='0.578'>0.578</td><td class='num' data-sort='3391.5'>3,391.5</td><td class='num' data-sort='49.3'>49.3</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='14'>14</td><td class='method' data-sort='LaplacianScore'>LaplacianScore</td><td class='' data-sort='Filter'><span class='pill'>π Filter</span></td><td class='num' data-sort='1026.0'>1026<span class='sub'>+59 / β71</span></td><td class='num' data-sort='11.83'>11.83</td><td class='num' data-sort='22.7'>22.7</td><td class='num' data-sort='40.2'>40.2</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.644'>0.644</td><td class='num' data-sort='0.85'>0.850</td><td class='num' data-sort='0.2'>0.2</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='15'>15</td><td class='method' data-sort='Random'>Random</td><td class='' data-sort='Baseline'><span class='pill'>βͺ Baseline</span></td><td class='num' data-sort='1000.0'>1000<span class='sub'>+52 / β64</span></td><td class='num' data-sort='11.66'>11.66</td><td class='num' data-sort='23.8'>23.8</td><td class='num' data-sort='37.5'>37.5</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.001'>0.001</td><td class='num' data-sort='0.499'>0.499</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='100.0'>100.0</td><td class='num' data-sort='71'>71</td></tr></tbody></table></div><script type='text/plain' id='sa-csv'>Rank,Type,TypeName,Method,Elo,Elo CI low,Elo CI high,Avg. rank,Win rate (%),Improvability (%),Error vs. all features,Stability,Validity,Selection time (s),Fallback rate (%),#Datasets,#Comparisons
|
| 31 |
-
|
| 32 |
-
2,π³,Embedded,Lasso,1377.0,1325.0,1434.0,5.96,64.6,15.2,,0.422,0.825,2.0,0.0,71,213
|
| 33 |
-
|
| 34 |
-
|
| 35 |
-
|
| 36 |
-
|
| 37 |
-
|
| 38 |
-
|
| 39 |
-
|
| 40 |
-
|
| 41 |
-
|
| 42 |
-
|
| 43 |
-
|
| 44 |
-
|
| 45 |
-
|
|
|
|
|
|
|
| 46 |
</script><script>
|
| 47 |
function saDownload(name, blob) {
|
| 48 |
var a = document.createElement('a');
|
|
|
|
| 26 |
td .sub { color:var(--muted); font-size:12px; font-weight:400; margin-left:6px; }
|
| 27 |
td.na { color:var(--muted); }
|
| 28 |
.pill { display:inline-block; padding:0 8px; border:1px solid var(--line); border-radius:999px;
|
| 29 |
+
font-size:12px; color:var(--fg); }
|
| 30 |
+
.pill.cat-search { border-color:var(--accent); }
|
| 31 |
+
.pill.cat-baseline { color:var(--muted); }
|
| 32 |
+
</style></head><body><div class='toolbar'><span class='hint'>Click a column to sort. Hover a header for its definition.</span><button onclick="saCsv('selectarena_leaderboard.csv')">CSV</button></div><div class='wrap'><table id='sa-table'><thead><tr><th class='num'>#</th><th class=''>Method</th><th class='' title='Taxonomy of the paper (Appendix A). Search methods evaluate many feature subsets with the model (forward, backward, leave-one-out); non-search methods use a proxy instead (correlation, distance, information theory, tree, regularization).'>Type</th><th class='num' title='Bradley-Terry Elo from pairwise matches within each dataset, budget and downstream model. Random = 1000. +/β give the 95% bootstrap interval.'>Elo β</th><th class='num' title='Mean rank of the downstream error.'>Avg. rank β</th><th class='num' title='Share of pairwise comparisons won, in %. Ties count half.'>Win rate β</th><th class='num' title='Mean relative gap (error β best error) / error to the best method, in %.'>Improvability β</th><th class='num' title='Mean ratio of the error to the same model trained on all features. Below 1: the selection helped.'>Error vs. all β</th><th class='num' title='Agreement of the selected features across bootstrap resamples (Nogueira et al., 2018).'>Stability β</th><th class='num' title='Share of the budget filled with original features when as many noise features are added.'>Validity β</th><th class='num' title='Median wall-clock time of the selection.'>Time (s) β</th><th class='num' title='Share of runs where the method ran out of time and random features filled the budget.'>Fallback (%)</th><th class='num'>#Datasets</th></tr></thead><tbody><tr><td class='num' data-sort='1'>1</td><td class='method' data-sort='ElasticNet'>ElasticNet</td><td class='' data-sort='Regularization'><span class='pill cat-non-search'>Regularization</span><span class='sub'>non-search</span></td><td class='num' data-sort='1405.0'>1405<span class='sub'>+56 / β57</span></td><td class='num' data-sort='5.71'>5.71</td><td class='num' data-sort='66.4'>66.4</td><td class='num' data-sort='15.8'>15.8</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.519'>0.519</td><td class='num' data-sort='0.873'>0.873</td><td class='num' data-sort='6.3'>6.3</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='2'>2</td><td class='method' data-sort='Lasso'>Lasso</td><td class='' data-sort='Regularization'><span class='pill cat-non-search'>Regularization</span><span class='sub'>non-search</span></td><td class='num' data-sort='1377.0'>1377<span class='sub'>+57 / β52</span></td><td class='num' data-sort='5.96'>5.96</td><td class='num' data-sort='64.6'>64.6</td><td class='num' data-sort='15.2'>15.2</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.422'>0.422</td><td class='num' data-sort='0.825'>0.825</td><td class='num' data-sort='2.0'>2.0</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='3'>3</td><td class='method' data-sort='RFImportance'>RFImportance</td><td class='' data-sort='Tree'><span class='pill cat-non-search'>Tree</span><span class='sub'>non-search</span></td><td class='num' data-sort='1365.0'>1365<span class='sub'>+54 / β51</span></td><td class='num' data-sort='6.6'>6.60</td><td class='num' data-sort='60.0'>60.0</td><td class='num' data-sort='16.0'>16.0</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.653'>0.653</td><td class='num' data-sort='0.741'>0.741</td><td class='num' data-sort='0.2'>0.2</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='4'>4</td><td class='method' data-sort='MI'>MI</td><td class='' data-sort='Information theory'><span class='pill cat-non-search'>Information theory</span><span class='sub'>non-search</span></td><td class='num' data-sort='1342.0'>1342<span class='sub'>+51 / β44</span></td><td class='num' data-sort='6.59'>6.59</td><td class='num' data-sort='60.1'>60.1</td><td class='num' data-sort='17.7'>17.7</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.673'>0.673</td><td class='num' data-sort='0.863'>0.863</td><td class='num' data-sort='0.4'>0.4</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='5'>5</td><td class='method' data-sort='F-Test'>F-Test</td><td class='' data-sort='Correlation'><span class='pill cat-non-search'>Correlation</span><span class='sub'>non-search</span></td><td class='num' data-sort='1326.0'>1326<span class='sub'>+46 / β42</span></td><td class='num' data-sort='6.86'>6.86</td><td class='num' data-sort='58.1'>58.1</td><td class='num' data-sort='19.4'>19.4</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.734'>0.734</td><td class='num' data-sort='0.932'>0.932</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='6'>6</td><td class='method' data-sort='CART'>CART</td><td class='' data-sort='Tree'><span class='pill cat-non-search'>Tree</span><span class='sub'>non-search</span></td><td class='num' data-sort='1318.0'>1318<span class='sub'>+54 / β49</span></td><td class='num' data-sort='7.01'>7.01</td><td class='num' data-sort='57.1'>57.1</td><td class='num' data-sort='20.0'>20.0</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.456'>0.456</td><td class='num' data-sort='0.638'>0.638</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='7'>7</td><td class='method' data-sort='MarkovBlanket'>MarkovBlanket</td><td class='' data-sort='Backward search'><span class='pill cat-search'>Backward search</span><span class='sub'>search</span></td><td class='num' data-sort='1301.0'>1301<span class='sub'>+43 / β44</span></td><td class='num' data-sort='7.49'>7.49</td><td class='num' data-sort='53.7'>53.7</td><td class='num' data-sort='21.9'>21.9</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.396'>0.396</td><td class='num' data-sort='0.467'>0.467</td><td class='num' data-sort='0.1'>0.1</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='8'>8</td><td class='method' data-sort='mRMR'>mRMR</td><td class='' data-sort='Information theory'><span class='pill cat-non-search'>Information theory</span><span class='sub'>non-search</span></td><td class='num' data-sort='1296.0'>1296<span class='sub'>+52 / β45</span></td><td class='num' data-sort='7.46'>7.46</td><td class='num' data-sort='53.9'>53.9</td><td class='num' data-sort='20.6'>20.6</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.578'>0.578</td><td class='num' data-sort='0.712'>0.712</td><td class='num' data-sort='0.3'>0.3</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='9'>9</td><td class='method' data-sort='GainRatio'>GainRatio</td><td class='' data-sort='Information theory'><span class='pill cat-non-search'>Information theory</span><span class='sub'>non-search</span></td><td class='num' data-sort='1280.0'>1280<span class='sub'>+46 / β39</span></td><td class='num' data-sort='7.86'>7.86</td><td class='num' data-sort='51.0'>51.0</td><td class='num' data-sort='23.5'>23.5</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.681'>0.681</td><td class='num' data-sort='0.915'>0.915</td><td class='num' data-sort='0.2'>0.2</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='10'>10</td><td class='method' data-sort='SFS'>SFS</td><td class='' data-sort='Forward search'><span class='pill cat-search'>Forward search</span><span class='sub'>search</span></td><td class='num' data-sort='1267.0'>1267<span class='sub'>+53 / β46</span></td><td class='num' data-sort='7.86'>7.86</td><td class='num' data-sort='51.0'>51.0</td><td class='num' data-sort='22.4'>22.4</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.278'>0.278</td><td class='num' data-sort='0.664'>0.664</td><td class='num' data-sort='1791.6'>1,791.6</td><td class='num' data-sort='43.7'>43.7</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='11'>11</td><td class='method' data-sort='(R)ReliefF'>(R)ReliefF</td><td class='' data-sort='Distance'><span class='pill cat-non-search'>Distance</span><span class='sub'>non-search</span></td><td class='num' data-sort='1216.0'>1216<span class='sub'>+45 / β47</span></td><td class='num' data-sort='9.07'>9.07</td><td class='num' data-sort='42.4'>42.4</td><td class='num' data-sort='23.2'>23.2</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.647'>0.647</td><td class='num' data-sort='0.808'>0.808</td><td class='num' data-sort='13.0'>13.0</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='12'>12</td><td class='method' data-sort='LOCO'>LOCO</td><td class='' data-sort='Leave-one-out'><span class='pill cat-search'>Leave-one-out</span><span class='sub'>search</span></td><td class='num' data-sort='1199.0'>1199<span class='sub'>+53 / β50</span></td><td class='num' data-sort='8.97'>8.97</td><td class='num' data-sort='43.1'>43.1</td><td class='num' data-sort='27.4'>27.4</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.22'>0.220</td><td class='num' data-sort='0.613'>0.613</td><td class='num' data-sort='235.0'>235.0</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='13'>13</td><td class='method' data-sort='SBE'>SBE</td><td class='' data-sort='Backward search'><span class='pill cat-search'>Backward search</span><span class='sub'>search</span></td><td class='num' data-sort='1184.0'>1184<span class='sub'>+50 / β51</span></td><td class='num' data-sort='9.08'>9.08</td><td class='num' data-sort='42.2'>42.2</td><td class='num' data-sort='29.2'>29.2</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.175'>0.175</td><td class='num' data-sort='0.578'>0.578</td><td class='num' data-sort='3391.5'>3,391.5</td><td class='num' data-sort='49.3'>49.3</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='14'>14</td><td class='method' data-sort='LaplacianScore'>LaplacianScore</td><td class='' data-sort='Distance'><span class='pill cat-non-search'>Distance</span><span class='sub'>non-search</span></td><td class='num' data-sort='1026.0'>1026<span class='sub'>+59 / β71</span></td><td class='num' data-sort='11.83'>11.83</td><td class='num' data-sort='22.7'>22.7</td><td class='num' data-sort='40.2'>40.2</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.644'>0.644</td><td class='num' data-sort='0.85'>0.850</td><td class='num' data-sort='0.2'>0.2</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='71'>71</td></tr><tr><td class='num' data-sort='15'>15</td><td class='method' data-sort='Random'>Random</td><td class='' data-sort='Baseline'><span class='pill cat-baseline'>Baseline</span></td><td class='num' data-sort='1000.0'>1000<span class='sub'>+52 / β64</span></td><td class='num' data-sort='11.66'>11.66</td><td class='num' data-sort='23.8'>23.8</td><td class='num' data-sort='37.5'>37.5</td><td class='num na' data-sort=''>β</td><td class='num' data-sort='0.001'>0.001</td><td class='num' data-sort='0.499'>0.499</td><td class='num' data-sort='0.0'>0.0</td><td class='num' data-sort='100.0'>100.0</td><td class='num' data-sort='71'>71</td></tr></tbody></table></div><script type='text/plain' id='sa-csv'>Rank,Method,Category,Subgroup,Elo,Elo CI low,Elo CI high,Avg. rank,Win rate (%),Improvability (%),Error vs. all features,Stability,Validity,Selection time (s),Fallback rate (%),#Datasets,#Comparisons
|
| 33 |
+
1,ElasticNet,Non-search,Regularization,1405.0,1348.0,1461.0,5.71,66.4,15.8,,0.519,0.873,6.3,0.0,71,213
|
| 34 |
+
2,Lasso,Non-search,Regularization,1377.0,1325.0,1434.0,5.96,64.6,15.2,,0.422,0.825,2.0,0.0,71,213
|
| 35 |
+
3,RFImportance,Non-search,Tree,1365.0,1314.0,1419.0,6.6,60.0,16.0,,0.653,0.741,0.2,0.0,71,213
|
| 36 |
+
4,MI,Non-search,Information theory,1342.0,1298.0,1393.0,6.59,60.1,17.7,,0.673,0.863,0.4,0.0,71,213
|
| 37 |
+
5,F-Test,Non-search,Correlation,1326.0,1284.0,1372.0,6.86,58.1,19.4,,0.734,0.932,0.0,0.0,71,213
|
| 38 |
+
6,CART,Non-search,Tree,1318.0,1269.0,1372.0,7.01,57.1,20.0,,0.456,0.638,0.0,0.0,71,213
|
| 39 |
+
7,MarkovBlanket,Search,Backward search,1301.0,1257.0,1344.0,7.49,53.7,21.9,,0.396,0.467,0.1,0.0,71,213
|
| 40 |
+
8,mRMR,Non-search,Information theory,1296.0,1251.0,1348.0,7.46,53.9,20.6,,0.578,0.712,0.3,0.0,71,213
|
| 41 |
+
9,GainRatio,Non-search,Information theory,1280.0,1241.0,1326.0,7.86,51.0,23.5,,0.681,0.915,0.2,0.0,71,213
|
| 42 |
+
10,SFS,Search,Forward search,1267.0,1221.0,1320.0,7.86,51.0,22.4,,0.278,0.664,1791.6,43.7,71,213
|
| 43 |
+
11,(R)ReliefF,Non-search,Distance,1216.0,1169.0,1261.0,9.07,42.4,23.2,,0.647,0.808,13.0,0.0,71,213
|
| 44 |
+
12,LOCO,Search,Leave-one-out,1199.0,1149.0,1252.0,8.97,43.1,27.4,,0.22,0.613,235.0,0.0,71,213
|
| 45 |
+
13,SBE,Search,Backward search,1184.0,1133.0,1234.0,9.08,42.2,29.2,,0.175,0.578,3391.5,49.3,71,213
|
| 46 |
+
14,LaplacianScore,Non-search,Distance,1026.0,955.0,1085.0,11.83,22.7,40.2,,0.644,0.85,0.2,0.0,71,213
|
| 47 |
+
15,Random,Baseline,Baseline,1000.0,936.0,1052.0,11.66,23.8,37.5,,0.001,0.499,0.0,100.0,71,213
|
| 48 |
</script><script>
|
| 49 |
function saDownload(name, blob) {
|
| 50 |
var a = document.createElement('a');
|
data/subsets/tasks_all/models_linearmodel/budgets_1/website_leaderboard.csv
CHANGED
|
@@ -1,16 +1,16 @@
|
|
| 1 |
-
Rank,
|
| 2 |
-
1,
|
| 3 |
-
2,
|
| 4 |
-
3,
|
| 5 |
-
4,
|
| 6 |
-
5,
|
| 7 |
-
6,
|
| 8 |
-
7,
|
| 9 |
-
8,
|
| 10 |
-
9,
|
| 11 |
-
10,
|
| 12 |
-
11,
|
| 13 |
-
12,
|
| 14 |
-
13,
|
| 15 |
-
14,
|
| 16 |
-
15,
|
|
|
|
| 1 |
+
Rank,Method,Category,Subgroup,Elo,Elo CI low,Elo CI high,Avg. rank,Win rate (%),Improvability (%),Error vs. all features,Stability,Validity,Selection time (s),Fallback rate (%),#Datasets,#Comparisons
|
| 2 |
+
1,ElasticNet,Non-search,Regularization,1405.0,1348.0,1461.0,5.71,66.4,15.8,,0.519,0.873,6.3,0.0,71,213
|
| 3 |
+
2,Lasso,Non-search,Regularization,1377.0,1325.0,1434.0,5.96,64.6,15.2,,0.422,0.825,2.0,0.0,71,213
|
| 4 |
+
3,RFImportance,Non-search,Tree,1365.0,1314.0,1419.0,6.6,60.0,16.0,,0.653,0.741,0.2,0.0,71,213
|
| 5 |
+
4,MI,Non-search,Information theory,1342.0,1298.0,1393.0,6.59,60.1,17.7,,0.673,0.863,0.4,0.0,71,213
|
| 6 |
+
5,F-Test,Non-search,Correlation,1326.0,1284.0,1372.0,6.86,58.1,19.4,,0.734,0.932,0.0,0.0,71,213
|
| 7 |
+
6,CART,Non-search,Tree,1318.0,1269.0,1372.0,7.01,57.1,20.0,,0.456,0.638,0.0,0.0,71,213
|
| 8 |
+
7,MarkovBlanket,Search,Backward search,1301.0,1257.0,1344.0,7.49,53.7,21.9,,0.396,0.467,0.1,0.0,71,213
|
| 9 |
+
8,mRMR,Non-search,Information theory,1296.0,1251.0,1348.0,7.46,53.9,20.6,,0.578,0.712,0.3,0.0,71,213
|
| 10 |
+
9,GainRatio,Non-search,Information theory,1280.0,1241.0,1326.0,7.86,51.0,23.5,,0.681,0.915,0.2,0.0,71,213
|
| 11 |
+
10,SFS,Search,Forward search,1267.0,1221.0,1320.0,7.86,51.0,22.4,,0.278,0.664,1791.6,43.7,71,213
|
| 12 |
+
11,(R)ReliefF,Non-search,Distance,1216.0,1169.0,1261.0,9.07,42.4,23.2,,0.647,0.808,13.0,0.0,71,213
|
| 13 |
+
12,LOCO,Search,Leave-one-out,1199.0,1149.0,1252.0,8.97,43.1,27.4,,0.22,0.613,235.0,0.0,71,213
|
| 14 |
+
13,SBE,Search,Backward search,1184.0,1133.0,1234.0,9.08,42.2,29.2,,0.175,0.578,3391.5,49.3,71,213
|
| 15 |
+
14,LaplacianScore,Non-search,Distance,1026.0,955.0,1085.0,11.83,22.7,40.2,,0.644,0.85,0.2,0.0,71,213
|
| 16 |
+
15,Random,Baseline,Baseline,1000.0,936.0,1052.0,11.66,23.8,37.5,,0.001,0.499,0.0,100.0,71,213
|