| INFO 10-26 02:40:28 [__init__.py:235] Automatically detected platform cuda. |
| [2025-10-26 02:40:30,363] [[32m INFO[0m]: --- INIT SEEDS --- (pipeline.py:249)[0m |
| [2025-10-26 02:40:30,364] [[32m INFO[0m]: --- LOADING TASKS --- (pipeline.py:210)[0m |
| [2025-10-26 02:40:30,368] [[33m WARNING[0m]: Careful, the task gpqa:diamond is using evaluation data to build the few shot examples. (lighteval_task.py:269)[0m |
| [2025-10-26 02:40:34,331] [[32m INFO[0m]: --- LOADING MODEL --- (pipeline.py:177)[0m |
| `torch_dtype` is deprecated! Use `dtype` instead! |
| [2025-10-26 02:40:40,484] [[32m INFO[0m]: Using max model len 32768 (config.py:1604)[0m |
| [2025-10-26 02:40:41,005] [[32m INFO[0m]: Chunked prefill is enabled with max_num_batched_tokens=2048. (config.py:2434)[0m |
| INFO 10-26 02:40:45 [__init__.py:235] Automatically detected platform cuda. |
| INFO 10-26 02:40:46 [core.py:572] Waiting for init message from front-end. |
| INFO 10-26 02:40:46 [core.py:71] Initializing a V1 LLM engine (v0.10.0) with config: model='/mnt/public/wucanhui/outputs/Qwen3-14B-math-reasoning/checkpoint-2562', speculative_config=None, tokenizer='/mnt/public/wucanhui/outputs/Qwen3-14B-math-reasoning/checkpoint-2562', skip_tokenizer_init=False, tokenizer_mode=auto, revision=main, override_neuron_config={}, tokenizer_revision=main, trust_remote_code=False, dtype=torch.bfloat16, max_seq_len=32768, download_dir=None, load_format=LoadFormat.AUTO, tensor_parallel_size=1, pipeline_parallel_size=1, disable_custom_all_reduce=False, quantization=None, enforce_eager=True, kv_cache_dtype=auto, device_config=cuda, decoding_config=DecodingConfig(backend='auto', disable_fallback=False, disable_any_whitespace=False, disable_additional_properties=False, reasoning_backend=''), observability_config=ObservabilityConfig(show_hidden_metrics_for_version=None, otlp_traces_endpoint=None, collect_detailed_traces=None), seed=1234, served_model_name=/mnt/public/wucanhui/outputs/Qwen3-14B-math-reasoning/checkpoint-2562, num_scheduler_steps=1, multi_step_stream_outputs=True, enable_prefix_caching=True, chunked_prefill_enabled=True, use_async_output_proc=True, pooler_config=None, compilation_config={"level":0,"debug_dump_path":"","cache_dir":"","backend":"","custom_ops":[],"splitting_ops":[],"use_inductor":true,"compile_sizes":[],"inductor_compile_config":{"enable_auto_functionalized_v2":false},"inductor_passes":{},"use_cudagraph":true,"cudagraph_num_of_warmups":0,"cudagraph_capture_sizes":[],"cudagraph_copy_inputs":false,"full_cuda_graph":false,"max_capture_size":0,"local_cache_dir":null} |
| INFO 10-26 02:40:49 [parallel_state.py:1102] rank 0 in world size 1 is assigned as DP rank 0, PP rank 0, TP rank 0, EP rank 0 |
| WARNING 10-26 02:40:49 [topk_topp_sampler.py:59] FlashInfer is not available. Falling back to the PyTorch-native implementation of top-p & top-k sampling. For the best performance, please install FlashInfer. |
| INFO 10-26 02:40:49 [gpu_model_runner.py:1843] Starting to load model /mnt/public/wucanhui/outputs/Qwen3-14B-math-reasoning/checkpoint-2562... |
| INFO 10-26 02:40:50 [gpu_model_runner.py:1875] Loading model from scratch... |
| INFO 10-26 02:40:50 [cuda.py:290] Using Flash Attention backend on V1 engine. |
|
Loading safetensors checkpoint shards: 0% Completed | 0/6 [00:00<?, ?it/s] |
|
Loading safetensors checkpoint shards: 17% Completed | 1/6 [00:36<03:03, 36.65s/it] |
|
Loading safetensors checkpoint shards: 33% Completed | 2/6 [01:15<02:31, 37.90s/it] |
|
Loading safetensors checkpoint shards: 50% Completed | 3/6 [01:50<01:50, 36.73s/it] |
|
Loading safetensors checkpoint shards: 67% Completed | 4/6 [02:25<01:11, 35.91s/it] |
|
Loading safetensors checkpoint shards: 83% Completed | 5/6 [03:01<00:35, 35.98s/it] |
|
Loading safetensors checkpoint shards: 100% Completed | 6/6 [03:35<00:00, 35.42s/it] |
|
Loading safetensors checkpoint shards: 100% Completed | 6/6 [03:35<00:00, 35.98s/it] |
|
|
| INFO 10-26 02:44:27 [default_loader.py:262] Loading weights took 216.85 seconds |
| INFO 10-26 02:44:27 [gpu_model_runner.py:1892] Model loading took 27.5185 GiB and 217.282319 seconds |
| INFO 10-26 02:44:28 [gpu_worker.py:255] Available KV cache memory: 97.63 GiB |
| INFO 10-26 02:44:28 [kv_cache_utils.py:833] GPU KV cache size: 639,792 tokens |
| INFO 10-26 02:44:28 [kv_cache_utils.py:837] Maximum concurrency for 32,768 tokens per request: 19.52x |
| INFO 10-26 02:44:28 [core.py:193] init engine (profile, create kv cache, warmup model) took 1.16 seconds |
| [2025-10-26 02:44:29,377] [[32m INFO[0m]: [CACHING] Initializing data cache (cache_management.py:105)[0m |
| [2025-10-26 02:44:29,384] [[32m INFO[0m]: --- RUNNING MODEL --- (pipeline.py:330)[0m |
| [2025-10-26 02:44:29,385] [[32m INFO[0m]: Running SamplingMethod.GENERATIVE requests (pipeline.py:313)[0m |
| [2025-10-26 02:44:35,145] [[32m INFO[0m]: Cache: Starting to process 198/198 samples (not found in cache) for tasks lighteval|gpqa:diamond|0 (cd454966f6c36e03, GENERATIVE) (cache_management.py:399)[0m |
| [2025-10-26 02:44:35,147] [[33m WARNING[0m]: You cannot select the number of dataset splits for a generative evaluation at the moment. Automatically inferring. (data.py:206)[0m |
|
Splits: 0%| | 0/1 [00:00<?, ?it/s][2025-10-26 02:44:35,245] [[33m WARNING[0m]: context_size + max_new_tokens=35595 which is greater than self.max_length=32768. Truncating context to 0 tokens. (vllm_model.py:367)[0m |
|
|
|
Adding requests: 0%| | 0/198 [00:00<?, ?it/s][A
Adding requests: 100%|██████████| 198/198 [00:00<00:00, 15591.33it/s] |
|
|
|
Processed prompts: 0%| | 0/198 [00:00<?, ?it/s, est. speed input: 0.00 toks/s, output: 0.00 toks/s][A |
|
Processed prompts: 1%| | 1/198 [00:06<20:02, 6.11s/it, est. speed input: 26.54 toks/s, output: 33.91 toks/s][A |
|
Processed prompts: 1%| | 2/198 [00:07<09:56, 3.04s/it, est. speed input: 52.97 toks/s, output: 65.96 toks/s][A |
|
Processed prompts: 2%|▏ | 3/198 [00:07<05:37, 1.73s/it, est. speed input: 71.05 toks/s, output: 100.44 toks/s][A |
|
Processed prompts: 3%|▎ | 5/198 [00:08<03:11, 1.01it/s, est. speed input: 114.03 toks/s, output: 161.21 toks/s][A |
|
Processed prompts: 3%|▎ | 6/198 [00:09<03:25, 1.07s/it, est. speed input: 123.89 toks/s, output: 178.78 toks/s][A |
|
Processed prompts: 4%|▎ | 7/198 [00:09<03:00, 1.06it/s, est. speed input: 138.22 toks/s, output: 207.82 toks/s][A |
|
Processed prompts: 5%|▍ | 9/198 [00:10<02:07, 1.49it/s, est. speed input: 163.56 toks/s, output: 272.03 toks/s][A |
|
Processed prompts: 6%|▌ | 11/198 [00:11<01:41, 1.83it/s, est. speed input: 207.99 toks/s, output: 335.91 toks/s][A |
|
Processed prompts: 6%|▌ | 12/198 [00:11<01:33, 1.98it/s, est. speed input: 223.89 toks/s, output: 367.29 toks/s][A |
|
Processed prompts: 7%|▋ | 13/198 [00:11<01:22, 2.24it/s, est. speed input: 259.40 toks/s, output: 401.44 toks/s][A |
|
Processed prompts: 8%|▊ | 15/198 [00:12<01:12, 2.54it/s, est. speed input: 275.04 toks/s, output: 461.94 toks/s][A |
|
Processed prompts: 8%|▊ | 16/198 [00:12<01:04, 2.83it/s, est. speed input: 292.84 toks/s, output: 496.30 toks/s][A |
|
Processed prompts: 9%|▊ | 17/198 [00:13<00:58, 3.08it/s, est. speed input: 305.18 toks/s, output: 529.03 toks/s][A |
|
Processed prompts: 9%|▉ | 18/198 [00:13<01:08, 2.62it/s, est. speed input: 320.49 toks/s, output: 550.15 toks/s][A |
|
Processed prompts: 10%|█ | 20/198 [00:14<01:02, 2.86it/s, est. speed input: 395.86 toks/s, output: 609.43 toks/s][A |
|
Processed prompts: 11%|█ | 21/198 [00:15<01:29, 1.98it/s, est. speed input: 384.63 toks/s, output: 610.69 toks/s][A |
|
Processed prompts: 12%|█▏ | 23/198 [00:15<01:00, 2.89it/s, est. speed input: 413.18 toks/s, output: 685.91 toks/s][A |
|
Processed prompts: 12%|█▏ | 24/198 [00:15<01:04, 2.71it/s, est. speed input: 427.66 toks/s, output: 708.79 toks/s][A |
|
Processed prompts: 13%|█▎ | 25/198 [00:17<01:36, 1.80it/s, est. speed input: 414.65 toks/s, output: 703.49 toks/s][A |
|
Processed prompts: 14%|█▎ | 27/198 [00:17<01:03, 2.68it/s, est. speed input: 456.50 toks/s, output: 779.65 toks/s][A |
|
Processed prompts: 14%|█▍ | 28/198 [00:17<00:54, 3.14it/s, est. speed input: 467.48 toks/s, output: 816.03 toks/s][A |
|
Processed prompts: 15%|█▍ | 29/198 [00:17<00:48, 3.46it/s, est. speed input: 485.48 toks/s, output: 849.72 toks/s][A |
|
Processed prompts: 15%|█▌ | 30/198 [00:17<00:42, 3.92it/s, est. speed input: 495.50 toks/s, output: 884.64 toks/s][A |
|
Processed prompts: 16%|█▌ | 32/198 [00:17<00:30, 5.46it/s, est. speed input: 519.97 toks/s, output: 919.09 toks/s][A |
|
Processed prompts: 17%|█▋ | 33/198 [00:18<00:30, 5.48it/s, est. speed input: 522.98 toks/s, output: 925.71 toks/s][A |
|
Processed prompts: 17%|█▋ | 34/198 [00:18<00:30, 5.38it/s, est. speed input: 534.08 toks/s, output: 958.51 toks/s][A |
|
Processed prompts: 18%|█▊ | 36/198 [00:18<00:22, 7.26it/s, est. speed input: 553.70 toks/s, output: 1035.51 toks/s][A |
|
Processed prompts: 19%|█▊ | 37/198 [00:18<00:25, 6.44it/s, est. speed input: 578.59 toks/s, output: 1066.50 toks/s][A |
|
Processed prompts: 20%|█▉ | 39/198 [00:19<00:28, 5.53it/s, est. speed input: 595.14 toks/s, output: 1126.48 toks/s][A |
|
Processed prompts: 21%|██ | 41/198 [00:19<00:22, 7.08it/s, est. speed input: 654.26 toks/s, output: 1203.45 toks/s][A |
|
Processed prompts: 22%|██▏ | 43/198 [00:19<00:19, 7.93it/s, est. speed input: 691.17 toks/s, output: 1276.92 toks/s][A |
|
Processed prompts: 22%|██▏ | 44/198 [00:19<00:24, 6.23it/s, est. speed input: 691.39 toks/s, output: 1299.86 toks/s][A |
|
Processed prompts: 23%|██▎ | 45/198 [00:20<00:34, 4.47it/s, est. speed input: 688.75 toks/s, output: 1313.52 toks/s][A |
|
Processed prompts: 23%|██▎ | 46/198 [00:21<01:19, 1.91it/s, est. speed input: 666.72 toks/s, output: 1266.54 toks/s][A |
|
Processed prompts: 24%|██▎ | 47/198 [00:22<01:18, 1.94it/s, est. speed input: 661.47 toks/s, output: 1269.39 toks/s][A |
|
Processed prompts: 24%|██▍ | 48/198 [00:22<01:09, 2.15it/s, est. speed input: 663.97 toks/s, output: 1294.05 toks/s][A |
|
Processed prompts: 25%|██▌ | 50/198 [00:22<00:42, 3.48it/s, est. speed input: 691.82 toks/s, output: 1374.49 toks/s][A |
|
Processed prompts: 26%|██▌ | 51/198 [00:22<00:42, 3.48it/s, est. speed input: 691.74 toks/s, output: 1400.24 toks/s][A |
|
Processed prompts: 27%|██▋ | 53/198 [00:23<00:32, 4.50it/s, est. speed input: 703.64 toks/s, output: 1459.76 toks/s][A |
|
Processed prompts: 27%|██▋ | 54/198 [00:23<00:32, 4.40it/s, est. speed input: 707.48 toks/s, output: 1487.45 toks/s][A |
|
Processed prompts: 28%|██▊ | 55/198 [00:23<00:29, 4.87it/s, est. speed input: 718.32 toks/s, output: 1522.14 toks/s][A |
|
Processed prompts: 28%|██▊ | 56/198 [00:23<00:26, 5.32it/s, est. speed input: 728.46 toks/s, output: 1556.49 toks/s][A |
|
Processed prompts: 29%|██▉ | 57/198 [00:23<00:26, 5.39it/s, est. speed input: 731.85 toks/s, output: 1587.85 toks/s][A |
|
Processed prompts: 29%|██▉ | 58/198 [00:25<01:17, 1.81it/s, est. speed input: 700.68 toks/s, output: 1536.05 toks/s][A |
|
Processed prompts: 30%|██▉ | 59/198 [00:25<01:16, 1.82it/s, est. speed input: 695.29 toks/s, output: 1547.24 toks/s][A |
|
Processed prompts: 30%|███ | 60/198 [00:26<01:06, 2.07it/s, est. speed input: 692.37 toks/s, output: 1542.04 toks/s][A |
|
Processed prompts: 31%|███ | 61/198 [00:26<01:14, 1.83it/s, est. speed input: 685.89 toks/s, output: 1545.55 toks/s][A |
|
Processed prompts: 31%|███▏ | 62/198 [00:27<01:20, 1.69it/s, est. speed input: 676.97 toks/s, output: 1550.11 toks/s][A |
|
Processed prompts: 32%|███▏ | 63/198 [00:28<01:24, 1.61it/s, est. speed input: 665.11 toks/s, output: 1526.16 toks/s][A |
|
Processed prompts: 32%|███▏ | 64/198 [00:28<01:07, 1.99it/s, est. speed input: 669.01 toks/s, output: 1557.98 toks/s][A |
|
Processed prompts: 33%|███▎ | 65/198 [00:28<00:59, 2.23it/s, est. speed input: 673.03 toks/s, output: 1583.85 toks/s][A |
|
Processed prompts: 34%|███▍ | 67/198 [00:29<00:40, 3.26it/s, est. speed input: 690.55 toks/s, output: 1655.15 toks/s][A |
|
Processed prompts: 34%|███▍ | 68/198 [00:30<01:08, 1.90it/s, est. speed input: 669.89 toks/s, output: 1608.48 toks/s][A |
|
Processed prompts: 35%|███▍ | 69/198 [00:30<00:56, 2.29it/s, est. speed input: 674.81 toks/s, output: 1623.51 toks/s][A |
|
Processed prompts: 35%|███▌ | 70/198 [00:30<00:51, 2.47it/s, est. speed input: 682.14 toks/s, output: 1650.26 toks/s][A |
|
Processed prompts: 36%|███▋ | 72/198 [00:31<00:46, 2.70it/s, est. speed input: 680.67 toks/s, output: 1689.48 toks/s][A |
|
Processed prompts: 37%|███▋ | 74/198 [00:31<00:37, 3.27it/s, est. speed input: 689.31 toks/s, output: 1724.62 toks/s][A |
|
Processed prompts: 38%|███▊ | 75/198 [00:32<00:47, 2.57it/s, est. speed input: 679.77 toks/s, output: 1731.23 toks/s][A |
|
Processed prompts: 40%|███▉ | 79/198 [00:33<00:39, 3.04it/s, est. speed input: 687.77 toks/s, output: 1807.53 toks/s][A |
|
Processed prompts: 40%|████ | 80/198 [00:34<00:42, 2.81it/s, est. speed input: 687.19 toks/s, output: 1824.92 toks/s][A |
|
Processed prompts: 41%|████ | 81/198 [00:34<00:43, 2.68it/s, est. speed input: 683.10 toks/s, output: 1816.33 toks/s][A |
|
Processed prompts: 41%|████▏ | 82/198 [00:34<00:41, 2.81it/s, est. speed input: 683.02 toks/s, output: 1831.42 toks/s][A |
|
Processed prompts: 42%|████▏ | 83/198 [00:35<00:39, 2.89it/s, est. speed input: 683.66 toks/s, output: 1846.42 toks/s][A |
|
Processed prompts: 42%|████▏ | 84/198 [00:35<00:34, 3.29it/s, est. speed input: 685.15 toks/s, output: 1880.53 toks/s][A |
|
Processed prompts: 43%|████▎ | 85/198 [00:35<00:28, 3.94it/s, est. speed input: 688.56 toks/s, output: 1897.77 toks/s][A |
|
Processed prompts: 44%|████▍ | 87/198 [00:36<00:38, 2.87it/s, est. speed input: 679.30 toks/s, output: 1902.28 toks/s][A |
|
Processed prompts: 44%|████▍ | 88/198 [00:36<00:32, 3.41it/s, est. speed input: 681.92 toks/s, output: 1939.88 toks/s][A |
|
Processed prompts: 45%|████▍ | 89/198 [00:36<00:27, 3.94it/s, est. speed input: 683.38 toks/s, output: 1950.99 toks/s][A |
|
Processed prompts: 45%|████▌ | 90/198 [00:37<00:34, 3.17it/s, est. speed input: 678.05 toks/s, output: 1938.97 toks/s][A |
|
Processed prompts: 46%|████▌ | 91/198 [00:37<00:44, 2.38it/s, est. speed input: 670.27 toks/s, output: 1939.71 toks/s][A |
|
Processed prompts: 47%|████▋ | 93/198 [00:38<00:29, 3.57it/s, est. speed input: 678.51 toks/s, output: 1994.43 toks/s][A |
|
Processed prompts: 48%|████▊ | 95/198 [00:38<00:30, 3.36it/s, est. speed input: 678.69 toks/s, output: 2022.18 toks/s][A |
|
Processed prompts: 49%|████▉ | 97/198 [00:39<00:35, 2.82it/s, est. speed input: 674.53 toks/s, output: 2028.21 toks/s][A |
|
Processed prompts: 49%|████▉ | 98/198 [00:39<00:34, 2.86it/s, est. speed input: 676.85 toks/s, output: 2055.08 toks/s][A |
|
Processed prompts: 50%|█████ | 99/198 [00:41<01:10, 1.41it/s, est. speed input: 652.16 toks/s, output: 1998.27 toks/s][A |
|
Processed prompts: 51%|█████ | 100/198 [00:42<00:59, 1.64it/s, est. speed input: 651.56 toks/s, output: 2007.36 toks/s][A |
|
Processed prompts: 51%|█████ | 101/198 [00:43<01:05, 1.47it/s, est. speed input: 641.44 toks/s, output: 1982.34 toks/s][A |
|
Processed prompts: 53%|█████▎ | 104/198 [00:43<00:32, 2.87it/s, est. speed input: 656.04 toks/s, output: 2106.45 toks/s][A |
|
Processed prompts: 53%|█████▎ | 105/198 [00:43<00:29, 3.18it/s, est. speed input: 658.67 toks/s, output: 2126.16 toks/s][A |
|
Processed prompts: 54%|█████▎ | 106/198 [00:44<00:43, 2.10it/s, est. speed input: 647.54 toks/s, output: 2101.88 toks/s][A |
|
Processed prompts: 54%|█████▍ | 107/198 [00:45<00:46, 1.97it/s, est. speed input: 644.22 toks/s, output: 2116.92 toks/s][A |
|
Processed prompts: 56%|█████▌ | 110/198 [00:46<00:39, 2.25it/s, est. speed input: 644.58 toks/s, output: 2174.78 toks/s][A |
|
Processed prompts: 56%|█████▌ | 111/198 [00:46<00:38, 2.28it/s, est. speed input: 644.48 toks/s, output: 2199.22 toks/s][A |
|
Processed prompts: 57%|█████▋ | 112/198 [00:48<01:06, 1.29it/s, est. speed input: 625.10 toks/s, output: 2153.44 toks/s][A |
|
Processed prompts: 57%|█████▋ | 113/198 [00:48<00:53, 1.58it/s, est. speed input: 626.11 toks/s, output: 2173.53 toks/s][A |
|
Processed prompts: 58%|█████▊ | 114/198 [00:49<00:44, 1.87it/s, est. speed input: 625.88 toks/s, output: 2186.69 toks/s][A |
|
Processed prompts: 58%|█████▊ | 115/198 [00:50<01:15, 1.09it/s, est. speed input: 608.49 toks/s, output: 2146.43 toks/s][A |
|
Processed prompts: 59%|█████▊ | 116/198 [00:52<01:23, 1.01s/it, est. speed input: 599.09 toks/s, output: 2138.02 toks/s][A |
|
Processed prompts: 59%|█████▉ | 117/198 [00:53<01:27, 1.08s/it, est. speed input: 589.29 toks/s, output: 2122.54 toks/s][A |
|
Processed prompts: 60%|█████▉ | 118/198 [00:53<01:10, 1.14it/s, est. speed input: 588.66 toks/s, output: 2133.24 toks/s][A |
|
Processed prompts: 61%|██████ | 120/198 [00:54<00:49, 1.57it/s, est. speed input: 588.37 toks/s, output: 2176.56 toks/s][A |
|
Processed prompts: 62%|██████▏ | 122/198 [00:54<00:35, 2.14it/s, est. speed input: 590.74 toks/s, output: 2215.17 toks/s][A |
|
Processed prompts: 62%|██████▏ | 123/198 [00:56<00:59, 1.26it/s, est. speed input: 574.09 toks/s, output: 2169.31 toks/s][A |
|
Processed prompts: 63%|██████▎ | 124/198 [00:57<00:53, 1.38it/s, est. speed input: 575.14 toks/s, output: 2194.77 toks/s][A |
|
Processed prompts: 63%|██████▎ | 125/198 [00:57<00:44, 1.64it/s, est. speed input: 580.13 toks/s, output: 2228.37 toks/s][A |
|
Processed prompts: 64%|██████▎ | 126/198 [00:58<00:49, 1.44it/s, est. speed input: 575.17 toks/s, output: 2237.53 toks/s][A |
|
Processed prompts: 65%|██████▍ | 128/198 [01:01<01:04, 1.09it/s, est. speed input: 560.16 toks/s, output: 2227.53 toks/s][A |
|
Processed prompts: 65%|██████▌ | 129/198 [01:01<00:51, 1.34it/s, est. speed input: 562.56 toks/s, output: 2264.73 toks/s][A |
|
Processed prompts: 66%|██████▌ | 130/198 [01:01<00:40, 1.69it/s, est. speed input: 565.13 toks/s, output: 2304.31 toks/s][A |
|
Processed prompts: 66%|██████▌ | 131/198 [01:01<00:35, 1.88it/s, est. speed input: 564.29 toks/s, output: 2314.97 toks/s][A |
|
Processed prompts: 67%|██████▋ | 132/198 [01:02<00:32, 2.04it/s, est. speed input: 567.39 toks/s, output: 2345.27 toks/s][A |
|
Processed prompts: 68%|██████▊ | 134/198 [01:02<00:19, 3.35it/s, est. speed input: 575.41 toks/s, output: 2421.49 toks/s][A |
|
Processed prompts: 68%|██████▊ | 135/198 [01:04<00:53, 1.18it/s, est. speed input: 556.75 toks/s, output: 2368.93 toks/s][A |
|
Processed prompts: 69%|██████▊ | 136/198 [01:05<00:49, 1.26it/s, est. speed input: 557.20 toks/s, output: 2390.68 toks/s][A |
|
Processed prompts: 69%|██████▉ | 137/198 [01:05<00:41, 1.48it/s, est. speed input: 557.41 toks/s, output: 2422.65 toks/s][A |
|
Processed prompts: 70%|██████▉ | 138/198 [01:06<00:47, 1.27it/s, est. speed input: 551.06 toks/s, output: 2428.03 toks/s][A |
|
Processed prompts: 70%|███████ | 139/198 [01:07<00:40, 1.46it/s, est. speed input: 552.07 toks/s, output: 2457.27 toks/s][A |
|
Processed prompts: 71%|███████ | 140/198 [01:09<00:59, 1.03s/it, est. speed input: 539.34 toks/s, output: 2419.90 toks/s][A |
|
Processed prompts: 71%|███████ | 141/198 [01:10<01:03, 1.11s/it, est. speed input: 531.56 toks/s, output: 2409.05 toks/s][A |
|
Processed prompts: 72%|███████▏ | 142/198 [01:11<00:59, 1.06s/it, est. speed input: 528.36 toks/s, output: 2423.06 toks/s][A |
|
Processed prompts: 72%|███████▏ | 143/198 [01:11<00:46, 1.19it/s, est. speed input: 528.83 toks/s, output: 2446.32 toks/s][A |
|
Processed prompts: 73%|███████▎ | 144/198 [01:13<00:56, 1.05s/it, est. speed input: 521.35 toks/s, output: 2434.72 toks/s][A |
|
Processed prompts: 74%|███████▎ | 146/198 [01:14<00:45, 1.13it/s, est. speed input: 517.49 toks/s, output: 2466.02 toks/s][A |
|
Processed prompts: 74%|███████▍ | 147/198 [01:14<00:36, 1.41it/s, est. speed input: 518.40 toks/s, output: 2494.15 toks/s][A |
|
Processed prompts: 75%|███████▍ | 148/198 [01:15<00:34, 1.47it/s, est. speed input: 515.73 toks/s, output: 2502.82 toks/s][A |
|
Processed prompts: 75%|███████▌ | 149/198 [01:16<00:44, 1.11it/s, est. speed input: 509.25 toks/s, output: 2499.55 toks/s][A |
|
Processed prompts: 76%|███████▌ | 150/198 [01:18<00:48, 1.02s/it, est. speed input: 503.65 toks/s, output: 2502.67 toks/s][A |
|
Processed prompts: 76%|███████▋ | 151/198 [01:20<00:59, 1.26s/it, est. speed input: 494.65 toks/s, output: 2486.68 toks/s][A |
|
Processed prompts: 77%|███████▋ | 153/198 [01:21<00:42, 1.06it/s, est. speed input: 492.68 toks/s, output: 2533.79 toks/s][A |
|
Processed prompts: 78%|███████▊ | 154/198 [01:24<01:09, 1.58s/it, est. speed input: 473.68 toks/s, output: 2457.96 toks/s][A |
|
Processed prompts: 78%|███████▊ | 155/198 [01:25<00:52, 1.22s/it, est. speed input: 474.55 toks/s, output: 2486.02 toks/s][A |
|
Processed prompts: 79%|███████▉ | 156/198 [01:25<00:39, 1.07it/s, est. speed input: 480.28 toks/s, output: 2527.83 toks/s][A |
|
Processed prompts: 79%|███████▉ | 157/198 [01:26<00:41, 1.02s/it, est. speed input: 476.28 toks/s, output: 2536.79 toks/s][A |
|
Processed prompts: 80%|███████▉ | 158/198 [01:30<01:16, 1.90s/it, est. speed input: 457.48 toks/s, output: 2461.68 toks/s][A |
|
Processed prompts: 80%|████████ | 159/198 [01:32<01:13, 1.89s/it, est. speed input: 450.62 toks/s, output: 2458.20 toks/s][A |
|
Processed prompts: 81%|████████ | 160/198 [01:41<02:29, 3.93s/it, est. speed input: 413.59 toks/s, output: 2285.91 toks/s][A |
|
Processed prompts: 81%|████████▏ | 161/198 [01:44<02:20, 3.80s/it, est. speed input: 402.17 toks/s, output: 2256.72 toks/s][A |
|
Processed prompts: 82%|████████▏ | 162/198 [01:46<01:55, 3.20s/it, est. speed input: 396.45 toks/s, output: 2253.14 toks/s][A |
|
Processed prompts: 82%|████████▏ | 163/198 [01:47<01:29, 2.57s/it, est. speed input: 394.63 toks/s, output: 2277.60 toks/s][A |
|
Processed prompts: 83%|████████▎ | 164/198 [01:48<01:08, 2.01s/it, est. speed input: 394.00 toks/s, output: 2303.91 toks/s][A |
|
Processed prompts: 83%|████████▎ | 165/198 [01:49<00:53, 1.62s/it, est. speed input: 394.22 toks/s, output: 2336.38 toks/s][A |
|
Processed prompts: 84%|████████▍ | 166/198 [01:49<00:39, 1.23s/it, est. speed input: 394.02 toks/s, output: 2365.56 toks/s][A |
|
Processed prompts: 84%|████████▍ | 167/198 [01:51<00:46, 1.52s/it, est. speed input: 388.74 toks/s, output: 2366.34 toks/s][A |
|
Processed prompts: 85%|████████▍ | 168/198 [01:52<00:41, 1.40s/it, est. speed input: 385.96 toks/s, output: 2378.80 toks/s][A |
|
Processed prompts: 85%|████████▌ | 169/198 [01:53<00:39, 1.37s/it, est. speed input: 382.94 toks/s, output: 2393.04 toks/s][A |
|
Processed prompts: 86%|████████▌ | 170/198 [01:55<00:35, 1.27s/it, est. speed input: 382.24 toks/s, output: 2418.53 toks/s][A |
|
Processed prompts: 86%|████████▋ | 171/198 [01:56<00:32, 1.19s/it, est. speed input: 381.35 toks/s, output: 2445.01 toks/s][A |
|
Processed prompts: 87%|████████▋ | 172/198 [01:56<00:28, 1.09s/it, est. speed input: 380.53 toks/s, output: 2473.96 toks/s][A |
|
Processed prompts: 87%|████████▋ | 173/198 [01:57<00:27, 1.10s/it, est. speed input: 378.37 toks/s, output: 2489.65 toks/s][A |
|
Processed prompts: 88%|████████▊ | 174/198 [01:59<00:26, 1.09s/it, est. speed input: 377.11 toks/s, output: 2514.84 toks/s][A |
|
Processed prompts: 88%|████████▊ | 175/198 [02:00<00:26, 1.13s/it, est. speed input: 374.87 toks/s, output: 2529.69 toks/s][A |
|
Processed prompts: 89%|████████▉ | 176/198 [02:03<00:35, 1.62s/it, est. speed input: 368.80 toks/s, output: 2520.52 toks/s][A |
|
Processed prompts: 89%|████████▉ | 177/198 [02:05<00:39, 1.88s/it, est. speed input: 364.23 toks/s, output: 2518.54 toks/s][A |
|
Processed prompts: 90%|████████▉ | 178/198 [02:06<00:29, 1.48s/it, est. speed input: 364.61 toks/s, output: 2555.24 toks/s][A |
|
Processed prompts: 90%|█████████ | 179/198 [02:06<00:23, 1.21s/it, est. speed input: 364.47 toks/s, output: 2585.76 toks/s][A |
|
Processed prompts: 91%|█████████ | 180/198 [02:07<00:18, 1.05s/it, est. speed input: 364.67 toks/s, output: 2616.91 toks/s][A |
|
Processed prompts: 92%|█████████▏| 182/198 [02:15<00:37, 2.34s/it, est. speed input: 348.77 toks/s, output: 2555.80 toks/s][A |
|
Processed prompts: 92%|█████████▏| 183/198 [02:17<00:36, 2.42s/it, est. speed input: 345.06 toks/s, output: 2555.33 toks/s][A |
|
Processed prompts: 93%|█████████▎| 184/198 [02:26<00:57, 4.10s/it, est. speed input: 325.93 toks/s, output: 2450.33 toks/s][A |
|
Processed prompts: 93%|█████████▎| 185/198 [02:31<00:57, 4.43s/it, est. speed input: 316.16 toks/s, output: 2414.11 toks/s][A |
|
Processed prompts: 94%|█████████▍| 186/198 [02:48<01:32, 7.73s/it, est. speed input: 286.36 toks/s, output: 2223.97 toks/s][A |
|
Processed prompts: 94%|█████████▍| 187/198 [02:48<01:01, 5.56s/it, est. speed input: 287.43 toks/s, output: 2268.40 toks/s][A |
|
Processed prompts: 95%|█████████▍| 188/198 [02:52<00:50, 5.07s/it, est. speed input: 282.20 toks/s, output: 2268.15 toks/s][A |
|
Processed prompts: 95%|█████████▌| 189/198 [03:01<00:57, 6.41s/it, est. speed input: 268.87 toks/s, output: 2199.32 toks/s][A |
|
Processed prompts: 96%|█████████▌| 190/198 [03:03<00:40, 5.07s/it, est. speed input: 269.00 toks/s, output: 2228.61 toks/s][A |
|
Processed prompts: 96%|█████████▋| 191/198 [03:08<00:35, 5.01s/it, est. speed input: 263.09 toks/s, output: 2219.39 toks/s][A |
|
Processed prompts: 97%|█████████▋| 192/198 [03:19<00:40, 6.81s/it, est. speed input: 249.51 toks/s, output: 2146.22 toks/s][A |
|
Processed prompts: 97%|█████████▋| 193/198 [03:40<00:55, 11.17s/it, est. speed input: 226.01 toks/s, output: 1987.52 toks/s][A |
|
Processed prompts: 98%|█████████▊| 194/198 [03:54<00:47, 11.84s/it, est. speed input: 225.13 toks/s, output: 1928.64 toks/s][A |
|
Processed prompts: 98%|█████████▊| 195/198 [08:40<04:41, 93.87s/it, est. speed input: 102.63 toks/s, output: 930.21 toks/s] [A |
|
Processed prompts: 99%|█████████▉| 196/198 [08:45<02:14, 67.16s/it, est. speed input: 102.40 toks/s, output: 983.73 toks/s][A |
|
Processed prompts: 99%|█████████▉| 197/198 [08:56<00:50, 50.48s/it, est. speed input: 100.51 toks/s, output: 1023.40 toks/s][A |
|
Processed prompts: 100%|██████████| 198/198 [09:04<00:00, 37.65s/it, est. speed input: 99.34 toks/s, output: 1068.98 toks/s] [A |
|
Processed prompts: 100%|██████████| 198/198 [09:04<00:00, 37.65s/it, est. speed input: 99.34 toks/s, output: 1068.98 toks/s][A
Processed prompts: 100%|██████████| 198/198 [09:04<00:00, 2.75s/it, est. speed input: 99.34 toks/s, output: 1068.98 toks/s] |
|
Splits: 100%|██████████| 1/1 [09:04<00:00, 544.35s/it]
Splits: 100%|██████████| 1/1 [09:04<00:00, 544.35s/it] |
|
Creating parquet from Arrow format: 0%| | 0/1 [00:00<?, ?ba/s]
Creating parquet from Arrow format: 100%|██████████| 1/1 [00:00<00:00, 19.91ba/s] |
| [2025-10-26 02:53:42,264] [[32m INFO[0m]: Cached 198 samples of lighteval|gpqa:diamond|0 (cd454966f6c36e03, GENERATIVE) at /mnt/public/wucanhui/outputs/Qwen3-14B-math-reasoning/checkpoint-2562/081b5a149587018c/lighteval|gpqa:diamond|0/cd454966f6c36e03/GENERATIVE.parquet. (cache_management.py:345)[0m |
|
Generating train split: 0 examples [00:00, ? examples/s]
Generating train split: 198 examples [00:00, 6150.51 examples/s] |
| [rank0]:[W1026 02:53:48.330838811 ProcessGroupNCCL.cpp:1479] Warning: WARNING: destroy_process_group() was not called before program exit, which can leak resources. For more info, please see https://pytorch.org/docs/stable/distributed.html#shutdown (function operator()) |
| [2025-10-26 02:53:49,110] [[32m INFO[0m]: --- POST-PROCESSING MODEL RESPONSES --- (pipeline.py:344)[0m |
| [2025-10-26 02:53:49,112] [[32m INFO[0m]: --- COMPUTING METRICS --- (pipeline.py:371)[0m |
| [2025-10-26 02:53:49,113] [[33m WARNING[0m]: n undefined in the pass@k. We assume it's the same as the sample's number of predictions. (metrics_sample.py:1302)[0m |
| [2025-10-26 02:53:49,224] [[32m INFO[0m]: --- DISPLAYING RESULTS --- (pipeline.py:432)[0m |
| [2025-10-26 02:53:49,234] [[32m INFO[0m]: --- SAVING AND PUSHING RESULTS --- (pipeline.py:422)[0m |
| [2025-10-26 02:53:49,234] [[32m INFO[0m]: Saving experiment tracker (evaluation_tracker.py:246)[0m |
| [2025-10-26 02:53:49,641] [[32m INFO[0m]: Saving results to /mnt/public/wucanhui/lighteval/results/results/mnt/public/wucanhui/outputs/Qwen3-14B-math-reasoning/checkpoint-2562/results_2025-10-26T02-53-49.235695.json (evaluation_tracker.py:310)[0m |
| | Task |Version| Metric |Value | |Stderr| |
| |------------------------|-------|------------------|-----:|---|-----:| |
| |all | |gpqa_pass@k_with_k|0.5455|± |0.0355| |
| |lighteval:gpqa:diamond:0| |gpqa_pass@k_with_k|0.5455|± |0.0355| |
|
|
|
|
