Search results

[EVAL] Add MathVista

Status: Open.

#975 In huggingface/lighteval;

· ThakurRajAnand opened on Sep 17, 2025

Can't reproduce the Chinese FineTasks results.

Status: Open.

#974 In huggingface/lighteval;

· bg51717 opened on Sep 17, 2025

[BUG] Cannot list tasks if the user doesn't install multilingual extras

bugSomething isn't workingSomething isn't working

Status: Open.

#973 In huggingface/lighteval;

· elichen3051 opened on Sep 17, 2025

How to use a configuration file?

Status: Open.

#970 In huggingface/lighteval;

· oluwandabira opened on Sep 16, 2025

[FT] Additional LiteLLM model config options and better context length estimate

Status: Open.

#966 In huggingface/lighteval;

· rolshoven opened on Sep 16, 2025

[BUG] lighteval does not support openai endpoints but says it does

bugSomething isn't workingSomething isn't working

Status: Open.

#960 In huggingface/lighteval;

· oluwandabira opened on Sep 13, 2025

[BUG] Cuda OOM with gpt-oss-20b

bugSomething isn't workingSomething isn't working

Status: Open.

#959 In huggingface/lighteval;

· vince62s opened on Sep 12, 2025

[BUG] bug prevents mixing batched and non-batched metrics

bugSomething isn't workingSomething isn't working

Status: Open.

#957 In huggingface/lighteval;

· rolshoven opened on Sep 12, 2025

[BUG] Evaluation tracker won't save task which relies on using metrics of 2 different types

bugSomething isn't workingSomething isn't working

Status: Open.

#950 In huggingface/lighteval;

· clefourrier opened on Sep 9, 2025

[BUG] PreTrainedTokenizerFast._batch_encode_plus() got multiple values for keyword argument 'truncation_strategy

bugSomething isn't workingSomething isn't working

Status: Open.

#949 In huggingface/lighteval;

· yujonglee opened on Sep 8, 2025

[FT] Support GGUF in vllm, and use HF tokenizer together

Status: Open.

#943 In huggingface/lighteval;

· JIElite opened on Aug 29, 2025

[FT] Support confidence reporting and sampling

science-team

Status: Open.

#940 In huggingface/lighteval;

· clefourrier opened on Aug 28, 2025