TASK
DATASET
MODEL
METRIC NAME
METRIC VALUE
GLOBAL RANK
EXTRA DATA
REMOVE
Arithmetic Reasoning
GSM8K
Minerva 62B (8-shot)
Accuracy
52.4
# 119
Arithmetic Reasoning
GSM8K
Minerva 62B (8-shot)
Parameters (Billion)
62
# 79
Arithmetic Reasoning
GSM8K
Minerva 540B (CoT)
Accuracy
78.5
# 69
Arithmetic Reasoning
GSM8K
Minerva 540B (CoT)
Parameters (Billion)
540
# 111
Arithmetic Reasoning
GSM8K
PaLM 540B (8-shot)
Accuracy
56.5
# 113
Arithmetic Reasoning
GSM8K
PaLM 540B (8-shot)
Parameters (Billion)
540
# 111
Arithmetic Reasoning
GSM8K
Minerva 8B (maj5@100)
Accuracy
56.8
# 111
Arithmetic Reasoning
GSM8K
Minerva 8B (maj5@100)
Parameters (Billion)
8
# 48
Arithmetic Reasoning
GSM8K
Minerva 62B (maj5@100)
Accuracy
89
# 24
Arithmetic Reasoning
GSM8K
Minerva 62B (maj5@100)
Parameters (Billion)
62
# 79
Arithmetic Reasoning
GSM8K
Minerva 62B (maj1@100)
Accuracy
68.5
# 96
Arithmetic Reasoning
GSM8K
Minerva 62B (maj1@100)
Parameters (Billion)
62
# 79
Arithmetic Reasoning
GSM8K
PaLM 62B (8-shot)
Accuracy
33.0
# 134
Arithmetic Reasoning
GSM8K
PaLM 62B (8-shot)
Parameters (Billion)
62
# 79
Arithmetic Reasoning
GSM8K
Minerva 8B (8-shot)
Accuracy
16.2
# 146
Arithmetic Reasoning
GSM8K
Minerva 8B (8-shot)
Parameters (Billion)
8
# 48
Arithmetic Reasoning
GSM8K
Minerva 8B-maj1@k (8-shot)
Accuracy
28.4
# 137
Arithmetic Reasoning
GSM8K
Minerva 8B-maj1@k (8-shot)
Parameters (Billion)
8
# 48
Arithmetic Reasoning
GSM8K
PaLM 8B (8-shot)
Accuracy
4.1
# 152
Arithmetic Reasoning
GSM8K
PaLM 8B (8-shot)
Parameters (Billion)
8
# 48
Math Word Problem Solving
MATH
davinci-002 175B
Accuracy
19.1
# 80
Math Word Problem Solving
MATH
davinci-002 175B
Parameters (Billions)
175
# 5
Math Word Problem Solving
MATH
Minerva 540B (maj1@k, k=64)
Accuracy
50.3
# 27
Math Word Problem Solving
MATH
Minerva 8B (maj1@k, k=64)
Accuracy
25.4
# 72
Math Word Problem Solving
MATH
Minerva 8B (maj1@k, k=64)
Parameters (Billions)
8
# 53
Math Word Problem Solving
MATH
Minerva 540B
Accuracy
33.6
# 57
Math Word Problem Solving
MATH
Minerva 540B
Parameters (Billions)
540
# 1
Math Word Problem Solving
MATH
Minerva 62B (maj1@k, k=64)
Accuracy
43.4
# 48
Math Word Problem Solving
MATH
Minerva 62B (maj1@k, k=64)
Parameters (Billions)
62
# 23
Math Word Problem Solving
MATH
Minerva 62B (maj5@256)
Accuracy
64.9
# 6
Math Word Problem Solving
MATH
Minerva 62B (maj5@256)
Parameters (Billions)
62
# 23
Math Word Problem Solving
MATH
Minerva 8B (maj5@256)
Accuracy
47.6
# 36
Math Word Problem Solving
MATH
Minerva 8B (maj5@256)
Parameters (Billions)
8
# 53
Math Word Problem Solving
MATH
PaLM 540B
Accuracy
8.8
# 92
Math Word Problem Solving
MATH
PaLM 540B
Parameters (Billions)
540
# 1
Math Word Problem Solving
MATH
PaLM 8B (fine-tuned)
Accuracy
5.6
# 100
Math Word Problem Solving
MATH
PaLM 8B (fine-tuned)
Parameters (Billions)
8
# 53
Math Word Problem Solving
MATH
Minerva 62B (4-shot)
Accuracy
27.6
# 69
Math Word Problem Solving
MATH
Minerva 62B (4-shot)
Parameters (Billions)
62
# 23
Math Word Problem Solving
MATH
PaLM 8B
Accuracy
1.5
# 110
Math Word Problem Solving
MATH
PaLM 8B
Parameters (Billions)
8
# 53
Math Word Problem Solving
MATH
PaLM 62B
Accuracy
4.4
# 105
Math Word Problem Solving
MATH
PaLM 62B
Parameters (Billions)
62
# 23
Math Word Problem Solving
MATH
Minerva 8B
Accuracy
14.1
# 84
Math Word Problem Solving
MATH
Minerva 8B
Parameters (Billions)
8
# 53