zechen-nlp commited on
Commit
32d730f
Β·
verified Β·
1 Parent(s): 8ad9f21

Update Automated MNLP evaluation report (2026-05-22)

Browse files
Files changed (1) hide show
  1. EVAL_REPORT.md +58 -7
EVAL_REPORT.md CHANGED
@@ -2,7 +2,7 @@
2
 
3
  - **Model repo:** [`cs-552-2026-thinkinsidethebox/general_knowledge_model`](https://huggingface.co/cs-552-2026-thinkinsidethebox/general_knowledge_model)
4
  - **Owner(s):** group **thinkinsidethebox**
5
- - **Generated at:** 2026-05-21T03:28:21+00:00 (UTC)
6
  - **Pipeline:** [mnlp-project-ci](https://github.com/eric11eca/mnlp-project-ci)
7
 
8
  _This PR is opened automatically by the course CI. It is **non-blocking** β€” you do not need to merge it. The next nightly run will refresh this file._
@@ -12,7 +12,7 @@ _This PR is opened automatically by the course CI. It is **non-blocking** β€” yo
12
  | Benchmark | Accuracy | Status |
13
  |---|---:|---|
14
  | Math | β€” | not run |
15
- | Knowledge | 0.3200 | ok |
16
  | Multilingual | β€” | not run |
17
  | Safety | β€” | not run |
18
 
@@ -24,9 +24,9 @@ _Prompts are intentionally omitted to avoid revealing benchmark contents. For mu
24
 
25
  **Correct** (1 shown)
26
 
27
- - **reference**: `B`
28
  - **overall** (1/1 completions correct)
29
- - **extracted** (βœ“): `B`
30
  - **completion**:
31
 
32
  ```text
@@ -34,14 +34,65 @@ _Prompts are intentionally omitted to avoid revealing benchmark contents. For mu
34
 
35
  </think>
36
 
37
- \boxed{B}
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
38
  ```
39
 
40
  **Incorrect** (1 shown)
41
 
42
  - **reference**: `C`
43
  - **overall** (0/1 completions correct)
44
- - **extracted** (βœ—): `H`
45
  - **completion**:
46
 
47
  ```text
@@ -49,5 +100,5 @@ _Prompts are intentionally omitted to avoid revealing benchmark contents. For mu
49
 
50
  </think>
51
 
52
- \boxed{H}
53
  ```
 
2
 
3
  - **Model repo:** [`cs-552-2026-thinkinsidethebox/general_knowledge_model`](https://huggingface.co/cs-552-2026-thinkinsidethebox/general_knowledge_model)
4
  - **Owner(s):** group **thinkinsidethebox**
5
+ - **Generated at:** 2026-05-22T03:04:12+00:00 (UTC)
6
  - **Pipeline:** [mnlp-project-ci](https://github.com/eric11eca/mnlp-project-ci)
7
 
8
  _This PR is opened automatically by the course CI. It is **non-blocking** β€” you do not need to merge it. The next nightly run will refresh this file._
 
12
  | Benchmark | Accuracy | Status |
13
  |---|---:|---|
14
  | Math | β€” | not run |
15
+ | Knowledge | 0.3500 | ok |
16
  | Multilingual | β€” | not run |
17
  | Safety | β€” | not run |
18
 
 
24
 
25
  **Correct** (1 shown)
26
 
27
+ - **reference**: `F`
28
  - **overall** (1/1 completions correct)
29
+ - **extracted** (βœ“): `F`
30
  - **completion**:
31
 
32
  ```text
 
34
 
35
  </think>
36
 
37
+ We are given the surface area of a human body as:
38
+
39
+ $$
40
+ S = 0.1091 w^{0.425} h^{0.725}
41
+ $$
42
+
43
+ and told that the measurement errors for both $w$ and $h$ are no greater than $2\%$. We are to estimate the largest possible percentage error in the computed value of $S$.
44
+
45
+ ---
46
+
47
+ ### Step 1: Compute the relative error in $S$
48
+
49
+ We first compute the relative error in $w$ and $h$, and then combine them using the properties of logarithms.
50
+
51
+ $$
52
+ \frac{\Delta S}{S} \approx \left( \frac{\partial S}{\partial w} \frac{\Delta w}{w} + \frac{\partial S}{\partial h} \frac{\Delta h}{h} \right)
53
+ $$
54
+
55
+ We compute the partial derivatives:
56
+
57
+ $$
58
+ \frac{\partial S}{\partial w} = 0.1091 \cdot 0.425 w^{-0.575} h^{0.725}
59
+ $$
60
+ $$
61
+ \frac{\partial S}{\partial h} = 0.1091 \cdot 0.725 w^{0.425} h^{-0.275}
62
+ $$
63
+
64
+ Now we compute the relative error:
65
+
66
+ $$
67
+ \frac{\Delta S}{S} \approx 0.425 \cdot \frac{\Delta w}{w} + 0.725 \cdot \frac{\Delta h}{h}
68
+ $$
69
+
70
+ Since the errors are no greater than $2\%$, we use:
71
+
72
+ $$
73
+ \frac{\Delta w}{w} \leq 0.02, \quad \frac{\Delta h}{h} \leq 0.02
74
+ $$
75
+
76
+ Then:
77
+
78
+ $$
79
+ \frac{\Delta S}{S} \leq 0.425 \cdot 0.02 + 0.725 \cdot 0.02 = 0.0085 + 0.0145 = 0.023 = 2.3\%
80
+ $$
81
+
82
+ ---
83
+
84
+ ### Final Answer
85
+
86
+ $$
87
+ \boxed{F}
88
+ $$
89
  ```
90
 
91
  **Incorrect** (1 shown)
92
 
93
  - **reference**: `C`
94
  - **overall** (0/1 completions correct)
95
+ - **extracted** (βœ—): `D`
96
  - **completion**:
97
 
98
  ```text
 
100
 
101
  </think>
102
 
103
+ \boxed{D}
104
  ```