zechen-nlp commited on
Commit
04ab59e
Β·
verified Β·
1 Parent(s): ab6cf3e

Update Automated MNLP evaluation report (2026-06-08)

Browse files
Files changed (1) hide show
  1. EVAL_REPORT.md +11 -19
EVAL_REPORT.md CHANGED
@@ -2,23 +2,23 @@
2
 
3
  - **Model repo:** [`cs-552-2026-thinkinsidethebox/general_knowledge_model`](https://huggingface.co/cs-552-2026-thinkinsidethebox/general_knowledge_model)
4
  - **Owner(s):** group **thinkinsidethebox**
5
- - **Generated at:** 2026-06-06T17:33:41+00:00 (UTC)
6
  - **Pipeline:** [mnlp-project-ci](https://github.com/eric11eca/mnlp-project-ci)
7
 
8
  _This PR is opened automatically by the course CI. It is **non-blocking** β€” you do not need to merge it. The next nightly run will refresh this file._
9
 
10
  ## Evaluated checkpoint
11
 
12
- - **Commit:** [`85504c2`](https://huggingface.co/cs-552-2026-thinkinsidethebox/general_knowledge_model/commit/85504c2e5486a253af83f73c379ccc2006ec6bb5)
13
- - **Message:** Upload tokenizer
14
- - **Committed:** 2026-06-04T11:02:30+00:00
15
 
16
  ## Summary
17
 
18
  | Benchmark | Accuracy | Status |
19
  |---|---:|---|
20
  | Math | β€” | not run |
21
- | Knowledge | 0.3500 | ok |
22
  | Multilingual | β€” | not run |
23
  | Safety | β€” | not run |
24
 
@@ -30,30 +30,22 @@ _Prompts are intentionally omitted to avoid revealing benchmark contents. For mu
30
 
31
  **Correct** (1 shown)
32
 
33
- - **reference**: `G`
34
  - **overall** (1/1 completions correct)
35
- - **extracted** (βœ“): `G`
36
  - **completion**:
37
 
38
  ```text
39
- <think>
40
-
41
- </think>
42
-
43
- \boxed{G}
44
  ```
45
 
46
  **Incorrect** (1 shown)
47
 
48
- - **reference**: `A`
49
  - **overall** (0/1 completions correct)
50
- - **extracted** (βœ—): `G_f(\omega) = |H(\omega)|^2 / G_{\textphi}(\omega)`
51
  - **completion**:
52
 
53
  ```text
54
- <think>
55
-
56
- </think>
57
-
58
- \boxed{G_f(\omega) = |H(\omega)|^2 / G_{\textphi}(\omega)}
59
  ```
 
2
 
3
  - **Model repo:** [`cs-552-2026-thinkinsidethebox/general_knowledge_model`](https://huggingface.co/cs-552-2026-thinkinsidethebox/general_knowledge_model)
4
  - **Owner(s):** group **thinkinsidethebox**
5
+ - **Generated at:** 2026-06-08T04:40:54+00:00 (UTC)
6
  - **Pipeline:** [mnlp-project-ci](https://github.com/eric11eca/mnlp-project-ci)
7
 
8
  _This PR is opened automatically by the course CI. It is **non-blocking** β€” you do not need to merge it. The next nightly run will refresh this file._
9
 
10
  ## Evaluated checkpoint
11
 
12
+ - **Commit:** [`a63624d`](https://huggingface.co/cs-552-2026-thinkinsidethebox/general_knowledge_model/commit/a63624d4ce801853890021d7e3284c3956f14b4b)
13
+ - **Message:** Phase 3 DPO
14
+ - **Committed:** 2026-06-06T19:50:56+00:00
15
 
16
  ## Summary
17
 
18
  | Benchmark | Accuracy | Status |
19
  |---|---:|---|
20
  | Math | β€” | not run |
21
+ | Knowledge | 0.3100 | ok |
22
  | Multilingual | β€” | not run |
23
  | Safety | β€” | not run |
24
 
 
30
 
31
  **Correct** (1 shown)
32
 
33
+ - **reference**: `A`
34
  - **overall** (1/1 completions correct)
35
+ - **extracted** (βœ“): `A`
36
  - **completion**:
37
 
38
  ```text
39
+ \boxed{A}
 
 
 
 
40
  ```
41
 
42
  **Incorrect** (1 shown)
43
 
44
+ - **reference**: `C`
45
  - **overall** (0/1 completions correct)
46
+ - **extracted** (βœ—): `D`
47
  - **completion**:
48
 
49
  ```text
50
+ \boxed{D}
 
 
 
 
51
  ```