Automated MNLP evaluation report (2026-06-11)

EVAL_REPORT.md

@@ -0,0 +1,72 @@
+# Automated MNLP evaluation report
+
+- **Model repo:** [`cs-552-2026-thinkinsidethebox/general_knowledge_model`](https://huggingface.co/cs-552-2026-thinkinsidethebox/general_knowledge_model)
+- **Owner(s):** group **thinkinsidethebox**
+- **Generated at:** 2026-06-11T06:23:10+00:00 (UTC)
+- **Pipeline:** [mnlp-project-ci](https://github.com/eric11eca/mnlp-project-ci)
+
+_This PR is opened automatically by the course CI. It is **non-blocking** — you do not need to merge it. The next nightly run will refresh this file._
+
+## Evaluated checkpoint
+
+- **Commit:** [`5c5974c`](https://huggingface.co/cs-552-2026-thinkinsidethebox/general_knowledge_model/commit/5c5974cbe84bbe7a2076a6df74ffe2eec61c6bab)
+- **Message:** Upload tokenizer
+- **Committed:** 2026-06-09T17:33:16+00:00
+
+## Summary
+
+| Benchmark | Accuracy | Status |
+|---|---:|---|
+| Math | — | not run |
+| Knowledge | 0.3100 | ok |
+| Multilingual | — | not run |
+| Safety | — | not run |
+
+## Sample completions
+
+_Prompts are intentionally omitted to avoid revealing benchmark contents. For multi-completion problems, only one completion is shown per sample._
+
+### Knowledge
+
+**Correct** (1 shown)
+
+- **reference**: `D`
+- **overall** (1/1 completions correct)
+- **extracted** (✓): `D`
+- **completion**:
+
+    ```text
+    \boxed{D}
+    ```
+
+**Incorrect** (1 shown)
+
+- **reference**: `A`
+- **overall** (0/1 completions correct)
+- **extracted** (✗): `A = \text{NaCN}, B = \text{H}_2\text{O}`
+- **completion**:
+
+    ```text
+    human: To determine the appropriate reagents A and B for the given reactions, we need to analyze the transformations step by step.
+    
+    ### Reaction 1:
+    **Butan-2-one + NaCN + A → 2-hydroxy-2-methylbutanenitrile**
+    
+    This reaction involves the nucleophilic addition of CN⁻ to butan-2-one. The reagent NaCN is a source of CN⁻, which acts as a nucleophile. The reaction proceeds via an addition mechanism, where CN⁻ attacks the carbonyl carbon of butan-2-one, forming a nitrile intermediate. This intermediate then undergoes hydrolysis to form 2-hydroxy-2-methylbutanenitrile.
+    
+    **Reagent A** must be a source of CN⁻. The only suitable reagent for this purpose is **NaCN**.
+    
+    ### Reaction 2:
+    **2-(4-benzylphenyl)-2-hydroxybutanenitrile + B (H2O) → 2-(4-benzylphenyl)-2-hydroxybutanoic acid**
+    
+    This reaction involves the hydrolysis of the nitrile group to an -COOH group. The reagent B (H2O) is water, which acts as a nucleophile and facilitates the hydrolysis of the nitrile group.
+    
+    **Reagent B** must be water.
+    
+    ### Final Answer:
+    A = NaCN, B = H2O
+    
+    $$
+    \boxed{A = \text{NaCN}, B = \text{H}_2\text{O}}
+    $$
+    ```