moxiaonian/ZiPingEightChar
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 3 Wiki Activity
Files
1a0eed199981d364c56c78c299c1ce14f6aeef58
ZiPingEightChar/评测/ai模型纬度得分/qoder-agent/with-skill/得分.md
T
moxiaonian 1a0eed1999 规则库 3.8.1 + 排盘修复 + 评测体系 
- 规则库 3.6→3.8.1（542→548 条）：剪枝上锁、R-杂-38 从格临界双轨、
  R-事-10~13 冲刑方向判别、R-禁-23 自造必要条件之禁
- 排盘修复随 3.6.2 已发：港澳时区夏令时双向误拨、晚子时双重换日
- 新增 评测/ 目录（BaziQA 题答分离、判分工具、记分卡、with/without-skill 对照）
  与 scripts/query_rules.py 条款检索；ZIPEC_EVAL 评测额度豁免
- manifest 重建并重新签名；VERSION 保持 3.6.2，不发版不打 tag
2026-10-13 12:34:36 +08:00

191 lines
11 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
# BaziQA 评测得分 — qoder-agent（with-skill）
- 模型：**qoder-agent**（Qoder 桌面端主代理 + 每位命主一个无上下文子代理）
- 模式：**with-skill**（加载子平八字技能排盘 + 规则库推理）
- 技能版本：3.6.2-未发版（规则库 3.7）
- 评测日期：2026-10-12 起跑，2026-10-13 判分收录
- 赛事：BaziQA-全球算命师大赛（2021–2025 五届 × 40 题，共 200 题）
- 判分：`评测/工具/run_eval.py`（脚本判分，非模型裁判）
- 求解留档：`评测/ai答案/qoder-agent/with-skill/`（41 位命主，含完整推理过程）
- 求解方式：技能副本 `/tmp/skill_bench`（rsync 自母版，**排除 `评测/` 与 `.git/`**，
物理隔绝答案键）+ `extend_cities.py` 地点扩展（追加 23 条，41 位命主 place 全解析）；
每命主**新开无上下文子代理**独立排盘 + 独立推理，跨命主不共享上下文；
真太阳时主盘 + 双盘纪律（境外/模糊地名与时辰边界 ±15 分钟内加排 120°E 对照盘）
- 额度：全程 `ZIPEC_EVAL=1` 豁免，副本 `data/` 仅 `cities.json`、无 `.license.json`，
**零授权额度消耗**
- 错因分类：暂未归并（错题清单错因列留空，待后续人工/脚本补录）
## 同模型对照
| 组别 | 正确/总题数 | 准确率 |
|---|---|---|
| **with-skill（本组）** | 79/200 | 39.5% |
| without-skill（裸答对照） | 78/200 | 39.0% |
| 官方基线（Gemini） | 69/200 | 34.5% |
如实呈现：加载技能比同模型裸答只高 **1 题（+0.5pp）**，配对 z = 0.10。
题级配对检验——两组都对 53 题、只 with-skill 对 26 题、只裸答对 25 题、都错 96 题；
McNemar 不一致对 51 组、26:25 几乎对半，校正 χ² = 0.00，p ≈ 1.0。
**技能未带来可检出的增益**。相对官方基线 +5.0pp，配对 z = 1.04（p ≈ 0.30），
单组标准误 ±3.46pp，同样不显著。200 题规模下这套基准尚不足以区分
「加载技能」与「裸答」两种口径。
## 总览
| 口径 | 正确/总题数 | 准确率 | 未答 |
|---|---|---|---|
| 本次评测 | 79/200 | 39.5% | 0 |
| 官方基线（Gemini） | 69/200 | 34.5% | 0 |
> 基线口径见 试题/BaziQA-全球算命师大赛/SOURCE.md。
## 分届准确率
| 届别 | 本次 | 官方 |
|---|---|---|
| contest8_2021 | 16/40 = 40.0%（未答 0） | 27.5% |
| contest8_2022 | 15/40 = 37.5%（未答 0） | 40.0% |
| contest8_2023 | 15/40 = 37.5%（未答 0） | 27.5% |
| contest8_2024 | 19/40 = 47.5%（未答 0） | 35.0% |
| contest8_2025 | 14/40 = 35.0%（未答 0） | 42.5% |
## 分维度准确率
| 维度 | 本次 | 官方 |
|---|---|---|
| 流年运势 | 15/37 = 40.5%（未答 0） | 40.5% |
| 感情 | 16/49 = 32.7%（未答 0） | 36.7% |
| 六亲 | 15/29 = 51.7%（未答 0） | 27.6% |
| 事业 | 10/25 = 40.0%（未答 0） | 24.0% |
| 性格 | 8/20 = 40.0%（未答 0） | 45.0% |
| 学业 | 5/12 = 41.7%（未答 0） | 41.7% |
| 财富 | 5/13 = 38.5%（未答 0） | 30.8% |
| 健康 | 4/13 = 30.8%（未答 0） | 23.1% |
| 其他 | 1/2 = 50.0%（未答 0） | 50.0% |
## 错题清单（错因留空，待人工/后续填写）
| 届别 | 命主 | 题序 | 维度 | 我答 | 正解 | 错因 |
|---|---|---|---|---|---|---|
| contest8_2021 | male_19540318_P025 | Q2 | 六亲 | B | A | |
| contest8_2021 | male_19540318_P025 | Q3 | 流年运势 | D | C | |
| contest8_2021 | male_19540318_P025 | Q5 | 六亲 | C | B | |
| contest8_2021 | male_19831019_P026 | Q1 | 事业 | B | C | |
| contest8_2021 | male_19831019_P026 | Q2 | 感情 | A | D | |
| contest8_2021 | male_19831019_P026 | Q3 | 六亲 | A | D | |
| contest8_2021 | male_19831019_P026 | Q4 | 流年运势 | D | B | |
| contest8_2021 | female_19841220_P027 | Q1 | 学业 | B | C | |
| contest8_2021 | female_19841220_P027 | Q2 | 事业 | C | A | |
| contest8_2021 | female_19841220_P027 | Q3 | 感情 | D | C | |
| contest8_2021 | female_19841220_P027 | Q4 | 流年运势 | C | D | |
| contest8_2021 | male_19770706_P028 | Q2 | 事业 | C | A | |
| contest8_2021 | male_19770706_P028 | Q4 | 感情 | C | D | |
| contest8_2021 | male_19700723_P029 | Q1 | 感情 | C | D | |
| contest8_2021 | male_19700723_P029 | Q3 | 性格 | C | A | |
| contest8_2021 | male_19700723_P029 | Q5 | 健康 | B | C | |
| contest8_2021 | female_19750107_P030 | Q2 | 感情 | A | D | |
| contest8_2021 | female_19750107_P030 | Q3 | 感情 | B | D | |
| contest8_2021 | male_19530127_P031 | Q1 | 事业 | A | C | |
| contest8_2021 | male_19530127_P031 | Q4 | 流年运势 | D | A | |
| contest8_2021 | male_19530127_P031 | Q5 | 六亲 | B | D | |
| contest8_2021 | male_19710910_P033 | Q3 | 流年运势 | A | D | |
| contest8_2021 | female_19670827_P034 | Q1 | 感情 | B | D | |
| contest8_2021 | female_19670827_P034 | Q2 | 事业 | A | C | |
| contest8_2022 | male_19740428_P017 | Q1 | 流年运势 | B | A | |
| contest8_2022 | male_19740428_P017 | Q2 | 感情 | B | C | |
| contest8_2022 | male_19740428_P017 | Q3 | 流年运势 | A | C | |
| contest8_2022 | female_19810526_P018 | Q1 | 学业 | C | B | |
| contest8_2022 | female_19810526_P018 | Q2 | 感情 | A | B | |
| contest8_2022 | female_19810526_P018 | Q4 | 性格 | A | D | |
| contest8_2022 | female_19900523_P019 | Q1 | 六亲 | B | D | |
| contest8_2022 | female_19900523_P019 | Q4 | 流年运势 | B | C | |
| contest8_2022 | female_19771026_P020 | Q1 | 事业 | C | A | |
| contest8_2022 | female_19771026_P020 | Q3 | 感情 | D | B | |
| contest8_2022 | female_19771026_P020 | Q4 | 财富 | C | A | |
| contest8_2022 | male_19850629_P021 | Q1 | 性格 | B | A | |
| contest8_2022 | male_19850629_P021 | Q4 | 流年运势 | B | C | |
| contest8_2022 | male_19850629_P021 | Q5 | 学业 | A | B | |
| contest8_2022 | male_19831101_P022 | Q2 | 学业 | A | D | |
| contest8_2022 | male_19831101_P022 | Q3 | 流年运势 | C | D | |
| contest8_2022 | male_19831101_P022 | Q5 | 财富 | D | B | |
| contest8_2022 | female_19841209_P023 | Q1 | 性格 | A | D | |
| contest8_2022 | female_19841209_P023 | Q2 | 财富 | A | C | |
| contest8_2022 | female_19841209_P023 | Q3 | 流年运势 | B | A | |
| contest8_2022 | female_19841209_P023 | Q4 | 感情 | A | C | |
| contest8_2022 | female_19841209_P023 | Q5 | 感情 | C | B | |
| contest8_2022 | male_19810617_P024 | Q2 | 感情 | D | C | |
| contest8_2022 | male_19810617_P024 | Q3 | 健康 | A | B | |
| contest8_2022 | male_19810617_P024 | Q5 | 财富 | C | B | |
| contest8_2023 | female_19661018_P009 | Q1 | 性格 | D | B | |
| contest8_2023 | female_19661018_P009 | Q2 | 事业 | D | C | |
| contest8_2023 | female_19661018_P009 | Q3 | 六亲 | D | B | |
| contest8_2023 | female_19661018_P009 | Q4 | 感情 | C | A | |
| contest8_2023 | male_19620806_P010 | Q4 | 流年运势 | C | D | |
| contest8_2023 | male_19620806_P010 | Q5 | 其他 | B | D | |
| contest8_2023 | male_19860424_P011 | Q2 | 六亲 | A | D | |
| contest8_2023 | male_19860424_P011 | Q4 | 感情 | D | C | |
| contest8_2023 | male_19560311_P012 | Q2 | 学业 | C | A | |
| contest8_2023 | male_19560311_P012 | Q3 | 感情 | C | A | |
| contest8_2023 | male_19560311_P012 | Q4 | 流年运势 | D | C | |
| contest8_2023 | female_19780405_P013 | Q2 | 流年运势 | A | C | |
| contest8_2023 | female_19780405_P013 | Q3 | 感情 | D | C | |
| contest8_2023 | female_19780405_P013 | Q4 | 六亲 | A | B | |
| contest8_2023 | female_19780405_P013 | Q5 | 六亲 | D | B | |
| contest8_2023 | female_19581027_P014 | Q1 | 事业 | D | C | |
| contest8_2023 | female_19581027_P014 | Q3 | 事业 | A | C | |
| contest8_2023 | female_19581027_P014 | Q5 | 健康 | B | D | |
| contest8_2023 | female_20020601_P015 | Q1 | 财富 | C | B | |
| contest8_2023 | female_20020601_P015 | Q3 | 感情 | D | C | |
| contest8_2023 | female_20020601_P015 | Q5 | 六亲 | C | A | |
| contest8_2023 | male_19800711_P016 | Q1 | 感情 | A | C | |
| contest8_2023 | male_19800711_P016 | Q2 | 感情 | B | C | |
| contest8_2023 | male_19800711_P016 | Q3 | 事业 | A | B | |
| contest8_2023 | male_19800711_P016 | Q4 | 性格 | A | B | |
| contest8_2024 | guangdong_female_19800824_P001 | Q2 | 学业 | C | B | |
| contest8_2024 | guangdong_female_19800824_P001 | Q3 | 感情 | B | D | |
| contest8_2024 | guangdong_female_19800824_P001 | Q5 | 感情 | B | C | |
| contest8_2024 | chaozhou_male_19720108_P002 | Q2 | 感情 | D | C | |
| contest8_2024 | chaozhou_male_19720108_P002 | Q4 | 六亲 | D | C | |
| contest8_2024 | chaozhou_male_19720108_P002 | Q5 | 财富 | C | A | |
| contest8_2024 | male_19611230_P003 | Q3 | 流年运势 | A | B | |
| contest8_2024 | male_19611230_P003 | Q4 | 流年运势 | D | A | |
| contest8_2024 | male_19611230_P003 | Q5 | 财富 | D | C | |
| contest8_2024 | female_19831028_P004 | Q2 | 感情 | B | A | |
| contest8_2024 | female_19831028_P004 | Q4 | 感情 | B | D | |
| contest8_2024 | female_19831028_P004 | Q5 | 感情 | B | A | |
| contest8_2024 | male_19710412_P005 | Q2 | 感情 | B | A | |
| contest8_2024 | male_19710412_P005 | Q3 | 财富 | B | C | |
| contest8_2024 | male_19710412_P005 | Q5 | 健康 | C | B | |
| contest8_2024 | female_19830326_P006 | Q1 | 感情 | C | B | |
| contest8_2024 | female_19830326_P006 | Q2 | 感情 | C | A | |
| contest8_2024 | female_19830326_P006 | Q4 | 流年运势 | A | D | |
| contest8_2024 | male_20070131_P008 | Q1 | 健康 | D | B | |
| contest8_2024 | male_20070131_P008 | Q3 | 性格 | D | A | |
| contest8_2024 | male_20070131_P008 | Q4 | 流年运势 | C | D | |
| contest8_2025 | guangdong_female_19511114_P001 | Q2 | 感情 | B | D | |
| contest8_2025 | guangdong_female_19511114_P001 | Q4 | 流年运势 | C | A | |
| contest8_2025 | hongkong_female_19870705_P002 | Q1 | 健康 | A | D | |
| contest8_2025 | hongkong_female_19870705_P002 | Q2 | 性格 | D | B | |
| contest8_2025 | hongkong_female_19870705_P002 | Q3 | 事业 | A | D | |
| contest8_2025 | miyazaki_male_19830421_P003 | Q1 | 六亲 | C | B | |
| contest8_2025 | miyazaki_male_19830421_P003 | Q2 | 性格 | D | B | |
| contest8_2025 | miyazaki_male_19830421_P003 | Q3 | 学业 | C | D | |
| contest8_2025 | miyazaki_male_19830421_P003 | Q4 | 感情 | B | C | |
| contest8_2025 | singapore_male_19930408_P004 | Q2 | 流年运势 | B | D | |
| contest8_2025 | singapore_male_19930408_P004 | Q3 | 事业 | C | B | |
| contest8_2025 | singapore_male_19930408_P004 | Q5 | 健康 | C | B | |
| contest8_2025 | malaysia_male_19880110_P005 | Q1 | 事业 | A | D | |
| contest8_2025 | malaysia_male_19880110_P005 | Q3 | 感情 | A | D | |
| contest8_2025 | malaysia_male_19880110_P005 | Q4 | 六亲 | C | A | |
| contest8_2025 | malaysia_male_19730824_P006 | Q1 | 性格 | C | A | |
| contest8_2025 | malaysia_male_19730824_P006 | Q2 | 性格 | C | B | |
| contest8_2025 | malaysia_male_19730824_P006 | Q3 | 事业 | C | B | |
| contest8_2025 | malaysia_male_19730824_P006 | Q4 | 流年运势 | A | D | |
| contest8_2025 | malaysia_male_19730824_P006 | Q5 | 健康 | D | C | |
| contest8_2025 | taiwan_female_19880215_P007 | Q1 | 六亲 | B | A | |
| contest8_2025 | taiwan_female_19880215_P007 | Q3 | 感情 | C | B | |
| contest8_2025 | taiwan_female_19880215_P007 | Q4 | 事业 | A | D | |
| contest8_2025 | taiwan_female_19880215_P007 | Q5 | 性格 | A | B | |
| contest8_2025 | beijing_male_19700722_P008 | Q3 | 流年运势 | A | D | |
| contest8_2025 | beijing_male_19700722_P008 | Q4 | 健康 | A | C | |
Reference in New Issue View Git Blame Copy Permalink