Add versioned prompt profiles and event-entity generation pipeline by Leon-Algo · Pull Request #192 · InternScience/GraphGen

Leon-Algo · 2026-10-10T03:10:00Z

Summary

Add versioned prompt profiles to all 10 generators while preserving upstream templates as defaults.
Add deterministic output gates and bounded generator concurrency as optional GenerateService parameters.
Add SAG-inspired event-entity extraction, deterministic star-graph merge, and same-project event-join partitioning.
Add support citation parsing and partition-node closure validation; invalid citations are stripped without dropping the QA.
Restore bounded empty-response retry, extraction prefill retry, and optional thinking disablement for logprob calls.

Validation

PMS contract suite: 38 passed.
Real 10-record event smoke: 7 partitions and 7 anchor-compliant QAs; support references closed against partition nodes; no internal event IDs leaked into question or answer.
50-record A/B: event arm 30/30 single-project partitions and 30/30 anchor-compliant QAs; 28 support-bearing QAs all passed closure validation. Baseline arm generated 168 QAs, 159 anchor-compliant; this change trades volume for project-pure, auditable multi-event questions.
Fork test results matched pristine upstream baseline (23 failed, 6 passed; failures were pre-existing upstream/environment issues).

Notes

Default behavior remains the upstream LightRAG + ECE path; event generation is opt-in via kg_method: event_entity and partition.method: event_join.
Existing concurrent edits in the fork checkout were not included.

🤖 Generated with Claude Code

- 新增版本化 prompt profile 加载器（结构 fail-closed 校验），10 个 generate method 全部支持 config 级模板覆盖 - pms_policy.v2 内置 profile：迁移 PMS 侧 v2 出题策略模板为正本 - 输出门注册表（pms_project_anchor）与 LLM 并发封顶成为 GenerateService 一等参数 - 复位历史上随 checkout 重建丢失的三个补丁：空响应重试、空抽取 prefill、logprob 调用关思考（env 开关语义不变） - 台账见 docs/PMS_PATCHES.md

- 事件抽取：SAG extract_document v3.1 提示词改编 + 严格响应合同 + 数字接地校验（MIT 归因） - 星型编码进图：事件=EVENT 节点+成员边（边属性含角色），merge/SEP 血缘复用官方 LightRAG - build_kg kg_method 分发：event_entity 与 light_rag 并存，默认行为逐字节不变 - event_join 分区器：共享实体 join + 同锚硬约束，产出官方 Community，metadata 带 event_ids - <support> 审计链：生成端引用分区节点集合校验，失败剥离保留题面；pms_policy.v3 profile - 台账 docs/PMS_PATCHES.md InternScience#7；fork 测试基线与上游一致（23F/6P，零回归） Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

- event_contract 抽为纯 stdlib 模块（rft 契约测试可按路径加载） - builder 补回 json 导入；EVENT 节点/成员边/参与者实体确定性 SEP 合并，不再触发 LightRAG LLM 摘要（摘要器曾把 event id 扩写成伪业务事实污染出题上下文） - EventJoinPartitioner 继承 BasePartitioner（community2batch）；models lazy export 补齐 - v3 profile support 大括号转义（str.format KeyError）；分区服务移除 metadata 依赖（support 校验以持久化分区节点集合为准，JSONL 往返一致） - 真实 10 记录端到端 smoke 全通：7 分区→7 题全锚点合规、cited 全部通过分区校验 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

引入版本化事件抽取profile，将实体类型、prompt、锚点和数字接地策略从PMS专用代码解耦。event_join基于显式锚点属性隔离，生成prompt隐藏内部事件ID并把support引用映射回canonical节点。\n\n新增多项目SDK与不可变tag发布文档，修正安装包资源声明和过时的partition integration测试；保留LightRAG+ECE默认路径及PMS事件ID兼容。完成真实非PMS抽取、分区和问答生成smoke，默认生成结果未泄漏内部事件ID。\n\nCo-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

新增可配置 Ray LLM actor 并发并保持默认串行兼容。为缺失 logger 上下文的 Ray 调用提供标准 logger fallback；加入并发与日志回归测试，更新版本及发行说明。 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

ChenZiHong-Gavin

Thanks for the detailed PR. The opt-in mechanisms here (versioned prompt profiles with fail-closed structural validation, output-gate registry, per-loop LLM concurrency cap, the env-gated LLM client fixes) are genuinely useful, and the default LightRAG + ECE path is indeed untouched. The partitioner test fixes (async → sync) are also correct; those tests are broken on main today.

That said, I don't think this can merge as-is:

Three functional bugs (inline, two of them reproduced locally): path-based event profiles fail on second load; edge citations in <support> are always stripped; the extraction prefill uses tuple format against a JSON parser.
Fork identity leaks into upstream: setup.py URL/author, version 1.1.0, fork README section, fork release docs.
PMS domain policy ships in the core package: pms_policy.v2/v3 prompts, pms_project_anchor gate, pms_event.v1 as the implicit default.
CI is red: Pylint exits 28 (line length, too-many-branches/statements, unused imports, duplicate validate_support import in generate_service.py, and arguments-differ on all 10 build_prompt overrides after the static → instance change).

Suggested split, smallest first:

PR A: partitioner test fixes only.
PR B: prompt-profile mechanism + gate registry + llm_concurrency + the four env-gated LLM client patches, with no pms_* assets.
PR C: event_entity builder + event_join partitioner + profile loader, shipping only generic_event.v1 as an example, with the three bugs fixed.
PMS profiles and fork release docs stay in the fork.

ChenZiHong-Gavin · 2026-10-16T06:05:29Z

+        if self.kg_method == "event_entity":
+            from graphgen.templates.event_profile_loader import load_event_profile
+
+            self.event_profile = load_event_profile(self.event_profile).profile_id


Bug (verified): this keeps only profile_id after loading, and build_event_entity_kg then calls load_event_profile(profile_id) again, which only looks in the packaged event_profiles/ directory. Any profile passed as a filesystem path (the flow docs/MULTI_PROJECT_SDK.md tells new projects to use) fails on the second load.

Repro: copy generic_event.v1 to a temp dir as myproj.v1, set event_profile: /tmp/.../profile.json → event_profile_not_found:myproj.v1.

Suggest passing the loaded EventExtractionProfile object down instead of the id.

ChenZiHong-Gavin · 2026-10-16T06:05:29Z

+        display_target = display_by_id.get(str(target), target)
+        edges.append((display_source, display_target, data))
+        if str(source) in display_by_id:
+            support_aliases[f"{display_source} - {display_target}"] = f"{source} - {target}"


Bug (verified): edge citations are resolved to "{source} - {target}", but validate_support (L103) only checks against the node-name set, so every edge citation is stripped. Also the alias separator here is a single -, while MultiHopGenerator.build_prompt renders edges with --, so the alias never matches what the model actually sees.

Tested: node citation passes; T - A and T -- A both come back None.

Either add edges to the validation set (and use one separator everywhere), or document that support.cited is node-only.

ChenZiHong-Gavin · 2026-10-16T06:05:29Z

+        # that seeds the opening of the expected output; non-empty responses
+        # are never retried or modified.
+        if not response.strip() and os.environ.get("GRAPHGEN_EMPTY_EXTRACTION_PREFILL", "0") == "1":
+            prefill = '("entity"<|>'


Bug: ("entity"<|> is the LightRAG tuple-format opener from kg_extraction.py. This extractor parses the response with json.loads a few lines below, so a response that starts with this prefill can never parse. The retry will always fail. This patch belongs in the LightRAGKGBuilder extraction path, not here.

ChenZiHong-Gavin · 2026-10-16T06:05:29Z

+            return await self._merge_event_node(entity_name, node_list, kg_instance)
+        # 星型事件参与者是图投影，不需要再次 LLM 摘要；只做确定性去重合并，
+        # 避免 LightRAG summarizer 把跨事件上下文扩写成伪业务事实。
+        return await self._merge_entity_node(entity_name, node_list, kg_instance)


docs/PMS_PATCHES.md #7 says entity descriptions and normal edges "continue to reuse the official merge", but this routes all entity nodes through the custom SEP-dedup merge and never calls the LightRAG summarizer. Code and docs disagree; please align one of them.

ChenZiHong-Gavin · 2026-10-16T06:05:29Z

+        event_ids = {
+            str(dp["src_id"])
+            for dp in edge_list
+            if dp.get("event_member") and dp.get("src_id")


merge_edges depends on event_member, but extract() never sets it. The flag is injected afterwards by build_event_kg.py (L47). Anyone using the builder directly gets the wrong branch. Set it in extract() where the edge is created.

ChenZiHong-Gavin · 2026-10-16T06:05:29Z

+
+from graphgen.templates.event_profile_loader import load_event_profile
+
+_pms_profile = load_event_profile("pms_event.v1")


This runs file I/O at import time for every import graphgen.templates, and the exported EVENT_ENTITY_EXTRACTION_PROMPT is not used by the builder itself. It is a compatibility shim for fork consumers; it should not be in upstream.

ChenZiHong-Gavin · 2026-10-16T06:05:29Z

+    import zipfile
+
+    output = tmp_path / "dist"
+    subprocess.run(["uv", "build", str(REPO), "--out-dir", str(output)], check=True, capture_output=True, text=True)


This shells out to uv build and builds the whole package inside a unit test. It requires uv on PATH and is slow. This belongs in a packaging CI job, not in pytest.

ChenZiHong-Gavin · 2026-10-16T06:05:29Z

        name="graphg",
        version=get_version(),
-        url="https://github.com/open-sciencelab/GraphGen",
+        url="https://github.com/Leon-Algo/GraphGen",


Package URL / author / version (_version.py → 1.1.0) are fork release metadata and must not be changed in an upstream PR. Same for the "Maintained fork" README section, docs/RELEASING.md and the two RELEASE_NOTES files.

ChenZiHong-Gavin · 2026-10-16T06:05:29Z

+- `get_current_logger` falls back to the standard `GraphGen` logger only when the ContextVar is absent. This does not catch or replace LLM/model errors and does not fabricate generation output.
+- Covered by `tests/unit_tests/utils/test_init_llm_actor_concurrency.py` and `tests/unit_tests/utils/test_log.py`.
+
+## #9 Ray LLM actor concurrency and logger context fallback (1.1.0)


Section #9 is duplicated (L97 and L103).

ChenZiHong-Gavin · 2026-10-16T06:05:29Z

+from collections import defaultdict
+from typing import Any
+
+PMS_ENTITY_TYPES: list[dict] = [


PMS-specific defaults are hard-coded into the core package: PMS_ENTITY_TYPES, PROJECT_ID_RE, and load_event_profile(None) falling back to pms_event.v1. For upstream, there should be no default profile; a missing profile should be an error. The pms_* prompt/gate/profile assets should stay in the fork.

内置 pms_event.v1 提示词要求锚点必须原样取自片段，并加入防照抄回归测试；与生成策略 v2_2 的加固一致。版本号升至 1.1.1。 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Leon-Algo · 2026-10-16T08:52:00Z

Code review

Found 1 issue:

External profile paths are reduced to profile_id and then loaded again as a built-in ID, so custom profiles that are not packaged fail with event_profile_not_found:. The SDK guide explicitly supports project-specific profile.json paths.

BuildKGService passes only profile_id; the loader switches to packaged lookup for ID-only references; the documented external-path contract.

Generated with Claude Code

Leon-Algo · 2026-10-16T09:04:51Z

Code review

Found 1 issue:

Persisted event membership edges drop the participant role. Extraction includes role, but the deterministic merge builds a fresh edge_data without carrying it through before upsert_edge, so every stored membership edge loses the structured role. This contradicts the documented edge contract.

Merge payload construction · Documented membership-edge role

修复文件型事件 profile 传递、双向 support 边引用、无锚事件分区、JSON prefill 和逐请求限流；保留事件成员标记与角色属性。升级 graphg 到 1.1.2，补充回归测试和发行记录。 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

登记最终候选提交、wheel/sdist 哈希及 Python 3.12 安装证据。 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

记录 fork tag、PMSAgent pin、锁文件同步与兼容测试最终结果。 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Leon-L20 and others added 3 commits September 18, 2026 08:40

github-actions Bot added documentation Improvements or additions to documentation core labels Oct 10, 2026

github-actions Bot added the tests label Oct 10, 2026

ChenZiHong-Gavin reviewed Oct 16, 2026

View reviewed changes

fix: 事件抽取提示词禁止照抄示例项目编号

0692702

内置 pms_event.v1 提示词要求锚点必须原样取自片段，并加入防照抄回归测试；与生成策略 v2_2 的加固一致。版本号升至 1.1.1。 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Leon-L20 and others added 3 commits October 16, 2026 09:27

docs: 记录 GraphGen 1.1.2 发行产物

3a2012d

登记最终候选提交、wheel/sdist 哈希及 Python 3.12 安装证据。 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

docs: 收口 GraphGen 1.1.2 发布验收记录

d45b17b

记录 fork tag、PMSAgent pin、锁文件同步与兼容测试最终结果。 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Navigation Menu

Repository navigation

Add versioned prompt profiles and event-entity generation pipeline - #192

Add versioned prompt profiles and event-entity generation pipeline#192
Leon-Algo wants to merge 9 commits into
InternScience:mainfrom
Leon-Algo:pms/main

Leon-Algo commented Oct 10, 2026 •

edited

Loading

Uh oh!

ChenZiHong-Gavin left a comment

Uh oh!

ChenZiHong-Gavin Oct 16, 2026

Uh oh!

ChenZiHong-Gavin Oct 16, 2026

Uh oh!

ChenZiHong-Gavin Oct 16, 2026

Uh oh!

ChenZiHong-Gavin Oct 16, 2026

Uh oh!

ChenZiHong-Gavin Oct 16, 2026

Uh oh!

ChenZiHong-Gavin Oct 16, 2026

Uh oh!

ChenZiHong-Gavin Oct 16, 2026

Uh oh!

ChenZiHong-Gavin Oct 16, 2026

Uh oh!

ChenZiHong-Gavin Oct 16, 2026

Uh oh!

ChenZiHong-Gavin Oct 16, 2026

Uh oh!

Leon-Algo commented Oct 16, 2026

Uh oh!

Leon-Algo commented Oct 16, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants


		from graphgen.templates.event_profile_loader import load_event_profile

		_pms_profile = load_event_profile("pms_event.v1")

Conversation

Leon-Algo commented Oct 10, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Validation

Notes

Uh oh!

ChenZiHong-Gavin left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Leon-Algo commented Oct 16, 2026

Code review

Uh oh!

Leon-Algo commented Oct 16, 2026

Code review

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Leon-Algo commented Oct 10, 2026 •

edited

Loading