构建知识图谱 · Semantica Cookbook

1概览

本 notebook 演示如何使用 Semantica 的图构建模块从实体和关系构建知识图谱。你将学习使用 GraphBuilder 和 EntityResolver。

学习目标

使用 GraphBuilder 构建知识图谱
使用 EntityResolver 消解实体冲突注意：对于去重，请使用 semantica.deduplication 模块。

2安装

从 PyPI 安装 Semantica：

pip install semantica
# 或安装所有可选依赖：
pip install semantica[all]

3步骤 1：构建知识图谱

从实体和关系构建知识图谱。

# 安装 semantica 包
!pip install semantica

# 从实体与关系构建知识图谱
from semantica.kg import GraphBuilder
from semantica.semantic_extract import NERExtractor, RelationExtractor

builder = GraphBuilder()
ner_extractor = NERExtractor()
relation_extractor = RelationExtractor()

# 包含实体与关系的示例文本
text = "Apple Inc. is a technology company. Tim Cook is the CEO of Apple Inc. Apple Inc. is headquartered in Cupertino, California."

# 先抽取实体，再基于实体抽取关系
entities_list = ner_extractor.extract(text)
relationships_list = relation_extractor.extract(text, entities_list)

# 将实体转换为图谱构建所需的字典格式
entities = []
for i, entity in enumerate(entities_list[:5], 1):
    entities.append({
        "id": f"e{i}",
        "type": entity.label,
        "name": entity.text,
        "properties": {}
    })

# 将关系转换为图谱构建所需的字典格式
relationships = []
for i, rel in enumerate(relationships_list[:3], 1):
    relationships.append({
        "source": f"e{1}",
        "target": f"e{i+1}",
        "type": rel.predicate,
        "properties": {}
    })

# 构建知识图谱
knowledge_graph = builder.build(entities, relationships)

print(f"Built knowledge graph with {len(knowledge_graph.get('entities', []))} entities")
print(f"Relationships: {len(knowledge_graph.get('relationships', []))}")

4步骤 2：实体消解

消解实体冲突和重复。

# 使用 EntityResolver 消解实体冲突与重复
from semantica.kg import EntityResolver

entity_resolver = EntityResolver()

# 对前面构建的实体列表进行消解
resolved_entities = entity_resolver.resolve_entities(entities)

print(f"Original entities: {len(entities)}")
print(f"Resolved entities: {len(resolved_entities)}")

5步骤 3：去重

从图中移除重复实体。

from semantica.deduplication import DuplicateDetector, EntityMerger, MergeStrategy

# 检测重复
detector = DuplicateDetector(similarity_threshold=0.8)
duplicate_groups = detector.detect_duplicate_groups(knowledge_graph.get('entities', []))

# 合并重复
merger = EntityMerger()
merge_operations = merger.merge_duplicates(
    knowledge_graph.get('entities', []),
    strategy=MergeStrategy.KEEP_MOST_COMPLETE
)

deduplicated_entities = [op.merged_entity for op in merge_operations]

print(f"Original entities: {len(knowledge_graph.get('entities', []))}")
print(f"Deduplicated entities: {len(deduplicated_entities)}")

6小结

你已经学会了如何构建知识图谱：

GraphBuilder：从实体和关系构建知识图谱
EntityResolver：消解实体冲突和重复
去重：使用 semantica.deduplication 模块移除重复实体

下一步：在 Graph_Analytics notebook 中学习如何分析图。