滚屏网站模板白度

概述

OpenMetadata 是一个开源的元数据管理平台，支持端到端的血缘关系追踪。对于 MySQL 数据库，OpenMetadata 通过解析表的外键约束、视图定义及查询日志（可选）构建表级血缘。本文结合源码分析其实现机制。

---

环境配置与数据摄取

1. 配置文件示例（YAML）

source:type: mysqlserviceName: mysql_devserviceConnection:config:type: Mysqlusername: adminpassword: passhostPort: localhost:3306databaseSchema: sales_dbsourceConfig:config:includeViews: trueincludeTables: truemarkDeletedTables: truelineageQuery: "SELECT * FROM information_schema.views WHERE view_definition LIKE '%{table}%';"
sink:type: metadata-restconfig: {}
workflowConfig:openMetadataServerConfig:hostPort: "http://localhost:8585/api"authProvider: openmetadatasecurityConfig:jwtToken: "token"

2. 关键配置项

• lineageQuery: 自定义血缘分析 SQL（可选）
• includeViews: 是否解析视图血缘
• markDeletedTables: 处理已删除表

---

源码解析与核心流程

1. 入口类：MysqlSource

路径：openmetadata-ingestion/src/metadata/ingestion/source/database/mysql/connection.py

class MysqlSource(RDBMSSource):def __init__(self, config: WorkflowSource, metadata_config: OpenMetadataConnection):super().__init__(config, metadata_config)self.connection = MysqlConnection(config.serviceConnection.__root__.config)

2. 血缘提取核心方法

路径：openmetadata-ingestion/src/metadata/ingestion/source/database/common_db_source.py

class CommonDbSourceService(ABC):def process_table_lineage(self, table: Table) -> None:# 通过外键解析直接血缘for column in table.columns:if column.foreignKeys:self._build_foreign_key_lineage(table, column)# 通过视图定义解析if self.config.sourceConfig.config.includeViews:view_def = self._get_view_definition(table.name)self._parse_view_lineage(view_def, table)

3. SQL 解析器

路径：openmetadata-ingestion/src/metadata/ingestion/source/database/lineage/parser.py

class LineageParser:@staticmethoddef parse(sql: str) -> List[LineageEdge]:# 使用 ANTLR 解析 SQL，生成语法树parser = SqlLineageParser(sql)return parser.get_lineage_edges()

---

4. 流程图