背景
最近看看开源的元数据管理项目，比较出名点的有open-metadata、datahub、OpenLineage、atlas。
open-metadata有1千多的贡献者，4.8K的stars，社区现在也比较活跃，支持的数据库类型还蛮多，基本市面上常见的都有支持，项目迭代也比较快，正好最近我自己也在搞数据资产治理这块，打算本地部署一下研究研究。
按照官网的介绍，采用docker部署最简单，打算采用docker部署的方式本地部署一下。但是由于各种不好直接说的原因，国内通过docker部署应用变得极其不方便（就像有个网友说的：没有困难我们要制造困难，然后迎难而上！）。
- open-metadata官网：https://open-metadata.org/
环境准备
我是在虚拟机里面弄了一个centos7来部署。
- centos7修改yum源为阿里源
sudo mv /etc/yum.repos.d/CentOS-Base.repo /etc/yum.repos.d/CentOS-Base.repo.backup
sudo curl -o /etc/yum.repos.d/CentOS-Base.repo http://mirrors.aliyun.com/repo/Centos-7.repo
sudo yum clean all
sudo yum makecache
docker安装
- 安装基础依赖
yum install -y yum-utils device-mapper-persistent-data lvm2 --skip-broken
- 设置docker镜像源
yum-config-manager --add-repo https://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.reposed -i 's/download.docker.com/mirrors.aliyun.com\/docker-ce/g' /etc/yum.repos.d/docker-ce.repoyum makecache fast
- 安装docker
yum install -y docker-ce## 启动docker
sudo systemctl start docker## 查看docker版本
docker -v## 查看docker基本信息
docker info
docker-compose安装
docker-compose是docker的服务编排工具，其安装依赖Python3.
- python3安装
yum install -y python3 python3-devel python3-libs python3-tools
python3 -m ensurepip
python3 -m pip install --upgrade pippython3 -V
Python 3.6.8
- 安装docker-compose
pip install docker-compose
镜像准备
国内docker镜像源几乎全军覆没，阿里还有个免费私有镜像服务可以用（良心啊），允许管理300个镜像，对个人来说也够用了。能够通过github的流水线（cicd）能力自动拉取docker hub的镜像然后推送到阿里私有镜像库。
阿里私有docker镜像服务地址：
https://cr.console.aliyun.com/
可以参看这位老铁的博客，https://blog.csdn.net/weixin_59164654/article/details/139601846。
github也有一个现成的开源项目，可以fork到自己名下配置使用。
- 项目地址
https://github.com/tech-shrimp/docker_image_pusher - 项目使用教学视频
https://www.bilibili.com/video/BV1Py411877t
下载open-metadata docker-compose.yml
可以到open-metadata 的github主页找到最新版本。
https://github.com/open-metadata/OpenMetadata/releases/tag/1.4.3-release
在docker-compose.yml中我们发现有4个镜像文件
- image: docker.getcollate.io/openmetadata/db:1.4.3
- image: docker.elastic.co/elasticsearch/elasticsearch:8.10.2
- image: docker.getcollate.io/openmetadata/server:1.4.3
- image: docker.getcollate.io/openmetadata/ingestion:1.4.3
这4个docker镜像用上面提到的docker_image_pusher方式转储到自己的阿里私有docker仓库中心。
在虚拟机登录阿里私有镜像库拉取转储后的镜像。
## 登录docker login --username=username@163.com registry.cn-hangzhou.aliyuncs.com## 拉取镜像docker pull registry.cn-hangzhou.aliyuncs.com/itclj/db:1.4.3docker pull registry.cn-hangzhou.aliyuncs.com/itclj/elasticsearch:8.10.2docker pull registry.cn-hangzhou.aliyuncs.com/itclj/server:1.4.3docker pull registry.cn-hangzhou.aliyuncs.com/itclj/ingestion:1.4.3
通过阿里私有镜像库拉取镜像还是非常快的，几分钟都拉完了。
- 修改open-metadata 的docker-compose.yml，把镜像名称改为转储后的。
| 原镜像名称 | 转储后镜像名称 |
|---|---|
| docker.getcollate.io/openmetadata/db:1.4.3 | registry.cn-hangzhou.aliyuncs.com/itclj/db:1.4.3 |
| docker.elastic.co/elasticsearch/elasticsearch:8.10.2 | registry.cn-hangzhou.aliyuncs.com/itclj/elasticsearch:8.10.2 |
| docker.getcollate.io/openmetadata/server:1.4.3 | registry.cn-hangzhou.aliyuncs.com/itclj/server:1.4.3 |
| docker.getcollate.io/openmetadata/ingestion:1.4.3 | registry.cn-hangzhou.aliyuncs.com/itclj/ingestion:1.4.3 |
- 修改前后的docker-compose.yml比较。
- 修改后的 itclj-docker-compose.yml
# Copyright 2021 Collate
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
# http://www.apache.org/licenses/LICENSE-2.0
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.version: "3.9"
volumes:ingestion-volume-dag-airflow:ingestion-volume-dags:ingestion-volume-tmp:es-data:
services:mysql:container_name: openmetadata_mysqlimage: registry.cn-hangzhou.aliyuncs.com/itclj/db:1.4.3command: "--sort_buffer_size=10M"restart: alwaysenvironment:MYSQL_ROOT_PASSWORD: itclj123456expose:- 3306ports:- "3306:3306"volumes:- ./docker-volume/db-data:/var/lib/mysqlnetworks:- app_nethealthcheck:test: mysql --user=root --password=$$MYSQL_ROOT_PASSWORD --silent --execute "use openmetadata_db"interval: 15stimeout: 10sretries: 10elasticsearch:container_name: openmetadata_elasticsearchimage: registry.cn-hangzhou.aliyuncs.com/itclj/elasticsearch:8.10.2environment:- discovery.type=single-node- ES_JAVA_OPTS=-Xms1024m -Xmx1024m- xpack.security.enabled=falsenetworks:- app_netports:- "9200:9200"- "9300:9300"healthcheck:test: "curl -s http://localhost:9200/_cluster/health?pretty | grep status | grep -qE 'green|yellow' || exit 1"interval: 15stimeout: 10sretries: 10volumes:- es-data:/usr/share/elasticsearch/dataexecute-migrate-all:container_name: execute_migrate_allimage: registry.cn-hangzhou.aliyuncs.com/itclj/server:1.4.3command: "./bootstrap/openmetadata-ops.sh migrate"environment:OPENMETADATA_CLUSTER_NAME: ${OPENMETADATA_CLUSTER_NAME:-openmetadata}SERVER_PORT: ${SERVER_PORT:-8585}SERVER_ADMIN_PORT: ${SERVER_ADMIN_PORT:-8586}LOG_LEVEL: ${LOG_LEVEL:-INFO}# Migration MIGRATION_LIMIT_PARAM: ${MIGRATION_LIMIT_PARAM:-1200}# OpenMetadata Server Authentication ConfigurationAUTHORIZER_CLASS_NAME: ${AUTHORIZER_CLASS_NAME:-org.openmetadata.service.security.DefaultAuthorizer}AUTHORIZER_REQUEST_FILTER: ${AUTHORIZER_REQUEST_FILTER:-org.openmetadata.service.security.JwtFilter}AUTHORIZER_ADMIN_PRINCIPALS: ${AUTHORIZER_ADMIN_PRINCIPALS:-[admin]}AUTHORIZER_ALLOWED_REGISTRATION_DOMAIN: ${AUTHORIZER_ALLOWED_REGISTRATION_DOMAIN:-["all"]}AUTHORIZER_INGESTION_PRINCIPALS: ${AUTHORIZER_INGESTION_PRINCIPALS:-[ingestion-bot]}AUTHORIZER_PRINCIPAL_DOMAIN: ${AUTHORIZER_PRINCIPAL_DOMAIN:-"openmetadata.org"}AUTHORIZER_ENFORCE_PRINCIPAL_DOMAIN: ${AUTHORIZER_ENFORCE_PRINCIPAL_DOMAIN:-false}AUTHORIZER_ENABLE_SECURE_SOCKET: ${AUTHORIZER_ENABLE_SECURE_SOCKET:-false}AUTHENTICATION_PROVIDER: ${AUTHENTICATION_PROVIDER:-basic}AUTHENTICATION_RESPONSE_TYPE: ${AUTHENTICATION_RESPONSE_TYPE:-id_token}CUSTOM_OIDC_AUTHENTICATION_PROVIDER_NAME: ${CUSTOM_OIDC_AUTHENTICATION_PROVIDER_NAME:-""}AUTHENTICATION_PUBLIC_KEYS: ${AUTHENTICATION_PUBLIC_KEYS:-[http://localhost:8585/api/v1/system/config/jwks]}AUTHENTICATION_AUTHORITY: ${AUTHENTICATION_AUTHORITY:-https://accounts.google.com}AUTHENTICATION_CLIENT_ID: ${AUTHENTICATION_CLIENT_ID:-""}AUTHENTICATION_CALLBACK_URL: ${AUTHENTICATION_CALLBACK_URL:-""}AUTHENTICATION_JWT_PRINCIPAL_CLAIMS: ${AUTHENTICATION_JWT_PRINCIPAL_CLAIMS:-[email,preferred_username,sub]}AUTHENTICATION_ENABLE_SELF_SIGNUP: ${AUTHENTICATION_ENABLE_SELF_SIGNUP:-true}AUTHENTICATION_CLIENT_TYPE: ${AUTHENTICATION_CLIENT_TYPE:-public}#For OIDC Authentication, when client is confidentialOIDC_CLIENT_ID: ${OIDC_CLIENT_ID:-""}OIDC_TYPE: ${OIDC_TYPE:-""} # google, azure etc.OIDC_CLIENT_SECRET: ${OIDC_CLIENT_SECRET:-""}OIDC_SCOPE: ${OIDC_SCOPE:-"openid email profile"}OIDC_DISCOVERY_URI: ${OIDC_DISCOVERY_URI:-""}OIDC_USE_NONCE: ${OIDC_USE_NONCE:-true}OIDC_PREFERRED_JWS: ${OIDC_PREFERRED_JWS:-"RS256"}OIDC_RESPONSE_TYPE: ${OIDC_RESPONSE_TYPE:-"code"}OIDC_DISABLE_PKCE: ${OIDC_DISABLE_PKCE:-true}OIDC_CALLBACK: ${OIDC_CALLBACK:-"http://localhost:8585/callback"}OIDC_SERVER_URL: ${OIDC_SERVER_URL:-"http://localhost:8585"}OIDC_CLIENT_AUTH_METHOD: ${OIDC_CLIENT_AUTH_METHOD:-"client_secret_post"}OIDC_TENANT: ${OIDC_TENANT:-""}OIDC_MAX_CLOCK_SKEW: ${OIDC_MAX_CLOCK_SKEW:-""}OIDC_CUSTOM_PARAMS: ${OIDC_CUSTOM_PARAMS:-{}}# For SAML Authentication# SAML_DEBUG_MODE: ${SAML_DEBUG_MODE:-false}# SAML_IDP_ENTITY_ID: ${SAML_IDP_ENTITY_ID:-""}# SAML_IDP_SSO_LOGIN_URL: ${SAML_IDP_SSO_LOGIN_URL:-""}# SAML_IDP_CERTIFICATE: ${SAML_IDP_CERTIFICATE:-""}# SAML_AUTHORITY_URL: ${SAML_AUTHORITY_URL:-"http://localhost:8585/api/v1/saml/login"}# SAML_IDP_NAME_ID: ${SAML_IDP_NAME_ID:-"urn:oasis:names:tc:SAML:2.0:nameid-format:emailAddress"}# SAML_SP_ENTITY_ID: ${SAML_SP_ENTITY_ID:-"http://localhost:8585/api/v1/saml/metadata"}# SAML_SP_ACS: ${SAML_SP_ACS:-"http://localhost:8585/api/v1/saml/acs"}# SAML_SP_CERTIFICATE: ${SAML_SP_CERTIFICATE:-""}# SAML_SP_CALLBACK: ${SAML_SP_CALLBACK:-"http://localhost:8585/saml/callback"}# SAML_STRICT_MODE: ${SAML_STRICT_MODE:-false}# SAML_SP_TOKEN_VALIDITY: ${SAML_SP_TOKEN_VALIDITY:-"3600"}# SAML_SEND_ENCRYPTED_NAME_ID: ${SAML_SEND_ENCRYPTED_NAME_ID:-false}# SAML_SEND_SIGNED_AUTH_REQUEST: ${SAML_SEND_SIGNED_AUTH_REQUEST:-false}# SAML_SIGNED_SP_METADATA: ${SAML_SIGNED_SP_METADATA:-false}# SAML_WANT_MESSAGE_SIGNED: ${SAML_WANT_MESSAGE_SIGNED:-false}# SAML_WANT_ASSERTION_SIGNED: ${SAML_WANT_ASSERTION_SIGNED:-false}# SAML_WANT_ASSERTION_ENCRYPTED: ${SAML_WANT_ASSERTION_ENCRYPTED:-false}# SAML_WANT_NAME_ID_ENCRYPTED: ${SAML_WANT_NAME_ID_ENCRYPTED:-false}# SAML_KEYSTORE_FILE_PATH: ${SAML_KEYSTORE_FILE_PATH:-""}# SAML_KEYSTORE_ALIAS: ${SAML_KEYSTORE_ALIAS:-""}# SAML_KEYSTORE_PASSWORD: ${SAML_KEYSTORE_PASSWORD:-""}# For LDAP Authentication# AUTHENTICATION_LDAP_HOST: ${AUTHENTICATION_LDAP_HOST:-}# AUTHENTICATION_LDAP_PORT: ${AUTHENTICATION_LDAP_PORT:-}# AUTHENTICATION_LOOKUP_ADMIN_DN: ${AUTHENTICATION_LOOKUP_ADMIN_DN:-""}# AUTHENTICATION_LOOKUP_ADMIN_PWD: ${AUTHENTICATION_LOOKUP_ADMIN_PWD:-""}# AUTHENTICATION_USER_LOOKUP_BASEDN: ${AUTHENTICATION_USER_LOOKUP_BASEDN:-""}# AUTHENTICATION_USER_MAIL_ATTR: ${AUTHENTICATION_USER_MAIL_ATTR:-}# AUTHENTICATION_LDAP_POOL_SIZE: ${AUTHENTICATION_LDAP_POOL_SIZE:-3}# AUTHENTICATION_LDAP_SSL_ENABLED: ${AUTHENTICATION_LDAP_SSL_ENABLED:-}# AUTHENTICATION_LDAP_TRUSTSTORE_TYPE: ${AUTHENTICATION_LDAP_TRUSTSTORE_TYPE:-TrustAll}# AUTHENTICATION_LDAP_TRUSTSTORE_PATH: ${AUTHENTICATION_LDAP_TRUSTSTORE_PATH:-}# AUTHENTICATION_LDAP_KEYSTORE_PASSWORD: ${AUTHENTICATION_LDAP_KEYSTORE_PASSWORD:-}# AUTHENTICATION_LDAP_SSL_KEY_FORMAT: ${AUTHENTICATION_LDAP_SSL_KEY_FORMAT:-}# AUTHENTICATION_LDAP_ALLOW_WILDCARDS: ${AUTHENTICATION_LDAP_ALLOW_WILDCARDS:-}# AUTHENTICATION_LDAP_ALLOWED_HOSTNAMES: ${AUTHENTICATION_LDAP_ALLOWED_HOSTNAMES:-[]}# AUTHENTICATION_LDAP_SSL_VERIFY_CERT_HOST: ${AUTHENTICATION_LDAP_SSL_VERIFY_CERT_HOST:-}# AUTHENTICATION_LDAP_EXAMINE_VALIDITY_DATES: ${AUTHENTICATION_LDAP_EXAMINE_VALIDITY_DATES:-true}# JWT ConfigurationRSA_PUBLIC_KEY_FILE_PATH: ${RSA_PUBLIC_KEY_FILE_PATH:-"./conf/public_key.der"}RSA_PRIVATE_KEY_FILE_PATH: ${RSA_PRIVATE_KEY_FILE_PATH:-"./conf/private_key.der"}JWT_ISSUER: ${JWT_ISSUER:-"open-metadata.org"}JWT_KEY_ID: ${JWT_KEY_ID:-"Gb389a-9f76-gdjs-a92j-0242bk94356"}# OpenMetadata Server Pipeline Service Client ConfigurationPIPELINE_SERVICE_CLIENT_ENDPOINT: ${PIPELINE_SERVICE_CLIENT_ENDPOINT:-http://ingestion:8080}PIPELINE_SERVICE_CLIENT_HEALTH_CHECK_INTERVAL: ${PIPELINE_SERVICE_CLIENT_HEALTH_CHECK_INTERVAL:-300}SERVER_HOST_API_URL: ${SERVER_HOST_API_URL:-http://openmetadata-server:8585/api}PIPELINE_SERVICE_CLIENT_VERIFY_SSL: ${PIPELINE_SERVICE_CLIENT_VERIFY_SSL:-"no-ssl"}PIPELINE_SERVICE_CLIENT_SSL_CERT_PATH: ${PIPELINE_SERVICE_CLIENT_SSL_CERT_PATH:-""}# Database configuration for MySQLDB_DRIVER_CLASS: ${DB_DRIVER_CLASS:-com.mysql.cj.jdbc.Driver}DB_SCHEME: ${DB_SCHEME:-mysql}DB_PARAMS: ${DB_PARAMS:-allowPublicKeyRetrieval=true&useSSL=false&serverTimezone=UTC}DB_USER: ${DB_USER:-openmetadata_user}DB_USER_PASSWORD: ${DB_USER_PASSWORD:-openmetadata_password}DB_HOST: ${DB_HOST:-mysql}DB_PORT: ${DB_PORT:-3306}OM_DATABASE: ${OM_DATABASE:-openmetadata_db}# ElasticSearch ConfigurationsELASTICSEARCH_HOST: ${ELASTICSEARCH_HOST:- elasticsearch}ELASTICSEARCH_PORT: ${ELASTICSEARCH_PORT:-9200}ELASTICSEARCH_SCHEME: ${ELASTICSEARCH_SCHEME:-http}ELASTICSEARCH_USER: ${ELASTICSEARCH_USER:-""}ELASTICSEARCH_PASSWORD: ${ELASTICSEARCH_PASSWORD:-""}SEARCH_TYPE: ${SEARCH_TYPE:- "elasticsearch"}ELASTICSEARCH_TRUST_STORE_PATH: ${ELASTICSEARCH_TRUST_STORE_PATH:-""}ELASTICSEARCH_TRUST_STORE_PASSWORD: ${ELASTICSEARCH_TRUST_STORE_PASSWORD:-""}ELASTICSEARCH_CONNECTION_TIMEOUT_SECS: ${ELASTICSEARCH_CONNECTION_TIMEOUT_SECS:-5}ELASTICSEARCH_SOCKET_TIMEOUT_SECS: ${ELASTICSEARCH_SOCKET_TIMEOUT_SECS:-60}ELASTICSEARCH_KEEP_ALIVE_TIMEOUT_SECS: ${ELASTICSEARCH_KEEP_ALIVE_TIMEOUT_SECS:-600}ELASTICSEARCH_BATCH_SIZE: ${ELASTICSEARCH_BATCH_SIZE:-10}ELASTICSEARCH_PAYLOAD_BYTES_SIZE: ${ELASTICSEARCH_PAYLOAD_BYTES_SIZE:-10485760} #max payLoadSize in BytesELASTICSEARCH_INDEX_MAPPING_LANG: ${ELASTICSEARCH_INDEX_MAPPING_LANG:-EN}#eventMonitoringConfigurationEVENT_MONITOR: ${EVENT_MONITOR:-prometheus}EVENT_MONITOR_BATCH_SIZE: ${EVENT_MONITOR_BATCH_SIZE:-10}EVENT_MONITOR_PATH_PATTERN: ${EVENT_MONITOR_PATH_PATTERN:-["/api/v1/tables/*", "/api/v1/health-check"]}EVENT_MONITOR_LATENCY: ${EVENT_MONITOR_LATENCY:-[]}#pipelineServiceClientConfigurationPIPELINE_SERVICE_CLIENT_ENABLED: ${PIPELINE_SERVICE_CLIENT_ENABLED:-true}PIPELINE_SERVICE_CLIENT_CLASS_NAME: ${PIPELINE_SERVICE_CLIENT_CLASS_NAME:-"org.openmetadata.service.clients.pipeline.airflow.AirflowRESTClient"}PIPELINE_SERVICE_IP_INFO_ENABLED: ${PIPELINE_SERVICE_IP_INFO_ENABLED:-false}PIPELINE_SERVICE_CLIENT_HOST_IP: ${PIPELINE_SERVICE_CLIENT_HOST_IP:-""}PIPELINE_SERVICE_CLIENT_SECRETS_MANAGER_LOADER: ${PIPELINE_SERVICE_CLIENT_SECRETS_MANAGER_LOADER:-"noop"}#airflow parametersAIRFLOW_USERNAME: ${AIRFLOW_USERNAME:-admin}AIRFLOW_PASSWORD: ${AIRFLOW_PASSWORD:-admin}AIRFLOW_TIMEOUT: ${AIRFLOW_TIMEOUT:-10}AIRFLOW_TRUST_STORE_PATH: ${AIRFLOW_TRUST_STORE_PATH:-""}AIRFLOW_TRUST_STORE_PASSWORD: ${AIRFLOW_TRUST_STORE_PASSWORD:-""}FERNET_KEY: ${FERNET_KEY:-jJ/9sz0g0OHxsfxOoSfdFdmk3ysNmPRnH3TUAbz3IHA=}#secretsManagerConfigurationSECRET_MANAGER: ${SECRET_MANAGER:-db}# AWS:OM_SM_REGION: ${OM_SM_REGION:-""}OM_SM_ACCESS_KEY_ID: ${OM_SM_ACCESS_KEY_ID:-""}OM_SM_ACCESS_KEY: ${OM_SM_ACCESS_KEY:-""}# Azure:OM_SM_VAULT_NAME: ${OM_SM_VAULT_NAME:-""}OM_SM_CLIENT_ID: ${OM_SM_CLIENT_ID:-""}OM_SM_CLIENT_SECRET: ${OM_SM_CLIENT_SECRET:-""}OM_SM_TENANT_ID: ${OM_SM_TENANT_ID:-""}#email configuration:OM_EMAIL_ENTITY: ${OM_EMAIL_ENTITY:-"OpenMetadata"}OM_SUPPORT_URL: ${OM_SUPPORT_URL:-"https://slack.open-metadata.org"}AUTHORIZER_ENABLE_SMTP : ${AUTHORIZER_ENABLE_SMTP:-false}OPENMETADATA_SERVER_URL: ${OPENMETADATA_SERVER_URL:-""}OPENMETADATA_SMTP_SENDER_MAIL: ${OPENMETADATA_SMTP_SENDER_MAIL:-""}SMTP_SERVER_ENDPOINT: ${SMTP_SERVER_ENDPOINT:-""}SMTP_SERVER_PORT: ${SMTP_SERVER_PORT:-""}SMTP_SERVER_USERNAME: ${SMTP_SERVER_USERNAME:-""}SMTP_SERVER_PWD: ${SMTP_SERVER_PWD:-""}SMTP_SERVER_STRATEGY: ${SMTP_SERVER_STRATEGY:-"SMTP_TLS"}# Heap OPTS ConfigurationsOPENMETADATA_HEAP_OPTS: ${OPENMETADATA_HEAP_OPTS:--Xmx1G -Xms1G}# Mask passwords values in UIMASK_PASSWORDS_API: ${MASK_PASSWORDS_API:-false}#OpenMetadata Web ConfigurationWEB_CONF_URI_PATH: ${WEB_CONF_URI_PATH:-"/api"}#HSTSWEB_CONF_HSTS_ENABLED: ${WEB_CONF_HSTS_ENABLED:-false}WEB_CONF_HSTS_MAX_AGE: ${WEB_CONF_HSTS_MAX_AGE:-"365 days"}WEB_CONF_HSTS_INCLUDE_SUBDOMAINS: ${WEB_CONF_HSTS_INCLUDE_SUBDOMAINS:-"true"}WEB_CONF_HSTS_PRELOAD: ${WEB_CONF_HSTS_PRELOAD:-"true"}#Frame OptionsWEB_CONF_FRAME_OPTION_ENABLED: ${WEB_CONF_FRAME_OPTION_ENABLED:-false}WEB_CONF_FRAME_OPTION: ${WEB_CONF_FRAME_OPTION:-"SAMEORIGIN"}WEB_CONF_FRAME_ORIGIN: ${WEB_CONF_FRAME_ORIGIN:-""}#Content TypeWEB_CONF_CONTENT_TYPE_OPTIONS_ENABLED: ${WEB_CONF_CONTENT_TYPE_OPTIONS_ENABLED:-false}#XSS-Protection WEB_CONF_XSS_PROTECTION_ENABLED: ${WEB_CONF_XSS_PROTECTION_ENABLED:-false}WEB_CONF_XSS_PROTECTION_ON: ${WEB_CONF_XSS_PROTECTION_ON:-true}WEB_CONF_XSS_PROTECTION_BLOCK: ${WEB_CONF_XSS_PROTECTION_BLOCK:-true}#CSP WEB_CONF_XSS_CSP_ENABLED: ${WEB_CONF_XSS_CSP_ENABLED:-false}WEB_CONF_XSS_CSP_POLICY: ${WEB_CONF_XSS_CSP_POLICY:-"default-src 'self'"}WEB_CONF_XSS_CSP_REPORT_ONLY_POLICY: ${WEB_CONF_XSS_CSP_REPORT_ONLY_POLICY:-""}#Referrer-PolicyWEB_CONF_REFERRER_POLICY_ENABLED: ${WEB_CONF_REFERRER_POLICY_ENABLED:-false}WEB_CONF_REFERRER_POLICY_OPTION: ${WEB_CONF_REFERRER_POLICY_OPTION:-"SAME_ORIGIN"}#Permission-PolicyWEB_CONF_PERMISSION_POLICY_ENABLED: ${WEB_CONF_PERMISSION_POLICY_ENABLED:-false}WEB_CONF_PERMISSION_POLICY_OPTION: ${WEB_CONF_PERMISSION_POLICY_OPTION:-""}depends_on:elasticsearch:condition: service_healthymysql:condition: service_healthynetworks:- app_netopenmetadata-server:container_name: openmetadata_serverrestart: alwaysimage: registry.cn-hangzhou.aliyuncs.com/itclj/server:1.4.3environment:OPENMETADATA_CLUSTER_NAME: ${OPENMETADATA_CLUSTER_NAME:-openmetadata}SERVER_PORT: ${SERVER_PORT:-8585}SERVER_ADMIN_PORT: ${SERVER_ADMIN_PORT:-8586}LOG_LEVEL: ${LOG_LEVEL:-INFO}# OpenMetadata Server Authentication ConfigurationAUTHORIZER_CLASS_NAME: ${AUTHORIZER_CLASS_NAME:-org.openmetadata.service.security.DefaultAuthorizer}AUTHORIZER_REQUEST_FILTER: ${AUTHORIZER_REQUEST_FILTER:-org.openmetadata.service.security.JwtFilter}AUTHORIZER_ADMIN_PRINCIPALS: ${AUTHORIZER_ADMIN_PRINCIPALS:-[admin]}AUTHORIZER_ALLOWED_REGISTRATION_DOMAIN: ${AUTHORIZER_ALLOWED_REGISTRATION_DOMAIN:-["all"]}AUTHORIZER_INGESTION_PRINCIPALS: ${AUTHORIZER_INGESTION_PRINCIPALS:-[ingestion-bot]}AUTHORIZER_PRINCIPAL_DOMAIN: ${AUTHORIZER_PRINCIPAL_DOMAIN:-"openmetadata.org"}AUTHORIZER_ENFORCE_PRINCIPAL_DOMAIN: ${AUTHORIZER_ENFORCE_PRINCIPAL_DOMAIN:-false}AUTHORIZER_ENABLE_SECURE_SOCKET: ${AUTHORIZER_ENABLE_SECURE_SOCKET:-false}AUTHENTICATION_PROVIDER: ${AUTHENTICATION_PROVIDER:-basic}AUTHENTICATION_RESPONSE_TYPE: ${AUTHENTICATION_RESPONSE_TYPE:-id_token}CUSTOM_OIDC_AUTHENTICATION_PROVIDER_NAME: ${CUSTOM_OIDC_AUTHENTICATION_PROVIDER_NAME:-""}AUTHENTICATION_PUBLIC_KEYS: ${AUTHENTICATION_PUBLIC_KEYS:-[http://localhost:8585/api/v1/system/config/jwks]}AUTHENTICATION_AUTHORITY: ${AUTHENTICATION_AUTHORITY:-https://accounts.google.com}AUTHENTICATION_CLIENT_ID: ${AUTHENTICATION_CLIENT_ID:-""}AUTHENTICATION_CALLBACK_URL: ${AUTHENTICATION_CALLBACK_URL:-""}AUTHENTICATION_JWT_PRINCIPAL_CLAIMS: ${AUTHENTICATION_JWT_PRINCIPAL_CLAIMS:-[email,preferred_username,sub]}AUTHENTICATION_ENABLE_SELF_SIGNUP: ${AUTHENTICATION_ENABLE_SELF_SIGNUP:-true}AUTHENTICATION_CLIENT_TYPE: ${AUTHENTICATION_CLIENT_TYPE:-public}#For OIDC Authentication, when client is confidentialOIDC_CLIENT_ID: ${OIDC_CLIENT_ID:-""}OIDC_TYPE: ${OIDC_TYPE:-""} # google, azure etc.OIDC_CLIENT_SECRET: ${OIDC_CLIENT_SECRET:-""}OIDC_SCOPE: ${OIDC_SCOPE:-"openid email profile"}OIDC_DISCOVERY_URI: ${OIDC_DISCOVERY_URI:-""}OIDC_USE_NONCE: ${OIDC_USE_NONCE:-true}OIDC_PREFERRED_JWS: ${OIDC_PREFERRED_JWS:-"RS256"}OIDC_RESPONSE_TYPE: ${OIDC_RESPONSE_TYPE:-"code"}OIDC_DISABLE_PKCE: ${OIDC_DISABLE_PKCE:-true}OIDC_CALLBACK: ${OIDC_CALLBACK:-"http://localhost:8585/callback"}OIDC_SERVER_URL: ${OIDC_SERVER_URL:-"http://localhost:8585"}OIDC_CLIENT_AUTH_METHOD: ${OIDC_CLIENT_AUTH_METHOD:-"client_secret_post"}OIDC_TENANT: ${OIDC_TENANT:-""}OIDC_MAX_CLOCK_SKEW: ${OIDC_MAX_CLOCK_SKEW:-""}OIDC_CUSTOM_PARAMS: ${OIDC_CUSTOM_PARAMS:-{}}# For SAML Authentication# SAML_DEBUG_MODE: ${SAML_DEBUG_MODE:-false}# SAML_IDP_ENTITY_ID: ${SAML_IDP_ENTITY_ID:-""}# SAML_IDP_SSO_LOGIN_URL: ${SAML_IDP_SSO_LOGIN_URL:-""}# SAML_IDP_CERTIFICATE: ${SAML_IDP_CERTIFICATE:-""}# SAML_AUTHORITY_URL: ${SAML_AUTHORITY_URL:-"http://localhost:8585/api/v1/saml/login"}# SAML_IDP_NAME_ID: ${SAML_IDP_NAME_ID:-"urn:oasis:names:tc:SAML:2.0:nameid-format:emailAddress"}# SAML_SP_ENTITY_ID: ${SAML_SP_ENTITY_ID:-"http://localhost:8585/api/v1/saml/metadata"}# SAML_SP_ACS: ${SAML_SP_ACS:-"http://localhost:8585/api/v1/saml/acs"}# SAML_SP_CERTIFICATE: ${SAML_SP_CERTIFICATE:-""}# SAML_SP_CALLBACK: ${SAML_SP_CALLBACK:-"http://localhost:8585/saml/callback"}# SAML_STRICT_MODE: ${SAML_STRICT_MODE:-false}# SAML_SP_TOKEN_VALIDITY: ${SAML_SP_TOKEN_VALIDITY:-"3600"}# SAML_SEND_ENCRYPTED_NAME_ID: ${SAML_SEND_ENCRYPTED_NAME_ID:-false}# SAML_SEND_SIGNED_AUTH_REQUEST: ${SAML_SEND_SIGNED_AUTH_REQUEST:-false}# SAML_SIGNED_SP_METADATA: ${SAML_SIGNED_SP_METADATA:-false}# SAML_WANT_MESSAGE_SIGNED: ${SAML_WANT_MESSAGE_SIGNED:-false}# SAML_WANT_ASSERTION_SIGNED: ${SAML_WANT_ASSERTION_SIGNED:-false}# SAML_WANT_ASSERTION_ENCRYPTED: ${SAML_WANT_ASSERTION_ENCRYPTED:-false}# SAML_WANT_NAME_ID_ENCRYPTED: ${SAML_WANT_NAME_ID_ENCRYPTED:-false}# SAML_KEYSTORE_FILE_PATH: ${SAML_KEYSTORE_FILE_PATH:-""}# SAML_KEYSTORE_ALIAS: ${SAML_KEYSTORE_ALIAS:-""}# SAML_KEYSTORE_PASSWORD: ${SAML_KEYSTORE_PASSWORD:-""}# For LDAP Authentication# AUTHENTICATION_LDAP_HOST: ${AUTHENTICATION_LDAP_HOST:-}# AUTHENTICATION_LDAP_PORT: ${AUTHENTICATION_LDAP_PORT:-}# AUTHENTICATION_LOOKUP_ADMIN_DN: ${AUTHENTICATION_LOOKUP_ADMIN_DN:-""}# AUTHENTICATION_LOOKUP_ADMIN_PWD: ${AUTHENTICATION_LOOKUP_ADMIN_PWD:-""}# AUTHENTICATION_USER_LOOKUP_BASEDN: ${AUTHENTICATION_USER_LOOKUP_BASEDN:-""}# AUTHENTICATION_USER_MAIL_ATTR: ${AUTHENTICATION_USER_MAIL_ATTR:-}# AUTHENTICATION_LDAP_POOL_SIZE: ${AUTHENTICATION_LDAP_POOL_SIZE:-3}# AUTHENTICATION_LDAP_SSL_ENABLED: ${AUTHENTICATION_LDAP_SSL_ENABLED:-}# AUTHENTICATION_LDAP_TRUSTSTORE_TYPE: ${AUTHENTICATION_LDAP_TRUSTSTORE_TYPE:-TrustAll}# AUTHENTICATION_LDAP_TRUSTSTORE_PATH: ${AUTHENTICATION_LDAP_TRUSTSTORE_PATH:-}# AUTHENTICATION_LDAP_KEYSTORE_PASSWORD: ${AUTHENTICATION_LDAP_KEYSTORE_PASSWORD:-}# AUTHENTICATION_LDAP_SSL_KEY_FORMAT: ${AUTHENTICATION_LDAP_SSL_KEY_FORMAT:-}# AUTHENTICATION_LDAP_ALLOW_WILDCARDS: ${AUTHENTICATION_LDAP_ALLOW_WILDCARDS:-}# AUTHENTICATION_LDAP_ALLOWED_HOSTNAMES: ${AUTHENTICATION_LDAP_ALLOWED_HOSTNAMES:-[]}# AUTHENTICATION_LDAP_SSL_VERIFY_CERT_HOST: ${AUTHENTICATION_LDAP_SSL_VERIFY_CERT_HOST:-}# AUTHENTICATION_LDAP_EXAMINE_VALIDITY_DATES: ${AUTHENTICATION_LDAP_EXAMINE_VALIDITY_DATES:-true}# JWT ConfigurationRSA_PUBLIC_KEY_FILE_PATH: ${RSA_PUBLIC_KEY_FILE_PATH:-"./conf/public_key.der"}RSA_PRIVATE_KEY_FILE_PATH: ${RSA_PRIVATE_KEY_FILE_PATH:-"./conf/private_key.der"}JWT_ISSUER: ${JWT_ISSUER:-"open-metadata.org"}JWT_KEY_ID: ${JWT_KEY_ID:-"Gb389a-9f76-gdjs-a92j-0242bk94356"}# OpenMetadata Server Pipeline Service Client ConfigurationPIPELINE_SERVICE_CLIENT_ENDPOINT: ${PIPELINE_SERVICE_CLIENT_ENDPOINT:-http://ingestion:8080}PIPELINE_SERVICE_CLIENT_HEALTH_CHECK_INTERVAL: ${PIPELINE_SERVICE_CLIENT_HEALTH_CHECK_INTERVAL:-300}SERVER_HOST_API_URL: ${SERVER_HOST_API_URL:-http://openmetadata-server:8585/api}PIPELINE_SERVICE_CLIENT_VERIFY_SSL: ${PIPELINE_SERVICE_CLIENT_VERIFY_SSL:-"no-ssl"}PIPELINE_SERVICE_CLIENT_SSL_CERT_PATH: ${PIPELINE_SERVICE_CLIENT_SSL_CERT_PATH:-""}# Database configuration for MySQLDB_DRIVER_CLASS: ${DB_DRIVER_CLASS:-com.mysql.cj.jdbc.Driver}DB_SCHEME: ${DB_SCHEME:-mysql}DB_PARAMS: ${DB_PARAMS:-allowPublicKeyRetrieval=true&useSSL=false&serverTimezone=UTC}DB_USER: ${DB_USER:-openmetadata_user}DB_USER_PASSWORD: ${DB_USER_PASSWORD:-openmetadata_password}DB_HOST: ${DB_HOST:-mysql}DB_PORT: ${DB_PORT:-3306}OM_DATABASE: ${OM_DATABASE:-openmetadata_db}# ElasticSearch ConfigurationsELASTICSEARCH_HOST: ${ELASTICSEARCH_HOST:- elasticsearch}ELASTICSEARCH_PORT: ${ELASTICSEARCH_PORT:-9200}ELASTICSEARCH_SCHEME: ${ELASTICSEARCH_SCHEME:-http}ELASTICSEARCH_USER: ${ELASTICSEARCH_USER:-""}ELASTICSEARCH_PASSWORD: ${ELASTICSEARCH_PASSWORD:-""}SEARCH_TYPE: ${SEARCH_TYPE:- "elasticsearch"}ELASTICSEARCH_TRUST_STORE_PATH: ${ELASTICSEARCH_TRUST_STORE_PATH:-""}ELASTICSEARCH_TRUST_STORE_PASSWORD: ${ELASTICSEARCH_TRUST_STORE_PASSWORD:-""}ELASTICSEARCH_CONNECTION_TIMEOUT_SECS: ${ELASTICSEARCH_CONNECTION_TIMEOUT_SECS:-5}ELASTICSEARCH_SOCKET_TIMEOUT_SECS: ${ELASTICSEARCH_SOCKET_TIMEOUT_SECS:-60}ELASTICSEARCH_KEEP_ALIVE_TIMEOUT_SECS: ${ELASTICSEARCH_KEEP_ALIVE_TIMEOUT_SECS:-600}ELASTICSEARCH_BATCH_SIZE: ${ELASTICSEARCH_BATCH_SIZE:-10}ELASTICSEARCH_PAYLOAD_BYTES_SIZE: ${ELASTICSEARCH_PAYLOAD_BYTES_SIZE:-10485760} #max payLoadSize in BytesELASTICSEARCH_INDEX_MAPPING_LANG: ${ELASTICSEARCH_INDEX_MAPPING_LANG:-EN}#eventMonitoringConfigurationEVENT_MONITOR: ${EVENT_MONITOR:-prometheus}EVENT_MONITOR_BATCH_SIZE: ${EVENT_MONITOR_BATCH_SIZE:-10}EVENT_MONITOR_PATH_PATTERN: ${EVENT_MONITOR_PATH_PATTERN:-["/api/v1/tables/*", "/api/v1/health-check"]}EVENT_MONITOR_LATENCY: ${EVENT_MONITOR_LATENCY:-[]}#pipelineServiceClientConfigurationPIPELINE_SERVICE_CLIENT_ENABLED: ${PIPELINE_SERVICE_CLIENT_ENABLED:-true}PIPELINE_SERVICE_CLIENT_CLASS_NAME: ${PIPELINE_SERVICE_CLIENT_CLASS_NAME:-"org.openmetadata.service.clients.pipeline.airflow.AirflowRESTClient"}PIPELINE_SERVICE_IP_INFO_ENABLED: ${PIPELINE_SERVICE_IP_INFO_ENABLED:-false}PIPELINE_SERVICE_CLIENT_HOST_IP: ${PIPELINE_SERVICE_CLIENT_HOST_IP:-""}PIPELINE_SERVICE_CLIENT_SECRETS_MANAGER_LOADER: ${PIPELINE_SERVICE_CLIENT_SECRETS_MANAGER_LOADER:-"noop"}#airflow parametersAIRFLOW_USERNAME: ${AIRFLOW_USERNAME:-admin}AIRFLOW_PASSWORD: ${AIRFLOW_PASSWORD:-admin}AIRFLOW_TIMEOUT: ${AIRFLOW_TIMEOUT:-10}AIRFLOW_TRUST_STORE_PATH: ${AIRFLOW_TRUST_STORE_PATH:-""}AIRFLOW_TRUST_STORE_PASSWORD: ${AIRFLOW_TRUST_STORE_PASSWORD:-""}FERNET_KEY: ${FERNET_KEY:-jJ/9sz0g0OHxsfxOoSfdFdmk3ysNmPRnH3TUAbz3IHA=}#secretsManagerConfigurationSECRET_MANAGER: ${SECRET_MANAGER:-db}#parameters:OM_SM_REGION: ${OM_SM_REGION:-""}OM_SM_ACCESS_KEY_ID: ${OM_SM_ACCESS_KEY_ID:-""}OM_SM_ACCESS_KEY: ${OM_SM_ACCESS_KEY:-""}#email configuration:OM_EMAIL_ENTITY: ${OM_EMAIL_ENTITY:-"OpenMetadata"}OM_SUPPORT_URL: ${OM_SUPPORT_URL:-"https://slack.open-metadata.org"}AUTHORIZER_ENABLE_SMTP : ${AUTHORIZER_ENABLE_SMTP:-false}OPENMETADATA_SERVER_URL: ${OPENMETADATA_SERVER_URL:-""}OPENMETADATA_SMTP_SENDER_MAIL: ${OPENMETADATA_SMTP_SENDER_MAIL:-""}SMTP_SERVER_ENDPOINT: ${SMTP_SERVER_ENDPOINT:-""}SMTP_SERVER_PORT: ${SMTP_SERVER_PORT:-""}SMTP_SERVER_USERNAME: ${SMTP_SERVER_USERNAME:-""}SMTP_SERVER_PWD: ${SMTP_SERVER_PWD:-""}SMTP_SERVER_STRATEGY: ${SMTP_SERVER_STRATEGY:-"SMTP_TLS"}# Heap OPTS ConfigurationsOPENMETADATA_HEAP_OPTS: ${OPENMETADATA_HEAP_OPTS:--Xmx1G -Xms1G}# Mask passwords values in UIMASK_PASSWORDS_API: ${MASK_PASSWORDS_API:-false}#OpenMetadata Web ConfigurationWEB_CONF_URI_PATH: ${WEB_CONF_URI_PATH:-"/api"}#HSTSWEB_CONF_HSTS_ENABLED: ${WEB_CONF_HSTS_ENABLED:-false}WEB_CONF_HSTS_MAX_AGE: ${WEB_CONF_HSTS_MAX_AGE:-"365 days"}WEB_CONF_HSTS_INCLUDE_SUBDOMAINS: ${WEB_CONF_HSTS_INCLUDE_SUBDOMAINS:-"true"}WEB_CONF_HSTS_PRELOAD: ${WEB_CONF_HSTS_PRELOAD:-"true"}#Frame OptionsWEB_CONF_FRAME_OPTION_ENABLED: ${WEB_CONF_FRAME_OPTION_ENABLED:-false}WEB_CONF_FRAME_OPTION: ${WEB_CONF_FRAME_OPTION:-"SAMEORIGIN"}WEB_CONF_FRAME_ORIGIN: ${WEB_CONF_FRAME_ORIGIN:-""}#Content TypeWEB_CONF_CONTENT_TYPE_OPTIONS_ENABLED: ${WEB_CONF_CONTENT_TYPE_OPTIONS_ENABLED:-false}#XSS-Protection WEB_CONF_XSS_PROTECTION_ENABLED: ${WEB_CONF_XSS_PROTECTION_ENABLED:-false}WEB_CONF_XSS_PROTECTION_ON: ${WEB_CONF_XSS_PROTECTION_ON:-true}WEB_CONF_XSS_PROTECTION_BLOCK: ${WEB_CONF_XSS_PROTECTION_BLOCK:-true}#CSP WEB_CONF_XSS_CSP_ENABLED: ${WEB_CONF_XSS_CSP_ENABLED:-false}WEB_CONF_XSS_CSP_POLICY: ${WEB_CONF_XSS_CSP_POLICY:-"default-src 'self'"}WEB_CONF_XSS_CSP_REPORT_ONLY_POLICY: ${WEB_CONF_XSS_CSP_REPORT_ONLY_POLICY:-""}expose:- 8585- 8586ports:- "8585:8585"- "8586:8586"depends_on:elasticsearch:condition: service_healthymysql:condition: service_healthyexecute-migrate-all:condition: service_completed_successfullynetworks:- app_nethealthcheck:test: [ "CMD", "wget", "-q", "--spider", "http://localhost:8586/healthcheck" ]ingestion:container_name: openmetadata_ingestionimage: registry.cn-hangzhou.aliyuncs.com/itclj/ingestion:1.4.3depends_on:elasticsearch:condition: service_startedmysql:condition: service_healthyopenmetadata-server:condition: service_startedenvironment:AIRFLOW__API__AUTH_BACKENDS: "airflow.api.auth.backend.basic_auth,airflow.api.auth.backend.session"AIRFLOW__CORE__EXECUTOR: LocalExecutorAIRFLOW__OPENMETADATA_AIRFLOW_APIS__DAG_GENERATED_CONFIGS: "/opt/airflow/dag_generated_configs"DB_HOST: ${AIRFLOW_DB_HOST:-mysql}DB_PORT: ${AIRFLOW_DB_PORT:-3306}AIRFLOW_DB: ${AIRFLOW_DB:-airflow_db}DB_SCHEME: ${AIRFLOW_DB_SCHEME:-mysql+pymysql}DB_USER: ${AIRFLOW_DB_USER:-airflow_user}DB_PASSWORD: ${AIRFLOW_DB_PASSWORD:-airflow_pass}# extra connection-string properties for the database# EXAMPLE # require SSL (only for Postgres)# properties: "?sslmode=require"DB_PROPERTIES: ${AIRFLOW_DB_PROPERTIES:-}# To test the lineage backend# AIRFLOW__LINEAGE__BACKEND: airflow_provider_openmetadata.lineage.backend.OpenMetadataLineageBackend# AIRFLOW__LINEAGE__AIRFLOW_SERVICE_NAME: local_airflow# AIRFLOW__LINEAGE__OPENMETADATA_API_ENDPOINT: http://openmetadata-server:8585/api# AIRFLOW__LINEAGE__JWT_TOKEN: ...entrypoint: /bin/bashcommand:- "/opt/airflow/ingestion_dependency.sh"expose:- 8080ports:- "8080:8080"networks:- app_netvolumes:- ingestion-volume-dag-airflow:/opt/airflow/dag_generated_configs- ingestion-volume-dags:/opt/airflow/dags- ingestion-volume-tmp:/tmpnetworks:app_net:ipam:driver: defaultconfig:- subnet: "192.168.11.0/24"在/opt目录下建openmetadata-docker目录把修改后的docker-compose.yml放入。
启动openmetadata服务。
docker-compose -f itclj-docker-compose.yml up --detach
在浏览器输入访问地址就可以进入open-metadata了。
http://192.168.10.165:8585/
默认用户名密码：admin/admin
