学习数仓工具 dbt

DBT 是一个有趣的工具，它通过一种结构化的方式定义了数仓中各种表、视图的构建和填充方式。

dbt 面相的对象是数据开发团队，提供了如下几个最有价值的能力：

  • 支持多种数据库
  • 通过 select 来定义数据，无需编写 DML
  • 构建数据时，自动分析数据依赖，按照依赖顺序生成模型
  • 版本化数据定义，支持团队 review
  • 支持数据质量测试

任何数据库都可以按照规范实现 DBT-XXX 适配器，使得你的数据库能够使用 DBT 来管理。

因为 OB 和 TiDB 都是兼容 MySQL 的，所以我先用 TiDB 的适配器来跑一下 OB，看看效果。因为 OB 对 information_schema.tables 表中的 table_type 字段内容做了一些扩展，除了 “VIEW” 和 “TABLE” 之外还引入了 “SYSTEM TABLE”, “SYSTEM VIEW”：

OceanBase(admin@test)>select distinct table_type from information_schema.tables;
+--------------+
| table_type   |
+--------------+
| BASE TABLE   |
| VIEW         |
| SYSTEM TABLE |
| SYSTEM VIEW  |
+--------------+
4 rows in set (0.008 sec)

这需要适配一下。改一下adapters.sql 即可。

/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/dbt/include/tidb/macros/adapters.sql

本文的 dbt 脚本见 https://github.com/dbt-labs/jaffle_shop

已经定义好 dbt 脚本后，数据开发工程师每次需要更新数仓中的数据时，只需要调用 dbt run 即可让所有数据更新，提升工作效率。可以反复执行 dbt run， dbt run背后会建立一些临时表，如：

customers__dbt_backup
stg_customers__dbt_tmp

运行(https://cn.pingcap.com/blog/when-tidb-meets-dbt/
) 里的例子，内部日志如下。仔细阅读，就能理解其工作原理了：

============================== 22:08:55.622663 | f43f4f98-1c20-4751-8101-18f6577098a1 ==============================
22:08:55.622663 [info ] [MainThread]: Running with dbt=1.6.9
22:08:55.622992 [debug] [MainThread]: running dbt with arguments {'printer_width': '80', 'indirect_selection': 'eager', 'log_cache_events': 'False', 'write_json': 'True', 'partial_parse': 'True', 'cache_selected_only': 'False', 'warn_error': 'None', 'fail_fast': 'False', 'debug': 'False', 'log_path': '/Users/raywill/code/raywill/jaffle_shop/logs', 'version_check': 'True', 'profiles_dir': '/Users/raywill/.dbt', 'use_colors': 'True', 'use_experimental_parser': 'False', 'no_print': 'None', 'quiet': 'False', 'warn_error_options': 'WarnErrorOptions(include=[], exclude=[])', 'introspect': 'True', 'static_parser': 'True', 'log_format': 'default', 'target_path': 'None', 'invocation_command': 'dbt run', 'send_anonymous_usage_stats': 'True'}
22:08:55.662363 [debug] [MainThread]: Sending event: {'category': 'dbt', 'action': 'project_id', 'label': 'f43f4f98-1c20-4751-8101-18f6577098a1', 'context': [<snowplow_tracker.self_describing_json.SelfDescribingJson object at 0x10afdad10>]}
22:08:55.669168 [debug] [MainThread]: Sending event: {'category': 'dbt', 'action': 'adapter_info', 'label': 'f43f4f98-1c20-4751-8101-18f6577098a1', 'context': [<snowplow_tracker.self_describing_json.SelfDescribingJson object at 0x10a9cd050>]}
22:08:55.669586 [info ] [MainThread]: Registered adapter: tidb=1.6.4
22:08:55.678545 [debug] [MainThread]: checksum: f795ff6322dabffafa5fc1f59b34557dea99c8bef60412d1b41cabcb4e97dcce, vars: {}, profile: , target: , version: 1.6.9
22:08:55.693348 [debug] [MainThread]: Partial parsing enabled: 0 files deleted, 0 files added, 0 files changed.
22:08:55.693590 [debug] [MainThread]: Partial parsing enabled, no changes found, skipping parsing
22:08:55.697272 [debug] [MainThread]: Sending event: {'category': 'dbt', 'action': 'load_project', 'label': 'f43f4f98-1c20-4751-8101-18f6577098a1', 'context': [<snowplow_tracker.self_describing_json.SelfDescribingJson object at 0x10767aa50>]}
22:08:55.703811 [debug] [MainThread]: Sending event: {'category': 'dbt', 'action': 'resource_counts', 'label': 'f43f4f98-1c20-4751-8101-18f6577098a1', 'context': [<snowplow_tracker.self_describing_json.SelfDescribingJson object at 0x10b01b950>]}
22:08:55.704055 [info ] [MainThread]: Found 5 models, 3 seeds, 20 tests, 0 sources, 0 exposures, 0 metrics, 349 macros, 0 groups, 0 semantic models
22:08:55.704224 [debug] [MainThread]: Sending event: {'category': 'dbt', 'action': 'runnable_timing', 'label': 'f43f4f98-1c20-4751-8101-18f6577098a1', 'context': [<snowplow_tracker.self_describing_json.SelfDescribingJson object at 0x10b008050>]}
22:08:55.705247 [info ] [MainThread]:
22:08:55.705615 [debug] [MainThread]: Acquiring new tidb connection 'master'
22:08:55.706190 [debug] [ThreadPool]: Acquiring new tidb connection 'list_schemas'
22:08:55.711040 [debug] [ThreadPool]: Using tidb connection "list_schemas"
22:08:55.711272 [debug] [ThreadPool]: On list_schemas: /* {"app": "dbt", "dbt_version": "1.6.9", "profile_name": "jaffle_shop", "target_name": "dev", "connection_name": "list_schemas"} */
select distinct schema_namefrom information_schema.schemata
22:08:55.711413 [debug] [ThreadPool]: Opening a new connection, currently in state init
22:08:55.791747 [debug] [ThreadPool]: SQL status: SUCCESS 7 in 0.0 seconds
22:08:55.793001 [debug] [ThreadPool]: On list_schemas: Close
22:08:55.794166 [debug] [ThreadPool]: Re-using an available connection from the pool (formerly list_schemas, now list_None_oceanbase)
22:08:55.797532 [debug] [ThreadPool]: Using tidb connection "list_None_oceanbase"
22:08:55.797766 [debug] [ThreadPool]: On list_None_oceanbase: BEGIN
22:08:55.797922 [debug] [ThreadPool]: Opening a new connection, currently in state closed
22:08:55.872273 [debug] [ThreadPool]: SQL status: SUCCESS 0 in 0.0 seconds
22:08:55.872627 [debug] [ThreadPool]: Using tidb connection "list_None_oceanbase"
22:08:55.872860 [debug] [ThreadPool]: On list_None_oceanbase: /* {"app": "dbt", "dbt_version": "1.6.9", "profile_name": "jaffle_shop", "target_name": "dev", "connection_name": "list_None_oceanbase"} */
selectnull as "database",table_name as name,table_schema as "schema",case when table_type = 'BASE TABLE' then 'table'when table_type = 'SYSTEM TABLE' then 'table'when table_type = 'SYSTEM VIEW' then 'view'when table_type = 'VIEW' then 'view'else table_typeend as table_typefrom information_schema.tableswhere table_schema = 'oceanbase'22:08:55.982141 [debug] [ThreadPool]: SQL status: SUCCESS 1067 in 0.0 seconds
22:08:55.991267 [debug] [ThreadPool]: On list_None_oceanbase: ROLLBACK
22:08:55.999188 [debug] [ThreadPool]: On list_None_oceanbase: Close
22:08:56.050320 [debug] [MainThread]: Sending event: {'category': 'dbt', 'action': 'runnable_timing', 'label': 'f43f4f98-1c20-4751-8101-18f6577098a1', 'context': [<snowplow_tracker.self_describing_json.SelfDescribingJson object at 0x10b26a3d0>]}
22:08:56.050653 [debug] [MainThread]: Using tidb connection "master"
22:08:56.050791 [debug] [MainThread]: On master: BEGIN
22:08:56.050913 [debug] [MainThread]: Opening a new connection, currently in state init
22:08:56.112223 [debug] [MainThread]: SQL status: SUCCESS 0 in 0.0 seconds
22:08:56.112550 [debug] [MainThread]: On master: COMMIT
22:08:56.112700 [debug] [MainThread]: Using tidb connection "master"
22:08:56.112843 [debug] [MainThread]: On master: COMMIT
22:08:56.127682 [debug] [MainThread]: SQL status: SUCCESS 0 in 0.0 seconds
22:08:56.127872 [debug] [MainThread]: On master: Close
22:08:56.128367 [info ] [MainThread]: Concurrency: 1 threads (target='dev')
22:08:56.128558 [info ] [MainThread]:
22:08:56.129995 [debug] [Thread-1 (]: Began running node model.jaffle_shop.stg_customers
22:08:56.130305 [info ] [Thread-1 (]: 1 of 5 START sql view model oceanbase.stg_customers ............................ [RUN]
22:08:56.130725 [debug] [Thread-1 (]: Re-using an available connection from the pool (formerly list_None_oceanbase, now model.jaffle_shop.stg_customers)
22:08:56.130937 [debug] [Thread-1 (]: Began compiling node model.jaffle_shop.stg_customers
22:08:56.132859 [debug] [Thread-1 (]: Writing injected SQL for node "model.jaffle_shop.stg_customers"
22:08:56.133337 [debug] [Thread-1 (]: Timing info for model.jaffle_shop.stg_customers (compile): 22:08:56.131077 => 22:08:56.133228
22:08:56.133526 [debug] [Thread-1 (]: Began executing node model.jaffle_shop.stg_customers
22:08:56.158597 [debug] [Thread-1 (]: Writing runtime sql for node "model.jaffle_shop.stg_customers"
22:08:56.159066 [debug] [Thread-1 (]: Using tidb connection "model.jaffle_shop.stg_customers"
22:08:56.159227 [debug] [Thread-1 (]: On model.jaffle_shop.stg_customers: BEGIN
22:08:56.159372 [debug] [Thread-1 (]: Opening a new connection, currently in state closed
22:08:56.217134 [debug] [Thread-1 (]: SQL status: SUCCESS 0 in 0.0 seconds
22:08:56.217351 [debug] [Thread-1 (]: Using tidb connection "model.jaffle_shop.stg_customers"
22:08:56.217555 [debug] [Thread-1 (]: On model.jaffle_shop.stg_customers: /* {"app": "dbt", "dbt_version": "1.6.9", "profile_name": "jaffle_shop", "target_name": "dev", "node_id": "model.jaffle_shop.stg_customers"} */create view `oceanbase`.`stg_customers__dbt_tmp`as (with source as (select * from `oceanbase`.`raw_customers`),renamed as (selectid as customer_id,first_name,last_namefrom source)select * from renamed);
22:08:56.329009 [debug] [Thread-1 (]: SQL status: SUCCESS 0 in 0.0 seconds
22:08:56.364055 [debug] [Thread-1 (]: Using tidb connection "model.jaffle_shop.stg_customers"
22:08:56.364679 [debug] [Thread-1 (]: On model.jaffle_shop.stg_customers: /* {"app": "dbt", "dbt_version": "1.6.9", "profile_name": "jaffle_shop", "target_name": "dev", "node_id": "model.jaffle_shop.stg_customers"} */drop view if exists `oceanbase`.`stg_customers__dbt_backup` cascade22:08:56.388533 [debug] [Thread-1 (]: SQL status: SUCCESS 0 in 0.0 seconds
22:08:56.390391 [debug] [Thread-1 (]: Using tidb connection "model.jaffle_shop.stg_customers"
22:08:56.390649 [debug] [Thread-1 (]: On model.jaffle_shop.stg_customers: /* {"app": "dbt", "dbt_version": "1.6.9", "profile_name": "jaffle_shop", "target_name": "dev", "node_id": "model.jaffle_shop.stg_customers"} */rename table `oceanbase`.`stg_customers` to `oceanbase`.`stg_customers__dbt_backup`22:08:56.482864 [debug] [Thread-1 (]: SQL status: SUCCESS 0 in 0.0 seconds
22:08:56.489215 [debug] [Thread-1 (]: Using tidb connection "model.jaffle_shop.stg_customers"
22:08:56.489815 [debug] [Thread-1 (]: On model.jaffle_shop.stg_customers: /* {"app": "dbt", "dbt_version": "1.6.9", "profile_name": "jaffle_shop", "target_name": "dev", "node_id": "model.jaffle_shop.stg_customers"} */drop view if exists `oceanbase`.`stg_customers` cascade22:08:56.510550 [debug] [Thread-1 (]: SQL status: SUCCESS 0 in 0.0 seconds
22:08:56.513581 [debug] [Thread-1 (]: Using tidb connection "model.jaffle_shop.stg_customers"
22:08:56.514166 [debug] [Thread-1 (]: On model.jaffle_shop.stg_customers: /* {"app": "dbt", "dbt_version": "1.6.9", "profile_name": "jaffle_shop", "target_name": "dev", "node_id": "model.jaffle_shop.stg_customers"} */rename table `oceanbase`.`stg_customers__dbt_tmp` to `oceanbase`.`stg_customers`22:08:56.600139 [debug] [Thread-1 (]: SQL status: SUCCESS 0 in 0.0 seconds
22:08:56.620564 [debug] [Thread-1 (]: On model.jaffle_shop.stg_customers: COMMIT
22:08:56.621050 [debug] [Thread-1 (]: Using tidb connection "model.jaffle_shop.stg_customers"
22:08:56.621314 [debug] [Thread-1 (]: On model.jaffle_shop.stg_customers: COMMIT
22:08:56.636866 [debug] [Thread-1 (]: SQL status: SUCCESS 0 in 0.0 seconds
22:08:56.642164 [debug] [Thread-1 (]: Using tidb connection "model.jaffle_shop.stg_customers"
22:08:56.642473 [debug] [Thread-1 (]: On model.jaffle_shop.stg_customers: /* {"app": "dbt", "dbt_version": "1.6.9", "profile_name": "jaffle_shop", "target_name": "dev", "node_id": "model.jaffle_shop.stg_customers"} */
drop view if exists `oceanbase`.`stg_customers__dbt_backup`
22:08:56.719959 [debug] [Thread-1 (]: SQL status: SUCCESS 0 in 0.0 seconds
22:08:56.724716 [debug] [Thread-1 (]: Timing info for model.jaffle_shop.stg_customers (execute): 22:08:56.133648 => 22:08:56.724313
22:08:56.725436 [debug] [Thread-1 (]: On model.jaffle_shop.stg_customers: Close
22:08:56.727114 [debug] [Thread-1 (]: Sending event: {'category': 'dbt', 'action': 'run_model', 'label': 'f43f4f98-1c20-4751-8101-18f6577098a1', 'context': [<snowplow_tracker.self_describing_json.SelfDescribingJson object at 0x10adad4d0>]}
22:08:56.728088 [info ] [Thread-1 (]: 1 of 5 OK created sql view model oceanbase.stg_customers ....................... [SUCCESS 0 in 0.60s]
22:08:56.728829 [debug] [Thread-1 (]: Finished running node model.jaffle_shop.stg_customers
22:08:56.729372 [debug] [Thread-1 (]: Began running node model.jaffle_shop.stg_orders
22:08:56.729887 [info ] [Thread-1 (]: 2 of 5 START sql view model oceanbase.stg_orders ............................... [RUN]
22:08:56.730682 [debug] [Thread-1 (]: Re-using an available connection from the pool (formerly model.jaffle_shop.stg_customers, now model.jaffle_shop.stg_orders)
22:08:56.731039 [debug] [Thread-1 (]: Began compiling node model.jaffle_shop.stg_orders
22:08:56.734512 [debug] [Thread-1 (]: Writing injected SQL for node "model.jaffle_shop.stg_orders"
22:08:56.735763 [debug] [Thread-1 (]: Timing info for model.jaffle_shop.stg_orders (compile): 22:08:56.731390 => 22:08:56.735555
22:08:56.736135 [debug] [Thread-1 (]: Began executing node model.jaffle_shop.stg_orders
22:08:56.752144 [debug] [Thread-1 (]: Writing runtime sql for node "model.jaffle_shop.stg_orders"
22:08:56.752639 [debug] [Thread-1 (]: Using tidb connection "model.jaffle_shop.stg_orders"
22:08:56.752858 [debug] [Thread-1 (]: On model.jaffle_shop.stg_orders: BEGIN
22:08:56.753063 [debug] [Thread-1 (]: Opening a new connection, currently in state closed
22:08:56.817242 [debug] [Thread-1 (]: SQL status: SUCCESS 0 in 0.0 seconds
22:08:56.817668 [debug] [Thread-1 (]: Using tidb connection "model.jaffle_shop.stg_orders"
22:08:56.818016 [debug] [Thread-1 (]: On model.jaffle_shop.stg_orders: /* {"app": "dbt", "dbt_version": "1.6.9", "profile_name": "jaffle_shop", "target_name": "dev", "node_id": "model.jaffle_shop.stg_orders"} */create view `oceanbase`.`stg_orders__dbt_tmp`as (with source as (select * from `oceanbase`.`raw_orders`),renamed as (selectid as order_id,user_id as customer_id,order_date,statusfrom source)select * from renamed);
22:08:56.896459 [debug] [Thread-1 (]: SQL status: SUCCESS 0 in 0.0 seconds
22:08:56.903640 [debug] [Thread-1 (]: Using tidb connection "model.jaffle_shop.stg_orders"
22:08:56.903963 [debug] [Thread-1 (]: On model.jaffle_shop.stg_orders: /* {"app": "dbt", "dbt_version": "1.6.9", "profile_name": "jaffle_shop", "target_name": "dev", "node_id": "model.jaffle_shop.stg_orders"} */drop view if exists `oceanbase`.`stg_orders__dbt_backup` cascade22:08:56.925139 [debug] [Thread-1 (]: SQL status: SUCCESS 0 in 0.0 seconds
22:08:56.926137 [debug] [Thread-1 (]: Using tidb connection "model.jaffle_shop.stg_orders"
22:08:56.926412 [debug] [Thread-1 (]: On model.jaffle_shop.stg_orders: /* {"app": "dbt", "dbt_version": "1.6.9", "profile_name": "jaffle_shop", "target_name": "dev", "node_id": "model.jaffle_shop.stg_orders"} */rename table `oceanbase`.`stg_orders` to `oceanbase`.`stg_orders__dbt_backup`22:08:56.994692 [debug] [Thread-1 (]: SQL status: SUCCESS 0 in 0.0 seconds
22:08:56.997962 [debug] [Thread-1 (]: Using tidb connection "model.jaffle_shop.stg_orders"
22:08:56.998295 [debug] [Thread-1 (]: On model.jaffle_shop.stg_orders: /* {"app": "dbt", "dbt_version": "1.6.9", "profile_name": "jaffle_shop", "target_name": "dev", "node_id": "model.jaffle_shop.stg_orders"} */drop view if exists `oceanbase`.`stg_orders` cascade22:08:57.016278 [debug] [Thread-1 (]: SQL status: SUCCESS 0 in 0.0 seconds
22:08:57.017341 [debug] [Thread-1 (]: Using tidb connection "model.jaffle_shop.stg_orders"
22:08:57.017666 [debug] [Thread-1 (]: On model.jaffle_shop.stg_orders: /* {"app": "dbt", "dbt_version": "1.6.9", "profile_name": "jaffle_shop", "target_name": "dev", "node_id": "model.jaffle_shop.stg_orders"} */rename table `oceanbase`.`stg_orders__dbt_tmp` to `oceanbase`.`stg_orders`22:08:57.091437 [debug] [Thread-1 (]: SQL status: SUCCESS 0 in 0.0 seconds
22:08:57.097185 [debug] [Thread-1 (]: On model.jaffle_shop.stg_orders: COMMIT
22:08:57.097653 [debug] [Thread-1 (]: Using tidb connection "model.jaffle_shop.stg_orders"
22:08:57.097973 [debug] [Thread-1 (]: On model.jaffle_shop.stg_orders: COMMIT
22:08:57.113205 [debug] [Thread-1 (]: SQL status: SUCCESS 0 in 0.0 seconds
22:08:57.119160 [debug] [Thread-1 (]: Using tidb connection "model.jaffle_shop.stg_orders"
22:08:57.119605 [debug] [Thread-1 (]: On model.jaffle_shop.stg_orders: /* {"app": "dbt", "dbt_version": "1.6.9", "profile_name": "jaffle_shop", "target_name": "dev", "node_id": "model.jaffle_shop.stg_orders"} */
drop view if exists `oceanbase`.`stg_orders__dbt_backup`
22:08:57.191539 [debug] [Thread-1 (]: SQL status: SUCCESS 0 in 0.0 seconds
22:08:57.196838 [debug] [Thread-1 (]: Timing info for model.jaffle_shop.stg_orders (execute): 22:08:56.736369 => 22:08:57.196496
22:08:57.197465 [debug] [Thread-1 (]: On model.jaffle_shop.stg_orders: Close
22:08:57.198946 [debug] [Thread-1 (]: Sending event: {'category': 'dbt', 'action': 'run_model', 'label': 'f43f4f98-1c20-4751-8101-18f6577098a1', 'context': [<snowplow_tracker.self_describing_json.SelfDescribingJson object at 0x10b00e090>]}
22:08:57.199806 [info ] [Thread-1 (]: 2 of 5 OK created sql view model oceanbase.stg_orders .......................... [SUCCESS 0 in 0.47s]
22:08:57.200556 [debug] [Thread-1 (]: Finished running node model.jaffle_shop.stg_orders
22:08:57.201094 [debug] [Thread-1 (]: Began running node model.jaffle_shop.stg_payments
22:08:57.201612 [info ] [Thread-1 (]: 3 of 5 START sql view model oceanbase.stg_payments ............................. [RUN]
22:08:57.202546 [debug] [Thread-1 (]: Re-using an available connection from the pool (formerly model.jaffle_shop.stg_orders, now model.jaffle_shop.stg_payments)
22:08:57.202993 [debug] [Thread-1 (]: Began compiling node model.jaffle_shop.stg_payments
22:08:57.206589 [debug] [Thread-1 (]: Writing injected SQL for node "model.jaffle_shop.stg_payments"
22:08:57.207501 [debug] [Thread-1 (]: Timing info for model.jaffle_shop.stg_payments (compile): 22:08:57.203264 => 22:08:57.207308
22:08:57.207855 [debug] [Thread-1 (]: Began executing node model.jaffle_shop.stg_payments
22:08:57.224227 [debug] [Thread-1 (]: Writing runtime sql for node "model.jaffle_shop.stg_payments"
22:08:57.224759 [debug] [Thread-1 (]: Using tidb connection "model.jaffle_shop.stg_payments"
22:08:57.224987 [debug] [Thread-1 (]: On model.jaffle_shop.stg_payments: BEGIN
22:08:57.225194 [debug] [Thread-1 (]: Opening a new connection, currently in state closed
22:08:57.286954 [debug] [Thread-1 (]: SQL status: SUCCESS 0 in 0.0 seconds
22:08:57.287364 [debug] [Thread-1 (]: Using tidb connection "model.jaffle_shop.stg_payments"
22:08:57.287725 [debug] [Thread-1 (]: On model.jaffle_shop.stg_payments: /* {"app": "dbt", "dbt_version": "1.6.9", "profile_name": "jaffle_shop", "target_name": "dev", "node_id": "model.jaffle_shop.stg_payments"} */create view `oceanbase`.`stg_payments__dbt_tmp`as (with source as (select * from `oceanbase`.`raw_payments`),renamed as (selectid as payment_id,order_id,payment_method,-- `amount` is currently stored in cents, so we convert it to dollarsamount / 100 as amountfrom source)select * from renamed);
22:08:57.369772 [debug] [Thread-1 (]: SQL status: SUCCESS 0 in 0.0 seconds
22:08:57.381916 [debug] [Thread-1 (]: Using tidb connection "model.jaffle_shop.stg_payments"
22:08:57.382347 [debug] [Thread-1 (]: On model.jaffle_shop.stg_payments: /* {"app": "dbt", "dbt_version": "1.6.9", "profile_name": "jaffle_shop", "target_name": "dev", "node_id": "model.jaffle_shop.stg_payments"} */drop view if exists `oceanbase`.`stg_payments__dbt_backup` cascade22:08:57.404650 [debug] [Thread-1 (]: SQL status: SUCCESS 0 in 0.0 seconds
22:08:57.406090 [debug] [Thread-1 (]: Using tidb connection "model.jaffle_shop.stg_payments"
22:08:57.406480 [debug] [Thread-1 (]: On model.jaffle_shop.stg_payments: /* {"app": "dbt", "dbt_version": "1.6.9", "profile_name": "jaffle_shop", "target_name": "dev", "node_id": "model.jaffle_shop.stg_payments"} */rename table `oceanbase`.`stg_payments` to `oceanbase`.`stg_payments__dbt_backup`22:08:57.476087 [debug] [Thread-1 (]: SQL status: SUCCESS 0 in 0.0 seconds
22:08:57.482015 [debug] [Thread-1 (]: Using tidb connection "model.jaffle_shop.stg_payments"
22:08:57.482508 [debug] [Thread-1 (]: On model.jaffle_shop.stg_payments: /* {"app": "dbt", "dbt_version": "1.6.9", "profile_name": "jaffle_shop", "target_name": "dev", "node_id": "model.jaffle_shop.stg_payments"} */drop view if exists `oceanbase`.`stg_payments` cascade22:08:57.504008 [debug] [Thread-1 (]: SQL status: SUCCESS 0 in 0.0 seconds
22:08:57.507457 [debug] [Thread-1 (]: Using tidb connection "model.jaffle_shop.stg_payments"
22:08:57.508634 [debug] [Thread-1 (]: On model.jaffle_shop.stg_payments: /* {"app": "dbt", "dbt_version": "1.6.9", "profile_name": "jaffle_shop", "target_name": "dev", "node_id": "model.jaffle_shop.stg_payments"} */rename table `oceanbase`.`stg_payments__dbt_tmp` to `oceanbase`.`stg_payments`22:08:57.584623 [debug] [Thread-1 (]: SQL status: SUCCESS 0 in 0.0 seconds
22:08:57.589099 [debug] [Thread-1 (]: On model.jaffle_shop.stg_payments: COMMIT
22:08:57.589588 [debug] [Thread-1 (]: Using tidb connection "model.jaffle_shop.stg_payments"
22:08:57.589984 [debug] [Thread-1 (]: On model.jaffle_shop.stg_payments: COMMIT
22:08:57.607837 [debug] [Thread-1 (]: SQL status: SUCCESS 0 in 0.0 seconds
22:08:57.614888 [debug] [Thread-1 (]: Using tidb connection "model.jaffle_shop.stg_payments"
22:08:57.615358 [debug] [Thread-1 (]: On model.jaffle_shop.stg_payments: /* {"app": "dbt", "dbt_version": "1.6.9", "profile_name": "jaffle_shop", "target_name": "dev", "node_id": "model.jaffle_shop.stg_payments"} */
drop view if exists `oceanbase`.`stg_payments__dbt_backup`
22:08:57.690624 [debug] [Thread-1 (]: SQL status: SUCCESS 0 in 0.0 seconds
22:08:57.695481 [debug] [Thread-1 (]: Timing info for model.jaffle_shop.stg_payments (execute): 22:08:57.208062 => 22:08:57.695113
22:08:57.696561 [debug] [Thread-1 (]: On model.jaffle_shop.stg_payments: Close
22:08:57.697780 [debug] [Thread-1 (]: Sending event: {'category': 'dbt', 'action': 'run_model', 'label': 'f43f4f98-1c20-4751-8101-18f6577098a1', 'context': [<snowplow_tracker.self_describing_json.SelfDescribingJson object at 0x10b02da10>]}
22:08:57.698487 [info ] [Thread-1 (]: 3 of 5 OK created sql view model oceanbase.stg_payments ........................ [SUCCESS 0 in 0.50s]
22:08:57.699066 [debug] [Thread-1 (]: Finished running node model.jaffle_shop.stg_payments
22:08:57.700156 [debug] [Thread-1 (]: Began running node model.jaffle_shop.customers
22:08:57.700680 [info ] [Thread-1 (]: 4 of 5 START sql table model oceanbase.customers ............................... [RUN]
22:08:57.701414 [debug] [Thread-1 (]: Re-using an available connection from the pool (formerly model.jaffle_shop.stg_payments, now model.jaffle_shop.customers)
22:08:57.701787 [debug] [Thread-1 (]: Began compiling node model.jaffle_shop.customers
22:08:57.707287 [debug] [Thread-1 (]: Writing injected SQL for node "model.jaffle_shop.customers"
22:08:57.708262 [debug] [Thread-1 (]: Timing info for model.jaffle_shop.customers (compile): 22:08:57.702033 => 22:08:57.708100
22:08:57.708561 [debug] [Thread-1 (]: Began executing node model.jaffle_shop.customers
22:08:57.737060 [debug] [Thread-1 (]: Using tidb connection "model.jaffle_shop.customers"
22:08:57.737355 [debug] [Thread-1 (]: On model.jaffle_shop.customers: BEGIN
22:08:57.737543 [debug] [Thread-1 (]: Opening a new connection, currently in state closed
22:08:57.794172 [debug] [Thread-1 (]: SQL status: SUCCESS 0 in 0.0 seconds
22:08:57.794493 [debug] [Thread-1 (]: Using tidb connection "model.jaffle_shop.customers"
22:08:57.794774 [debug] [Thread-1 (]: On model.jaffle_shop.customers: /* {"app": "dbt", "dbt_version": "1.6.9", "profile_name": "jaffle_shop", "target_name": "dev", "node_id": "model.jaffle_shop.customers"} */create viewoceanbase.temp_viewas (with customers as (select * from `oceanbase`.`stg_customers`),orders as (select * from `oceanbase`.`stg_orders`),payments as (select * from `oceanbase`.`stg_payments`),customer_orders as (selectcustomer_id,min(order_date) as first_order,max(order_date) as most_recent_order,count(order_id) as number_of_ordersfrom ordersgroup by customer_id),customer_payments as (selectorders.customer_id,sum(amount) as total_amountfrom paymentsleft join orders onpayments.order_id = orders.order_idgroup by orders.customer_id),final as (selectcustomers.customer_id,customers.first_name,customers.last_name,customer_orders.first_order,customer_orders.most_recent_order,customer_orders.number_of_orders,customer_payments.total_amount as customer_lifetime_valuefrom customersinner join customer_orderson customers.customer_id = customer_orders.customer_idinner join customer_paymentson  customers.customer_id = customer_payments.customer_id)select * from final)22:08:57.895098 [debug] [Thread-1 (]: SQL status: SUCCESS 0 in 0.0 seconds
22:08:57.898817 [debug] [Thread-1 (]: Using tidb connection "model.jaffle_shop.customers"
22:08:57.899487 [debug] [Thread-1 (]: On model.jaffle_shop.customers: /* {"app": "dbt", "dbt_version": "1.6.9", "profile_name": "jaffle_shop", "target_name": "dev", "node_id": "model.jaffle_shop.customers"} */show columns from oceanbase.temp_view22:08:57.924479 [debug] [Thread-1 (]: SQL status: SUCCESS 7 in 0.0 seconds
22:08:57.926493 [debug] [Thread-1 (]: Using tidb connection "model.jaffle_shop.customers"
22:08:57.926786 [debug] [Thread-1 (]: On model.jaffle_shop.customers: /* {"app": "dbt", "dbt_version": "1.6.9", "profile_name": "jaffle_shop", "target_name": "dev", "node_id": "model.jaffle_shop.customers"} */create   table if not exists`oceanbase`.`customers__dbt_tmp`(`customer_id`int(11)not null,`first_name`textnot null,`last_name`textnot null,`first_order`datenot null,`most_recent_order`datenot null,`number_of_orders`bigint(20)not null,`customer_lifetime_value`decimal(37,4)not null)22:08:58.097120 [debug] [Thread-1 (]: SQL status: SUCCESS 0 in 0.0 seconds
22:08:58.101002 [debug] [Thread-1 (]: Using tidb connection "model.jaffle_shop.customers"
22:08:58.101631 [debug] [Thread-1 (]: On model.jaffle_shop.customers: /* {"app": "dbt", "dbt_version": "1.6.9", "profile_name": "jaffle_shop", "target_name": "dev", "node_id": "model.jaffle_shop.customers"} */insert into `oceanbase`.`customers__dbt_tmp` select * from oceanbase.temp_view22:08:58.153432 [debug] [Thread-1 (]: SQL status: SUCCESS 62 in 0.0 seconds
22:08:58.156432 [debug] [Thread-1 (]: Using tidb connection "model.jaffle_shop.customers"
22:08:58.157239 [debug] [Thread-1 (]: On model.jaffle_shop.customers: /* {"app": "dbt", "dbt_version": "1.6.9", "profile_name": "jaffle_shop", "target_name": "dev", "node_id": "model.jaffle_shop.customers"} */drop view if exists oceanbase.temp_view22:08:58.227587 [debug] [Thread-1 (]: SQL status: SUCCESS 0 in 0.0 seconds
22:08:58.229403 [debug] [Thread-1 (]: Writing runtime sql for node "model.jaffle_shop.customers"
22:08:58.230580 [debug] [Thread-1 (]: Using tidb connection "model.jaffle_shop.customers"
22:08:58.231106 [debug] [Thread-1 (]: On model.jaffle_shop.customers: /* {"app": "dbt", "dbt_version": "1.6.9", "profile_name": "jaffle_shop", "target_name": "dev", "node_id": "model.jaffle_shop.customers"} */22:08:58.245210 [debug] [Thread-1 (]: SQL status: SUCCESS 0 in 0.0 seconds
22:08:58.252855 [debug] [Thread-1 (]: Using tidb connection "model.jaffle_shop.customers"
22:08:58.253059 [debug] [Thread-1 (]: On model.jaffle_shop.customers: /* {"app": "dbt", "dbt_version": "1.6.9", "profile_name": "jaffle_shop", "target_name": "dev", "node_id": "model.jaffle_shop.customers"} */drop table if exists `oceanbase`.`customers__dbt_backup` cascade22:08:58.274641 [debug] [Thread-1 (]: SQL status: SUCCESS 0 in 0.0 seconds
22:08:58.276792 [debug] [Thread-1 (]: Using tidb connection "model.jaffle_shop.customers"
22:08:58.277412 [debug] [Thread-1 (]: On model.jaffle_shop.customers: /* {"app": "dbt", "dbt_version": "1.6.9", "profile_name": "jaffle_shop", "target_name": "dev", "node_id": "model.jaffle_shop.customers"} */rename table `oceanbase`.`customers` to `oceanbase`.`customers__dbt_backup`22:08:58.348851 [debug] [Thread-1 (]: SQL status: SUCCESS 0 in 0.0 seconds
22:08:58.354702 [debug] [Thread-1 (]: Using tidb connection "model.jaffle_shop.customers"
22:08:58.355026 [debug] [Thread-1 (]: On model.jaffle_shop.customers: /* {"app": "dbt", "dbt_version": "1.6.9", "profile_name": "jaffle_shop", "target_name": "dev", "node_id": "model.jaffle_shop.customers"} */drop table if exists `oceanbase`.`customers` cascade22:08:58.375974 [debug] [Thread-1 (]: SQL status: SUCCESS 0 in 0.0 seconds
22:08:58.378890 [debug] [Thread-1 (]: Using tidb connection "model.jaffle_shop.customers"
22:08:58.379742 [debug] [Thread-1 (]: On model.jaffle_shop.customers: /* {"app": "dbt", "dbt_version": "1.6.9", "profile_name": "jaffle_shop", "target_name": "dev", "node_id": "model.jaffle_shop.customers"} */rename table `oceanbase`.`customers__dbt_tmp` to `oceanbase`.`customers`22:08:58.449860 [debug] [Thread-1 (]: SQL status: SUCCESS 0 in 0.0 seconds
22:08:58.461118 [debug] [Thread-1 (]: On model.jaffle_shop.customers: COMMIT
22:08:58.461648 [debug] [Thread-1 (]: Using tidb connection "model.jaffle_shop.customers"
22:08:58.461999 [debug] [Thread-1 (]: On model.jaffle_shop.customers: COMMIT
22:08:58.476014 [debug] [Thread-1 (]: SQL status: SUCCESS 0 in 0.0 seconds
22:08:58.479457 [debug] [Thread-1 (]: Using tidb connection "model.jaffle_shop.customers"
22:08:58.479875 [debug] [Thread-1 (]: On model.jaffle_shop.customers: /* {"app": "dbt", "dbt_version": "1.6.9", "profile_name": "jaffle_shop", "target_name": "dev", "node_id": "model.jaffle_shop.customers"} */
drop table if exists `oceanbase`.`customers__dbt_backup`
22:08:58.583995 [debug] [Thread-1 (]: SQL status: SUCCESS 0 in 0.0 seconds
22:08:58.585602 [debug] [Thread-1 (]: Timing info for model.jaffle_shop.customers (execute): 22:08:57.708757 => 22:08:58.585436
22:08:58.585951 [debug] [Thread-1 (]: On model.jaffle_shop.customers: Close
22:08:58.586683 [debug] [Thread-1 (]: Sending event: {'category': 'dbt', 'action': 'run_model', 'label': 'f43f4f98-1c20-4751-8101-18f6577098a1', 'context': [<snowplow_tracker.self_describing_json.SelfDescribingJson object at 0x10c884f10>]}
22:08:58.587183 [info ] [Thread-1 (]: 4 of 5 OK created sql table model oceanbase.customers .......................... [SUCCESS 0 in 0.89s]
22:08:58.587656 [debug] [Thread-1 (]: Finished running node model.jaffle_shop.customers
22:08:58.587999 [debug] [Thread-1 (]: Began running node model.jaffle_shop.orders
22:08:58.588335 [info ] [Thread-1 (]: 5 of 5 START sql table model oceanbase.orders .................................. [RUN]
22:08:58.588989 [debug] [Thread-1 (]: Re-using an available connection from the pool (formerly model.jaffle_shop.customers, now model.jaffle_shop.orders)
22:08:58.589298 [debug] [Thread-1 (]: Began compiling node model.jaffle_shop.orders
22:08:58.592665 [debug] [Thread-1 (]: Writing injected SQL for node "model.jaffle_shop.orders"
22:08:58.593351 [debug] [Thread-1 (]: Timing info for model.jaffle_shop.orders (compile): 22:08:58.589484 => 22:08:58.593212
22:08:58.593607 [debug] [Thread-1 (]: Began executing node model.jaffle_shop.orders
22:08:58.607013 [debug] [Thread-1 (]: Using tidb connection "model.jaffle_shop.orders"
22:08:58.607297 [debug] [Thread-1 (]: On model.jaffle_shop.orders: BEGIN
22:08:58.607492 [debug] [Thread-1 (]: Opening a new connection, currently in state closed
22:08:58.660110 [debug] [Thread-1 (]: SQL status: SUCCESS 0 in 0.0 seconds
22:08:58.660481 [debug] [Thread-1 (]: Using tidb connection "model.jaffle_shop.orders"
22:08:58.660790 [debug] [Thread-1 (]: On model.jaffle_shop.orders: /* {"app": "dbt", "dbt_version": "1.6.9", "profile_name": "jaffle_shop", "target_name": "dev", "node_id": "model.jaffle_shop.orders"} */create viewoceanbase.temp_viewas (with orders as (select * from `oceanbase`.`stg_orders`),payments as (select * from `oceanbase`.`stg_payments`),order_payments as (selectorder_id,sum(case when payment_method = 'credit_card' then amount else 0 end) as credit_card_amount,sum(case when payment_method = 'coupon' then amount else 0 end) as coupon_amount,sum(case when payment_method = 'bank_transfer' then amount else 0 end) as bank_transfer_amount,sum(case when payment_method = 'gift_card' then amount else 0 end) as gift_card_amount,sum(amount) as total_amountfrom paymentsgroup by order_id),final as (selectorders.order_id,orders.customer_id,orders.order_date,orders.status,order_payments.credit_card_amount,order_payments.coupon_amount,order_payments.bank_transfer_amount,order_payments.gift_card_amount,order_payments.total_amount as amountfrom ordersleft join order_paymentson orders.order_id = order_payments.order_id)select * from final)22:08:58.746833 [debug] [Thread-1 (]: SQL status: SUCCESS 0 in 0.0 seconds
22:08:58.749953 [debug] [Thread-1 (]: Using tidb connection "model.jaffle_shop.orders"
22:08:58.750953 [debug] [Thread-1 (]: On model.jaffle_shop.orders: /* {"app": "dbt", "dbt_version": "1.6.9", "profile_name": "jaffle_shop", "target_name": "dev", "node_id": "model.jaffle_shop.orders"} */show columns from oceanbase.temp_view22:08:58.774376 [debug] [Thread-1 (]: SQL status: SUCCESS 9 in 0.0 seconds
22:08:58.777285 [debug] [Thread-1 (]: Using tidb connection "model.jaffle_shop.orders"
22:08:58.778144 [debug] [Thread-1 (]: On model.jaffle_shop.orders: /* {"app": "dbt", "dbt_version": "1.6.9", "profile_name": "jaffle_shop", "target_name": "dev", "node_id": "model.jaffle_shop.orders"} */create   table if not exists`oceanbase`.`orders__dbt_tmp`(`order_id`int(11)not null,`customer_id`int(11)not null,`order_date`datenot null,`status`textnot null,`credit_card_amount`decimal(37,4)not null,`coupon_amount`decimal(37,4)not null,`bank_transfer_amount`decimal(37,4)not null,`gift_card_amount`decimal(37,4)not null,`amount`decimal(37,4)not null)22:08:58.901266 [debug] [Thread-1 (]: SQL status: SUCCESS 0 in 0.0 seconds
22:08:58.904156 [debug] [Thread-1 (]: Using tidb connection "model.jaffle_shop.orders"
22:08:58.904774 [debug] [Thread-1 (]: On model.jaffle_shop.orders: /* {"app": "dbt", "dbt_version": "1.6.9", "profile_name": "jaffle_shop", "target_name": "dev", "node_id": "model.jaffle_shop.orders"} */insert into `oceanbase`.`orders__dbt_tmp` select * from oceanbase.temp_view22:08:58.942316 [debug] [Thread-1 (]: SQL status: SUCCESS 99 in 0.0 seconds
22:08:58.944322 [debug] [Thread-1 (]: Using tidb connection "model.jaffle_shop.orders"
22:08:58.944929 [debug] [Thread-1 (]: On model.jaffle_shop.orders: /* {"app": "dbt", "dbt_version": "1.6.9", "profile_name": "jaffle_shop", "target_name": "dev", "node_id": "model.jaffle_shop.orders"} */drop view if exists oceanbase.temp_view22:08:59.018315 [debug] [Thread-1 (]: SQL status: SUCCESS 0 in 0.0 seconds
22:08:59.021627 [debug] [Thread-1 (]: Writing runtime sql for node "model.jaffle_shop.orders"
22:08:59.023735 [debug] [Thread-1 (]: Using tidb connection "model.jaffle_shop.orders"
22:08:59.024275 [debug] [Thread-1 (]: On model.jaffle_shop.orders: /* {"app": "dbt", "dbt_version": "1.6.9", "profile_name": "jaffle_shop", "target_name": "dev", "node_id": "model.jaffle_shop.orders"} */22:08:59.039661 [debug] [Thread-1 (]: SQL status: SUCCESS 0 in 0.0 seconds
22:08:59.052185 [debug] [Thread-1 (]: Using tidb connection "model.jaffle_shop.orders"
22:08:59.052598 [debug] [Thread-1 (]: On model.jaffle_shop.orders: /* {"app": "dbt", "dbt_version": "1.6.9", "profile_name": "jaffle_shop", "target_name": "dev", "node_id": "model.jaffle_shop.orders"} */drop table if exists `oceanbase`.`orders__dbt_backup` cascade22:08:59.071808 [debug] [Thread-1 (]: SQL status: SUCCESS 0 in 0.0 seconds
22:08:59.073143 [debug] [Thread-1 (]: Using tidb connection "model.jaffle_shop.orders"
22:08:59.073565 [debug] [Thread-1 (]: On model.jaffle_shop.orders: /* {"app": "dbt", "dbt_version": "1.6.9", "profile_name": "jaffle_shop", "target_name": "dev", "node_id": "model.jaffle_shop.orders"} */rename table `oceanbase`.`orders` to `oceanbase`.`orders__dbt_backup`22:08:59.150592 [debug] [Thread-1 (]: SQL status: SUCCESS 0 in 0.0 seconds
22:08:59.156193 [debug] [Thread-1 (]: Using tidb connection "model.jaffle_shop.orders"
22:08:59.156664 [debug] [Thread-1 (]: On model.jaffle_shop.orders: /* {"app": "dbt", "dbt_version": "1.6.9", "profile_name": "jaffle_shop", "target_name": "dev", "node_id": "model.jaffle_shop.orders"} */drop table if exists `oceanbase`.`orders` cascade22:08:59.176531 [debug] [Thread-1 (]: SQL status: SUCCESS 0 in 0.0 seconds
22:08:59.179595 [debug] [Thread-1 (]: Using tidb connection "model.jaffle_shop.orders"
22:08:59.180285 [debug] [Thread-1 (]: On model.jaffle_shop.orders: /* {"app": "dbt", "dbt_version": "1.6.9", "profile_name": "jaffle_shop", "target_name": "dev", "node_id": "model.jaffle_shop.orders"} */rename table `oceanbase`.`orders__dbt_tmp` to `oceanbase`.`orders`22:08:59.248180 [debug] [Thread-1 (]: SQL status: SUCCESS 0 in 0.0 seconds
22:08:59.253036 [debug] [Thread-1 (]: On model.jaffle_shop.orders: COMMIT
22:08:59.253797 [debug] [Thread-1 (]: Using tidb connection "model.jaffle_shop.orders"
22:08:59.254205 [debug] [Thread-1 (]: On model.jaffle_shop.orders: COMMIT
22:08:59.268802 [debug] [Thread-1 (]: SQL status: SUCCESS 0 in 0.0 seconds
22:08:59.275387 [debug] [Thread-1 (]: Using tidb connection "model.jaffle_shop.orders"
22:08:59.275828 [debug] [Thread-1 (]: On model.jaffle_shop.orders: /* {"app": "dbt", "dbt_version": "1.6.9", "profile_name": "jaffle_shop", "target_name": "dev", "node_id": "model.jaffle_shop.orders"} */
drop table if exists `oceanbase`.`orders__dbt_backup`
22:08:59.460776 [debug] [Thread-1 (]: SQL status: SUCCESS 0 in 0.0 seconds
22:08:59.465491 [debug] [Thread-1 (]: Timing info for model.jaffle_shop.orders (execute): 22:08:58.593775 => 22:08:59.465044
22:08:59.466131 [debug] [Thread-1 (]: On model.jaffle_shop.orders: Close
22:08:59.467397 [debug] [Thread-1 (]: Sending event: {'category': 'dbt', 'action': 'run_model', 'label': 'f43f4f98-1c20-4751-8101-18f6577098a1', 'context': [<snowplow_tracker.self_describing_json.SelfDescribingJson object at 0x10c885950>]}
22:08:59.468191 [info ] [Thread-1 (]: 5 of 5 OK created sql table model oceanbase.orders ............................. [SUCCESS 0 in 0.88s]
22:08:59.468896 [debug] [Thread-1 (]: Finished running node model.jaffle_shop.orders
22:08:59.471036 [debug] [MainThread]: Using tidb connection "master"
22:08:59.471461 [debug] [MainThread]: On master: BEGIN
22:08:59.471741 [debug] [MainThread]: Opening a new connection, currently in state closed
22:08:59.556370 [debug] [MainThread]: SQL status: SUCCESS 0 in 0.0 seconds
22:08:59.557436 [debug] [MainThread]: On master: COMMIT
22:08:59.557989 [debug] [MainThread]: Using tidb connection "master"
22:08:59.558425 [debug] [MainThread]: On master: COMMIT
22:08:59.574008 [debug] [MainThread]: SQL status: SUCCESS 0 in 0.0 seconds
22:08:59.574684 [debug] [MainThread]: On master: Close
22:08:59.576154 [debug] [MainThread]: Connection 'master' was properly closed.
22:08:59.576798 [debug] [MainThread]: Connection 'model.jaffle_shop.orders' was properly closed.
22:08:59.577387 [info ] [MainThread]:
22:08:59.578056 [info ] [MainThread]: Finished running 3 view models, 2 table models in 0 hours 0 minutes and 3.87 seconds (3.87s).
22:08:59.579442 [debug] [MainThread]: Command end result
22:08:59.591150 [info ] [MainThread]:
22:08:59.591525 [info ] [MainThread]: Completed successfully
22:08:59.591792 [info ] [MainThread]:
22:08:59.592064 [info ] [MainThread]: Done. PASS=5 WARN=0 ERROR=0 SKIP=0 TOTAL=5
22:08:59.592567 [debug] [MainThread]: Command `dbt run` succeeded at 22:08:59.592489 after 4.01 seconds
22:08:59.592899 [debug] [MainThread]: Sending event: {'category': 'dbt', 'action': 'invocation', 'label': 'end', 'context': [<snowplow_tracker.self_describing_json.SelfDescribingJson object at 0x10a4eff10>, <snowplow_tracker.self_describing_json.SelfDescribingJson object at 0x1046a7f50>, <snowplow_tracker.self_describing_json.SelfDescribingJson object at 0x1046b4dd0>]}
22:08:59.593216 [debug] [MainThread]: Flushing usage events

对于 view stg_customers，dbt 的处理过程如下：

create view `oceanbase`.`stg_customers__dbt_tmp` as ...;
drop view if exists `oceanbase`.`stg_customers__dbt_backup` cascade;
rename table `oceanbase`.`stg_customers` to `oceanbase`.`stg_customers__dbt_backup`;
drop view if exists `oceanbase`.`stg_customers` cascade;
rename table `oceanbase`.`stg_customers__dbt_tmp` to `oceanbase`.`stg_customers`;

可见，它依靠创建临时 view，操作无误后通过 rename 的方式保证“原子性”。

对于 table orders，dbt 的处理过程如下：

create view   oceanbase.temp_view      as ( ... select * from final  );
create   table if not exists  `oceanbase`.`orders__dbt_tmp`  (...);
insert into `oceanbase`.`orders__dbt_tmp` select * from oceanbase.temp_view 
drop view if exists oceanbase.temp_view;
drop table if exists `oceanbase`.`orders__dbt_backup` cascade;
rename table `oceanbase`.`orders` to `oceanbase`.`orders__dbt_backup`;
drop table if exists `oceanbase`.`orders` cascade;
rename table `oceanbase`.`orders__dbt_tmp` to `oceanbase`.`orders`;

这里面，它创建了一个临时 temp_view，然后把 temp_view 的内容插入到 orders 表里。为了保证原子性，也使用了类似 view 的手法，创建了一些临时表。

作为一个完整的产品，dbt 还提供了一些贴心功能给数据使用者：
dbt docs generate 能够给数仓表生成文档，dbt docs serve 能启动一个服务用于浏览这些文档。真的很贴心。
在这里插入图片描述
在这里插入图片描述

本文来自互联网用户投稿，该文观点仅代表作者本人，不代表本站立场。本站仅提供信息存储空间服务，不拥有所有权，不承担相关法律责任。如若转载，请注明出处：http://www.rhkb.cn/news/262437.html

如若内容造成侵权/违法违规/事实不符，请联系长河编程网进行投诉反馈email:809451989@qq.com，一经查实，立即删除！

相关文章

C语言-指针初学速成

C语言-指针初学速成

1.指针是什么 C语言指针是一种特殊的变量&#xff0c;用于存储内存地址。它可以指向其他变量或者其他数据结构&#xff0c;通过指针可以直接访问或修改存储在指定地址的值。指针可以帮助我们在程序中动态地分配和释放内存&#xff0c;以及进行复杂的数据操作。在C语言中&#…
阅读更多...
冒泡排序法的名字由来，排序步骤是什么，最坏情况下的排序次数如何计算得来的呢？

冒泡排序法的名字由来，排序步骤是什么，最坏情况下的排序次数如何计算得来的呢？

问题描述&#xff1a;冒泡排序法的名字由来&#xff0c;排序步骤是什么&#xff0c;最坏情况下的排序次数如何计算得来的呢&#xff1f; 问题解答&#xff1a; 冒泡排序法的名字来源于排序过程中较大的元素会像气泡一样逐渐“冒”到序列的顶端&#xff0c;而较小的元素则会逐…
阅读更多...
BI 数据分析，数据库，Office，可视化，数据仓库

BI 数据分析，数据库，Office，可视化，数据仓库

AIGC ChatGPT 职场案例 AI 绘画 与 短视频制作 PowerBI 商业智能 68集 Mysql 8.0 54集 Oracle 21C 142集 Office 2021实战应用 Python 数据分析实战&#xff0c; ETL Informatica 数据仓库案例实战 51集 Excel 2021实操 100集&#xff0c; Excel 2021函数大全 80集 Excel 2021…
阅读更多...
谷歌掀桌子！开源Gemma：可商用，性能超过Llama 2！

谷歌掀桌子！开源Gemma：可商用，性能超过Llama 2！

2月22日&#xff0c;谷歌在官网宣布&#xff0c;开源大语言模型Gemma。 Gemma与谷歌最新发布的Gemini 使用了同一架构&#xff0c;有20亿、70亿两种参数&#xff0c;每种参数都有预训练和指令调优两个版本。 根据谷歌公布的测试显示&#xff0c;在MMLU、BBH、GSM8K等主流测试…
阅读更多...
PostgreSQL与MySQL，谁更胜一筹

PostgreSQL与MySQL，谁更胜一筹

前言 PostgreSQL与MySQL都是优秀的开源数据库。在日常学习中&#xff0c;新手可能接触最多的是MySql,但是实际工作中&#xff0c;两者的应用场景其实都很广。我之前的做过上网流量销售业务&#xff0c;用的是MySQL,现在接触广告业务&#xff0c;用的是pg数据库&#xff0c;每天…
阅读更多...
C# cass10 面积计算

C# cass10 面积计算

运行环境Visual Studio 2022 c# cad2016 cass10 通过面积计算得到扩展数据&#xff0c;宗地面积 &#xff0c;房屋占地面积&#xff0c;房屋使用面积 一、主要步骤 获取当前AutoCAD应用中的活动文档、数据库和编辑器对象。创建一个选择过滤器&#xff0c;限制用户只能选择&q…
阅读更多...
K8S—Pod详解

K8S—Pod详解

目录 一 Pod基础概念 1.1 Pod是什么 1.2 为什么要使用Pod&#xff1f;Pod在K8S集群中的使用方式&#xff1f; 1.3 基础容器pause 二 Pod的分类 2.1 自主式Pod和控制器管理的Pod 2.2 容器的分类 2.2.1 基础容器&#xff08;infrastructure container&#xff09; 2.2.2…
阅读更多...
igolang学习3，golang 项目中配置gin的web框架

igolang学习3，golang 项目中配置gin的web框架

1.go 初始化 mod文件 go mod init gin-ranking 2.gin的crm框架 go get -u github.com/gin-gonic/gin 3.go.mod爆红解决
阅读更多...
在word中将latex格式的公式转化为带有编号的mathtype公式

在word中将latex格式的公式转化为带有编号的mathtype公式

在word中将latex格式的公式转化为带有编号的mathtype公式 1.先在word里面配置好mathtype2.在word中设置mathtype的格式3.先将latex格式的公式转化为mathml格式4.读到这里&#xff0c;是不是觉得这个方法麻烦 1.先在word里面配置好mathtype 注意&#xff1a;1.word的版本应该是 …
阅读更多...
【动态规划】【回文】【字符串】1147. 段式回文

【动态规划】【回文】【字符串】1147. 段式回文

作者推荐 【广度优先搜索】【网格】【割点】【 推荐】1263. 推箱子 本文涉及知识点 动态规划汇总 LeetCode1147段式回文 你会得到一个字符串 text 。你应该把它分成 k 个子字符串 (subtext1, subtext2&#xff0c;…&#xff0c; subtextk) &#xff0c;要求满足: subtext…
阅读更多...
微服务-微服务Nacos配置中心

微服务-微服务Nacos配置中心

1.1 配置中心架构 1.2 Config Client源码分析 配置中心核心接口ConfigService public class ConfigServerDemo {public static void main(String[] args) throws NacosException, InterruptedException {String serverAddr "localhost";String dataId "naco…
阅读更多...
汽车常识网：电脑主机如何算功率的计算方法？

汽车常识网：电脑主机如何算功率的计算方法？

今天汽车知识网就给大家讲解一下如何计算一台主机的功率。 它还会解释如何计算计算机主机所需的功率&#xff1f; &#xff1f; &#xff08;如何计算电脑主机所需的功率&#xff09;进行说明。 如果它恰好解决了您现在面临的问题&#xff0c;请不要忘记关注本站。 让我们现在就…
阅读更多...
Elasticsearch：什么是 kNN?

Elasticsearch：什么是 kNN?

kNN - K-nearest neighbor 定义 kNN&#xff08;即 k 最近邻算法&#xff09;是一种机器学习算法&#xff0c;它使用邻近度将一个数据点与其训练并记忆的一组数据进行比较以进行预测。 这种基于实例的学习为 kNN 提供了 “惰性学习&#xff08;lazy learning&#xff09;” 名…
阅读更多...
2.5网安学习第二阶段第五周回顾（个人学习记录使用）

2.5网安学习第二阶段第五周回顾（个人学习记录使用）

本周重点 ①多进程和多线程 1、进程和线程 2、多线程爆破 ②Redis数据库 1、Redis的使用 2、Redis持久化 3、Redis未授权免密登录 ③嗅探和Python攻击脚本 1、嗅探&#xff08;端口扫描和IP扫描&#xff09; 2、SCAPY的应用 3、Python攻击脚本&#xff08;SYN半连接…
阅读更多...
前端数据可视化：ECharts使用

前端数据可视化：ECharts使用

可视化介绍 ​  ​  应对现在数据可视化的趋势&#xff0c;越来越多企业需要在很多场景(营销数据&#xff0c;生产数据&#xff0c;用户数据)下使用&#xff0c;可视化图表来展示体现数据&#xff0c;让数据更加直观&#xff0c;数据特点更加突出。   ​  数据可视化主要目…
阅读更多...
Qt_快速安装指南

Qt_快速安装指南

下载Qt在线安装程序&#xff08;不仔细介绍&#xff09;注册Qt账号&#xff08;不仔细介绍&#xff09;使用快速运行的命令&#xff0c;按照指定的下载地址下载 在Qt指定目录打开cmd命令窗口.\eqt-unified-windows-x86-4.0.1-1-online. exe --mirror https://mirrors.ustc.edu.…
阅读更多...
华清远见嵌入式学习——驱动开发——作业1

华清远见嵌入式学习——驱动开发——作业1

作业要求&#xff1a; 通过字符设备驱动分步注册过程实现LED驱动的编写&#xff0c;编写应用程序测试&#xff0c;发布到CSDN 作业答案&#xff1a; 运行效果&#xff1a; 驱动代码&#xff1a; #include <linux/init.h> #include <linux/module.h> #include &l…
阅读更多...
代理模式笔记

代理模式笔记

代理模式 代理模式代理模式的应用场景先理解什么是代理&#xff0c;再理解动静态举例举例所用代码 动静态的区别静态代理动态代理 动态代理的优点代理模式与装饰者模式的区别 代理模式 代理模式在设计模式中是7种结构型模式中的一种&#xff0c;而代理模式有分动态代理&#x…
阅读更多...
Nginx 配置前端工程项目二级目录

Nginx 配置前端工程项目二级目录

前提&#xff1a; 前端工程技术框架: vue 后端工程技术工程&#xff1a;spring boot 需求&#xff1a;需要通过二级目录访问前端工程&#xff1a; 如之前&#xff1a;http://127.0.0.1:80/ 改成 http://127.0.0.1/secondDirectory:80/ 一.前端工程支持二级目录 1.编译文…
阅读更多...
（十八）devops持续集成开发——使用docker安装部署jenkins流水线服务

（十八）devops持续集成开发——使用docker安装部署jenkins流水线服务

前言 本节内容介绍如何使用docker容器来部署安装jenkins流水线服务。关于docker容器的安装本节内容不做介绍。请读者提前安装。 正文 ①使用docker查找jenkins官方镜像 ② 拉取jenkins官方镜像jenkins/jenkins&#xff0c;选择一个最新稳定版本&#xff0c;避免一些插件不兼…
阅读更多...
最新文章
推荐文章

Copyright @ 2022~2023