物流实时数仓:数仓搭建(DWD)一

系列文章目录

物流实时数仓:采集通道搭建
物流实时数仓:数仓搭建
物流实时数仓:数仓搭建(DIM)
物流实时数仓:数仓搭建(DWD)一


文章目录

  • 系列文章目录
  • 前言
  • 一、文件编写
    • 1.目录创建
    • 2.bean文件
      • 1.DwdOrderDetailOriginBean
      • 2.DwdOrderInfoOriginBean
      • 3.DwdTradeCancelDetailBean
      • 4.DwdTradeOrderDetailBean
      • 5.DwdTradePaySucDetailBean
      • 6.DwdTransBoundFinishDetailBean
      • 7.DwdTransDeliverSucDetailBean
      • 8.DwdTransDispatchDetailBean
      • 9.DwdTransReceiveDetailBean
      • 10.DwdTransSignDetailBean
    • 3.DwdOrderRelevantApp
  • 二、代码测试
    • 1.环境启动
    • 2.kafka消费者
    • 3.修改配置
    • 4.测试结果
  • 总结


前言

这次博客我们进行DWD层的搭建,内容比较多,一次可能写不完。
在这里插入图片描述
以上就是本次博客需要完成的内容,简单来说就是,从kafka读取数据,然后根据不同的关键字,将其从主流中进行分离,然后在写入各自的kafka中以便后续的操作


一、文件编写

1.目录创建

我们现在beans中创建后边需要的的bean
在这里插入图片描述
然后在dwd目录中创建此次需要的app
在这里插入图片描述

2.bean文件

1.DwdOrderDetailOriginBean

package com.atguigu.tms.realtime.beans;import lombok.Data;import java.math.BigDecimal;/***订单货物明细实体类*/@Data
public class DwdOrderDetailOriginBean {// 编号(主键)String id;// 运单idString orderId;// 货物类型String cargoType;// 长cmInteger volumnLength;// 宽cmInteger volumnWidth;// 高cmInteger volumnHeight;// 重量 kgBigDecimal weight;// 创建时间String createTime;// 更新时间String updateTime;// 是否删除String isDeleted;
}

2.DwdOrderInfoOriginBean

package com.atguigu.tms.realtime.beans;import lombok.Data;import java.math.BigDecimal;/*** 订单实体类*/
@Data
public class DwdOrderInfoOriginBean {// 编号(主键)String id;// 运单号String orderNo;// 运单状态String status;// 取件类型,1为网点自寄,2为上门取件String collectType;// 客户idString userId;// 收件人小区idString receiverComplexId;// 收件人省份idString receiverProvinceId;// 收件人城市idString receiverCityId;// 收件人区县idString receiverDistrictId;// 收件人姓名String receiverName;// 发件人小区idString senderComplexId;// 发件人省份idString senderProvinceId;// 发件人城市idString senderCityId;// 发件人区县idString senderDistrictId;// 发件人姓名String senderName;// 支付方式String paymentType;// 货物个数Integer cargoNum;// 金额BigDecimal amount;// 预计到达时间Long estimateArriveTime;// 距离,单位:公里BigDecimal distance;// 创建时间String createTime;// 更新时间String updateTime;// 是否删除String isDeleted;
}

3.DwdTradeCancelDetailBean

package com.atguigu.tms.realtime.beans;import com.atguigu.tms.realtime.utils.DateFormatUtil;
import lombok.Data;import java.math.BigDecimal;/*** 交易域:取消运单事务事实表实体类*/
@Data
public class DwdTradeCancelDetailBean {// 运单明细IDString id;// 运单idString orderId;// 货物类型String cargoType;// 长cmInteger volumeLength;// 宽cmInteger volumeWidth;// 高cmInteger volumeHeight;// 重量 kgBigDecimal weight;// 取消时间String cancelTime;// 运单号String orderNo;// 运单状态String status;// 取件类型,1为网点自寄,2为上门取件String collectType;// 客户idString userId;// 收件人小区idString receiverComplexId;// 收件人省份idString receiverProvinceId;// 收件人城市idString receiverCityId;// 收件人区县idString receiverDistrictId;// 收件人姓名String receiverName;// 发件人小区idString senderComplexId;// 发件人省份idString senderProvinceId;// 发件人城市idString senderCityId;// 发件人区县idString senderDistrictId;// 发件人姓名String senderName;// 支付方式String paymentType;// 货物个数Integer cargoNum;// 金额BigDecimal amount;// 预计到达时间String estimateArriveTime;// 距离,单位:公里BigDecimal distance;// 时间戳Long ts;public void mergeBean(DwdOrderDetailOriginBean detailOriginBean, DwdOrderInfoOriginBean infoOriginBean) {// 合并原始明细字段this.id = detailOriginBean.id;this.orderId = detailOriginBean.orderId;this.cargoType = detailOriginBean.cargoType;this.volumeLength = detailOriginBean.volumnLength;this.volumeWidth = detailOriginBean.volumnWidth;this.volumeHeight = detailOriginBean.volumnHeight;this.weight = detailOriginBean.weight;// 合并原始订单字段this.orderNo = infoOriginBean.orderNo;this.status = infoOriginBean.status;this.collectType = infoOriginBean.collectType;this.userId = infoOriginBean.userId;this.receiverComplexId = infoOriginBean.receiverComplexId;this.receiverProvinceId = infoOriginBean.receiverProvinceId;this.receiverCityId = infoOriginBean.receiverCityId;this.receiverDistrictId = infoOriginBean.receiverDistrictId;this.receiverName = infoOriginBean.receiverName;this.senderComplexId = infoOriginBean.senderComplexId;this.senderProvinceId = infoOriginBean.senderProvinceId;this.senderCityId = infoOriginBean.senderCityId;this.senderDistrictId = infoOriginBean.senderDistrictId;this.senderName = infoOriginBean.senderName;this.paymentType = infoOriginBean.paymentType;this.cargoNum = infoOriginBean.cargoNum;this.amount = infoOriginBean.amount;this.estimateArriveTime = DateFormatUtil.toYmdHms(infoOriginBean.estimateArriveTime - 8 * 60 * 60 * 1000);this.distance = infoOriginBean.distance;this.cancelTime =DateFormatUtil.toYmdHms(DateFormatUtil.toTs(infoOriginBean.updateTime.replaceAll("T", " ").replaceAll("Z", ""), true)+ 8 * 60 * 60 * 1000);this.ts = DateFormatUtil.toTs(infoOriginBean.updateTime.replaceAll("T", " ").replaceAll("Z", ""), true)+ 8 * 60 * 60 * 1000;}
}

4.DwdTradeOrderDetailBean

package com.atguigu.tms.realtime.beans;import com.atguigu.tms.realtime.utils.DateFormatUtil;
import lombok.Data;import java.math.BigDecimal;/***交易域:下单事务事实表实体类*/
@Data
public class DwdTradeOrderDetailBean {// 运单明细IDString id;// 运单idString orderId;// 货物类型String cargoType;// 长cmInteger volumeLength;// 宽cmInteger volumeWidth;// 高cmInteger volumeHeight;// 重量 kgBigDecimal weight;// 下单时间String orderTime;// 运单号String orderNo;// 运单状态String status;// 取件类型,1为网点自寄,2为上门取件String collectType;// 客户idString userId;// 收件人小区idString receiverComplexId;// 收件人省份idString receiverProvinceId;// 收件人城市idString receiverCityId;// 收件人区县idString receiverDistrictId;// 收件人姓名String receiverName;// 发件人小区idString senderComplexId;// 发件人省份idString senderProvinceId;// 发件人城市idString senderCityId;// 发件人区县idString senderDistrictId;// 发件人姓名String senderName;// 支付方式String paymentType;// 货物个数Integer cargoNum;// 金额BigDecimal amount;// 预计到达时间String estimateArriveTime;// 距离,单位:公里BigDecimal distance;// 时间戳Long ts;public void mergeBean(DwdOrderDetailOriginBean detailOriginBean, DwdOrderInfoOriginBean infoOriginBean) {// 合并原始明细字段this.id = detailOriginBean.id;this.orderId = detailOriginBean.orderId;this.cargoType = detailOriginBean.cargoType;this.volumeLength = detailOriginBean.volumnLength;this.volumeWidth = detailOriginBean.volumnWidth;this.volumeHeight = detailOriginBean.volumnHeight;this.weight = detailOriginBean.weight;this.orderTime =DateFormatUtil.toYmdHms(DateFormatUtil.toTs(detailOriginBean.createTime.replaceAll("T", " ").replaceAll("Z", ""), true)+ 8 * 60 * 60 * 1000);this.ts = DateFormatUtil.toTs(detailOriginBean.createTime.replaceAll("T", " ").replaceAll("Z", ""), true)+ 8 * 60 * 60 * 1000;// 合并原始订单字段this.orderNo = infoOriginBean.orderNo;this.status = infoOriginBean.status;this.collectType = infoOriginBean.collectType;this.userId = infoOriginBean.userId;this.receiverComplexId = infoOriginBean.receiverComplexId;this.receiverProvinceId = infoOriginBean.receiverProvinceId;this.receiverCityId = infoOriginBean.receiverCityId;this.receiverDistrictId = infoOriginBean.receiverDistrictId;this.receiverName = infoOriginBean.receiverName;this.senderComplexId = infoOriginBean.senderComplexId;this.senderProvinceId = infoOriginBean.senderProvinceId;this.senderCityId = infoOriginBean.senderCityId;this.senderDistrictId = infoOriginBean.senderDistrictId;this.senderName = infoOriginBean.senderName;this.paymentType = infoOriginBean.paymentType;this.cargoNum = infoOriginBean.cargoNum;this.amount = infoOriginBean.amount;this.estimateArriveTime = DateFormatUtil.toYmdHms(infoOriginBean.estimateArriveTime - 8 * 60 * 60 * 1000);this.distance = infoOriginBean.distance;}
}

5.DwdTradePaySucDetailBean

package com.atguigu.tms.realtime.beans;
import com.atguigu.tms.realtime.utils.DateFormatUtil;
import lombok.Data;import java.math.BigDecimal;/***交易域:支付成功事务事实表实体类*/
@Data
public class DwdTradePaySucDetailBean {// 运单明细IDString id;// 运单idString orderId;// 货物类型String cargoType;// 长cmInteger volumeLength;// 宽cmInteger volumeWidth;// 高cmInteger volumeHeight;// 重量 kgBigDecimal weight;// 支付时间String payTime;// 运单号String orderNo;// 运单状态String status;// 取件类型,1为网点自寄,2为上门取件String collectType;// 客户idString userId;// 收件人小区idString receiverComplexId;// 收件人省份idString receiverProvinceId;// 收件人城市idString receiverCityId;// 收件人区县idString receiverDistrictId;// 收件人姓名String receiverName;// 发件人小区idString senderComplexId;// 发件人省份idString senderProvinceId;// 发件人城市idString senderCityId;// 发件人区县idString senderDistrictId;// 发件人姓名String senderName;// 支付方式String paymentType;// 货物个数Integer cargoNum;// 金额BigDecimal amount;// 预计到达时间String estimateArriveTime;// 距离,单位:公里BigDecimal distance;// 时间戳Long ts;public void mergeBean(DwdOrderDetailOriginBean detailOriginBean, DwdOrderInfoOriginBean infoOriginBean) {// 合并原始明细字段this.id = detailOriginBean.id;this.orderId = detailOriginBean.orderId;this.cargoType = detailOriginBean.cargoType;this.volumeLength = detailOriginBean.volumnLength;this.volumeWidth = detailOriginBean.volumnWidth;this.volumeHeight = detailOriginBean.volumnHeight;this.weight = detailOriginBean.weight;// 合并原始订单字段this.orderNo = infoOriginBean.orderNo;this.status = infoOriginBean.status;this.collectType = infoOriginBean.collectType;this.userId = infoOriginBean.userId;this.receiverComplexId = infoOriginBean.receiverComplexId;this.receiverProvinceId = infoOriginBean.receiverProvinceId;this.receiverCityId = infoOriginBean.receiverCityId;this.receiverDistrictId = infoOriginBean.receiverDistrictId;this.receiverName = infoOriginBean.receiverName;this.senderComplexId = infoOriginBean.senderComplexId;this.senderProvinceId = infoOriginBean.senderProvinceId;this.senderCityId = infoOriginBean.senderCityId;this.senderDistrictId = infoOriginBean.senderDistrictId;this.senderName = infoOriginBean.senderName;this.paymentType = infoOriginBean.paymentType;this.cargoNum = infoOriginBean.cargoNum;this.amount = infoOriginBean.amount;this.estimateArriveTime = DateFormatUtil.toYmdHms(infoOriginBean.estimateArriveTime - 8 * 60 * 60 * 1000);this.distance = infoOriginBean.distance;this.payTime =DateFormatUtil.toYmdHms(DateFormatUtil.toTs(infoOriginBean.updateTime.replaceAll("T", " ").replaceAll("Z", ""), true)+ 8 * 60 * 60 * 1000);this.ts = DateFormatUtil.toTs(infoOriginBean.updateTime.replaceAll("T", " ").replaceAll("Z", ""), true)+ 8 * 60 * 60 * 1000;}
}

6.DwdTransBoundFinishDetailBean

package com.atguigu.tms.realtime.beans;import com.atguigu.tms.realtime.utils.DateFormatUtil;
import lombok.Data;import java.math.BigDecimal;
/***物流域:转运完成事务事实表实体类*/
@Data
public class DwdTransBoundFinishDetailBean {// 运单明细IDString id;// 运单idString orderId;// 货物类型String cargoType;// 长cmInteger volumeLength;// 宽cmInteger volumeWidth;// 高cmInteger volumeHeight;// 重量 kgBigDecimal weight;// 发单时间String boundFinishTime;// 运单号String orderNo;// 运单状态String status;// 取件类型,1为网点自寄,2为上门取件String collectType;// 客户idString userId;// 收件人小区idString receiverComplexId;// 收件人省份idString receiverProvinceId;// 收件人城市idString receiverCityId;// 收件人区县idString receiverDistrictId;// 收件人姓名String receiverName;// 发件人小区idString senderComplexId;// 发件人省份idString senderProvinceId;// 发件人城市idString senderCityId;// 发件人区县idString senderDistrictId;// 发件人姓名String senderName;// 支付方式String paymentType;// 货物个数Integer cargoNum;// 金额BigDecimal amount;// 预计到达时间String estimateArriveTime;// 距离,单位:公里BigDecimal distance;// 时间戳Long ts;public void mergeBean(DwdOrderDetailOriginBean detailOriginBean, DwdOrderInfoOriginBean infoOriginBean) {// 合并原始明细字段this.id = detailOriginBean.id;this.orderId = detailOriginBean.orderId;this.cargoType = detailOriginBean.cargoType;this.volumeLength = detailOriginBean.volumnLength;this.volumeWidth = detailOriginBean.volumnWidth;this.volumeHeight = detailOriginBean.volumnHeight;this.weight = detailOriginBean.weight;// 合并原始订单字段this.orderNo = infoOriginBean.orderNo;this.status = infoOriginBean.status;this.collectType = infoOriginBean.collectType;this.userId = infoOriginBean.userId;this.receiverComplexId = infoOriginBean.receiverComplexId;this.receiverProvinceId = infoOriginBean.receiverProvinceId;this.receiverCityId = infoOriginBean.receiverCityId;this.receiverDistrictId = infoOriginBean.receiverDistrictId;this.receiverName = infoOriginBean.receiverName;this.senderComplexId = infoOriginBean.senderComplexId;this.senderProvinceId = infoOriginBean.senderProvinceId;this.senderCityId = infoOriginBean.senderCityId;this.senderDistrictId = infoOriginBean.senderDistrictId;this.senderName = infoOriginBean.senderName;this.paymentType = infoOriginBean.paymentType;this.cargoNum = infoOriginBean.cargoNum;this.amount = infoOriginBean.amount;this.estimateArriveTime = DateFormatUtil.toYmdHms(infoOriginBean.estimateArriveTime - 8 * 60 * 60 * 1000);this.distance = infoOriginBean.distance;this.boundFinishTime =DateFormatUtil.toYmdHms(DateFormatUtil.toTs(infoOriginBean.updateTime.replaceAll("T", " ").replaceAll("Z", ""), true)+ 8 * 60 * 60 * 1000);this.ts = DateFormatUtil.toTs(infoOriginBean.updateTime.replaceAll("T", " ").replaceAll("Z", ""), true)+ 8 * 60 * 60 * 1000;}
}

7.DwdTransDeliverSucDetailBean

package com.atguigu.tms.realtime.beans;import com.atguigu.tms.realtime.utils.DateFormatUtil;
import lombok.Data;import java.math.BigDecimal;
/***物流域:派送成功事务事实表实体类*/
@Data
public class DwdTransDeliverSucDetailBean {// 运单明细IDString id;// 运单idString orderId;// 货物类型String cargoType;// 长cmInteger volumeLength;// 宽cmInteger volumeWidth;// 高cmInteger volumeHeight;// 重量 kgBigDecimal weight;// 派送成功时间String deliverTime;// 运单号String orderNo;// 运单状态String status;// 取件类型,1为网点自寄,2为上门取件String collectType;// 客户idString userId;// 收件人小区idString receiverComplexId;// 收件人省份idString receiverProvinceId;// 收件人城市idString receiverCityId;// 收件人区县idString receiverDistrictId;// 收件人姓名String receiverName;// 发件人小区idString senderComplexId;// 发件人省份idString senderProvinceId;// 发件人城市idString senderCityId;// 发件人区县idString senderDistrictId;// 发件人姓名String senderName;// 支付方式String paymentType;// 货物个数Integer cargoNum;// 金额BigDecimal amount;// 预计到达时间String estimateArriveTime;// 距离,单位:公里BigDecimal distance;// 时间戳Long ts;public void mergeBean(DwdOrderDetailOriginBean detailOriginBean, DwdOrderInfoOriginBean infoOriginBean) {// 合并原始明细字段this.id = detailOriginBean.id;this.orderId = detailOriginBean.orderId;this.cargoType = detailOriginBean.cargoType;this.volumeLength = detailOriginBean.volumnLength;this.volumeWidth = detailOriginBean.volumnWidth;this.volumeHeight = detailOriginBean.volumnHeight;this.weight = detailOriginBean.weight;// 合并原始订单字段this.orderNo = infoOriginBean.orderNo;this.status = infoOriginBean.status;this.collectType = infoOriginBean.collectType;this.userId = infoOriginBean.userId;this.receiverComplexId = infoOriginBean.receiverComplexId;this.receiverProvinceId = infoOriginBean.receiverProvinceId;this.receiverCityId = infoOriginBean.receiverCityId;this.receiverDistrictId = infoOriginBean.receiverDistrictId;this.receiverName = infoOriginBean.receiverName;this.senderComplexId = infoOriginBean.senderComplexId;this.senderProvinceId = infoOriginBean.senderProvinceId;this.senderCityId = infoOriginBean.senderCityId;this.senderDistrictId = infoOriginBean.senderDistrictId;this.senderName = infoOriginBean.senderName;this.paymentType = infoOriginBean.paymentType;this.cargoNum = infoOriginBean.cargoNum;this.amount = infoOriginBean.amount;this.estimateArriveTime = DateFormatUtil.toYmdHms(infoOriginBean.estimateArriveTime - 8 * 60 * 60 * 1000);this.distance = infoOriginBean.distance;this.deliverTime =DateFormatUtil.toYmdHms(DateFormatUtil.toTs(infoOriginBean.updateTime.replaceAll("T", " ").replaceAll("Z", ""), true)+ 8 * 60 * 60 * 1000);this.ts = DateFormatUtil.toTs(infoOriginBean.updateTime.replaceAll("T", " ").replaceAll("Z", ""), true)+ 8 * 60 * 60 * 1000;}
}

8.DwdTransDispatchDetailBean

package com.atguigu.tms.realtime.beans;import com.atguigu.tms.realtime.utils.DateFormatUtil;import lombok.Data;import java.math.BigDecimal;/***物流域:发单事务事实表实体类*/
@Data
public class DwdTransDispatchDetailBean {// 运单明细IDString id;// 运单idString orderId;// 货物类型String cargoType;// 长cmInteger volumeLength;// 宽cmInteger volumeWidth;// 高cmInteger volumeHeight;// 重量 kgBigDecimal weight;// 发单时间String dispatchTime;// 运单号String orderNo;// 运单状态String status;// 取件类型,1为网点自寄,2为上门取件String collectType;// 客户idString userId;// 收件人小区idString receiverComplexId;// 收件人省份idString receiverProvinceId;// 收件人城市idString receiverCityId;// 收件人区县idString receiverDistrictId;// 收件人姓名String receiverName;// 发件人小区idString senderComplexId;// 发件人省份idString senderProvinceId;// 发件人城市idString senderCityId;// 发件人区县idString senderDistrictId;// 发件人姓名String senderName;// 支付方式String paymentType;// 货物个数Integer cargoNum;// 金额BigDecimal amount;// 预计到达时间String estimateArriveTime;// 距离,单位:公里BigDecimal distance;// 时间戳Long ts;public void mergeBean(DwdOrderDetailOriginBean detailOriginBean, DwdOrderInfoOriginBean infoOriginBean) {// 合并原始明细字段this.id = detailOriginBean.id;this.orderId = detailOriginBean.orderId;this.cargoType = detailOriginBean.cargoType;this.volumeLength = detailOriginBean.volumnLength;this.volumeWidth = detailOriginBean.volumnWidth;this.volumeHeight = detailOriginBean.volumnHeight;this.weight = detailOriginBean.weight;// 合并原始订单字段this.orderNo = infoOriginBean.orderNo;this.status = infoOriginBean.status;this.collectType = infoOriginBean.collectType;this.userId = infoOriginBean.userId;this.receiverComplexId = infoOriginBean.receiverComplexId;this.receiverProvinceId = infoOriginBean.receiverProvinceId;this.receiverCityId = infoOriginBean.receiverCityId;this.receiverDistrictId = infoOriginBean.receiverDistrictId;this.receiverName = infoOriginBean.receiverName;this.senderComplexId = infoOriginBean.senderComplexId;this.senderProvinceId = infoOriginBean.senderProvinceId;this.senderCityId = infoOriginBean.senderCityId;this.senderDistrictId = infoOriginBean.senderDistrictId;this.senderName = infoOriginBean.senderName;this.paymentType = infoOriginBean.paymentType;this.cargoNum = infoOriginBean.cargoNum;this.amount = infoOriginBean.amount;this.estimateArriveTime = DateFormatUtil.toYmdHms(infoOriginBean.estimateArriveTime - 8 * 60 * 60 * 1000);this.distance = infoOriginBean.distance;this.dispatchTime =DateFormatUtil.toYmdHms(DateFormatUtil.toTs(infoOriginBean.updateTime.replaceAll("T", " ").replaceAll("Z", ""), true)+ 8 * 60 * 60 * 1000);this.ts = DateFormatUtil.toTs(infoOriginBean.updateTime.replaceAll("T", " ").replaceAll("Z", ""), true)+ 8 * 60 * 60 * 1000;}
}

9.DwdTransReceiveDetailBean

package com.atguigu.tms.realtime.beans;import com.atguigu.tms.realtime.utils.DateFormatUtil;
import lombok.Data;import java.math.BigDecimal;/***物流域:揽收(接单)事务事实表实体类*/
@Data
public class DwdTransReceiveDetailBean {// 运单明细IDString id;// 运单idString orderId;// 货物类型String cargoType;// 长cmInteger volumeLength;// 宽cmInteger volumeWidth;// 高cmInteger volumeHeight;// 重量 kgBigDecimal weight;// 揽收时间String receiveTime;// 运单号String orderNo;// 运单状态String status;// 取件类型,1为网点自寄,2为上门取件String collectType;// 客户idString userId;// 收件人小区idString receiverComplexId;// 收件人省份idString receiverProvinceId;// 收件人城市idString receiverCityId;// 收件人区县idString receiverDistrictId;// 收件人姓名String receiverName;// 发件人小区idString senderComplexId;// 发件人省份idString senderProvinceId;// 发件人城市idString senderCityId;// 发件人区县idString senderDistrictId;// 发件人姓名String senderName;// 支付方式String paymentType;// 货物个数Integer cargoNum;// 金额BigDecimal amount;// 预计到达时间String estimateArriveTime;// 距离,单位:公里BigDecimal distance;// 时间戳Long ts;public void mergeBean(DwdOrderDetailOriginBean detailOriginBean, DwdOrderInfoOriginBean infoOriginBean) {// 合并原始明细字段this.id = detailOriginBean.id;this.orderId = detailOriginBean.orderId;this.cargoType = detailOriginBean.cargoType;this.volumeLength = detailOriginBean.volumnLength;this.volumeWidth = detailOriginBean.volumnWidth;this.volumeHeight = detailOriginBean.volumnHeight;this.weight = detailOriginBean.weight;// 合并原始订单字段this.orderNo = infoOriginBean.orderNo;this.status = infoOriginBean.status;this.collectType = infoOriginBean.collectType;this.userId = infoOriginBean.userId;this.receiverComplexId = infoOriginBean.receiverComplexId;this.receiverProvinceId = infoOriginBean.receiverProvinceId;this.receiverCityId = infoOriginBean.receiverCityId;this.receiverDistrictId = infoOriginBean.receiverDistrictId;this.receiverName = infoOriginBean.receiverName;this.senderComplexId = infoOriginBean.senderComplexId;this.senderProvinceId = infoOriginBean.senderProvinceId;this.senderCityId = infoOriginBean.senderCityId;this.senderDistrictId = infoOriginBean.senderDistrictId;this.senderName = infoOriginBean.senderName;this.paymentType = infoOriginBean.paymentType;this.cargoNum = infoOriginBean.cargoNum;this.amount = infoOriginBean.amount;this.estimateArriveTime = DateFormatUtil.toYmdHms(infoOriginBean.estimateArriveTime - 8 * 60 * 60 * 1000);this.distance = infoOriginBean.distance;this.receiveTime =DateFormatUtil.toYmdHms(DateFormatUtil.toTs(infoOriginBean.updateTime.replaceAll("T", " ").replaceAll("Z", ""), true)+ 8 * 60 * 60 * 1000);this.ts = DateFormatUtil.toTs(infoOriginBean.updateTime.replaceAll("T", " ").replaceAll("Z", ""), true)+ 8 * 60 * 60 * 1000;}
}

10.DwdTransSignDetailBean

package com.atguigu.tms.realtime.beans;import com.atguigu.tms.realtime.utils.DateFormatUtil;
import lombok.Data;import java.math.BigDecimal;/*** 物流域:签收事务事实表实体类*/
@Data
public class DwdTransSignDetailBean {// 运单明细IDString id;// 运单idString orderId;// 货物类型String cargoType;// 长cmInteger volumeLength;// 宽cmInteger volumeWidth;// 高cmInteger volumeHeight;// 重量 kgBigDecimal weight;// 签收时间String signTime;// 运单号String orderNo;// 运单状态String status;// 取件类型,1为网点自寄,2为上门取件String collectType;// 客户idString userId;// 收件人小区idString receiverComplexId;// 收件人省份idString receiverProvinceId;// 收件人城市idString receiverCityId;// 收件人区县idString receiverDistrictId;// 收件人姓名String receiverName;// 发件人小区idString senderComplexId;// 发件人省份idString senderProvinceId;// 发件人城市idString senderCityId;// 发件人区县idString senderDistrictId;// 发件人姓名String senderName;// 支付方式String paymentType;// 货物个数Integer cargoNum;// 金额BigDecimal amount;// 预计到达时间String estimateArriveTime;// 距离,单位:公里BigDecimal distance;// 时间戳Long ts;public void mergeBean(DwdOrderDetailOriginBean detailOriginBean, DwdOrderInfoOriginBean infoOriginBean) {// 合并原始明细字段this.id = detailOriginBean.id;this.orderId = detailOriginBean.orderId;this.cargoType = detailOriginBean.cargoType;this.volumeLength = detailOriginBean.volumnLength;this.volumeWidth = detailOriginBean.volumnWidth;this.volumeHeight = detailOriginBean.volumnHeight;this.weight = detailOriginBean.weight;// 合并原始订单字段this.orderNo = infoOriginBean.orderNo;this.status = infoOriginBean.status;this.collectType = infoOriginBean.collectType;this.userId = infoOriginBean.userId;this.receiverComplexId = infoOriginBean.receiverComplexId;this.receiverProvinceId = infoOriginBean.receiverProvinceId;this.receiverCityId = infoOriginBean.receiverCityId;this.receiverDistrictId = infoOriginBean.receiverDistrictId;this.receiverName = infoOriginBean.receiverName;this.senderComplexId = infoOriginBean.senderComplexId;this.senderProvinceId = infoOriginBean.senderProvinceId;this.senderCityId = infoOriginBean.senderCityId;this.senderDistrictId = infoOriginBean.senderDistrictId;this.senderName = infoOriginBean.senderName;this.paymentType = infoOriginBean.paymentType;this.cargoNum = infoOriginBean.cargoNum;this.amount = infoOriginBean.amount;this.estimateArriveTime = DateFormatUtil.toYmdHms(infoOriginBean.estimateArriveTime - 8 * 60 * 60 * 1000);this.distance = infoOriginBean.distance;this.signTime =DateFormatUtil.toYmdHms(DateFormatUtil.toTs(infoOriginBean.updateTime.replaceAll("T", " ").replaceAll("Z", ""), true)+ 8 * 60 * 60 * 1000);this.ts = DateFormatUtil.toTs(infoOriginBean.updateTime.replaceAll("T", " ").replaceAll("Z", ""), true)+ 8 * 60 * 60 * 1000;}
}

3.DwdOrderRelevantApp

package com.atguigu.tms.realtime.app.dwd;import com.alibaba.fastjson.JSON;
import com.alibaba.fastjson.JSONObject;
import com.atguigu.tms.realtime.beans.*;
import com.atguigu.tms.realtime.utils.CreateEnvUtil;
import com.atguigu.tms.realtime.utils.KafkaUtil;
import org.apache.flink.api.common.eventtime.WatermarkStrategy;
import org.apache.flink.api.common.functions.FilterFunction;
import org.apache.flink.api.common.functions.MapFunction;
import org.apache.flink.api.common.state.StateTtlConfig;
import org.apache.flink.api.common.state.ValueState;
import org.apache.flink.api.common.state.ValueStateDescriptor;
import org.apache.flink.api.common.time.Time;
import org.apache.flink.api.java.functions.KeySelector;
import org.apache.flink.configuration.Configuration;
import org.apache.flink.connector.kafka.sink.KafkaSink;
import org.apache.flink.connector.kafka.source.KafkaSource;
import org.apache.flink.streaming.api.datastream.KeyedStream;
import org.apache.flink.streaming.api.datastream.SideOutputDataStream;
import org.apache.flink.streaming.api.datastream.SingleOutputStreamOperator;
import org.apache.flink.streaming.api.environment.StreamExecutionEnvironment;
import org.apache.flink.streaming.api.functions.KeyedProcessFunction;
import org.apache.flink.util.Collector;
import org.apache.flink.util.OutputTag;public class DwdOrderRelevantApp {public static void main(String[] args) throws Exception {// 1.环境准备StreamExecutionEnvironment env = CreateEnvUtil.getStreamEnv(args);env.setParallelism(4);// 2.从Kafka读数据String topic = "tms_ods";String groupId = "dwd_order_relevant_group";KafkaSource<String> kafkaSource = KafkaUtil.getKafkaSource(topic, groupId, args);SingleOutputStreamOperator<String> kafkaStrDS = env.fromSource(kafkaSource, WatermarkStrategy.noWatermarks(), "kafka_source").uid("kafka_source");// 3.筛选订单和订单明细数据SingleOutputStreamOperator<String> filterDS = kafkaStrDS.filter((FilterFunction<String>) jsonStr -> {JSONObject jsonObj = JSON.parseObject(jsonStr);String tableName = jsonObj.getJSONObject("source").getString("table");return "order_info".equals(tableName) || "order_cargo".equals(tableName);});
//        filterDS.print(">>>");// 4.对流中的数据类型进行转换 jsonStr->jsonObjSingleOutputStreamOperator<JSONObject> jsonObjDS = filterDS.map((MapFunction<String, JSONObject>) jsonStr -> {JSONObject jsonObj = JSON.parseObject(jsonStr);String tableName = jsonObj.getJSONObject("source").getString("table");jsonObj.put("table", tableName);jsonObj.remove("source");jsonObj.remove("transaction");return jsonObj;});//        jsonObjDS.print(">>>");// 5.按照order_id进行分组KeyedStream<JSONObject, String> keyDS = jsonObjDS.keyBy((KeySelector<JSONObject, String>) jsonObj -> {String table = jsonObj.getString("table");if ("order_info".equals(table)) {return jsonObj.getJSONObject("after").getString("id");}return jsonObj.getJSONObject("after").getString("order_id");});
//        keyDS.print(">>>");// 6.定义侧输出流标签 下单放到主流,支付成功、取消运单、揽收(接单)、发单 转运完成、派送成功、签收放到侧输出流// 支付成功明细流标签OutputTag<String> paySucTag = new OutputTag<String>("dwd_trade_pay_suc_detail") {};// 取消运单明细流标签OutputTag<String> cancelDetailTag = new OutputTag<String>("dwd_trade_cancel_detail") {};// 揽收明细流标签OutputTag<String> receiveDetailTag = new OutputTag<String>("dwd_trans_receive_detail") {};// 发单明细流标签OutputTag<String> dispatchDetailTag = new OutputTag<String>("dwd_trans_dispatch_detail") {};// 转运完成明细流标签OutputTag<String> boundFinishDetailTag = new OutputTag<String>("dwd_trans_bound_finish_detail") {};// 派送成功明细流标签OutputTag<String> deliverSucDetailTag = new OutputTag<String>("dwd_trans_deliver_detail") {};// 签收明细流标签OutputTag<String> signDetailTag = new OutputTag<String>("dwd_trans_sign_detail") {};// 7.分流SingleOutputStreamOperator<String> orderDetailDS = keyDS.process(new KeyedProcessFunction<String, JSONObject, String>() {private ValueState<DwdOrderInfoOriginBean> infoBeanState;private ValueState<DwdOrderDetailOriginBean> detailBeanState;@Overridepublic void open(Configuration parameters) {ValueStateDescriptor<DwdOrderInfoOriginBean> InfoOriginBeanStateDescriptor= new ValueStateDescriptor<>("infoBeanState", DwdOrderInfoOriginBean.class);InfoOriginBeanStateDescriptor.enableTimeToLive(StateTtlConfig.newBuilder(Time.seconds(5)).build());infoBeanState = getRuntimeContext().getState(InfoOriginBeanStateDescriptor);ValueStateDescriptor detailBeanStateDescriptor= new ValueStateDescriptor<>("detailBeanState", DwdOrderDetailOriginBean.class);detailBeanState = getRuntimeContext().getState(detailBeanStateDescriptor);}@Overridepublic void processElement(JSONObject jsonObj, KeyedProcessFunction<String, JSONObject, String>.Context ctx, Collector<String> out) throws Exception {String table = jsonObj.getString("table");String op = jsonObj.getString("op");JSONObject data = jsonObj.getJSONObject("after");if ("order_info".equals(table)) {//处理的是订单数据DwdOrderInfoOriginBean infoOriginBean = data.toJavaObject(DwdOrderInfoOriginBean.class);// 脱敏String senderName = infoOriginBean.getSenderName();String receiverName = infoOriginBean.getReceiverName();senderName = senderName.charAt(0) + senderName.substring(1).replaceAll(".", "\\*");receiverName = receiverName.charAt(0) + receiverName.substring(1).replaceAll(".", "\\*");infoOriginBean.setSenderName(senderName);infoOriginBean.setReceiverName(receiverName);DwdOrderDetailOriginBean detailOriginBean = detailBeanState.value();if ("c".equals(op)) {// 下单操作if (detailOriginBean == null) {// 订单数据 比明细数据先到,将订单数据放到状态中infoBeanState.update(infoOriginBean);} else {// 说明订单数据来之前,明细数据已经来到了,直接关联DwdTradeOrderDetailBean dwdTradeOrderDetailBean = new DwdTradeOrderDetailBean();dwdTradeOrderDetailBean.mergeBean(detailOriginBean, infoOriginBean);// 将下单业务过程数据 放到主流中out.collect(JSON.toJSONString(dwdTradeOrderDetailBean));}} else if ("u".equals(op) && detailOriginBean != null) {// 其它操作// 获取修改前的数据JSONObject oldData = jsonObj.getJSONObject("before");// 获取修改前的状态值String oldStatus = oldData.getString("status");String status = infoOriginBean.getStatus();if (!oldStatus.equals(status)) {// 说明修改的是status字段String changeLog = oldStatus + " -> " + status;switch (changeLog) {case "60010 -> 60020":// 处理支付成功数据DwdTradePaySucDetailBean dwdTradePaySucDetailBean = new DwdTradePaySucDetailBean();dwdTradePaySucDetailBean.mergeBean(detailOriginBean, infoOriginBean);ctx.output(paySucTag, JSON.toJSONString(dwdTradePaySucDetailBean));break;case "60020 -> 60030":// 处理揽收明细数据DwdTransReceiveDetailBean dwdTransReceiveDetailBean = new DwdTransReceiveDetailBean();dwdTransReceiveDetailBean.mergeBean(detailOriginBean, infoOriginBean);ctx.output(receiveDetailTag, JSON.toJSONString(dwdTransReceiveDetailBean));break;case "60040 -> 60050":// 处理发单明细数据DwdTransDispatchDetailBean dispatchDetailBean = new DwdTransDispatchDetailBean();dispatchDetailBean.mergeBean(detailOriginBean, infoOriginBean);ctx.output(dispatchDetailTag, JSON.toJSONString(dispatchDetailBean));break;case "60050 -> 60060":// 处理转运完成明细数据DwdTransBoundFinishDetailBean boundFinishDetailBean = new DwdTransBoundFinishDetailBean();boundFinishDetailBean.mergeBean(detailOriginBean, infoOriginBean);ctx.output(boundFinishDetailTag, JSON.toJSONString(boundFinishDetailBean));break;case "60060 -> 60070":// 处理派送成功数据DwdTransDeliverSucDetailBean dwdTransDeliverSucDetailBean = new DwdTransDeliverSucDetailBean();dwdTransDeliverSucDetailBean.mergeBean(detailOriginBean, infoOriginBean);ctx.output(deliverSucDetailTag, JSON.toJSONString(dwdTransDeliverSucDetailBean));break;case "60070 -> 60080":// 处理签收明细数据DwdTransSignDetailBean dwdTransSignDetailBean = new DwdTransSignDetailBean();dwdTransSignDetailBean.mergeBean(detailOriginBean, infoOriginBean);ctx.output(signDetailTag, JSON.toJSONString(dwdTransSignDetailBean));// 签收后订单数据不会再发生变化,状态可以清除detailBeanState.clear();break;default:if (status.equals("60999")) {DwdTradeCancelDetailBean dwdTradeCancelDetailBean = new DwdTradeCancelDetailBean();dwdTradeCancelDetailBean.mergeBean(detailOriginBean, infoOriginBean);ctx.output(cancelDetailTag, JSON.toJSONString(dwdTradeCancelDetailBean));// 取消后订单数据不会再发生变化,状态可以清除detailBeanState.clear();}break;}}}} else {// 处理订单明细DwdOrderDetailOriginBean detailOriginBean = data.toJavaObject(DwdOrderDetailOriginBean.class);if ("c".equals(op)) {detailBeanState.update(detailOriginBean);// 获取状态中存放的订单数据 注意:只有下单操作,并且订单数据先到,明细数据后到的情况,才会从状态中拿到订单数据DwdOrderInfoOriginBean infoOriginBean = infoBeanState.value();if (infoOriginBean != null) {//属于下单业务过程DwdTradeOrderDetailBean dwdTradeOrderDetailBean = new DwdTradeOrderDetailBean();dwdTradeOrderDetailBean.mergeBean(detailOriginBean, infoOriginBean);// 将下单业务过程数据 放到主流中out.collect(JSON.toJSONString(dwdTradeOrderDetailBean));}}}}}).uid("process_data");// 8.从主流中提取侧输出流// 支付成功明细流//8.1 支付成功明细流SideOutputDataStream<String> paySucDS = orderDetailDS.getSideOutput(paySucTag);// 8.2 取消运单明细流SideOutputDataStream<String> cancelDetailDS = orderDetailDS.getSideOutput(cancelDetailTag);// 8.3 揽收明细流SideOutputDataStream<String> receiveDetailDS = orderDetailDS.getSideOutput(receiveDetailTag);// 8.4 发单明细流SideOutputDataStream<String> dispatchDetailDS = orderDetailDS.getSideOutput(dispatchDetailTag);// 8.5 转运成功明细流SideOutputDataStream<String> boundFinishDetailDS = orderDetailDS.getSideOutput(boundFinishDetailTag);// 8.6 派送成功明细流SideOutputDataStream<String> deliverSucDetailDS = orderDetailDS.getSideOutput(deliverSucDetailTag);// 8.7 签收明细流SideOutputDataStream<String> signDetailDS = orderDetailDS.getSideOutput(signDetailTag);// 9.将不同流的数据写到kafka的不同主题中// 9.1.1 交易域下单明细主题String detailTopic = "tms_dwd_trade_order_detail";// 9.1.2 交易域支付成功明细主题String paySucDetailTopic = "tms_dwd_trade_pay_suc_detail";// 9.1.3 交易域取消运单明细主题String cancelDetailTopic = "tms_dwd_trade_cancel_detail";// 9.1.4 物流域接单(揽收)明细主题String receiveDetailTopic = "tms_dwd_trans_receive_detail";// 9.1.5 物流域发单明细主题String dispatchDetailTopic = "tms_dwd_trans_dispatch_detail";// 9.1.6 物流域转运完成明细主题String boundFinishDetailTopic = "tms_dwd_trans_bound_finish_detail";// 9.1.7 物流域派送成功明细主题String deliverSucDetailTopic = "tms_dwd_trans_deliver_detail";// 9.1.8 物流域签收明细主题String signDetailTopic = "tms_dwd_trans_sign_detail";// 9.2 发送数据到 Kafka// 9.2.1 运单明细数据KafkaSink<String> kafkaProducer = KafkaUtil.getKafkaSink(detailTopic, args);orderDetailDS.print("~~");orderDetailDS.sinkTo(kafkaProducer).uid("order_detail_sink");// 9.2.2 支付成功明细数据KafkaSink<String> paySucKafkaProducer = KafkaUtil.getKafkaSink(paySucDetailTopic, args);paySucDS.print("!!");paySucDS.sinkTo(paySucKafkaProducer).uid("pay_suc_detail_sink");// 9.2.3 取消运单明细数据KafkaSink<String> cancelKafkaProducer = KafkaUtil.getKafkaSink(cancelDetailTopic, args);cancelDetailDS.print("@@");cancelDetailDS.sinkTo(cancelKafkaProducer).uid("cancel_detail_sink");// 9.2.4 揽收明细数据KafkaSink<String> receiveKafkaProducer = KafkaUtil.getKafkaSink(receiveDetailTopic, args);receiveDetailDS.print("##");receiveDetailDS.sinkTo(receiveKafkaProducer).uid("reveive_detail_sink");// 9.2.5 发单明细数据KafkaSink<String> dispatchKafkaProducer = KafkaUtil.getKafkaSink(dispatchDetailTopic, args);dispatchDetailDS.print("$$");dispatchDetailDS.sinkTo(dispatchKafkaProducer).uid("dispatch_detail_sink");// 9.2.6 转运完成明细主题KafkaSink<String> boundFinishKafkaProducer = KafkaUtil.getKafkaSink(boundFinishDetailTopic, args);boundFinishDetailDS.print("%%");boundFinishDetailDS.sinkTo(boundFinishKafkaProducer).uid("bound_finish_detail_sink");// 9.2.7 派送成功明细数据KafkaSink<String> deliverSucKafkaProducer = KafkaUtil.getKafkaSink(deliverSucDetailTopic, args);deliverSucDetailDS.print("^^");deliverSucDetailDS.sinkTo(deliverSucKafkaProducer).uid("deliver_suc_detail_sink");// 9.2.8 签收明细数据KafkaSink<String> signKafkaProducer = KafkaUtil.getKafkaSink(signDetailTopic, args);signDetailDS.print("&&");signDetailDS.sinkTo(signKafkaProducer).uid("sign_detail_sink");env.execute();}
}

二、代码测试

1.环境启动

hadoop,zk,kf全部启动
根据流程图可以看到,流程中没有使用到dim层的内容,所以我们不需要启动hbase。

2.kafka消费者

kafka-console-consumer.sh --bootstrap-server hadoop102:9092 --topic tms_dwd_trade_order_detail
kafka-console-consumer.sh --bootstrap-server hadoop102:9092 --topic tms_dwd_trade_pay_suc_detail
kafka-console-consumer.sh --bootstrap-server hadoop102:9092 --topic tms_dwd_trade_cancel_detail
kafka-console-consumer.sh --bootstrap-server hadoop102:9092 --topic tms_dwd_trans_receive_detail
kafka-console-consumer.sh --bootstrap-server hadoop102:9092 --topic tms_dwd_trans_dispatch_detail
kafka-console-consumer.sh --bootstrap-server hadoop102:9092 --topic tms_dwd_trans_bound_finish_detail
kafka-console-consumer.sh --bootstrap-server hadoop102:9092 --topic tms_dwd_trans_deliver_detail
kafka-console-consumer.sh --bootstrap-server hadoop102:9092 --topic tms_dwd_trans_sign_detail

一共需要查看8个消费者主题,你可以开8个窗口,也可以一个一个看,kafka如果没有消费者,会先将数据保存,等待消费,所以不需要8个主题同时消费。

3.修改配置

在这里插入图片描述

4.测试结果

先启动OdsApp和DwdOrderRelevantApp,然后生成模拟数据,之后查看kakfa消费者,有些数据可能要多生成几次才行。

kafka-console-consumer.sh --bootstrap-server hadoop102:9092 --topic tms_dwd_trade_order_detail
在这里插入图片描述

kafka-console-consumer.sh --bootstrap-server hadoop102:9092 --topic tms_dwd_trade_pay_suc_detail
在这里插入图片描述
kafka-console-consumer.sh --bootstrap-server hadoop102:9092 --topic tms_dwd_trade_cancel_detail

kafka-console-consumer.sh --bootstrap-server hadoop102:9092 --topic tms_dwd_trans_receive_detail
在这里插入图片描述
kafka-console-consumer.sh --bootstrap-server hadoop102:9092 --topic tms_dwd_trans_dispatch_detail
这个主题是特殊情况,正常可能没有输出。

kafka-console-consumer.sh --bootstrap-server hadoop102:9092 --topic tms_dwd_trans_bound_finish_detail
在这里插入图片描述

kafka-console-consumer.sh --bootstrap-server hadoop102:9092 --topic tms_dwd_trans_deliver_detail
在这里插入图片描述
kafka-console-consumer.sh --bootstrap-server hadoop102:9092 --topic tms_dwd_trans_sign_detail
在这里插入图片描述


总结

至此这篇博客的内容结束。

本文来自互联网用户投稿,该文观点仅代表作者本人,不代表本站立场。本站仅提供信息存储空间服务,不拥有所有权,不承担相关法律责任。如若转载,请注明出处:http://www.rhkb.cn/news/219287.html

如若内容造成侵权/违法违规/事实不符,请联系长河编程网进行投诉反馈email:809451989@qq.com,一经查实,立即删除!

相关文章

分类预测 | Matlab实现DBO-SVM蜣螂算法优化支持向量机的数据分类预测【23年新算法】

分类预测 | Matlab实现DBO-SVM蜣螂算法优化支持向量机的数据分类预测【23年新算法】 目录 分类预测 | Matlab实现DBO-SVM蜣螂算法优化支持向量机的数据分类预测【23年新算法】分类效果基本描述程序设计参考资料 分类效果 基本描述 1.Matlab实现DBO-SVM蜣螂算法优化支持向量机的…

基于FFmpeg,实现播放器功能

一、客户端选择音视频文件 MainActivity package com.anniljing.ffmpegnative;import android.Manifest; import android.content.ContentResolver; import android.content.Context; import android.content.Intent; import android.database.Cursor; import android.net.Ur…

世微 锂电池保护IC DW01 充电器检测过充保护SOT23-6

一、 描述 DW01A 是一个锂电池保护电路&#xff0c;为避免锂电池因过充电、过放电、电流过大导致电池寿命缩短或电池被损坏而设计的。它具有高精确度的电压检测与时间延迟电路。 二、 主要特点 工作电流低 过充检测 4.3V&#xff0c;过充释放 4.05V&#xff1b; 过放检测 2.4…

使用Audition录制电脑内部声音

在电脑上播放的媒体文件&#xff0c;包括视频和声音&#xff0c;很多是可以播放却无法保存的。例如一些网页播放的视频&#xff0c;或者在线播放的音乐。 视频的话&#xff0c;可以使用工具来截图&#xff0c;抓取GIF或录屏。 声音的话&#xff0c;也可以使用工具进行录制。这里…

为养宠家庭量身打造,352 X63 Pet宠物专效空气净化器“养宠安馨,人宠共护”

当下,养宠人群日益增多,宠物在给家庭带来了欢乐的同时,也产生了一系列困扰,如何在健康环境中快乐养宠,成为很多家养宠家庭的新需求。成立于2014年的北京三五二环保科技有限公司是一家立足于家庭洁净空气和安全用水领域的科技创新型公司。以“安全、健康、舒适”等消费需求为核心…

uniapp - 简单版本自定义tab栏切换

tab切换是APP开发最常见的功能之一&#xff0c;uniapp中提供了多种形式的tab组件供我们使用。对于简单的页面而言&#xff0c;使用tabbar组件非常方便快捷&#xff0c;可以快速实现底部导航栏的效果。对于比较复杂的页面&#xff0c;我们可以使用tab组件自由定义样式和内容 目录…

Python框架批量数据抓取的高级教程

一、背景介绍 批量数据抓取是一种常见的数据获取方式&#xff0c;能够帮助我们快速、高效地获取网络上的大量信息。本文将介绍如何使用Python框架进行大规模抽象数据&#xff0c;以及如何处理这个过程中可能遇到的问题。 二、项目需求 我们将爬取大量知乎文章&#xff0c;讨…

vue使用el-tag完成添加标签操作

需求&#xff1a;做一个添加标签的功能&#xff0c;点击添加后输入内容后回车可以添加&#xff0c;并且标签可以删除 1.效果 2.主要代码讲解 鼠标按下后触发handleLabel函数&#xff0c;根据回车的keycode判断用户是不是按下的回车键&#xff0c;回车键键值为13&#xff0c;用…

HiveSql语法优化二 :join算法

Hive拥有多种join算法&#xff0c;包括Common Join&#xff0c;Map Join&#xff0c;Bucket Map Join&#xff0c;Sort Merge Buckt Map Join等&#xff0c;下面对每种join算法做简要说明&#xff1a; Common Join Common Join是Hive中最稳定的join算法&#xff0c;其通过一个M…

【MyBatis-Plus】常用的插件介绍(乐观锁、逻辑删除、分页)

&#x1f973;&#x1f973;Welcome Huihuis Code World ! !&#x1f973;&#x1f973; 接下来看看由辉辉所写的关于MyBatis-Plus的相关操作吧 目录 &#x1f973;&#x1f973;Welcome Huihuis Code World ! !&#x1f973;&#x1f973; 一.为什么要使用MyBatis-Plus中的插…

C语言学习----字符串数组和字符串指针

&#x1f308;在C Primer Plus 第六版中第11章的字符串和字符串函数讲的很好~ 对于字符串和字符串指针的用法又更深入了解了一些~ &#x1f42c; 本blog为 C Primer Plus 的记录~ ☘️对于字符串指针和数组更加深入了解~ &#x1f33a;省流&#xff1a;1️⃣字符串常量被储存在…

HNU-计算机网络-实验3-应用层和传输层协议分析(PacketTracer)

计算机网络 课程基础实验三应用层和传输层协议分析&#xff08;PacketTracer&#xff09; 计科210X 甘晴void 202108010XXX 【给助教的验收建议】 如果是助教&#xff0c;比起听同学读报告&#xff0c;更好的验收方式是随机抽取一个场景&#xff08;URL/HTTPS/FTP&#xff09…

Linux线程的设计

文章目录 一.理解Linux线程的本质进程地址空间是进程访问系统资源的窗口Linux系统中,线程是比进程更轻量级的执行流 二.Linux线程独立运行的原理三.基础线程控制 一.理解Linux线程的本质 进程地址空间是进程访问系统资源的窗口 Linux系统中,线程是比进程更轻量级的执行流 线程…

Mac 中文版 Navicat Premium 16 下载安装详细教程

哈喽朋友们大家好&#xff0c;今天做一期 Mac 数据库连接工具 Navicat Premium 16 的安装教程&#xff0c;很多朋友不知道怎么安装的&#xff0c;要不就是有试用期无法正常使用&#xff0c;要不就是英文的&#xff0c;改不了中文&#xff0c;大家可以跟着我的步骤安装&#xff…

RabbitMQ插件详解:rabbitmq_message_timestamp【Rabbitmq 五】

欢迎来到我的博客&#xff0c;代码的世界里&#xff0c;每一行都是一个故事 RabbitMQ时空之旅&#xff1a;rabbitmq_message_timestamp的奇妙世界 前言什么是rabbitmq_message_timestamprabbitmq_message_timestamp 的定义与作用&#xff1a;如何在 RabbitMQ 中启用消息时间戳&…

视觉检测系统在半导体行业的应用

一、半导体产业链概述 半导体产业链是现代电子工业的核心组成部分&#xff0c;涵盖了从原材料到最终产品的整个生产过程。这个产业链主要分为以下几个环节&#xff1a; 1.原材料供应&#xff1a;半导体行业的基石是半导体材料&#xff0c;如硅片、化合物半导体等。这些材料需要…

挑战52天学小猪佩奇笔记--day24

52天学完小猪佩奇--day24 ​【本文说明】 本文内容来源于对B站UP 脑洞部长 的系列视频 挑战52天背完小猪佩奇----day24 的视频内容总结&#xff0c;方便复习。强烈建议大家去关注一波UP&#xff0c;配合UP视频学习。 注&#xff1a;这集开始变成一段一段的猜台词&#xff0c;加…

python学习,1.变量和简单的数据类型

一、编写文章的目的 1.这是为了初学者而写的&#xff0c;学习python比较简单然后上手&#xff0c;也会过滤一些&#xff0c;如果没有提起到的&#xff0c;可以在学习的时候进行补充 2.相对来说&#xff0c;上手难度不会很难。 二、内容 1.让首字母大写&#xff1b;字母都大写…

STM32F103C8T6—烧录程序

STM32F103C8T6烧录程序方法 1. ST-Link烧录程序ST-Link软件下载ST-Link软件安装程序下载 2. 串口烧录程序CH340驱动下载安装连接程序下载 1. ST-Link烧录程序 该USB驱动程序(STSW-LINK009)适用于ST-LINK/V2, ST-LINK/V2-1和STLINK-V3板及其衍生物 首先下载ST-link驱动&#xf…

华为配置基本QinQ示例

组网需求 如图1所示&#xff0c;网络中有两个企业&#xff0c;企业1有两个分支&#xff0c;企业2有两个分支。这两个企业的各办公地的企业网都分别和运营商网络中的SwitchA和SwitchB相连&#xff0c;且公网中存在其它厂商设备&#xff0c;其外层VLAN Tag的TPID值为0x9100。 现…