1 服务器环境准备
我们这里以3台服务器为列
1.1 硬件配置
| Centos | 7.1及以上 |
| Ubuntu | 16.04及以上 |
| java | 1.8及以上 |
| GCC | 4.8.2及以上 |
每台服务器磁盘大小最小50G及时间相差不超或5秒
1.2 环境配置
1.2.1 修改limits.conf文件
vim /etc/security/limit.conf
#在文件最后添加,*号也要添加
* soft nofile 65535
* hard nofile 65535
* soft nproc 65535
* hard nproc 655351.2.2 修改sysctl.conf文件
vim /etc/sysctl.conf
#在文件最后添加
vm.max_map_count = 2000000
vm.swappiness = 0
让该文件永久生效：sysctl -p
1.2.3 关闭swap(交互分区)
# 临时关闭：
swapoff -a
# 或永久关闭, 修改fstab文件, 注册带swap的行
vim /etc/fstab
1.2.4 关闭THP
echo "never" >> /sys/kernel/mm/transparent_hugepage/enabled
echo "never" >> /sys/kernel/mm/transparent_hugepage/defrag将上面两句命令加入到开机启动中, 防止重启失效
vi /etc/rc.d/rc.local # 给文件授权 chmod +x /etc/rc.d/rc.local
1.2.5 开发防火墙
如果你是直接关闭防火墙可以跳过该步骤
# 使用root用户, 开放Doris数据库的端口
firewall-cmd --zone=public --add-port=9001/tcp --permanent
firewall-cmd --zone=public --add-port=9020/tcp --permanent
firewall-cmd --zone=public --add-port=9030/tcp --permanent
firewall-cmd --zone=public --add-port=9010/tcp --permanent
firewall-cmd --zone=public --add-port=9060/tcp --permanent
firewall-cmd --zone=public --add-port=8040/tcp --permanent
firewall-cmd --zone=public --add-port=9050/tcp --permanent
firewall-cmd --zone=public --add-port=8060/tcp --permanent
# 参考命令 ,重启防火墙
关闭防火墙服务：systemctl stop firewalld
开启防火墙服务：systemctl start firewalld
查看防火墙服务状态：systemctl status firewalld
自启动防火墙服务：systemctl enable firewalld
查看防火墙端口：firewall-cmd --list-ports1.3 安装JAVA JDK
将JDK(版本1.8)压缩包拷贝到 /usr/local目录下(存放位置不强制要求)
# cd 到存放目录 cd /usr/local #解压 tar -xzvf jdk-8u391-linux-x64.tar.gz配置java环境
#修改profile文件 vim /etc/profile #在文件最后添加如下内容 export JAVA_HOME=/usr/local/jdk1.8.0_391 export PATH=$JAVA_HOME/bin:$PATH export CLASSPATH=.:$JAVA_HOME/lib/dt.jar:$JAVA_HOME/lib/tools.jar#刷新配置文件 source /etc/profile #验证是否安装成功 java -version
出现上面图片则表示成功
2 安装Doris数据库
2.1安装数据包
#判断PCU是否支持avx2
lscpu | grep -o 'avx2'
#如果输出为avx2则表示支持，否则表示不支持
#支持avx2下载不带noavx2的如：apache-doris-2.1.8-bin-x64.tar.gz
#不支持avx2下载带noavx2的如：apache-doris-2.1.4-bin-x64-noavx2.tar.gzdoris下载地址：Apache Doris - Download | Easily deploy Doris anywhere - Apache Doris
将安装包上传到对应的目录下,我这里以 /usr/local目录下
# cd到对应的目录
cd /usr/local
# 解压安装包
tar -xzvf apache-doris-2.1.8-bin-x64.tar.gz
#可以修改一个名称(可选)
mv apache-doris-2.1.8-bin-x64 ./doris-2.1.8
#可以配置环境变量
vim /etc/profile
#将下面内容添加到profile文件的最后
export DORIS_HOME=/usr/local/doris-2.1.8
export PATH=$DORIS_HOME/fe/bin:$DORIS_HOME/be/bin:$PATH2.2 修改FE与BE的配置参数
2.2.1 修改FE配置参数
#进入doris的根目录 cd /usr/local/doris-2.1.8 #修改fe.conf配置文件 vim fe/conf/fe.confCUR_DATE=`date +%Y%m%d-%H%M%S`
# Log dir
LOG_DIR = ${DORIS_HOME}/log #可修改日志文件存放地址# CMS JAVA OPTS
# JAVA_OPTS="-Dsun.security.krb5.debug=true -Djavax.security.auth.useSubjectCredsOnly=false -Xss4m -Xmx8192m -XX:+UseMembar -XX:SurvivorRatio=8 -XX:MaxTenuringThreshold=7 -XX:+PrintGCDateStamps -XX:+PrintGCDetails -XX:+UseConcMarkSweepGC -XX:+UseParNewGC -XX:+CMSClassUnloadingEnabled -XX:-CMSParallelRemarkEnabled -XX:CMSInitiatingOccupancyFraction=80 -XX:SoftRefLRUPolicyMSPerMB=0 -Xloggc:$DORIS_HOME/log/fe.gc.log.$CUR_DATE"# G1 JAVA OPTS
JAVA_OPTS="-Dfile.encoding=UTF-8 -Djavax.security.auth.useSubjectCredsOnly=false -Xss4m -Xmx8192m -XX:+UnlockExperimentalVMOptions -XX:+UseG1GC -XX:MaxGCPauseMillis=200 -XX:+PrintGCDateStamps -XX:+PrintGCDetails -Xloggc:$LOG_DIR/fe.gc.log.$CUR_DATE -Dlog4j2.formatMsgNoLookups=true"# For jdk 9+, this JAVA_OPTS_FOR_JDK_9 will be used as default CMS JVM options
# JAVA_OPTS_FOR_JDK_9="-Dsun.security.krb5.debug=true -Djavax.security.auth.useSubjectCredsOnly=false -Xss4m -Xmx8192m -XX:SurvivorRatio=8 -XX:MaxTenuringThreshold=7 -XX:+CMSClassUnloadingEnabled -XX:-CMSParallelRemarkEnabled -XX:CMSInitiatingOccupancyFraction=80 -XX:SoftRefLRUPolicyMSPerMB=0 -Xlog:gc*:$DORIS_HOME/log/fe.gc.log.$CUR_DATE:time"# For jdk 9+, this JAVA_OPTS_FOR_JDK_9 will be used as default G1 JVM options
JAVA_OPTS_FOR_JDK_9="-Dfile.encoding=UTF-8 -Djavax.security.auth.useSubjectCredsOnly=false -Xss4m -Xmx8192m -XX:+UseG1GC -XX:MaxGCPauseMillis=200 -Xlog:gc*:$LOG_DIR/fe.gc.log.$CUR_DATE:time -Dlog4j2.formatMsgNoLookups=true"# For jdk 17+, this JAVA_OPTS will be used as default JVM options
JAVA_OPTS_FOR_JDK_17="-Dfile.encoding=UTF-8 -Djavax.security.auth.useSubjectCredsOnly=false -XX:+UseG1GC -Xmx8192m -Xms8192m -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=$LOG_DIR/ -Xlog:gc*:$LOG_DIR/fe.gc.log.$CUR_DATE:time"##
## the lowercase properties are read by main program.
### store metadata, must be created before start FE.
# Default value is ${DORIS_HOME}/doris-meta
# meta_dir = ${DORIS_HOME}/doris-meta# Default dirs to put jdbc drivers,default value is ${DORIS_HOME}/jdbc_drivers
# jdbc_drivers_dir = ${DORIS_HOME}/jdbc_drivershttp_port = 8030
rpc_port = 9020
query_port = 9030
edit_log_port = 9010
arrow_flight_sql_port = -1# Choose one if there are more than one ip except loopback address.
# Note that there should at most one ip match this list.
# If no ip match this rule, will choose one randomly.
# use CIDR format, e.g. 10.10.10.0/24 or IP format, e.g. 10.10.10.1
# Default value is empty.
# priority_networks = 10.10.10.0/24;192.168.0.0/16
priority_networks = 192.168.88.101 #本机ip地址
# Advanced configurations
# log_roll_size_mb = 1024
# INFO, WARN, ERROR, FATAL
sys_log_level = WARN #可选 ,日志文件级别
# NORMAL, BRIEF, ASYNC
sys_log_mode = BRIEF #可选 ,日志文件级别
# sys_log_roll_num = 10
# sys_log_verbose_modules = org.apache.doris
# audit_log_dir = $LOG_DIR
# audit_log_modules = slow_query, query
# audit_log_roll_num = 10
# meta_delay_toleration_second = 10
# qe_max_connection = 1024
# qe_query_timeout_second = 300
# qe_slow_log_ms = 5000#保存退出ESC :wq
2.2.2 修改be参数
#修改be.conf配置文件 vim be/conf/be.confCUR_DATE=`date +%Y%m%d-%H%M%S`
# Log dir
LOG_DIR="${DORIS_HOME}/log/" #日志存放目录,可修改# For jdk 8 根据实际可以是不是需要添加内存大小
JAVA_OPTS="-Dfile.encoding=UTF-8 -Xmx8192m -DlogPath=$LOG_DIR/jni.log -Xloggc:$DORIS_HOME/log/be.gc.log.$CUR_DATE -Djavax.security.auth.useSubjectCredsOnly=false -Dsun.security.krb5.debug=true -Dsun.java.command=DorisBE -XX:-CriticalJNINatives -Darrow.enable_null_check_for_get=false"# For jdk 9+, this JAVA_OPTS will be used as default JVM options
JAVA_OPTS_FOR_JDK_9="-Dfile.encoding=UTF-8 -Xmx2048m -DlogPath=$DORIS_HOME/log/jni.log -Xlog:gc:$LOG_DIR/be.gc.log.$CUR_DATE -Djavax.security.auth.useSubjectCredsOnly=false -Dsun.security.krb5.debug=true -Dsun.java.command=DorisBE -XX:-CriticalJNINatives --add-opens=java.base/java.nio=ALL-UNNAMED -Darrow.enable_null_check_for_get=false"# For jdk 17+, this JAVA_OPTS will be used as default JVM options
JAVA_OPTS_FOR_JDK_17="-Dfile.encoding=UTF-8 -Xmx2048m -DlogPath=$LOG_DIR/jni.log -Xlog:gc:$LOG_DIR/be.gc.log.$CUR_DATE -Djavax.security.auth.useSubjectCredsOnly=false -Dsun.security.krb5.debug=true -Dsun.java.command=DorisBE -XX:-CriticalJNINatives --add-opens=java.base/java.net=ALL-UNNAMED --add-opens=java.base/java.nio=ALL-UNNAMED -Darrow.enable_null_check_for_get=false"# since 1.2, the JAVA_HOME need to be set to run BE process.
# JAVA_HOME=/path/to/jdk/# https://github.com/apache/doris/blob/master/docs/zh-CN/community/developer-guide/debug-tool.md#jemalloc-heap-profile
# https://jemalloc.net/jemalloc.3.html
JEMALLOC_CONF="percpu_arena:percpu,background_thread:true,metadata_thp:auto,muzzy_decay_ms:5000,dirty_decay_ms:5000,oversize_threshold:0,prof:true,prof_active:false,lg_prof_interval:-1"
JEMALLOC_PROF_PRFIX="jemalloc_heap_profile_"# ports for admin, web, heartbeat service
be_port = 9060
webserver_port = 8040
heartbeat_service_port = 9050
brpc_port = 8060
arrow_flight_sql_port = -1# HTTPS configures
enable_https = false
# path of certificate in PEM format.
ssl_certificate_path = "$DORIS_HOME/conf/cert.pem"
# path of private key in PEM format.
ssl_private_key_path = "$DORIS_HOME/conf/key.pem"
# Choose one if there are more than one ip except loopback address.
# Note that there should at most one ip match this list.
# If no ip match this rule, will choose one randomly.
# use CIDR format, e.g. 10.10.10.0/24 or IP format, e.g. 10.10.10.1
# Default value is empty.
# priority_networks = 10.10.10.0/24;192.168.0.0/16
priority_networks = 192.168.88.101 #ip地址
# data root path, separate by ';'
# You can specify the storage type for each root path, HDD (cold data) or SSD (hot data)
# eg:
# storage_root_path = /home/disk1/doris;/home/disk2/doris;/home/disk2/doris
# storage_root_path = /home/disk1/doris,medium:SSD;/home/disk2/doris,medium:SSD;/home/disk2/doris,medium:HDD
# /home/disk2/doris,medium:HDD(default)
#
# you also can specify the properties by setting '<property>:<value>', separate by ','
# property 'medium' has a higher priority than the extension of path
#
# Default value is ${DORIS_HOME}/storage, you should create it by hand.
# storage_root_path = ${DORIS_HOME}/storage# Default dirs to put jdbc drivers,default value is ${DORIS_HOME}/jdbc_drivers
# jdbc_drivers_dir = ${DORIS_HOME}/jdbc_drivers# Advanced configurations
# INFO, WARNING, ERROR, FATAL
sys_log_level = WARNING #日志级别可选择
# sys_log_roll_mode = SIZE-MB-1024
# sys_log_roll_num = 10
# sys_log_verbose_modules = *
# log_buffer_level = -1
# palo_cgroups# aws sdk log level
# Off = 0,
# Fatal = 1,
# Error = 2,
# Warn = 3,
# Info = 4,
# Debug = 5,
# Trace = 6
# Default to turn off aws sdk log, because aws sdk errors that need to be cared will be output through Doris logs
aws_log_level=0
## If you are not running in aws cloud, you can disable EC2 metadata
AWS_EC2_METADATA_DISABLED=true
string_type_length_soft_limit_bytes = 10485760 #修改单字段数据大小为10M,可选
2.3 主节点启动
2.3.1 启动fe
#没有配doris环境, 在doris根目录下 fe/bin/start_fe.sh --daemon be/bin/start_be.sh --daemon #如果配置了doris环境 ,二选一执行即可 start_fe.sh --daemon start_be.sh --daemon#查看是否启动成功 jps
如果出现这个图片的内容,则代表成功。
可以前往游览器查看：
#FE的web端,默认用户root,密码空
http://IP:8030
#BE的web端
http://IP:8040/
通过FE、BE的网址查看，如果正常，则安装成功。
2.4 数据库初始化
#如果服务器安装有mysql,可以mysql客服端连接FE mysql -hhadoop102 -P9030 -uroot #设置密码 SET PASSWORD FOR 'root' = PASSWORD('密码');#如果没有客户端,可以通过DBeaver连接,端口为9030#配置高可用 #扩容FE节点，新节点添加为follower ALTER SYSTEM ADD FOLLOWER "192.168.88.102:9010"; #或新节点添加为observer ALTER SYSTEM ADD OBSERVER "192.168.88.103:9010";#将BE关联到FE,如果只添加一个本机的BE,那么只就是个单价 ALTER SYSTEM ADD BACKEND "192.168.88.101:9050"; ALTER SYSTEM ADD BACKEND "192.168.88.102:9050"; ALTER SYSTEM ADD BACKEND "192.168.88.103:9050";#修改密码 alter user 'root' identified by '密码';下面配置可选
开启 SQL 跟踪日志set global enable_profile=true;
#查看跟踪是否开启
show variables like '%enable_profile%';
#关闭跟踪日志
set global enable_profile = false;
注：为避免影响性能日常运行中可以关闭跟踪日志，查问题时再打开。
开启审计日志
set global enable_audit_plugin = true;
其它命令
#查看审计日志是否开启
show variables like '%enable_audit_plugin%';
#关闭审计日志
set global enable_audit_plugin = false;
注：为避免影响性能日常运行中可以关闭审计日志，查问题时再打开。
3 配置高可用
#将主节点的be和fe分发其他两个节点 # 1.先到另外两个节点创建对应目录 mkdir -r /usr/local/doris-2.1.8.1 # 2.把主节点配置分发给另外两个节点 rsync -avz /usr/local/doris-2.1.8.1/* root@192.168.88.102:/usr/local/doris-2.1.8 rsync -avz /usr/local/doris-2.1.8.1/* root@192.168.88.103:/usr/local/doris-2.1.8分发完成后到另外两个节点的doris根目录下修改各自的ip地址
vim fe/conf/fe.conf vim be/conf/be.conf #修改如下内容, 修改为本机对应的ip priority_networks = 192.168.88.102#删除doris的fe目录下的元数据 rm -rf doris-meta另一个节点与上面步骤一致。
第一次启动时候需要在两个从节点执行该命令
#进入Doris的根目录, 第一次启动FE执行下面命令 主节点ip地址 fe/bin/start_fe.sh --helper 192.168.88.101:9010 --daemon#进入Doris的根目录,启动BE be/bin/start_be.sh --daemon
如果状态都为true则代表成功。
