开发doris的团队厉害,这个百度工程师确实也干了一些实事,不像领导层只会跑火车。
1 参数配置
1.1 文件句柄数
vim /etc/security/limits.conf
* soft nofile 655350
* hard nofile 655350
1.2 关闭透明大页
echo never > /sys/kernel/mm/transparent_hugepage/enabled
echo never > /sys/kernel/mm/transparent_hugepage/defrag
1.3 配置max_map_count
vi /etc/sysctl.conf
vm.max_map_count=2000000
# 生效
sysctl -p
1.4 关闭swap
临时关闭:swapoff -a
永久生效:vim /etc/fstab
注释掉带swap分区
1.5 下载安装包
doris下载,输出中有包含avx2
字样的行,则说明处理器支持AVX2。
# 下载
wget https://apache-doris-releases.oss-accelerate.aliyuncs.com/apache-doris-2.1.8-bin-x64.tar.gz
# 解压,有点慢,这个压缩文件就有2.7G
mkdir -p /home/doris
tar -zxf apache-doris-2.1.8-bin-x64.tar.gz -C /home/doris
1.6 jdk
# doris2还是使用jdk1.8
tar -zxf /home/download/jdk1.8.0_181.tar.gz -C /home/java
vi /etc/profile
export JAVA_HOME=/home/java/jdk1.8.0_181
export CLASSPATH=$JAVA_HOME/lib
export PATH=$JAVA_HOME/bin:$PATH# 生效
source /etc/profile
2 FE
2.1 安装mysql客户端
直接安装mysql5.7已经不维护了,直接装个mariadb得了。
sudo yum install mariadb
2.2 配置
2.2.1 fe.conf
cd /home/doris/apache-doris-2.1.8-bin-x64/fe
vi fe.conf
priority_networks = 10.101.12.0/24;
2.2.2 root密码
默认 root 无密码,通过以下命令修改 root 密码,
mysql -uroot -P9030 -hmaster1
SET PASSWORD FOR 'root' = PASSWORD('123456');
这个密码也是http://10.101.12.82:8030/home的密码
2.3 集群
参考【大数据系列】一、Apache Doris集群部署,当集群可用性要求很高时(比如提供在线业务),可以部署 3 个Follower 和 1-3 个 Observer。如果是离线业务,议部署 1 个 Follower 和 1-3 个 Observer
host | 角色 | 说明 |
master1 | FE | q |
master2 | FE((FOLLOWER) | |
master3 | FE(LEADER) | |
worker1 | BE | |
worker2 | BE | |
worker3 | BE |
2.3.1 添加BE
ALTER SYSTEM ADD BACKEND "worker1:9050";
ALTER SYSTEM ADD BACKEND "worker2:9050";
ALTER SYSTEM ADD BACKEND "worker3:9050";
# 查看BE是否添加成功
show proc '/backends'\G
2.3.2 添加FE
在各自的FE节点启动,然后再主节点mysql客户端执行下面的
ALTER SYSTEM ADD FOLLOWER "master2:9010";
ALTER SYSTEM ADD OBSERVER "master3:9010";
2.4 启动
cat > /etc/systemd/system/doris-fe.service <<EOF
[Unit]
Description=Apache Doris Frontend Service
After=network.target[Service]
Type=forking
ExecStart=/home/doris/apache-doris-2.1.8-bin-x64/fe/bin/start_fe.sh --daemon
ExecStop=/home/doris/apache-doris-2.1.8-bin-x64/fe/bin/stop_fe.sh
Restart=on-failure
Environment=JAVA_HOME=/home/java/jdk1.8.0_181[Install]
WantedBy=multi-user.target
EOFsudo systemctl daemon-reload
systemctl start doris-fe
sudo systemctl enable doris-fe
3 BE
3.1 配置
priority_networks = 10.101.12.0/24
# 20表示/home/doris/data1限制20G
storage_root_path = /home/doris/data1,20;
3.2 启动
cd /home/doris/apache-doris-2.1.8-bin-x64/be/bin
./start_be.sh --daemon
cat > /etc/systemd/system/doris-be.service <<EOF
[Unit]
Description=Apache Doris Backend Service
After=network.target[Service]
Type=forking
ExecStart=/home/doris/apache-doris-2.1.8-bin-x64/be/bin/start_be.sh --daemon
ExecStop=/home/doris/apache-doris-2.1.8-bin-x64/be/bin/stop_be.sh
Restart=on-failure
Environment=JAVA_HOME=/home/java/jdk1.8.0_181[Install]
WantedBy=multi-user.target
EOFsudo systemctl daemon-reload
systemctl start doris-be
sudo systemctl enable doris-be