Gerrit | 重磅! 2.x 版本升级到 3.x 版本
为什么要做版本升级？
2.x known bugs
重大问题不一一列举，这里仅仅是举几个例子：
- 安全或权限问题：普通用户能看到敏感数据，例如看到其他用户的 hashed api 密码，
问题传送门：https://bugs.chromium.org/p/gerrit/issues/detail?id=13514；
- 再例如 已经设置为失效的账号，还可以登录 gerrit ；
- 升级时索引建立失败：重建索引未完成，插件就 load 了，索引建立过程 invoked，https://bugs.chromium.org/p/gerrit/issues/detail?id=6472；
3.0 的重大更新
- ReviewDB 寿终正寝，NoteDB 上台
- UI 界面全面更新，GWT 被 PolyGerrit 取代
- 支持 REST-API
- 其他重大更新，等等
新版本大概长这样
升级 Migration 实操记录
目前待升级的 gerrit 版本为 2.13.9，根据官方文档，必须确保每个中间版本都被升级，所以升级路线暂定为：
- 2.13 -- 2.14 -- 2.15 -- 2.16
- 到了 2.16 后将review DB 迁移到NoteDB
- 迁移 2.16 到 3.0
注意：v2.16是唯一包含和支持 reviewDB 到 NoteDB 的关键版本
Reference Doc：https://gitenterprise.me/2019/05/20/gerrit-v3-0-is-here/
提前测试
测试原版 gerrit 升级路径，不携带任何数据，为的是快速探索成功路径
升级路径：2.13.9 -- 2.15.21 -- 2.16.25 -- 3.2.3
s0：升级前，确认 gerrit 的 replication 和 HA 的插件和配置文件，被禁用(可移走或重命名)
s1:起点版本 2.13.9
$ java -jar gerrit-2.13.9.war init --batch --dev -d gerrit_test
s2:升级到 2.15.21
$ java -jar gerrit-2.15.21.war init --batch --dev -d gerrit_test
$ java -jar gerrit-2.15.21.war reindex --index groups -d gerrit_test
$ java -jar gerrit-2.15.21.war reindex --index accounts -d gerrit_test
$ java -jar gerrit-2.15.21.war reindex --index changes -d gerrit_test
执行以上三步，gerrit 服务才能启动成功
其中第三步 reindex，耗时较长，可以先 skip ，升级到 3.x 再做
s3:升级到 2.16.25
$ java -jar gerrit-2.16.25.war init --batch --dev -d gerrit_test
直接启动 gerrit 可以成功
s4:升级 3.x 之前，将 jdk 11 下载解压，并加入到 环境变量中，source
$ java -version
openjdk version "11.0.9.1" 2020-11-04
OpenJDK Runtime Environment AdoptOpenJDK (build 11.0.9.1+1)
OpenJDK 64-Bit Server VM AdoptOpenJDK (build 11.0.9.1+1, mixed mode)
s5:在2.16.25 上进行数据库 migrate
$ java -jar gerrit-2.16.25.war migrate-to-note-db -d gerrit_test
可以加上 --reindex false 选择跳过 reindex
s6:升级到 3.2.3
$ java -jar gerrit-3.2.3.war init --batch --dev -d gerrit_test
s7:进行 reindex
$ java -jar gerrit-3.2.3.war reindex --index accounts -d gerrit_test
$ java -jar gerrit-3.2.3.war reindex --index changes -d gerrit_test
启动 gerrit 服务，启动 OK, awsome！！
验证一下：
$ curl localhost:29418
SSH-2.0-GerritCodeReview_3.2.3 (APACHE-SSHD-2.4.0)
Unsupported protocol version: GET / HTTP/1.1
到此基本的操作步骤都涵盖在上面了，分割线以下，是我的实际操作日志，仅供参考。
如果有任何问题，可私信咨询。
实际 Tips
- 升级前的准备
检查 gerrit.config
[cache "diff"]
timeout = 200ms # 减少timeout 提升整体 reindex 速度
- 导入数据
进入 MySQL ： set GLOBAL max_connections=1000;
本地搭建 mysql， 并导入数据
mysql -u username -p < ...sql脚本.sql ，
例如
$ mysql -u gerrit_rw -p --verbose < c3-inf-gerrit-db01.bj.sql
- 拷贝 线上的 gerrit 目录并停掉 gerrit 的 HA 和 replication，然后修改 gerrit.config 的 database 为本地数据库配置
[database]
type = mysql
hostname = localhost
database = xxx
username = x
password = x
port = 3306
useSSL = false
[auth] type = DEVELOPMENT_BECOME_ANY_ACCOUNT
- 同步线上代码
$ ssh -p 29418 gerrit@localhost replication start --url 目标地址 --all
- 2.13 --> 2.14
#停止 gerrit 服务
java -jar gerrit-2.14.22.war init --batch --dev -d ${gerrit-site}
Get an error：
Exception in thread "main" java.nio.file.NoSuchFileException: gerrit/ha/1/gerrit/shared
解决方式：去掉或重命名etc目录下的 ha config 和 replication config ，去掉 plugins 文件下的 ha 和 replication 的 jar 包
这一步也可以在执行 升级命令 之前完成继续上面的 java 命令
$ java -jar gerrit-2.14.22.war init --batch --dev -d ${gerrit-site}
Upgrading schema to 130 ...
Upgrading schema to 131 ...
Upgrading schema to 132 ...
迁移正常的打印信息为：
Migrating 5263 repositories ...
Migration completed: 3 repositories updated:
All-Projects platform/packages/apps/ContactsCommon platform/packages/apps/OneAccountSdk
> Done (44.769 s)
Migrating data to schema 131 ...
Migrating 5263 repositories ...
Migration completed: 16 repositories updated:
... ... ...
Migrating data to schema 141 ...
> Done (0.022 s)
Migrating data to schema 142 ...
> Done (137.173 s)
Execute the following SQL to drop unused objects:
... ... ...
Initialized /home/work/gerrit
报错：直接启动 gerrit 服务会失败：No index versions ready;
解决方式：进行 reindex 操作
$ java -jar gerrit-2.14.22.war reindex --index groups -d ${gerrit-site}
... ... ...
Finishing 5 disk cache updates
启动 gerrit 服务， 查看升级后的版本
$ curl localhost:29418
SSH-2.0-GerritCodeReview_2.14.22 (SSHD-CORE-1.4.0)
Unsupported protocol version: GET / HTTP/1.1
或者查看一下新UI 左下角的版本信息（2.14.22）
- 继续下一步：2.14 --> 2.15
migration to 2.15 与上述内容基本一致
$ java -jar gerrit-2.15.21.war init --batch --dev -d ${gerrit-site}
Upgrading schema to 143 ...
Upgrading schema to 144 ...
... ... ...
Initialized /home/work/gerrit
这一步想偷懒不做 reindex，但是失败了
No index versions for index 'changes' ready;
run java -jar /home/work/gerrit/bin/gerrit.war reindex --index changes
先升级到 3.2.3 再做 reindex
- 下一步：2.15 --> 2.16
$ java -jar gerrit-2.16.25.war init --batch --dev -d ${gerrit-site}
$ java -jar gerrit-2.16.25.war reindex --index projects -d ${gerrit-site}
$ java -jar gerrit-2.16.25.war reindex --index groups -d ${gerrit-site}
$ java -jar gerrit-2.16.25.war reindex --index changes -d ${gerrit-site}
注意这一步可能会耗时很久， 取决于Gerrit上change的数量大小
- 启动 gerrit 服务
$ curl localhost:29418
SSH-2.0-GerritCodeReview_2.16.25 (SSHD-CORE-2.0.0)
Unsupported protocol version: GET / HTTP/1.1
- 继续下一步：2.16 --> 3.0
出现错误ERROR com.google.gerrit.pgm.init.BaseInit : Couldn't upgrade schema. Expected if slave and read-only database
com.google.gerrit.exceptions.StorageException: You appear to be upgrading from a 2.x site, but the NoteDb change migration was not completed. See documentation:
https://gerrit-review.googlesource.com/Documentation/note-db.html#migration
根据文档，必须先进行 NoteDB 迁移
$ java -jar gerrit-2.16.25.war migrate-to-note-db -d ${gerrit-site}
这一步报错了： failed on some changes ,see log
于是需要把 失败的 changid 的相关信息从数据库里删除掉；
# 举一个changeid 的例子
use database_name;
delete from patch_sets where change_id=183663;
delete from patch_set_approvals where change_id=183663;
delete from patch_comments where change_id=183663;
delete from changes where change_id=183663;
delete from change_messages where change_id=183663;
delete from change_id where s=183663;
... ... ...
执行完后，继续进行 migrate
- 为了提升效（偷）率（懒），写了个自动化脚本
接受命令行输入参数的脚本
#!/bin/bash
#
#********************************************************************
#Author: 奈文摩尔
#ID: xxxxx
#Created: 2021-02-02
#Modified: 2021-02-04 10:27:27
#Email:
#Description: Upgrade gerrit from 2.x to 3.x
#Copyright (C): 2021 All rights reserved
#********************************************************************
# run as work
# Input Paras :
# 1:gerrit folder (local path)
# 2:dumped sql script (local path)
# 3:remote mysql host/port/user/password
``` Example:
bash gerrit_upgrate.sh -g /home/work/gerrit -d /home/work/gerrit/c3-inf-gerrit-db01.bj.sql -h 10.172.28.40 -p 3315 -u root -w 123456
```
set -e
if [ ! -n "$1" ] ;thenecho "invalid input , bash xxx -help to get demo"exit -1
fi
if [[ $1 = "--help" ]] || [[ $1 = "-h" ]]
thenecho "-----------------------------------------------------g gerrit folder path-d dump sql script path-h mysql host name-p mysql host port-u mysql login username-w mysql username passwordExample: bash gerrit_upgrate.sh -g /home/work/gerrit -d /home/work/gerrit/c3-inf-gerrit-db01.bj.sql -h 10.172.28.40 -p 3315 -u root -w 123456-----------------------------------------------------"exit 0
fi
while getopts "g:d:h:p:u:w:" opt; docase $opt ing)GERRIT_SITE=$OPTARGecho "GERRIT_SITE is $GERRIT_SITE" ;;d)DUMP_SQL=$OPTARGecho "DUMP_SQL is $DUMP_SQL" ;;h)DB_HOST=$OPTARGecho "DB_HOST is $DB_HOST" ;;p)DB_PORT=$OPTARGecho "DB_PORT is $DB_PORT" ;;u)DB_USER=$OPTARGecho "DB_USER is $DB_USER" ;;w)DB_PWD=$OPTARGecho "DB_PWD is $DB_PWD" ;;\?)echo "invalid arg" ;;esac
done
# step 0 : modify gerrit.config
cd ${GERRIT_SITE}/etc/
if [ -f gerrit.config ]; thenmv gerrit.config gerrit-bak.config
fi
if [ -f secure.config ]; thenmv secure.config secure-bak.config
fi
wget https://staging-cnbj2-fds.api.zzz.net/files/gerrit.config
wget https://staging-cnbj2-fds.api.zzz.net/files/secure.config
JDBC_STR="jdbc:mysql://${DB_HOST}:${DB_PORT}/reviewmiui?autoReconnect=true&characterEncoding=UTF-8&useSSL=false"
sed -i "0,/hostname =/{s/hostname =.*/hostname = ${DB_HOST}/}" gerrit.config
sed -i "0,/username =/{s/username =.*/username = ${DB_USER}/}" gerrit.config
sed -i "0,/password =/{s/password =.*/password = ${DB_PWD}/}" gerrit.config
sed -i "0,/port =/{s/port =.*/port = ${DB_PORT}/}" gerrit.config
sed -i "/url = /c\ url = jdbc:mysql://${DB_HOST}:${DB_PORT}/reviewmiui?autoReconnect=true&characterEncoding=UTF-8&useSSL=false" gerrit.config
sed -i "0,/password =/{s/password =.*/password = ${DB_PWD}/}" secure.config
# sed -i "s/hostname =/ hostname = ${DB_HOST}/g" gerrit.config
# sed -i "s/username =/ username = ${DB_USER}/g" gerrit.config
# sed -i "s/password =/ password = ${DB_PWD}/g" gerrit.config
# sed -i "s/port =/ port = ${DB_PORT}/g" gerrit.config
# sed -i "s/url =/ url = ${JDBC_STR}/g" gerrit.config
# sed -i "s/password =*/ password = ${DB_PWD}/g" secure.config
# step 1 : delete replication / hA config
RPC_CONFIG=${GERRIT_SITE}/etc/replication.config
HA_CONFIG=${GERRIT_SITE}/etc/high-availability.config
rm -rf ${RPC_CONFIG}
rm -rf ${HA_CONFIG}
# delete confirm
if [ -f "${RPC_CONFIG}"] || [ -f "${HA_CONFIG}" ];
thenecho "config file delete failed: ${RPC_CONFIG} or ${HA_CONFIG}"exit -1
fi
# step 2 : download pkg
cd ${GERRIT_SITE}
cd ..
rm -rf gerrit-*.war
wget https://staging-cnbj2-fds.api.zzzzz.net/files/gerrit-2.15.21.war
wget https://staging-cnbj2-fds.api.zzzzz.net/files/gerrit-2.16.25.war
wget https://staging-cnbj2-fds.api.zzzzz.net/files/gerrit-3.2.3.war
if [ ! -f "delete_dirty_changeids.sql" ]; thenwget https://staging-cnbj2-fds.api.zzzzz.net/files/delete_dirty_changeids.sql
fi
# step 3 : Loading dump data into DB
echo "-------------------- split line ---------------------------"
echo "start loading data :$(date "+%Y-%m-%d %H:%M:%S")" >> upgrade_time.log
mysql -h${DB_HOST} -P${DB_PORT} -u${DB_USER} -p${DB_PWD} --verbose < ${DUMP_SQL}
mysql -h${DB_HOST} -P${DB_PORT} -u${DB_USER} -p${DB_PWD} --verbose < delete_dirty_changeids.sql
# step 4 : upgrate from 2.13 --> 2.15
echo "start 2.15 :$(date "+%Y-%m-%d %H:%M:%S")" >> upgrade_time.log
cd ${GERRIT_SITE}/lib
wget https:///mysql-connector-java-5.1.43.jar
mkdir -p ${GERRIT_SITE}/ha/1/gerrit/shared
cd -
java -jar gerrit-2.15.21.war init --batch --dev -d ${GERRIT_SITE}
# step 5 : upgrate from 2.15 --> 2.16
echo "start 2.16 :$(date "+%Y-%m-%d %H:%M:%S")" >> upgrade_time.log
cd ${GERRIT_SITE}/lib
wget https://staging-cnbj2-fds.api.zzzzz.net/files/mysql-connector-java-5.1.48.jar
cd -
java -jar gerrit-2.16.25.war init --batch --dev -d ${GERRIT_SITE}
# step 6 : migrate on 2.16
echo "start migrate :$(date "+%Y-%m-%d %H:%M:%S")" >> upgrade_time.log
rm -rf ${GERRIT_SITE}/etc/notedb.config
nohup java -jar gerrit-2.16.25.war migrate-to-note-db --reindex false -d ${GERRIT_SITE} > mi.log 2>&1 &
while true
doecho "Waiting for NoteDB Migration complete: $(date "+%Y-%m-%d %H:%M:%S")"if test ! -z "$(tail -n 20 mi.log | grep 'NoteDB Migration complete.')"; thenecho "NoteDB Migration complete: $(date "+%Y-%m-%d %H:%M:%S")" >> upgrade_time.logbreakfiif test ! -z "$(tail -n 20 mi.log | grep 'fatal: Migrating primary storage for some changes failed, see log')"; thenecho "fatal: Migrating primary storage for some changes failed, see log: $(date "+%Y-%m-%d %H:%M:%S")" >> upgrade_time.logexit -1fisleep 1m
done
# step 7 : upgrate from 2.16 --> 3.2.3
echo "start 3.2.3 : $(date "+%Y-%m-%d %H:%M:%S")" >> upgrade_time.log
java -jar gerrit-3.2.3.war init --batch --dev -d ${GERRIT_SITE}
# step 8 : reindex on 3.2.3
echo "Reindex accounts : $(date "+%Y-%m-%d %H:%M:%S")" >> upgrade_time.log
java -jar gerrit-3.2.3.war reindex --index accounts -d ${GERRIT_SITE}
echo "Reindex changes : $(date "+%Y-%m-%d %H:%M:%S")" >> upgrade_time.log
nohup java -jar gerrit-3.2.3.war reindex --index changes -d ${GERRIT_SITE} > "reindex-$(date "+%Y-%m-%d-%H-%M-%S").log" 2>&1 &可固定参数的脚本：
#!/bin/bash
set -eexport GERRIT_SITE=/home/work/hdd1/gerrit_odm
cd ${GERRIT_SITE}/lib
rm -rf ${GERRIT_SITE}/lib/*
wget https://staging-cnbj2-fds.api.xiaomi.net/files/mysql-connector-java-5.1.43.jarcd /home/work/hdd1
echo "start 2.15 :$(date "+%Y-%m-%d %H:%M:%S")" >> upgrade_time.log
java -jar gerrit-2.15.21.war init --batch --dev -d ${GERRIT_SITE}cd ${GERRIT_SITE}/lib
rm -rf ${GERRIT_SITE}/lib/*
wget https://staging-cnbj2-fds.api.xiaomi.net/files/mysql-connector-java-5.1.48.jarcd -
echo "start 2.16 :$(date "+%Y-%m-%d %H:%M:%S")" >> upgrade_time.log
java -jar gerrit-2.16.25.war init --batch --dev -d ${GERRIT_SITE}echo "start migrate :$(date "+%Y-%m-%d %H:%M:%S")" >> upgrade_time.log
nohup java -jar gerrit-2.16.25.war migrate-to-note-db --reindex false -d ${GERRIT_SITE} > mi.log 2>&1 &while true
doecho "Waiting for NoteDB Migration complete: $(date "+%Y-%m-%d %H:%M:%S")"if test ! -z "$(tail -n 20 mi.log | grep 'NoteDB Migration complete.')"; thenecho "NoteDB Migration complete: $(date "+%Y-%m-%d %H:%M:%S")" >> upgrade_time.logbreakfiif test ! -z "$(tail -n 20 mi.log | grep 'fatal: Migrating primary storage for some changes failed, see log')"; thenecho "fatal: Migrating primary storage for some changes failed, see log: $(date "+%Y-%m-%d %H:%M:%S")" >> upgrade_time.logexit -1fisleep 1m
done编辑于 2021-05-13 11:47
![一、PHP环境搭建[phpstorm]](https://img-blog.csdnimg.cn/2e76ed6278314a99890a21337e7fa99a.png)
