Easysearch 使用 AWS S3 进行快照备份与还原：完整指南及常见错误排查

Easysearch 可以使用 AWS S3 作为远程存储库，进行索引的快照（Snapshot）备份和恢复。同时，Easysearch 内置了 S3 插件，无需额外安装。以下是完整的配置和操作步骤。

1. 在 AWS S3 上创建存储桶

登录 AWS 控制台，进入 S3 服务。
创建一个新存储桶（例如 easysearch-backups）。
启用版本控制（可选，但推荐）。
权限配置：确保 IAM 角色具有访问 S3 的权限。

{
"Version": "2012-10-17",
"Statement": [{"Action": ["s3:ListBucket"],"Effect": "Allow","Resource": ["arn:aws:s3:::s3-bucket-name"]},{"Action": ["s3:GetObject","s3:PutObject","s3:DeleteObject"],"Effect": "Allow","Resource": ["arn:aws:s3:::s3-bucket-name/*"]}
]
}

2. 在 Console 上注册 S3 作为快照存储库

使用 Console DevTools 或 API

在 Easysearch 的 DevTools 执行：

PUT _snapshot/my_s3_repository
{"type": "s3","settings": {"bucket": "easysearch-backups","base_path": ""}
}

注意：

bucket 需要填写你的 S3 存储桶名称。
region 需要替换成你的 AWS S3 所在区域，SDK 默认美东区。
如果 Bucket 在中国区，还需添加 endpoint: https://s3.<region>.amazonaws.com.cn 参数。

3. 创建快照

一旦 my_s3_repository 注册完成，就可以创建快照：

PUT _snapshot/my_s3_repository/my_snapshot_001
{"indices": "my_index","include_global_state": false
}

查看当前存储的快照：

GET _snapshot/my_s3_repository/_all

4. 从 AWS S3 还原快照

当你需要恢复索引时：

POST _snapshot/my_s3_repository/my_snapshot_001/_restore
{"indices": "my_index","rename_pattern": "my_index","rename_replacement": "restored_my_index"
}

说明：这会从 my_snapshot_001 还原 my_index，但以 restored_my_index 命名，避免与现有索引冲突。

如果要直接覆盖原索引（确保 my_index 为空或已删除）：

POST _snapshot/my_s3_repository/my_snapshot_001/_restore
{"indices": "my_index","ignore_unavailable": true,"include_global_state": false
}

5. 可能的错误与解决方案

错误信息	可能原因	解决方案
`repository_s3 plugin not installed`	没有安装 `repository-s3` 插件	运行 `bin/elasticsearch-plugin install repository-s3` 并重启
`NoSuchBucket`	S3 存储桶不存在	确保 S3 存储桶名称正确
`AccessDenied`	权限不足	确保 S3 存储桶策略正确，检查 IAM 角色
`index_closed_exception`	目标索引已关闭	先 `POST my_index/_open` 再恢复
`index_already_exists_exception`	目标索引已存在	先 `DELETE my_index` 再恢复

6. 快照恢复常见错误排查

报错 1：无法连接到 S3


{"error": {"root_cause": [{"type": "repository_verification_exception","reason": "[my_s3_repository] path [/] is not accessible on master node"}],"type": "repository_verification_exception","reason": "[my_s3_repository] path [/] is not accessible on master node","caused_by": {"type": "i_o_exception","reason": "Unable to upload object [//tests-sXkmh3q5ThCCIX2VJp609g/master.dat] using a single upload","caused_by": {"type": "sdk_client_exception","reason": "Failed to connect to service endpoint: ","caused_by": {"type": "socket_timeout_exception","reason": "Connect timed out"}}}},"status": 500
}

解决方案：

在 keystore 中添加 AWS 凭证：

sudo ./bin/easysearch-keystore add s3.client.default.access_key
sudo ./bin/easysearch-keystore add s3.client.default.secret_key

如果运行在 EC2 上，确保实例挂载了 IAM Role。

{"error": {"root_cause": [{"type": "repository_verification_exception","reason": "[my_s3_repositor1] path  is not accessible on master node"}],"type": "repository_verification_exception","reason": "[my_s3_repositor1] path  is not accessible on master node","caused_by": {"type": "i_o_exception","reason": "Unable to upload object [tests-sUUzs-mTSZeYw1qk372DkQ/master.dat] using a single upload","caused_by": {"type": "sdk_client_exception","reason": "The requested metadata is not found at http://169.254.169.254/latest/meta-data/iam/security-credentials/"}}},"status": 500
}

报错 2：索引已存在，无法恢复

{"error": {"root_cause": [{"type": "snapshot_restore_exception","reason": "[my_s3_repository:1/9gIDCgSySwKzQqEYvaGM_w] cannot restore index [my_index] because an open index with same name already exists in the cluster. Either close or delete the existing index or restore the index under a different name by providing a rename pattern and replacement name"}],"type": "snapshot_restore_exception","reason": "[my_s3_repository:1/9gIDCgSySwKzQqEYvaGM_w] cannot restore index [my_index] because an open index with same name already exists in the cluster. Either close or delete the existing index or restore the index under a different name by providing a rename pattern and replacement name"},"status": 500
}

解决方案：

删除现有索引后恢复：
```
DELETE /my_index
```
关闭索引后恢复：
```
POST /my_index/_close
```

恢复为新的索引名称：

POST _snapshot/my_s3_repository/1/_restore
{"indices": "my_index","rename_pattern": "my_index","rename_replacement": "restored_my_index"
}

报错 3：权限错误

{"error": {"root_cause": [{"type": "security_exception","reason": "no permissions for [] and User [name=admin, external_roles=[admin]]"}],"type": "security_exception","reason": "no permissions for [] and User [name=admin, external_roles=[admin]]"},"status": 403
}

解决方案：

确保用户有 manage_snapshots 角色权限。
排除 .security 索引或全局状态，否则无法恢复。

在这里插入图片描述

POST _snapshot/my_s3_repositor1/snapshot_002/_restore
{"indices": "-.security","ignore_unavailable": true,"include_global_state": false
}

📌 存储库（Repository）管理 API

存储库用于存储快照，Elasticsearch 支持 AWS S3、GCS、本地等存储。

1️⃣ 查看所有已注册的存储库

GET _snapshot/_all

示例返回：

{"my_s3_repository": {"type": "s3","settings": {"bucket": "es-snapshots-bucket","region": "us-east-1"}}
}

2️⃣ 查看特定存储库信息

GET _snapshot/my_s3_repository

3️⃣ 创建存储库（AWS S3 示例）

PUT _snapshot/my_s3_repository
{"type": "s3","settings": {"bucket": "es-snapshots-bucket",}
}

4️⃣ 删除存储库

DELETE _snapshot/my_s3_repository

⚠ 删除存储库不会删除快照，需要手动删除快照！

📌 快照（Snapshot）管理 API

快照用于备份和恢复索引数据。

1️⃣ 创建快照

备份特定索引

PUT _snapshot/my_s3_repository/snapshot_001
{"indices": "my_index","include_global_state": false
}

备份所有索引

PUT _snapshot/my_s3_repository/snapshot_002
{"include_global_state": true
}

2️⃣ 查看所有快照

GET _snapshot/my_s3_repository/_all

3️⃣ 查看特定快照信息

GET _snapshot/my_s3_repository/snapshot_001

4️⃣ 删除快照

DELETE _snapshot/my_s3_repository/snapshot_001

📌 快照恢复（Restore）API

恢复已备份的索引。

1️⃣ 还原单个索引

POST _snapshot/my_s3_repository/snapshot_001/_restore
{"indices": "my_index","ignore_unavailable": true,"include_global_state": false
}

2️⃣ 还原索引并重命名

POST _snapshot/my_s3_repository/snapshot_001/_restore
{"indices": "my_index","rename_pattern": "my_index","rename_replacement": "restored_my_index"
}

3️⃣ 还原所有索引

POST _snapshot/my_s3_repository/snapshot_002/_restore

📌 快照状态 API

查询快照的执行状态。

1️⃣ 查看当前快照任务

GET _snapshot/_status

2️⃣ 查看特定快照状态

GET _snapshot/my_s3_repository/snapshot_001/_status

API	用途
`GET _snapshot/_all`	查看所有存储库
`GET _snapshot/my_s3_repository`	查看特定存储库
`PUT _snapshot/my_s3_repository`	创建存储库
`DELETE _snapshot/my_s3_repository`	删除存储库
`PUT _snapshot/my_s3_repository/snapshot_001`	创建快照
`GET _snapshot/my_s3_repository/_all`	查看所有快照
`GET _snapshot/my_s3_repository/snapshot_001`	查看快照详情
`DELETE _snapshot/my_s3_repository/snapshot_001`	删除快照
`POST _snapshot/my_s3_repository/snapshot_001/_restore`	还原快照
`GET _snapshot/_status`	查看快照状态