作者:来自 Elastic Gustavo Llermaly
将我们的 Jira 内容索引到 Elaasticsearch 中以创建统一的数据源并使用文档级别安全性进行搜索。
在本文中,我们将回顾 Elastic Jira 原生连接器的一个用例。我们将使用一个模拟项目,其中一家银行正在开发一款汇款应用,需要将 Jira 中的信息集成到 Elastic 中。
原生连接器允许我们从票据、任务和其他文档中获取 Elastic 集群信息,集中数据并启用高级搜索功能。
使用此连接器的主要好处是:
- Jira 中的数据与 Elasticsearch 同步。
- 访问高级搜索功能。
- 文档级安全性 (DLS) 匹配源安全性。你只能搜索 Jira 中允许你查看的内容。
步骤
- 配置 Jira 连接器
- 将文档索引到 Elasticsearch
- 查询数据
- 文档级安全性 (document level security - DLS)
配置 Jira 连接器
首先,你需要从 Jira 获取 API token 以连接到 Elasticsearch。转到此链接了解如何创建它。
将其命名为 “elastic-connector”。它应该如下所示:
获取 token 并进入 Kibana 仪表板。然后,转到原生连接器并选择 New Jira Cloud connector。
https://<YOUR_KIBANA_URL>/app/enterprise_search/content/connectors/new_connector?service_type=jira&connector_type=native
用 Kibana 端点替换 YOUR_KIBANA_URL。
将连接器命名为 “bank”,然后点击 “Create and attach an index named bank”,即可创建一个同名的新索引。
完成了!现在我们需要配置我们的 Jira 数据。
由于我们不会使用自己的 SSL 证书,因此我们将保持 “Enable SSL” 关闭。
你可以在官方文档中查看每个字段的详细信息。
激活文档级安全性 (DLS),以便你获得有权查看文档的用户和组。
正确配置连接器后,你可以继续同步数据,如下所示。从 Jira 获取数据可能需要几分钟。
- 完整内容(Full Content):索引所有 Jira 文档。
- 增量内容(Incremental Content):仅索引自上次完整内容同步以来的更改。
- 访问控制(Access Control):在安全索引中索引 Jira 用户以激活 DLS。
我们可以检查连接器的概述来查看同步是否成功。
在 “Documents” 选项卡中,我们可以准确地看到使用连接器获取的数据。第一次同步的对象是:
- Projects
- Issues
- Attachments
将文档索引到 Elasticsearch
我们不仅限于跨连接器文档进行搜索。Elasticsearch 允许你使用单个查询搜索多个索引。
在我们的示例中,我们将其他文档索引到 galactic_documents 索引中,以了解搜索如何与多个数据源配合使用:
- GBFF 合规手册
- Galactic Banking App 用户指南
- 技术规格报告
但在索引之前,我们将为每个字段创建优化映射:
PUT /galactic_documents
{"mappings": {"properties": {"document_id": {"type": "keyword"},"title": {"type": "text","fields": {"raw": {"type": "keyword"}}},"content": {"type": "text"},"release_date": {"type": "date","format": "yyyy-MM-dd"},"page_count": {"type": "integer"},"tags": {"type": "keyword"}}}
}
配置映射后,我们现在可以索引:
POST galactic_documents/_bulk
{ "index": { "_index": "galactic_documents", "_id": "1" } }
{ "document_id": "GBFF-001", "title": "Compliance Manual of the GBFF", "content": "This document sets forth the compliance standards for intergalactic financial entities: Quantum-level data encryption to guarantee security in all transactions. Mandatory multi-factor authentication for all users and administrators. Quarterly reviews of security policies and access audit logs.", "release_date": "2024-01-01", "page_count": 5, "tags": ["compliance", "security"] }
{ "index": { "_index": "galactic_documents", "_id": "2" } }
{ "document_id": "GBFF-002", "title": "User Guide for the Galactic Banking App", "content": "Welcome to the Galactic Banking application by Interstellar Finance Corp. Here you can: Transfer galactic credits to any registered account across the Milky Way. Check your balance and manage your investments in real-time. Access interplanetary loans with ease. For your security, use multi-factor authentication each time you log in.", "release_date": "2024-01-01", "page_count": 3, "tags": ["user guide", "application"] }
{ "index": { "_index": "galactic_documents", "_id": "3" } }
{ "document_id": "GBFF-003", "title": "Technical Specifications Report - Galactic Banking Project", "content": "This report details the technical architecture of the Galactic Banking application: Microservices-based backend for scalability and performance. Secure communication protocols utilizing quantum encryption. Transaction management adapted to environments with gravity variations and time dilation.", "release_date": "2024-01-01", "page_count": 7, "tags": ["technical", "specifications", "architecture"] }
查询数据
现在我们有了 Jira 对象和文档,我们可以一起搜索它们。
GET bank,galactic_documents/_search
{"query": {"multi_match": {"query": "galactic moon","fields": ["content","title","*description","*summary"]}}
}
查询 “galactic moon” 将为我们获取 Jira 对象和我们索引的文档:
{"took": 3,"timed_out": false,"_shards": {"total": 3,"successful": 3,"skipped": 0,"failed": 0},"hits": {"total": {"value": 3,"relation": "eq"},"max_score": 1.2613049,"hits": [{"_index": "bank","_id": "Marketing Mars-MM-2","_score": 1.2613049,"_source": {"Type": "Task","Custom_Fields": {"Satisfaction": null,"Approvals": null,"Change reason": null,"Epic Link": null,"Actual end": null,"Design": null,"Campaign assets": null,"Department": null,"Story point estimate": null,"Approver groups": null,"[CHART] Date of First Response": null,"Request Type": null,"Campaign goals": null,"Project overview key": null,"Related projects": null,"Campaign type": null,"Impact": null,"Request participants": [],"Locked forms": null,"Time to first response": null,"Work category": null,"Audience": null,"Open forms": null,"Details": null,"Sprint": null,"Stakeholders": null,"Marketing asset type": null,"Submitted forms": null,"Start date": null,"Actual start": null,"Category": null,"Change risk": null,"Target start": null,"Issue color": null,"Parent Link": {"hasEpicLinkFieldDependency": false,"showField": false,"nonEditableReason": {"reason": "EPIC_LINK_SHOULD_BE_USED","message": "To set an epic as the parent, use the epic link instead"}},"Format": null,"Target end": null,"Approvers": null,"Team": null,"Change type": null,"Satisfaction date": null,"Request language": null,"Amount": null,"Rank": "0|i0003j:","Affected services": null,"Type": null,"Time to resolution": null,"Total forms": null,"[CHART] Time in Status": null,"Organizations": [],"Flagged": null,"Project overview status": null},"Issue": {"statuscategorychangedate": "2024-11-01T17:52:30.550-0300","issuetype": {"avatarId": 10318,"hierarchyLevel": 0,"name": "Task","self": "https://xxxx.atlassian.net/rest/api/2/issuetype/10017","description": "A small, distinct piece of work.","entityId": "f30ea676-7b3d-44ad-9858-558081742a2e","id": "10017","iconUrl": "https://xxxx.atlassian.net/rest/api/2/universal_avatar/view/type/issuetype/avatar/10318?size=medium","subtask": false},"components": [],"timespent": null,"timeoriginalestimate": null,"project": {"simplified": true,"avatarUrls": {"48x48": "https://xxxx.atlassian.net/rest/api/2/universal_avatar/view/type/project/avatar/10418","24x24": "https://xxxx.atlassian.net/rest/api/2/universal_avatar/view/type/project/avatar/10418?size=small","16x16": "https://xxxx.atlassian.net/rest/api/2/universal_avatar/view/type/project/avatar/10418?size=xsmall","32x32": "https://xxxx.atlassian.net/rest/api/2/universal_avatar/view/type/project/avatar/10418?size=medium"},"name": "Marketing Mars","self": "https://xxxx.atlassian.net/rest/api/2/project/10003","id": "10003","projectTypeKey": "business","key": "MM"},"description": null,"fixVersions": [],"aggregatetimespent": null,"resolution": null,"timetracking": {},"security": null,"aggregatetimeestimate": null,"attachment": [],"resolutiondate": null,"workratio": -1,"summary": "Conquer the moon","issuerestriction": {"issuerestrictions": {},"shouldDisplay": true},"watches": {"self": "https://xxxx.atlassian.net/rest/api/2/issue/MM-2/watchers","isWatching": true,"watchCount": 1},"lastViewed": "2024-11-01T17:52:34.925-0300","creator": {"accountId": "712020:88983800-6c97-469a-9451-79c2dd3732b5","emailAddress": "contornan_cliche.0y@icloud.com","avatarUrls": {"48x48": "https://secure.gravatar.com/avatar/f098101294d1a0da282bb2388df8c257?d=https%3A%2F%2Favatar-management--avatars.us-west-2.prod.public.atl-paas.net%2Finitials%2FTM-3.png","24x24": "https://secure.gravatar.com/avatar/f098101294d1a0da282bb2388df8c257?d=https%3A%2F%2Favatar-management--avatars.us-west-2.prod.public.atl-paas.net%2Finitials%2FTM-3.png","16x16": "https://secure.gravatar.com/avatar/f098101294d1a0da282bb2388df8c257?d=https%3A%2F%2Favatar-management--avatars.us-west-2.prod.public.atl-paas.net%2Finitials%2FTM-3.png","32x32": "https://secure.gravatar.com/avatar/f098101294d1a0da282bb2388df8c257?d=https%3A%2F%2Favatar-management--avatars.us-west-2.prod.public.atl-paas.net%2Finitials%2FTM-3.png"},"displayName": "Tomas Murua","accountType": "atlassian","self": "https://xxxx.atlassian.net/rest/api/2/user?accountId=712020%3A88983800-6c97-469a-9451-79c2dd3732b5","active": true,"timeZone": "Chile/Continental"},"subtasks": [],"created": "2024-11-01T17:52:30.289-0300","reporter": {"accountId": "712020:88983800-6c97-469a-9451-79c2dd3732b5","emailAddress": "contornan_cliche.0y@icloud.com","avatarUrls": {"48x48": "https://secure.gravatar.com/avatar/f098101294d1a0da282bb2388df8c257?d=https%3A%2F%2Favatar-management--avatars.us-west-2.prod.public.atl-paas.net%2Finitials%2FTM-3.png","24x24": "https://secure.gravatar.com/avatar/f098101294d1a0da282bb2388df8c257?d=https%3A%2F%2Favatar-management--avatars.us-west-2.prod.public.atl-paas.net%2Finitials%2FTM-3.png","16x16": "https://secure.gravatar.com/avatar/f098101294d1a0da282bb2388df8c257?d=https%3A%2F%2Favatar-management--avatars.us-west-2.prod.public.atl-paas.net%2Finitials%2FTM-3.png","32x32": "https://secure.gravatar.com/avatar/f098101294d1a0da282bb2388df8c257?d=https%3A%2F%2Favatar-management--avatars.us-west-2.prod.public.atl-paas.net%2Finitials%2FTM-3.png"},"displayName": "Tomas Murua","accountType": "atlassian","self": "https://xxxx.atlassian.net/rest/api/2/user?accountId=712020%3A88983800-6c97-469a-9451-79c2dd3732b5","active": true,"timeZone": "Chile/Continental"},"aggregateprogress": {"total": 0,"progress": 0},"priority": {"name": "Medium","self": "https://xxxx.atlassian.net/rest/api/2/priority/3","iconUrl": "https://xxxx.atlassian.net/images/icons/priorities/medium.svg","id": "3"},"labels": [],"environment": null,"timeestimate": null,"aggregatetimeoriginalestimate": null,"versions": [],"duedate": null,"progress": {"total": 0,"progress": 0},"issuelinks": [],"votes": {"hasVoted": false,"self": "https://xxxx.atlassian.net/rest/api/2/issue/MM-2/votes","votes": 0},"comment": {"total": 0,"comments": [],"maxResults": 0,"self": "https://xxxx.atlassian.net/rest/api/2/issue/10018/comment","startAt": 0},"assignee": null,"worklog": {"total": 0,"maxResults": 20,"startAt": 0,"worklogs": []},"updated": "2024-11-01T17:52:42.711-0300","status": {"name": "To Do","self": "https://xxxx.atlassian.net/rest/api/2/status/10014","description": "","iconUrl": "https://xxxx.atlassian.net/","id": "10014","statusCategory": {"colorName": "blue-gray","name": "To Do","self": "https://xxxx.atlassian.net/rest/api/2/statuscategory/2","id": 2,"key": "new"}}},"id": "Marketing Mars-MM-2","_timestamp": "2024-11-01T17:52:42.711-0300","Key": "MM-2","_allow_access_control": ["account_id:712020:88983800-6c97-469a-9451-79c2dd3732b5","name:Tomas-Murua"]}},{"_index": "galactic_documents","_id": "2","_score": 0.61183906,"_source": {"document_id": "GBFF-002","title": "User Guide for the Galactic Banking App","content": "Welcome to the Galactic Banking application by Interstellar Finance Corp. Here you can: Transfer galactic credits to any registered account across the Milky Way. Check your balance and manage your investments in real-time. Access interplanetary loans with ease. For your security, use multi-factor authentication each time you log in.","release_date": "2024-01-01","page_count": 3,"tags": ["user guide","application"]}},{"_index": "galactic_documents","_id": "3","_score": 0.5029222,"_source": {"document_id": "GBFF-003","title": "Technical Specifications Report - Galactic Banking Project","content": "This report details the technical architecture of the Galactic Banking application: Microservices-based backend for scalability and performance. Secure communication protocols utilizing quantum encryption. Transaction management adapted to environments with gravity variations and time dilation.","release_date": "2024-01-01","page_count": 7,"tags": ["technical","specifications","architecture"]}}]}
}
如果文档太长,你可以将选项 _source 添加到查询中,以仅包含你需要的字段。如果你只想删除一些字段,我们将在本系列的第二部分介绍该选项。
文档级安全性 (DLS)
我们现在将配置文档级安全性 (DLS),以将 Jira 权限与 Elasticsearch 中的权限相匹配,以便用户搜索时只能看到他们在 Jira 中被允许看到的内容。
首先,我们将转到 Elastic Cloud 中连接器的控制面板,然后单击 Access Control Sync。
此同步将带来 Jira 用户的访问和权限信息。为了测试这一点,我创建了另一个 Jira 板(board),但用户 “Gustavo” 无权访问该板。
注意:创建板后,不要忘记运行内容同步。你可以运行一次性同步,也可以按计划运行。
让我们开始检查新板中的文档是否存在:
GET bank/_search
{"_source": ["Issue.summary"],"query": {"match": {"Issue.project.name": "Marketing Mars"}}
}
我们可以有效地看到以下问题:
{"took": 2,"timed_out": false,"_shards": {"total": 2,"successful": 2,"skipped": 0,"failed": 0},"hits": {"total": {"value": 3,"relation": "eq"},"max_score": 0.7473189,"hits": [{"_index": "bank","_id": "Marketing Mars-MM-1","_score": 0.7473189,"_source": {"Issue": {"summary": "Conquer Mars"}}},{"_index": "bank","_id": "Marketing Mars-MM-3","_score": 0.7473189,"_source": {"Issue": {"summary": "Conquering Earth"}}},{"_index": "bank","_id": "Marketing Mars-MM-2","_score": 0.7473189,"_source": {"Issue": {"summary": "Conquer the moon"}}}]}
}
但是,由于用户 “Gustavo” 没有访问权限,因此他应该无法看到它们。
让我们在 ACL 过滤器索引中查找用户的文档以查看他们的权限。
GET .search-acl-filter-bank/_search
响应:
{"_index": ".search-acl-filter-bank","_id": "63c04b092341bff4fff6e0cb","_score": 1,"_source": {"created_at": "2024-11-01T23:19:35.784996+00:00","id": "63c04b092341bff4fff6e0cb","_timestamp": "2024-11-01T05:42:04.410478+00:00","identity": {"account_id": "account_id:63c04b092341bff4fff6e0cb","email_address": null,"display_name": "name:Gustavo","locale": "locale:en_US"},"query": {"template": {"source": """{"bool": {"should": [{"bool": {"must_not": {"exists": {"field": "_allow_access_control"}}}},{"terms": {"_allow_access_control.enum": {{#toJson}}access_control{{/toJson}}}}]}}""","params": {"access_control": ["account_id:63c04b092341bff4fff6e0cb","group_id:d3f28403-7e99-4262-8f11-77a75bcd33d8","role_key:jira-software"]}}}}
}
该索引包括用户 ID 及其所有 Jira 组。我们需要将用户访问控制中的内容与每个文档中的字段 _allowed_access_control 进行匹配。
我们将使用以下命令为 Gustavo 创建 API 密钥。你必须从上一步复制 query.template 值:
POST /_security/api_key
{"name": "gustavo","expiration": "30d","role_descriptors": {"jira-role": {"index": [{"names": ["bank","galactic_documents"],"privileges": ["read","view_index_metadata"],"query": {"template": {"params": {"access_control": ["account_id:63c04b092341bff4fff6e0cb","group_id:d3f28403-7e99-4262-8f11-77a75bcd33d8","role_key:jira-software"]},"source": """{"bool": {"should": [{"bool": {"must_not": {"exists": {"field": "_allow_access_control"}}}},{"terms": {"_allow_access_control.enum": {{#toJson}}access_control{{/toJson}}}}]}}"""}}}]}}
}
请注意,我们仅通过此选项授予对本文中索引的访问权限。
为 Gustavo 创建 API 密钥的响应如下:
{"id": "yLa1FJMBU4bZPaw5Stnl","name": "gustavo","expiration": 1733811245816,"api_key": "UrGdsnDFSyGxjQvLayw5jQ","encoded": "eUxhMUZKTUJVNGJaUGF3NVN0bmw6VXJHZHNuREZTeUd4alF2TGF5dzVqUQ=="
}
你可以使用 curl 来测试我们是否可以使用 API KEY 运行搜索,并且它不会从 Marketing board 获取信息,因为 Gustavo 无权访问它。
curl --location --request GET 'https://interstellar-finance-corp.es.us-central1.gcp.cloud.es.io/bank/_search' \
--header 'Authorization: ApiKey eUxhMUZKTUJVNGJaUGF3NVN0bmw6VXJHZHNuREZTeUd4alF2TGF5dzVqUQ==' \
--header 'Content-Type: application/json' \
--data '{"_source": ["Issue.summary"],"query": {"match": {"Issue.project.name": "Marketing Mars"}}
}'
响应:
{"took": 0,"timed_out": false,"_shards": {"total": 1,"successful": 1,"skipped": 0,"failed": 0},"hits": {"total": {"value": 0,"relation": "eq"},"max_score": null,"hits": []}
}
我们可以看到,Gustavo 没有获得任何信息,因为他没有访问权限。现在,让我们用他被允许查看的 board 文件进行测试:
curl --location --request GET 'https://interstellar-finance-corp.es.us-central1.gcp.cloud.es.io/bank/_search?pretty=true' \
--header 'Authorization: ApiKey eUxhMUZKTUJVNGJaUGF3NVN0bmw6VXJHZHNuREZTeUd4alF2TGF5dzVqUQ==' \
--header 'Content-Type: application/json' \
--data '{"_source": ["Issue.summary"],"query": {"match": {"Issue.project.name": "Galactic Banking Project"}}
}'
响应:
{"took" : 7,"timed_out" : false,"_shards" : {"total" : 2,"successful" : 2,"skipped" : 0,"failed" : 0},"hits" : {"total" : {"value" : 3,"relation" : "eq"},"max_score" : 3.1784885,"hits" : [{"_index" : "bank","_id" : "Galactic Banking Project-GBP-3","_score" : 3.1784885,"_source" : {"Issue" : {"summary" : "Intergalactic Security and Compliance"}}},{"_index" : "bank","_id" : "Galactic Banking Project-GBP-2","_score" : 0.5469647,"_source" : {"Issue" : {"summary" : "Bank Application Frontend"}}},{"_index" : "bank","_id" : "Galactic Banking Project-GBP-1","_score" : 0.5469647,"_source" : {"Issue" : {"summary" : "Development of API for International Transfers"}}}]}
}
结论
如你所见,将 Elasticsearch 与 Jira 集成有许多好处,例如能够对你正在处理的所有项目进行统一搜索,以及能够在多个数据源中运行更高级的搜索。添加的 DLS 是一种快速简便的方法,可确保用户保持他们在原始源中已有的访问权限。
想要获得 Elastic 认证?了解下一次 Elasticsearch 工程师培训何时开始!
Elasticsearch 包含新功能,可帮助你为你的用例构建最佳搜索解决方案。深入了解我们的示例笔记本以了解更多信息,开始免费云试用,或立即在你的本地机器上试用 Elastic。
原文:Jira connector tutorial part I - Elasticsearch Labs