1. 学习背景
在LangChain for LLM应用程序开发中课程中，学习了LangChain框架扩展应用程序开发中语言模型的用例和功能的基本技能，遂做整理为后面的应用做准备。视频地址：基于LangChain的大语言模型应用开发+构建和评估。
2. Q&A的作用
基于文档的问答系统是LLM的典型应用，给定一段可能从PDF文件、网页或某公司的内部文档库中提取的文本，可以使用LLM检索文档对问题进行回答。以下代码基于jupyternotebook运行。
1.导入环境
import osfrom dotenv import load_dotenv, find_dotenv
_ = load_dotenv(find_dotenv()) # read local .env file
from langchain.chains import RetrievalQA
from langchain.chat_models import ChatOpenAI
from langchain.document_loaders import CSVLoader
from langchain.vectorstores import DocArrayInMemorySearch
from IPython.display import display, Markdown
2.2 读取数据进行查询
from langchain.indexes import VectorstoreIndexCreator
# 没有docarray环境需要安装。命令：!pip install docarray# 要用到的数据文件
file = 'OutdoorClothingCatalog_1000.csv'
loader = CSVLoader(file_path=file, encoding='utf-8')# 此处我们已完成了文档的向量存储
index = VectorstoreIndexCreator(vectorstore_cls=DocArrayInMemorySearch).from_loaders([loader])# 创建提问语句
query ="Please list all your shirts with sun protection in a table in markdown and summarize each one."# 传入query内容，使用index生成响应
response = index.query(query)# 以markdown方式进行呈现，注意LLM生成的样式可能存在差异
display(Markdown(response))
输出如下：
| Name | Description | Sun Protection Rating |
|---|---|---|
| Men’s Tropical Plaid Short-Sleeve Shirt | Made of 100% polyester, UPF 50+ rating, front and back cape venting, two front bellows pockets | SPF 50+, blocks 98% of harmful UV rays |
| Men’s Plaid Tropic Shirt, Short-Sleeve | Made of 52% polyester and 48% nylon, UPF 50+ rating, front and back cape venting, two front bellows pockets | SPF 50+, blocks 98% of harmful UV rays |
| Men’s TropicVibe Shirt, Short-Sleeve | Made of 71% nylon and 29% polyester, UPF 50+ rating, front and back cape venting, two front bellows pockets | SPF 50+, blocks 98% of harmful UV rays |
| Sun Shield Shirt | Made of 78% nylon and 22% Lycra Xtra Life fiber, UPF 50+ rating, wicks moisture, abrasion resistant | SPF 50+, blocks 98% of harmful UV rays |
All four shirts provide UPF 50+ sun protection, blocking 98% of the sun’s harmful rays. The Men’s Tropical Plaid Short-Sleeve Shirt is made of 100% polyester and is wrinkle-resistant。
至此，内容已经查出来了，并生成了一小段总结的话。那么底层的原理又是什么呢？
2.3 底层原理
2.3.1向量化
一般的大模型一次只能接收几千个单词，如图：
如果有个很大的文档，我们要怎样让LLM对文档进行问答呢？这里就需要Embedding和向量存储发挥作用了。
什么是Embedding？Embedding将一段文本转换成数字，用一组数字表示这段文本。这组数字捕捉了它所代表的文字片段的肉容含义。内容相似的文本片段会有相似的向量值，这样我们可以在向量空间中比较文本片段。例如，我们有三段话：
- My dog Rover likes to chase squirrels.
- Fluffy, my cat, refuses to eat from a can.
- The Chevy Bolt accelerates to 60 mph in 6.7 seconds.
三段话前两个描述宠物，第三个描述汽车，向量化后如图：
如果我们观察数值空间中的表示，可以看到当我们比较关于两个关于宠物的句子的向量时，它们相似度非常高。将其与汽车相关的语句进行比对，可以看到相关程度非常低。利用向量可以很轻松的让我们找出哪些片段是相似的。利用这种技术，我们可以从文档中找出与提问相似的片段，传递给LLM进行解答。
2.3.2向量数据库
向量数据库是一种存储方法，可以存储我们在前面创建的那种矢量数字数组。往向量数据库中新建数据的方式，就是将文档拆分成块，每块生成Embedding，然后把Embedding和原始块一起存储到数据库中。
因为有些大文档无法整个传给文档，因此要先切块，然后只把最相关的内容存入，然后，把每个文本块生成一个Embedding，然后将这些Embedding存储在向量数据库中。如图：
当查询过来，我们先将查询内容embedding，得到一个数组，然后将这个数字数组与向量数据库中的所有向量进行比较，选择最相似的前若干个文本块。
拿到这些文本块后，将这些文本块和原始的查询内容一起传递给语言模型，这样可以让语言模型根据检索出来的文档内容生成最终答案。
2.4 再了解底层原理
loader = CSVLoader(file_path=file, encoding='utf-8')
docs = loader.load()
docs[0]
输出如下：
Document(page_content=": 0\nname: Women's Campside Oxfords\ndescription: This ultracomfortable lace-to-toe Oxford boasts a super-soft canvas, thick cushioning, and quality construction for a broken-in feel from the first time you put them on. \n\nSize & Fit: Order regular shoe size. For half sizes not offered, order up to next whole size. \n\nSpecs: Approx. weight: 1 lb.1 oz. per pair. \n\nConstruction: Soft canvas material for a broken-in feel and look. Comfortable EVA innersole with Cleansport NXT® antimicrobial odor control. Vintage hunt, fish and camping motif on innersole. Moderate arch contour of innersole. EVA foam midsole for cushioning and support. Chain-tread-inspired molded rubber outsole with modified chain-tread pattern. Imported. \n\nQuestions? Please contact us for any inquiries.", metadata={'source': 'OutdoorClothingCatalog_1000.csv', 'row': 0})
接着
# 使用OpenAIEmbeddings完成embedding
from langchain.embeddings import OpenAIEmbeddings
embeddings = OpenAIEmbeddings()
#使用embed_query模拟生成embeddings向量
embed = embeddings.embed_query("Hi my name is Harrison")
print(len(embed))
print(embed[:5])
输出如下：
1536[-0.021900920197367668, 0.006746490020304918, -0.018175246194005013, -0.039119575172662735, -0.014097143895924091]
可以看到，embedding向量的长度为1536，数组的前五个向量如上。
# 接着我们将刚刚加载的所有文本片段生成Embedding，并将它们存储在一个向量数据库中
db = DocArrayInMemorySearch.from_documents(docs, embeddings
)
# 创建对话查询语句
query = "Please suggest a shirt with sunblocking"
# 向量数据库中使用similarity_search方法得到查询的文档列表
docs = db.similarity_search(query)
print(len(docs))
print(docs[0])
输出如下：
4
Document(page_content=': 255\nname: Sun Shield Shirt by\ndescription: "Block the sun, not the fun – our high-performance sun shirt is guaranteed to protect from harmful UV rays. \n\nSize & Fit: Slightly Fitted: Softly shapes the body. Falls at hip.\n\nFabric & Care: 78% nylon, 22% Lycra Xtra Life fiber. UPF 50+ rated – the highest rated sun protection possible. Handwash, line dry.\n\nAdditional Features: Wicks moisture for quick-drying comfort. Fits comfortably over your favorite swimsuit. Abrasion resistant for season after season of wear. Imported.\n\nSun Protection That Won\'t Wear Off\nOur high-performance fabric provides SPF 50+ sun protection, blocking 98% of the sun\'s harmful rays. This fabric is recommended by The Skin Cancer Foundation as an effective UV protectant.', metadata={'source': 'OutdoorClothingCatalog_1000.csv', 'row': 255})
可以看到，得到了4个相关的文档列表内容，第一个内容如上所示。
2.5 如何利用这个来回答得到提问的结果
# 首先，需要从这个向量存储器创建一个检索器(Retriever)
retriever = db.as_retriever()
# 定义一个LLM模型
llm = ChatOpenAI(temperature = 0.0)
# 手动将检索出来的内容合并成一段话
qdocs = "".join([docs[i].page_content for i in range(len(docs))])
# 将提问和检索出来的内容一起交给LLM，并让其生成一段摘要
response = llm.call_as_llm(f"{qdocs} Question: Please list all your \
shirts with sun protection in a table in markdown and summarize each one.")
display(Markdown(response))
输出如下：
| Name | Description |
|---|---|
| Sun Shield Shirt | High-performance sun shirt with UPF 50+ sun protection, moisture-wicking, and abrasion-resistant fabric. Fits comfortably over swimsuits. Recommended by The Skin Cancer Foundation. |
| Men’s Plaid Tropic Shirt | Ultracomfortable shirt with UPF 50+ sun protection, wrinkle-free fabric, and front/back cape venting. Made with 52% polyester and 48% nylon. |
| Men’s TropicVibe Shirt | Men’s sun-protection shirt with built-in UPF 50+ and front/back cape venting. Made with 71% nylon and 29% polyester. |
| Men’s Tropical Plaid Short-Sleeve Shirt | Lightest hot-weather shirt with UPF 50+ sun protection, front/back cape venting, and two front bellows pockets. Made with 100% polyester and is wrinkle-resistant. |
All of these shirts provide UPF 50+ sun protection, blocking 98% of the sun’s harmful rays. They are made with high-performance fabrics that are moisture-wicking, abrasion-resistant, and/or wrinkle-free. Some have front/back cape venting for added comfort in hot weather. The Sun Shield Shirt is recommended by The Skin Cancer Foundation.
2.6使用langchain进行封装运行
qa_stuff = RetrievalQA.from_chain_type(llm=llm, chain_type="stuff", retriever=retriever, verbose=True
)
query = "Please list all your shirts with sun protection in a table in markdown and summarize each one."
response = qa_stuff.run(query)
输出如下：
| Shirt Name | Description |
|---|---|
| Men’s Tropical Plaid Short-Sleeve Shirt | Rated UPF 50+ for superior protection from the sun’s UV rays. Made of 100% polyester and is wrinkle-resistant. With front and back cape venting that lets in cool breezes and two front bellows pockets. Provides the highest rated sun protection possible. |
| Men’s Plaid Tropic Shirt, Short-Sleeve | Rated to UPF 50+, helping you stay cool and dry. Made with 52% polyester and 48% nylon, this shirt is machine washable and dryable. Additional features include front and back cape venting, two front bellows pockets and an imported design. With UPF 50+ coverage, you can limit sun exposure and feel secure with the highest rated sun protection available. |
| Men’s TropicVibe Shirt, Short-Sleeve | Built-in UPF 50+ has the lightweight feel you want and the coverage you need when the air is hot and the UV rays are strong. Made with Shell: 71% Nylon, 29% Polyester. Lining: 100% Polyester knit mesh. Wrinkle resistant. Front and back cape venting lets in cool breezes. Two front bellows pockets. Imported. |
| Sun Shield Shirt | High-performance sun shirt is guaranteed to protect from harmful UV rays. Made with 78% nylon, 22% Lycra Xtra Life fiber. Fits comfortably over your favorite swimsuit. Abrasion resistant for season after season of wear. |
All of the shirts listed have sun protection with a UPF rating of 50+ and block 98% of the sun’s harmful rays. The Men’s Tropical Plaid Short-Sleeve Shirt is made of 100% polyester and has front and back cape venting and two front bellows pockets. The Men’s Plaid Tropic Shirt, Short-Sleeve is made with 52% polyester and 48% nylon and has front and back cape venting and two front bellows pockets. The Men’s TropicVibe Shirt, Short-Sleeve is made with Shell: 71% Nylon, 29% Polyester. Lining: 100% Polyester knit mesh and has front and back cape venting and two front bellows pockets. The Sun Shield Shirt is made with 78% nylon, 22% Lycra Xtra Life fiber and fits comfortably over your favorite swimsuit.
同样的，我们尝试用index.query也会得到同样的内容。
response = index.query(query, llm=llm)
输出结果和之前的一致
3.总结
Q&A可以用一行代码完成，也可以把它分成五个详细的步骤，可以查看每一步的详细结果。五个步骤可以详细的让我们理解到它底层到底是如何执行的。此外，chain_type="stuff" 参数还有其他三种，可以根据实际情况选取合适的参数，另外三种如图，有需要可以根据实际情况选取合适的参数进行实验。
