使用Pydantic输出的查询引擎¶
每个查询引擎都支持在RetrieverQueryEngine
中使用以下response_mode
来集成结构化响应:
refine
compact
tree_summarize
accumulate
(测试版,需要额外解析以转换为对象)compact_accumulate
(测试版,需要额外解析以转换为对象)
在这个笔记本中,我们将通过一个小例子来演示使用方法。
在底层,每个LLM响应都将是一个pydantic对象。如果该响应需要被精炼或总结,它将被转换为下一个响应的JSON字符串。然后,最终的响应将作为一个pydantic对象返回。
注意: 从技术上讲,这可以与任何LLM一起使用,但非OpenAI的支持仍在开发中,被视为测试版。
设置¶
如果您在colab上打开这个笔记本,您可能需要安装LlamaIndex 🦙。
In [ ]:
Copied!
%pip install llama-index-llms-anthropic
%pip install llama-index-llms-openai
%pip install llama-index-llms-anthropic
%pip install llama-index-llms-openai
In [ ]:
Copied!
!pip install llama-index
!pip install llama-index
In [ ]:
Copied!
import os
import openai
os.environ["OPENAI_API_KEY"] = "sk-..."
openai.api_key = os.environ["OPENAI_API_KEY"]
import os
import openai
os.environ["OPENAI_API_KEY"] = "sk-..."
openai.api_key = os.environ["OPENAI_API_KEY"]
下载数据
In [ ]:
Copied!
!mkdir -p 'data/paul_graham/'
!wget 'https://raw.githubusercontent.com/run-llama/llama_index/main/docs/docs/examples/data/paul_graham/paul_graham_essay.txt' -O 'data/paul_graham/paul_graham_essay.txt'
!mkdir -p 'data/paul_graham/'
!wget 'https://raw.githubusercontent.com/run-llama/llama_index/main/docs/docs/examples/data/paul_graham/paul_graham_essay.txt' -O 'data/paul_graham/paul_graham_essay.txt'
In [ ]:
Copied!
from llama_index.core import SimpleDirectoryReader
documents = SimpleDirectoryReader("./data/paul_graham").load_data()
from llama_index.core import SimpleDirectoryReader
documents = SimpleDirectoryReader("./data/paul_graham").load_data()
创建我们的Pydantic输出对象¶
In [ ]:
Copied!
from typing import List
from pydantic import BaseModel
class Biography(BaseModel):
"""传记的数据模型。"""
name: str
best_known_for: List[str]
extra_info: str
from typing import List
from pydantic import BaseModel
class Biography(BaseModel):
"""传记的数据模型。"""
name: str
best_known_for: List[str]
extra_info: str
创建索引 + 查询引擎(OpenAI)¶
在使用OpenAI时,将利用函数调用API来获得可靠的结构化输出。
In [ ]:
Copied!
from llama_index.core import VectorStoreIndex
from llama_index.llms.openai import OpenAI
llm = OpenAI(model="gpt-3.5-turbo", temperature=0.1)
index = VectorStoreIndex.from_documents(
documents,
)
from llama_index.core import VectorStoreIndex
from llama_index.llms.openai import OpenAI
llm = OpenAI(model="gpt-3.5-turbo", temperature=0.1)
index = VectorStoreIndex.from_documents(
documents,
)
In [ ]:
Copied!
query_engine = index.as_query_engine(
output_cls=Biography, response_mode="compact", llm=llm
)
query_engine = index.as_query_engine(
output_cls=Biography, response_mode="compact", llm=llm
)
In [ ]:
Copied!
response = query_engine.query("Who is Paul Graham?")
response = query_engine.query("Who is Paul Graham?")
In [ ]:
Copied!
print(response.name)
print(response.best_known_for)
print(response.extra_info)
print(response.name)
print(response.best_known_for)
print(response.extra_info)
Paul Graham ['working on Bel', 'co-founding Viaweb', 'creating the programming language Arc'] Paul Graham is a computer scientist, entrepreneur, and writer. He is best known for his work on Bel, a programming language, and for co-founding Viaweb, an early web application company that was later acquired by Yahoo. Graham also created the programming language Arc. He has written numerous essays on topics such as startups, programming, and life.
In [ ]:
Copied!
# 获取完整的pydanitc对象
print(type(response.response))
# 获取完整的pydanitc对象
print(type(response.response))
<class '__main__.Biography'>
创建索引 + 查询引擎(非OpenAI,Beta)¶
当使用不支持函数调用的LLM时,我们依赖LLM自己编写JSON,并将JSON解析为适当的pydantic对象。
In [ ]:
Copied!
import os
os.environ["ANTHROPIC_API_KEY"] = "sk-..."
import os
os.environ["ANTHROPIC_API_KEY"] = "sk-..."
In [ ]:
Copied!
from llama_index.core import VectorStoreIndex
from llama_index.llms.anthropic import Anthropic
llm = Anthropic(model="claude-instant-1.2", temperature=0.1)
index = VectorStoreIndex.from_documents(
documents,
)
from llama_index.core import VectorStoreIndex
from llama_index.llms.anthropic import Anthropic
llm = Anthropic(model="claude-instant-1.2", temperature=0.1)
index = VectorStoreIndex.from_documents(
documents,
)
In [ ]:
Copied!
query_engine = index.as_query_engine(
output_cls=Biography, response_mode="tree_summarize", llm=llm
)
query_engine = index.as_query_engine(
output_cls=Biography, response_mode="tree_summarize", llm=llm
)
In [ ]:
Copied!
response = query_engine.query("Who is Paul Graham?")
response = query_engine.query("Who is Paul Graham?")
In [ ]:
Copied!
print(response.name)
print(response.best_known_for)
print(response.extra_info)
print(response.name)
print(response.best_known_for)
print(response.extra_info)
Paul Graham ['Co-founder of Y Combinator', 'Essayist and programmer'] He is known for creating Viaweb, one of the first web application builders, and for founding Y Combinator, one of the world's top startup accelerators. Graham has also written extensively about technology, investing, and philosophy.
In [ ]:
Copied!
# 获取完整的pydanitc对象
print(type(response.response))
# 获取完整的pydanitc对象
print(type(response.response))
<class '__main__.Biography'>
累积示例(Beta)¶
使用pydantic对象进行累积需要一些额外的解析。这仍然是一个测试功能,但仍然可以对pydantic对象进行累积。
In [ ]:
Copied!
from typing import List
from pydantic import BaseModel
class Company(BaseModel):
"""公司提及的数据模型。"""
company_name: str
context_info: str
from typing import List
from pydantic import BaseModel
class Company(BaseModel):
"""公司提及的数据模型。"""
company_name: str
context_info: str
In [ ]:
Copied!
from llama_index.core import VectorStoreIndex,
from llama_index.llms.openai import OpenAI
llm = OpenAI(model="gpt-3.5-turbo", temperature=0.1)
index = VectorStoreIndex.from_documents(
documents,
)
from llama_index.core import VectorStoreIndex,
from llama_index.llms.openai import OpenAI
llm = OpenAI(model="gpt-3.5-turbo", temperature=0.1)
index = VectorStoreIndex.from_documents(
documents,
)
In [ ]:
Copied!
query_engine = index.as_query_engine(
output_cls=Company, response_mode="accumulate", llm=llm
)
query_engine = index.as_query_engine(
output_cls=Company, response_mode="accumulate", llm=llm
)
In [ ]:
Copied!
response = query_engine.query("What companies are mentioned in the text?")
response = query_engine.query("What companies are mentioned in the text?")
在accumulate中,默认情况下,响应是由一个默认分隔符分隔,并在前面加上一个前缀。
In [ ]:
Copied!
companies = []
# 以默认分隔符拆分
for response_str in str(response).split("\n---------------------\n"):
# 移除前缀 -- 每个响应都以`Response 1: {...}`开头
# 因此,我们找到第一个括号并移除它之前的所有内容
response_str = response_str[response_str.find("{") :]
companies.append(Company.parse_raw(response_str))
companies = []
# 以默认分隔符拆分
for response_str in str(response).split("\n---------------------\n"):
# 移除前缀 -- 每个响应都以`Response 1: {...}`开头
# 因此,我们找到第一个括号并移除它之前的所有内容
response_str = response_str[response_str.find("{") :]
companies.append(Company.parse_raw(response_str))
In [ ]:
Copied!
print(companies)
print(companies)
[Company(company_name='Yahoo', context_info='Yahoo bought us'), Company(company_name='Yahoo', context_info="I'd been meaning to since Yahoo bought us")]