Weaviate向量存储 - 混合搜索¶
如果您在Colab上打开这个笔记本,您可能需要安装LlamaIndex 🦙。
In [ ]:
Copied!
%pip install llama-index-vector-stores-weaviate
%pip install llama-index-vector-stores-weaviate
In [ ]:
Copied!
!pip install llama-index
!pip install llama-index
In [ ]:
Copied!
import logging
import sys
logging.basicConfig(stream=sys.stdout, level=logging.INFO)
logging.getLogger().addHandler(logging.StreamHandler(stream=sys.stdout))
import logging
import sys
logging.basicConfig(stream=sys.stdout, level=logging.INFO)
logging.getLogger().addHandler(logging.StreamHandler(stream=sys.stdout))
创建一个Weaviate客户端¶
In [ ]:
Copied!
import os
import openai
os.environ["OPENAI_API_KEY"] = ""
openai.api_key = os.environ["OPENAI_API_KEY"]
import os
import openai
os.environ["OPENAI_API_KEY"] = ""
openai.api_key = os.environ["OPENAI_API_KEY"]
In [ ]:
Copied!
import weaviate
import weaviate
In [ ]:
Copied!
# 连接到云实例cluster_url = ""api_key = ""client = weaviate.connect_to_wcs( cluster_url=cluster_url, auth_credentials=weaviate.auth.AuthApiKey(api_key),)# 连接到本地实例# client = weaviate.connect_to_local()
# 连接到云实例cluster_url = ""api_key = ""client = weaviate.connect_to_wcs( cluster_url=cluster_url, auth_credentials=weaviate.auth.AuthApiKey(api_key),)# 连接到本地实例# client = weaviate.connect_to_local()
In [ ]:
Copied!
from llama_index.core import VectorStoreIndex, SimpleDirectoryReader
from llama_index.vector_stores.weaviate import WeaviateVectorStore
from llama_index.core.response.notebook_utils import display_response
from llama_index.core import VectorStoreIndex, SimpleDirectoryReader
from llama_index.vector_stores.weaviate import WeaviateVectorStore
from llama_index.core.response.notebook_utils import display_response
下载数据¶
In [ ]:
Copied!
!mkdir -p 'data/paul_graham/'
!wget 'https://raw.githubusercontent.com/run-llama/llama_index/main/docs/docs/examples/data/paul_graham/paul_graham_essay.txt' -O 'data/paul_graham/paul_graham_essay.txt'
!mkdir -p 'data/paul_graham/'
!wget 'https://raw.githubusercontent.com/run-llama/llama_index/main/docs/docs/examples/data/paul_graham/paul_graham_essay.txt' -O 'data/paul_graham/paul_graham_essay.txt'
加载文档¶
In [ ]:
Copied!
# 加载文档documents = SimpleDirectoryReader("./data/paul_graham/").load_data()
# 加载文档documents = SimpleDirectoryReader("./data/paul_graham/").load_data()
使用WeaviateVectorStore构建VectorStoreIndex¶
In [ ]:
Copied!
# 注意:您也可以选择手动定义索引名称。# index_name = "test_prefix"# vector_store = WeaviateVectorStore(weaviate_client=client, index_name=index_name)
# 注意:您也可以选择手动定义索引名称。# index_name = "test_prefix"# vector_store = WeaviateVectorStore(weaviate_client=client, index_name=index_name)
使用默认向量搜索查询索引¶
在这个示例中,我们将展示如何使用默认向量搜索来查询索引。
In [ ]:
Copied!
# 将日志级别设置为DEBUG,以获得更详细的输出query_engine = index.as_query_engine(similarity_top_k=2)response = query_engine.query("作者在成长过程中做了什么?")
# 将日志级别设置为DEBUG,以获得更详细的输出query_engine = index.as_query_engine(similarity_top_k=2)response = query_engine.query("作者在成长过程中做了什么?")
In [ ]:
Copied!
display_response(response)
display_response(response)
使用混合搜索查询索引¶
使用BM25和向量混合搜索。
alpha
参数决定加权(alpha = 0 -> BM25,alpha = 1 -> 向量搜索)。
默认情况下,使用alpha=0.75
(与向量搜索非常相似)¶
In [ ]:
Copied!
# 将日志级别设置为DEBUG,以获得更详细的输出query_engine = index.as_query_engine( vector_store_query_mode="hybrid", similarity_top_k=2)response = query_engine.query( "作者在成长过程中做了什么?",)
# 将日志级别设置为DEBUG,以获得更详细的输出query_engine = index.as_query_engine( vector_store_query_mode="hybrid", similarity_top_k=2)response = query_engine.query( "作者在成长过程中做了什么?",)
In [ ]:
Copied!
display_response(response)
display_response(response)
将alpha=0.
设置为偏爱bm25¶
In [ ]:
Copied!
# 将日志级别设置为DEBUG,以获得更详细的输出query_engine = index.as_query_engine( vector_store_query_mode="hybrid", similarity_top_k=2, alpha=0.0)response = query_engine.query( "作者在成长过程中做了什么?",)
# 将日志级别设置为DEBUG,以获得更详细的输出query_engine = index.as_query_engine( vector_store_query_mode="hybrid", similarity_top_k=2, alpha=0.0)response = query_engine.query( "作者在成长过程中做了什么?",)
In [ ]:
Copied!
display_response(response)
display_response(response)