Faiss向量存储¶

如果您在colab上打开这个笔记本，您可能需要安装LlamaIndex 🦙。

In [ ]:

%pip install llama-index-vector-stores-faiss

%pip install llama-index-vector-stores-faiss

In [ ]:

!pip install llama-index

!pip install llama-index

创建Faiss索引¶

In [ ]:

import logging
import sys

logging.basicConfig(stream=sys.stdout, level=logging.INFO)
logging.getLogger().addHandler(logging.StreamHandler(stream=sys.stdout))

import logging import sys logging.basicConfig(stream=sys.stdout, level=logging.INFO) logging.getLogger().addHandler(logging.StreamHandler(stream=sys.stdout))

In [ ]:

import faiss

# 文本ada嵌入002的维度
d = 1536
faiss_index = faiss.IndexFlatL2(d)

import faiss # 文本ada嵌入002的维度 d = 1536 faiss_index = faiss.IndexFlatL2(d)

加载文档，构建VectorStoreIndex¶

In [ ]:

from llama_index.core import (
    SimpleDirectoryReader,
    load_index_from_storage,
    VectorStoreIndex,
    StorageContext,
)
from llama_index.vector_stores.faiss import FaissVectorStore
from IPython.display import Markdown, display

from llama_index.core import ( SimpleDirectoryReader, load_index_from_storage, VectorStoreIndex, StorageContext, ) from llama_index.vector_stores.faiss import FaissVectorStore from IPython.display import Markdown, display

下载数据¶

In [ ]:

!mkdir -p 'data/paul_graham/'
!wget 'https://raw.githubusercontent.com/run-llama/llama_index/main/docs/docs/examples/data/paul_graham/paul_graham_essay.txt' -O 'data/paul_graham/paul_graham_essay.txt'

!mkdir -p 'data/paul_graham/' !wget 'https://raw.githubusercontent.com/run-llama/llama_index/main/docs/docs/examples/data/paul_graham/paul_graham_essay.txt' -O 'data/paul_graham/paul_graham_essay.txt'

In [ ]:

# 加载文档
documents = SimpleDirectoryReader("./data/paul_graham/").load_data()

# 加载文档 documents = SimpleDirectoryReader("./data/paul_graham/").load_data()

In [ ]:

vector_store = FaissVectorStore(faiss_index=faiss_index)
storage_context = StorageContext.from_defaults(vector_store=vector_store)
index = VectorStoreIndex.from_documents(
    documents, storage_context=storage_context
)

vector_store = FaissVectorStore(faiss_index=faiss_index) storage_context = StorageContext.from_defaults(vector_store=vector_store) index = VectorStoreIndex.from_documents( documents, storage_context=storage_context )

In [ ]:

# 将索引保存到磁盘
index.storage_context.persist()

# 将索引保存到磁盘 index.storage_context.persist()

In [ ]:

# 从磁盘加载索引
vector_store = FaissVectorStore.from_persist_dir("./storage")
storage_context = StorageContext.from_defaults(
    vector_store=vector_store, persist_dir="./storage"
)
index = load_index_from_storage(storage_context=storage_context)

# 从磁盘加载索引 vector_store = FaissVectorStore.from_persist_dir("./storage") storage_context = StorageContext.from_defaults( vector_store=vector_store, persist_dir="./storage" ) index = load_index_from_storage(storage_context=storage_context)

查询索引¶

In [ ]:

# 将日志级别设置为DEBUG，以获得更详细的输出
query_engine = index.as_query_engine()
response = query_engine.query("作者在成长过程中做了什么？")

# 将日志级别设置为DEBUG，以获得更详细的输出 query_engine = index.as_query_engine() response = query_engine.query("作者在成长过程中做了什么？")

In [ ]:

display(Markdown(f"<b>{response}</b>"))

display(Markdown(f"{response}"))

In [ ]:

# 将日志级别设置为DEBUG，以获得更详细的输出
query_engine = index.as_query_engine()
response = query_engine.query(
    "作者在Y Combinator结束后做了什么？"
)

# 将日志级别设置为DEBUG，以获得更详细的输出 query_engine = index.as_query_engine() response = query_engine.query( "作者在Y Combinator结束后做了什么？" )

In [ ]:

display(Markdown(f"<b>{response}</b>"))

display(Markdown(f"{response}"))