如果您在colab上打开这个笔记本,您可能需要安装LlamaIndex 🦙。
In [ ]:
Copied!
%pip install llama-index-postprocessor-openvino-rerank
%pip install llama-index-embeddings-openvino
%pip install llama-index-postprocessor-openvino-rerank
%pip install llama-index-embeddings-openvino
In [ ]:
Copied!
!pip install llama-index
!pip install llama-index
下载数据¶
In [ ]:
Copied!
!mkdir -p 'data/paul_graham/'
!wget 'https://raw.githubusercontent.com/run-llama/llama_index/main/docs/docs/examples/data/paul_graham/paul_graham_essay.txt' -O 'data/paul_graham/paul_graham_essay.txt'
!mkdir -p 'data/paul_graham/'
!wget 'https://raw.githubusercontent.com/run-llama/llama_index/main/docs/docs/examples/data/paul_graham/paul_graham_essay.txt' -O 'data/paul_graham/paul_graham_essay.txt'
In [ ]:
Copied!
from llama_index.core import VectorStoreIndex和SimpleDirectoryReader# 加载文档documents = SimpleDirectoryReader("./data/paul_graham/").load_data()# 构建索引index = VectorStoreIndex.from_documents(documents=documents)
from llama_index.core import VectorStoreIndex和SimpleDirectoryReader# 加载文档documents = SimpleDirectoryReader("./data/paul_graham/").load_data()# 构建索引index = VectorStoreIndex.from_documents(documents=documents)
下载嵌入、重新排序模型和LLM¶
In [ ]:
Copied!
from llama_index.embeddings.huggingface_openvino import OpenVINOEmbedding
OpenVINOEmbedding.create_and_save_openvino_model(
"BAAI/bge-small-en-v1.5", "./embedding_ov"
)
from llama_index.embeddings.huggingface_openvino import OpenVINOEmbedding
OpenVINOEmbedding.create_and_save_openvino_model(
"BAAI/bge-small-en-v1.5", "./embedding_ov"
)
In [ ]:
Copied!
from llama_index.postprocessor.openvino_rerank import OpenVINORerank
OpenVINORerank.create_and_save_openvino_model(
"BAAI/bge-reranker-large", "./rerank_ov"
)
from llama_index.postprocessor.openvino_rerank import OpenVINORerank
OpenVINORerank.create_and_save_openvino_model(
"BAAI/bge-reranker-large", "./rerank_ov"
)
In [ ]:
Copied!
!optimum-cli export openvino --model HuggingFaceH4/zephyr-7b-beta --weight-format int4 llm_ov
!optimum-cli export openvino --model HuggingFaceH4/zephyr-7b-beta --weight-format int4 llm_ov
下载模型¶
In [ ]:
Copied!
from llama_index.llms.openvino import OpenVINOLLM
from llama_index.core import Settings
Settings.embed_model = OpenVINOEmbedding(folder_name="./embedding_ov")
Settings.llm = OpenVINOLLM(model_name="./llm_ov", tokenizer_name="./llm_ov")
ov_rerank = OpenVINORerank(model="./rerank_ov", top_n=2)
from llama_index.llms.openvino import OpenVINOLLM
from llama_index.core import Settings
Settings.embed_model = OpenVINOEmbedding(folder_name="./embedding_ov")
Settings.llm = OpenVINOLLM(model_name="./llm_ov", tokenizer_name="./llm_ov")
ov_rerank = OpenVINORerank(model="./rerank_ov", top_n=2)
检索前10个最相关的节点,然后使用OpenVINO重新排序进行过滤¶
In [ ]:
Copied!
import os
from llama_index.postprocessor.openvino_rerank import OpenVINORerank
ov_rerank = OpenVINORerank(top_n=2)
import os
from llama_index.postprocessor.openvino_rerank import OpenVINORerank
ov_rerank = OpenVINORerank(top_n=2)
In [ ]:
Copied!
index = VectorStoreIndex.from_documents(documents=documents)
index = VectorStoreIndex.from_documents(documents=documents)
In [ ]:
Copied!
query_engine = index.as_query_engine(
similarity_top_k=10,
node_postprocessors=[ov_rerank],
)
response = query_engine.query(
"What did Sam Altman do in this essay?",
)
query_engine = index.as_query_engine(
similarity_top_k=10,
node_postprocessors=[ov_rerank],
)
response = query_engine.query(
"What did Sam Altman do in this essay?",
)
In [ ]:
Copied!
print(response)
print(response)
Sam Altman was asked by the author, Paul Graham, to become the president of Y Combinator (YC), a startup accelerator. Initially, Sam declined the offer as he wanted to start a startup to make nuclear reactors. However, the author continued to persuade him, and in October 2013, Sam agreed to take over YC starting with the winter 2014 batch. The author then stepped back from running YC and focused on other activities, including painting and writing essays.
In [ ]:
Copied!
print(response.get_formatted_sources(length=200))
print(response.get_formatted_sources(length=200))
> Source (Doc id: ae4297fa-670c-403c-a355-6fffe7e16835): Why not organize a summer program where they'd start startups instead? We wouldn't feel guilty for being in a sense fake investors, because they would in a similar sense be fake founders. So while ... > Source (Doc id: c55eddb9-33f8-46bb-82a1-cb7fa0c7f5b6): This seemed strange advice, because YC was doing great. But if there was one thing rarer than Rtm offering advice, it was Rtm being wrong. So this set me thinking. It was true that on my current tr...
直接检索前两个最相似的节点¶
In [ ]:
Copied!
query_engine = index.as_query_engine(
similarity_top_k=2,
)
response = query_engine.query(
"What did Sam Altman do in this essay?",
)
query_engine = index.as_query_engine(
similarity_top_k=2,
)
response = query_engine.query(
"What did Sam Altman do in this essay?",
)
检索到的上下文不相关,回复是虚构的。
In [ ]:
Copied!
print(response)
print(response)
Sam Altman is mentioned in the essay as the person who was asked to become the president of Y Combinator. He initially declined the offer but later agreed to take over starting with the winter 2014 batch. The author also mentions that they left running Y Combinator more and more to Sam, partly so he could learn the job, and partly because they were focused on their mother, who had cancer and passed away in January 2014.
In [ ]:
Copied!
print(response.get_formatted_sources(length=200))
print(response.get_formatted_sources(length=200))
> Source (Doc id: c55eddb9-33f8-46bb-82a1-cb7fa0c7f5b6): This seemed strange advice, because YC was doing great. But if there was one thing rarer than Rtm offering advice, it was Rtm being wrong. So this set me thinking. It was true that on my current tr... > Source (Doc id: 6b2c335f-1390-4e92-9171-3ba5d24b3826): I knew that online essays would be a marginal medium at first. Socially they'd seem more like rants posted by nutjobs on their GeoCities sites than the genteel and beautifully typeset compositions ...
有关更多信息,请参考: