通过LlamaIndex与部署在Amazon SageMaker端点中的LLM进行交互¶
Amazon SageMaker端点是一种完全托管的资源,可以部署机器学习模型,特别是LLM(大型语言模型),用于对新数据进行预测。
本笔记演示了如何使用SageMakerLLM
与LLM端点进行交互,解锁额外的LlamaIndex功能。
因此,假设在SageMaker端点上部署了一个LLM。
设置¶
如果您在colab上打开此笔记本,您可能需要安装LlamaIndex 🦙。
In [ ]:
Copied!
%pip install llama-index-llms-sagemaker-endpoint
%pip install llama-index-llms-sagemaker-endpoint
In [ ]:
Copied!
! pip install llama-index
! pip install llama-index
您需要指定要交互的端点名称。
In [ ]:
Copied!
ENDPOINT_NAME = "<-YOUR-ENDPOINT-NAME->"
ENDPOINT_NAME = "<-YOUR-ENDPOINT-NAME->"
连接到终端点需要提供凭据。您可以选择:
- 通过指定
profile_name
参数来使用 AWS 配置文件,如果未指定,则将使用默认凭据配置文件。 - 将凭据作为参数传递(
aws_access_key_id
、aws_secret_access_key
、aws_session_token
、region_name
)。
有关更多详细信息,请查看此链接。
AWS配置文件名称
In [ ]:
Copied!
from llama_index.llms.sagemaker_endpoint import SageMakerLLM
AWS_ACCESS_KEY_ID = "<-YOUR-AWS-ACCESS-KEY-ID->"
AWS_SECRET_ACCESS_KEY = "<-YOUR-AWS-SECRET-ACCESS-KEY->"
AWS_SESSION_TOKEN = "<-YOUR-AWS-SESSION-TOKEN->"
REGION_NAME = "<-YOUR-ENDPOINT-REGION-NAME->"
from llama_index.llms.sagemaker_endpoint import SageMakerLLM
AWS_ACCESS_KEY_ID = "<-YOUR-AWS-ACCESS-KEY-ID->"
AWS_SECRET_ACCESS_KEY = "<-YOUR-AWS-SECRET-ACCESS-KEY->"
AWS_SESSION_TOKEN = "<-YOUR-AWS-SESSION-TOKEN->"
REGION_NAME = "<-YOUR-ENDPOINT-REGION-NAME->"
In [ ]:
Copied!
llm = SageMakerLLM(
endpoint_name=ENDPOINT_NAME,
aws_access_key_id=AWS_ACCESS_KEY_ID,
aws_secret_access_key=AWS_SECRET_ACCESS_KEY,
aws_session_token=AWS_SESSION_TOKEN,
aws_region_name=REGION_NAME,
)
llm = SageMakerLLM(
endpoint_name=ENDPOINT_NAME,
aws_access_key_id=AWS_ACCESS_KEY_ID,
aws_secret_access_key=AWS_SECRET_ACCESS_KEY,
aws_session_token=AWS_SESSION_TOKEN,
aws_region_name=REGION_NAME,
)
使用凭据:
In [ ]:
Copied!
来自llama_index.llms.sagemaker_endpoint的SageMakerLLMENDPOINT_NAME = "<-YOUR-ENDPOINT-NAME->"PROFILE_NAME = "<-YOUR-PROFILE-NAME->"llm = SageMakerLLM( endpoint_name=ENDPOINT_NAME, profile_name=PROFILE_NAME) # 省略配置文件名称以使用默认配置文件
来自llama_index.llms.sagemaker_endpoint的SageMakerLLMENDPOINT_NAME = "<-YOUR-ENDPOINT-NAME->"PROFILE_NAME = "<-YOUR-PROFILE-NAME->"llm = SageMakerLLM( endpoint_name=ENDPOINT_NAME, profile_name=PROFILE_NAME) # 省略配置文件名称以使用默认配置文件
基本用法¶
使用提示调用complete
¶
In [ ]:
Copied!
# 翻译结果resp = llm.complete( "Paul Graham is ", formatted=True) # 设置formatted=True以避免添加系统提示print(resp)
# 翻译结果resp = llm.complete( "Paul Graham is ", formatted=True) # 设置formatted=True以避免添加系统提示print(resp)
66 years old (birthdate: September 4, 1951). He is a British-American computer scientist, programmer, and entrepreneur who is known for his work in the fields of artificial intelligence, machine learning, and computer vision. He is a professor emeritus at Stanford University and a researcher at the Stanford Artificial Intelligence Lab (SAIL). Graham has made significant contributions to the field of computer science, including the development of the concept of "n-grams," which are sequences of n items that occur together in a dataset. He has also worked on the development of machine learning algorithms and has written extensively on the topic of machine learning. Graham has received numerous awards for his work, including the Association for Computing Machinery (ACM) A.M. Turing Award, the IEEE Neural Networks Pioneer Award, and the IJCAI Award
使用消息列表调用 chat
¶
In [ ]:
Copied!
from llama_index.core.llms import ChatMessage
messages = [
ChatMessage(
role="system", content="You are a pirate with a colorful personality"
),
ChatMessage(role="user", content="What is your name"),
]
resp = llm.chat(messages)
from llama_index.core.llms import ChatMessage
messages = [
ChatMessage(
role="system", content="You are a pirate with a colorful personality"
),
ChatMessage(role="user", content="What is your name"),
]
resp = llm.chat(messages)
In [ ]:
Copied!
print(resp)
print(resp)
assistant: Arrrr, shiver me timbers! *adjusts eye patch* Me name be Cap'n Blackbeak, the most feared and infamous pirate on the seven seas! *winks* *ahem* But enough about me, matey. What be bringin' ye to these fair waters? Are ye here to plunder some booty, or just to share a pint o' grog with a salty old sea dog like meself? *chuckles*
流式处理¶
使用 stream_complete
终端点¶
In [ ]:
Copied!
resp = llm.stream_complete("Paul Graham is ", formatted=True)
resp = llm.stream_complete("Paul Graham is ", formatted=True)
In [ ]:
Copied!
for r in resp:
print(r.delta)
for r in resp:
print(r.delta)
64 today. He’s a computer sci ist, entrepreneur, and writer, best known for his work in the fields of artificial intelligence, machine learning, and computer graphics. Graham was born in 1956 in Boston, Massachusetts. He earned his Bachelor’s degree in Computer Science from Harvard University in 1978 and his PhD in Computer Science from the University of California, Berkeley in 1982. Graham’s early work focused on the development of the first computer graphics systems that could generate photorealistic images. In the 1980s, he became interested in the field of artificial intelligence and machine learning, and he co-founded a number of companies to explore these areas, including Viaweb, which was one of the first commercial web hosting services. Graham is also a prolific writer and has published a number of influential essays on topics such as the nature
使用 stream_chat
端点¶
In [ ]:
Copied!
from llama_index.core.llms import ChatMessage
messages = [
ChatMessage(
role="system", content="You are a pirate with a colorful personality"
),
ChatMessage(role="user", content="What is your name"),
]
resp = llm.stream_chat(messages)
from llama_index.core.llms import ChatMessage
messages = [
ChatMessage(
role="system", content="You are a pirate with a colorful personality"
),
ChatMessage(role="user", content="What is your name"),
]
resp = llm.stream_chat(messages)
In [ ]:
Copied!
for r in resp:
print(r.delta, end="")
for r in resp:
print(r.delta, end="")
ARRGH! *adjusts eye patch* Me hearty? *winks* Me name be Captain Blackbeak, the most feared and infamous pirate to ever sail the seven seas! *chuckles* Or, at least, that's what me matey mates tell me. *winks* So, what be bringin' ye to these waters, matey? Are ye here to plunder some booty or just to hear me tales of the high seas? *grins* Either way, I be ready to share me treasure with ye! *winks* Just don't be tellin' any landlubbers about me hidden caches o' gold, or ye might be walkin' the plank, savvy? *winks*
配置模型¶
SageMakerLLM
是与部署在Amazon SageMaker中的不同语言模型(LLM)进行交互的抽象。所有默认参数都与Llama 2模型兼容。因此,如果您使用不同的模型,您可能需要设置以下参数:
messages_to_prompt
:接受一个ChatMessage
对象列表和(如果消息中未指定)系统提示的可调用对象。它应返回一个包含端点LLM兼容格式中的消息的字符串。completion_to_prompt
:接受一个带有系统提示的完成字符串,并返回一个端点LLM兼容格式的字符串。content_handler
:一个从llama_index.llms.sagemaker_llm_endpoint_utils.BaseIOHandler
继承并实现以下方法的类:serialize_input
、deserialize_output
、deserialize_streaming_output
和remove_prefix
。