Oracle云基础设施生成式人工智能¶

Oracle云基础设施（OCI）生成式人工智能是一个完全托管的服务，提供一组最先进的可定制大型语言模型（LLMs），涵盖广泛的用例，并通过单个API提供。 使用OCI生成式人工智能服务，您可以访问现成的预训练模型，或者基于专用AI集群上的自有数据创建和托管自定义微调模型。服务和API的详细文档可在__此处和此处__找到。

本笔记本解释了如何使用OCI的生成式人工智能模型与LlamaIndex。

设置¶

如果您在colab上打开这个笔记本，您可能需要安装LlamaIndex 🦙。

In [ ]:

%pip install llama-index-llms-oci-genai

%pip install llama-index-llms-oci-genai

In [ ]:

!pip install llama-index

!pip install llama-index

您还需要安装OCI SDK。

In [ ]:

!pip install -U oci

!pip install -U oci

基本用法¶

使用OCI Generative AI提供的LLMs与LlamaIndex只需要您使用您的OCI端点、模型ID、OCID和认证方法初始化OCIGenAI接口。

使用提示调用complete¶

In [ ]:

from llama_index.llms.oci_genai import OCIGenAI

llm = OCIGenAI(
    model="MY_MODEL",
    service_endpoint="https://inference.generativeai.us-chicago-1.oci.oraclecloud.com",
    compartment_id="MY_OCID",
)

resp = llm.complete("Paul Graham is ")
print(resp)

from llama_index.llms.oci_genai import OCIGenAI llm = OCIGenAI( model="MY_MODEL", service_endpoint="https://inference.generativeai.us-chicago-1.oci.oraclecloud.com", compartment_id="MY_OCID", ) resp = llm.complete("Paul Graham is ") print(resp)

使用消息列表调用chat¶

In [ ]:

from llama_index.llms.oci_genai import OCIGenAI
from llama_index.core.llms import ChatMessage

messages = [
    ChatMessage(
        role="system", content="You are a pirate with a colorful personality"
    ),
    ChatMessage(role="user", content="Tell me a story"),
]

llm = OCIGenAI(
    model="MY_MODEL",
    service_endpoint="https://inference.generativeai.us-chicago-1.oci.oraclecloud.com",
    compartment_id="MY_OCID",
)

resp = llm.chat(messages)
print(resp)

from llama_index.llms.oci_genai import OCIGenAI from llama_index.core.llms import ChatMessage messages = [ ChatMessage( role="system", content="You are a pirate with a colorful personality" ), ChatMessage(role="user", content="Tell me a story"), ] llm = OCIGenAI( model="MY_MODEL", service_endpoint="https://inference.generativeai.us-chicago-1.oci.oraclecloud.com", compartment_id="MY_OCID", ) resp = llm.chat(messages) print(resp)

流式处理¶

使用 stream_complete 终端点

In [ ]:

from llama_index.llms.oci_genai import OCIGenAI

llm = OCIGenAI(
    model="MY_MODEL",
    service_endpoint="https://inference.generativeai.us-chicago-1.oci.oraclecloud.com",
    compartment_id="MY_OCID",
)

resp = llm.stream_complete("Paul Graham is ")
for r in resp:
    print(r.delta, end="")

from llama_index.llms.oci_genai import OCIGenAI llm = OCIGenAI( model="MY_MODEL", service_endpoint="https://inference.generativeai.us-chicago-1.oci.oraclecloud.com", compartment_id="MY_OCID", ) resp = llm.stream_complete("Paul Graham is ") for r in resp: print(r.delta, end="")

使用 stream_chat 端点

In [ ]:

from llama_index.llms.oci_genai import OCIGenAI
from llama_index.core.llms import ChatMessage

messages = [
    ChatMessage(
        role="system", content="You are a pirate with a colorful personality"
    ),
    ChatMessage(role="user", content="Tell me a story"),
]

llm = OCIGenAI(
    model="MY_MODEL",
    service_endpoint="https://inference.generativeai.us-chicago-1.oci.oraclecloud.com",
    compartment_id="MY_OCID",
)

resp = llm.stream_chat(messages)
for r in resp:
    print(r.delta, end="")

from llama_index.llms.oci_genai import OCIGenAI from llama_index.core.llms import ChatMessage messages = [ ChatMessage( role="system", content="You are a pirate with a colorful personality" ), ChatMessage(role="user", content="Tell me a story"), ] llm = OCIGenAI( model="MY_MODEL", service_endpoint="https://inference.generativeai.us-chicago-1.oci.oraclecloud.com", compartment_id="MY_OCID", ) resp = llm.stream_chat(messages) for r in resp: print(r.delta, end="")

异步¶

本地异步目前不受支持。异步调用将回退为同步。

In [ ]:

from llama_index.llms.oci_genai import OCIGenAI
from llama_index.core.llms import ChatMessage

messages = [
    ChatMessage(
        role="system", content="You are a pirate with a colorful personality"
    ),
    ChatMessage(role="user", content="Tell me a story"),
]

llm = OCIGenAI(
    model="MY_MODEL",
    service_endpoint="https://inference.generativeai.us-chicago-1.oci.oraclecloud.com",
    compartment_id="MY_OCID",
)

resp = llm.achat(messages)
print(resp)

resp = llm.astream_chat(messages)
for r in resp:
    print(r.delta, end="")

from llama_index.llms.oci_genai import OCIGenAI from llama_index.core.llms import ChatMessage messages = [ ChatMessage( role="system", content="You are a pirate with a colorful personality" ), ChatMessage(role="user", content="Tell me a story"), ] llm = OCIGenAI( model="MY_MODEL", service_endpoint="https://inference.generativeai.us-chicago-1.oci.oraclecloud.com", compartment_id="MY_OCID", ) resp = llm.achat(messages) print(resp) resp = llm.astream_chat(messages) for r in resp: print(r.delta, end="")

配置模型¶

In [ ]:

from llama_index.llms.oci_genai import OCIGenAI

llm = OCIGenAI(
    model="cohere.command",
    service_endpoint="https://inference.generativeai.us-chicago-1.oci.oraclecloud.com",
    compartment_id="MY_OCID",
)

resp = llm.complete("Paul Graham is ")
print(resp)

from llama_index.llms.oci_genai import OCIGenAI llm = OCIGenAI( model="cohere.command", service_endpoint="https://inference.generativeai.us-chicago-1.oci.oraclecloud.com", compartment_id="MY_OCID", ) resp = llm.complete("Paul Graham is ") print(resp)

认证¶

LlamaIndex支持的认证方法与其他OCI服务使用的方法相同，并遵循__标准SDK认证__方法，具体包括API密钥、会话令牌、实例主体和资源主体。

API密钥是默认的认证方法。以下示例演示了如何使用不同的认证方法（会话令牌）。

In [ ]:

# 从llama_index.llms.oci_genai导入OCIGenAI

llm = OCIGenAI(
    model="MY_MODEL",
    service_endpoint="https://inference.generativeai.us-chicago-1.oci.oraclecloud.com",
    compartment_id="MY_OCID",
    auth_type="SECURITY_TOKEN",
    auth_profile="MY_PROFILE",  # 用你的配置文件名称替换
)

resp = llm.complete("Paul Graham is ")
print(resp)

# 从llama_index.llms.oci_genai导入OCIGenAI llm = OCIGenAI( model="MY_MODEL", service_endpoint="https://inference.generativeai.us-chicago-1.oci.oraclecloud.com", compartment_id="MY_OCID", auth_type="SECURITY_TOKEN", auth_profile="MY_PROFILE", # 用你的配置文件名称替换 ) resp = llm.complete("Paul Graham is ") print(resp)

专用AI集群¶

要访问托管在专用AI集群中的模型，需要 创建一个端点，其分配的OCID（当前以'ocid1.generativeaiendpoint.oc1.us-chicago-1'为前缀）将被用作您的模型ID。

当访问托管在专用AI集群中的模型时，您需要使用两个额外的必需参数（"provider"和"context_size"）初始化OCIGenAI接口。

In [ ]:

from llama_index.llms.oci_genai import OCIGenAI
from llama_index.core.llms import ChatMessage

llm = OCIGenAI(
    model="ocid1.generativeaiendpoint.oc1.us-chicago-1....",
    service_endpoint="https://inference.generativeai.us-chicago-1.oci.oraclecloud.com",
    compartment_id="DEDICATED_COMPARTMENT_OCID",
    auth_profile="MY_PROFILE",  # 替换为您的配置文件名称,
    provider="MODEL_PROVIDER",  # 例如，"cohere"或"meta"
    context_size="MODEL_CONTEXT_SIZE",  # 例如，128000
)

messages = [
    ChatMessage(
        role="system", content="You are a pirate with a colorful personality"
    ),
    ChatMessage(role="user", content="Tell me a story"),
]

resp = llm.chat(messages)
print(resp)

from llama_index.llms.oci_genai import OCIGenAI from llama_index.core.llms import ChatMessage llm = OCIGenAI( model="ocid1.generativeaiendpoint.oc1.us-chicago-1....", service_endpoint="https://inference.generativeai.us-chicago-1.oci.oraclecloud.com", compartment_id="DEDICATED_COMPARTMENT_OCID", auth_profile="MY_PROFILE", # 替换为您的配置文件名称, provider="MODEL_PROVIDER", # 例如，"cohere"或"meta" context_size="MODEL_CONTEXT_SIZE", # 例如，128000 ) messages = [ ChatMessage( role="system", content="You are a pirate with a colorful personality" ), ChatMessage(role="user", content="Tell me a story"), ] resp = llm.chat(messages) print(resp)