In [ ]:

<a href="https://colab.research.google.com/github/ulan-yisaev/llama_index/blob/main/docs/docs/examples/llm/deepinfra.ipynb" target="_parent"><img src="https://colab.research.google.com/assets/colab-badge.svg" alt="Open In Colab"/></a>
[![nbviewer](https://raw.githubusercontent.com/jupyter/design/master/logos/Badges/nbviewer_badge.svg)](https://nbviewer.org/github/aidoczh/llama_index_examples_zh/blob/main/examples/llm/deepinfra.ipynb)

[](https://nbviewer.org/github/aidoczh/llama_index_examples_zh/blob/main/examples/llm/deepinfra.ipynb)

DeepInfra¶

安装¶

首先，安装必要的包：

%pip install llama-index-llms-deepinfra

In [ ]:

%pip install llama-index-llms-deepinfra

%pip install llama-index-llms-deepinfra

初始化¶

使用您的API密钥和所需的参数设置DeepInfraLLM类：

In [ ]:

from llama_index.llms.deepinfra import DeepInfraLLM
import asyncio

llm = DeepInfraLLM(
    model="mistralai/Mixtral-8x22B-Instruct-v0.1",  # Default model name
    api_key="your-deepinfra-api-key",  # Replace with your DeepInfra API key
    temperature=0.5,
    max_tokens=50,
    additional_kwargs={"top_p": 0.9},
)

from llama_index.llms.deepinfra import DeepInfraLLM import asyncio llm = DeepInfraLLM( model="mistralai/Mixtral-8x22B-Instruct-v0.1", # Default model name api_key="your-deepinfra-api-key", # Replace with your DeepInfra API key temperature=0.5, max_tokens=50, additional_kwargs={"top_p": 0.9}, )

同步完成¶

使用complete方法同步生成文本完成。

In [ ]:

response = llm.complete("Hello World!")
print(response.text)

response = llm.complete("Hello World!") print(response.text)

同步流完成¶

使用stream_complete方法同步生成流式文本完成。

In [ ]:

content = ""
for completion in llm.stream_complete("Once upon a time"):
    content += completion.delta
    print(completion.delta, end="")

content = "" for completion in llm.stream_complete("Once upon a time"): content += completion.delta print(completion.delta, end="")

同步聊天¶

使用chat方法同步生成聊天回复：

In [ ]:

from llama_index.core.base.llms.types import ChatMessage

messages = [
    ChatMessage(role="user", content="Tell me a joke."),
]
chat_response = llm.chat(messages)
print(chat_response.message.content)

from llama_index.core.base.llms.types import ChatMessage messages = [ ChatMessage(role="user", content="Tell me a joke."), ] chat_response = llm.chat(messages) print(chat_response.message.content)

同步流式聊天¶

使用stream_chat方法同步生成流式聊天响应：

In [ ]:

messages = [
    ChatMessage(role="system", content="You are a helpful assistant."),
    ChatMessage(role="user", content="Tell me a story."),
]
content = ""
for chat_response in llm.stream_chat(messages):
    content += chat_response.message.delta
    print(chat_response.message.delta, end="")

messages = [ ChatMessage(role="system", content="You are a helpful assistant."), ChatMessage(role="user", content="Tell me a story."), ] content = "" for chat_response in llm.stream_chat(messages): content += chat_response.message.delta print(chat_response.message.delta, end="")

异步完成¶

使用acomplete方法异步生成文本完成。

In [ ]:

async def async_complete():
    response = await llm.acomplete("Hello Async World!")
    print(response.text)


asyncio.run(async_complete())

async def async_complete(): response = await llm.acomplete("Hello Async World!") print(response.text) asyncio.run(async_complete())

异步流完成¶

使用astream_complete方法异步生成流式文本完成。

In [ ]:

async def async_stream_complete():
    content = ""
    response = await llm.astream_complete("Once upon an async time")
    async for completion in response:
        content += completion.delta
        print(completion.delta, end="")


asyncio.run(async_stream_complete())

async def async_stream_complete(): content = "" response = await llm.astream_complete("Once upon an async time") async for completion in response: content += completion.delta print(completion.delta, end="") asyncio.run(async_stream_complete())

异步聊天¶

使用achat方法异步生成聊天回复：

In [ ]:

async def async_chat():
    messages = [
        ChatMessage(role="user", content="Tell me an async joke."),
    ]
    chat_response = await llm.achat(messages)
    print(chat_response.message.content)


asyncio.run(async_chat())

async def async_chat(): messages = [ ChatMessage(role="user", content="Tell me an async joke."), ] chat_response = await llm.achat(messages) print(chat_response.message.content) asyncio.run(async_chat())

异步流式聊天¶

使用astream_chat方法异步生成流式聊天响应：

In [ ]:

async def async_stream_chat():
    messages = [
        ChatMessage(role="system", content="You are a helpful assistant."),
        ChatMessage(role="user", content="Tell me an async story."),
    ]
    content = ""
    response = await llm.astream_chat(messages)
    async for chat_response in response:
        content += chat_response.message.delta
        print(chat_response.message.delta, end="")


asyncio.run(async_stream_chat())

async def async_stream_chat(): messages = [ ChatMessage(role="system", content="You are a helpful assistant."), ChatMessage(role="user", content="Tell me an async story."), ] content = "" response = await llm.astream_chat(messages) async for chat_response in response: content += chat_response.message.delta print(chat_response.message.delta, end="") asyncio.run(async_stream_chat())

---

如有任何问题或反馈，请通过邮件联系我们：feedback@deepinfra.com。