开放式路由器¶
OpenRouter提供了一个标准化的API,以最优惠的价格访问许多LLM。您可以在他们的主页上找到更多信息。
如果您在colab上打开这个笔记本,您可能需要安装LlamaIndex 🦙。
In [ ]:
Copied!
%pip install llama-index-llms-openrouter
%pip install llama-index-llms-openrouter
In [ ]:
Copied!
!pip install llama-index
!pip install llama-index
In [ ]:
Copied!
from llama_index.llms.openrouter import OpenRouter
from llama_index.core.llms import ChatMessage
from llama_index.llms.openrouter import OpenRouter
from llama_index.core.llms import ChatMessage
使用chat
与ChatMessage列表¶
您需要设置环境变量OPENROUTER_API_KEY
或在类构造函数中设置api_key
In [ ]:
Copied!
# 导入 os 模块# 设置环境变量 'OPENROUTER_API_KEY' 为 '<your-api-key>'llm = OpenRouter( api_key="<your-api-key>", max_tokens=256, context_window=4096, model="gryphe/mythomax-l2-13b",)
# 导入 os 模块# 设置环境变量 'OPENROUTER_API_KEY' 为 ''llm = OpenRouter( api_key="", max_tokens=256, context_window=4096, model="gryphe/mythomax-l2-13b",)
In [ ]:
Copied!
message = ChatMessage(role="user", content="Tell me a joke")
resp = llm.chat([message])
print(resp)
message = ChatMessage(role="user", content="Tell me a joke")
resp = llm.chat([message])
print(resp)
assistant: Why did the tomato turn red? Because it saw the salad dressing!
流式处理¶
In [ ]:
Copied!
message = ChatMessage(role="user", content="Tell me a story in 250 words")
resp = llm.stream_chat([message])
for r in resp:
print(r.delta, end="")
message = ChatMessage(role="user", content="Tell me a story in 250 words")
resp = llm.stream_chat([message])
for r in resp:
print(r.delta, end="")
Once upon a time, there was a young girl named Maria who lived in a small village surrounded by lush green forests. Maria was a kind and gentle soul, loved by everyone in the village. She spent most of her days exploring the forests, discovering new species of plants and animals, and helping the villagers with their daily chores. One day, while Maria was out on a walk, she stumbled upon a hidden path she had never seen before. The path was overgrown with weeds and vines, but something about it called to her. She decided to follow it, and it led her deeper and deeper into the forest. As she walked, the trees grew taller and the air grew colder. Maria began to feel a sense of unease, but she was determined to see where the path led. Finally, she came to a clearing, and in the center of it stood an enormous tree, its trunk as wide as a house. Maria approached the tree and saw that it was covered in strange symbols. She reached out to touch one of the symbols, and suddenly, the tree began to glow. The glow grew brighter and brighter, until Maria
使用Prompt调用complete
¶
In [ ]:
Copied!
resp = llm.complete("Tell me a joke")
print(resp)
resp = llm.complete("Tell me a joke")
print(resp)
Sure, here's a joke for you: Why couldn't the bicycle stand up by itself? Because it was two-tired! I hope that brought a smile to your face!
In [ ]:
Copied!
resp = llm.stream_complete("Tell me a story in 250 words")
for r in resp:
print(r.delta, end="")
resp = llm.stream_complete("Tell me a story in 250 words")
for r in resp:
print(r.delta, end="")
Once upon a time, there was a young girl named Maria. She lived in a small village surrounded by lush green forests and sparkling rivers. Maria was a kind and gentle soul, loved by everyone in the village. She spent her days helping her parents with their farm work and exploring the surrounding nature. One day, while wandering in the forest, Maria stumbled upon a hidden path she had never seen before. She decided to follow it, and it led her to a beautiful meadow filled with wildflowers. In the center of the meadow, she found a small pond, where she saw her own reflection in the water. As she gazed into the pond, Maria saw a figure approaching her. It was a wise old woman, who introduced herself as the guardian of the meadow. The old woman told Maria that she had been chosen to receive a special gift, one that would bring her great joy and happiness. The old woman then presented Maria with a small, delicate flower. She told her that this flower had the power to heal any wound, both physical and emotional. Maria was amazed and grateful, and she promised to use the flower wisely.
模型配置¶
In [ ]:
Copied!
# 查看https://openrouter.ai/models上的视图选项# 此示例使用Mistral的MoE,Mixtral:llm = OpenRouter(model="mistralai/mixtral-8x7b-instruct")
# 查看https://openrouter.ai/models上的视图选项# 此示例使用Mistral的MoE,Mixtral:llm = OpenRouter(model="mistralai/mixtral-8x7b-instruct")
In [ ]:
Copied!
resp = llm.complete("Write a story about a dragon who can code in Rust")
print(resp)
resp = llm.complete("Write a story about a dragon who can code in Rust")
print(resp)