Trubrics
Trubrics 是一个LLM用户分析平台，允许您收集、分析和管理用户对AI模型的提示和反馈。
查看Trubrics 仓库以获取更多关于
Trubrics的信息。
在本指南中，我们将介绍如何设置TrubricsCallbackHandler。
安装与设置
%pip install --upgrade --quiet trubrics langchain langchain-community
获取Trubrics凭证
如果您没有Trubrics账户，请在这里创建一个。在本教程中，我们将使用账户创建时构建的default项目。
现在将您的凭据设置为环境变量：
import os
os.environ["TRUBRICS_EMAIL"] = "***@***"
os.environ["TRUBRICS_PASSWORD"] = "***"
from langchain_community.callbacks.trubrics_callback import TrubricsCallbackHandler
API Reference:TrubricsCallbackHandler
用法
TrubricsCallbackHandler 可以接收各种可选参数。请参阅 这里 了解可以传递给 Trubrics 提示的 kwargs。
class TrubricsCallbackHandler(BaseCallbackHandler):
"""
Callback handler for Trubrics.
Args:
project: a trubrics project, default project is "default"
email: a trubrics account email, can equally be set in env variables
password: a trubrics account password, can equally be set in env variables
**kwargs: all other kwargs are parsed and set to trubrics prompt variables, or added to the `metadata` dict
"""
示例
以下是两个示例，展示了如何在Langchain的LLMs或Chat Models中使用TrubricsCallbackHandler。我们将使用OpenAI模型，因此请在此处设置您的OPENAI_API_KEY密钥：
os.environ["OPENAI_API_KEY"] = "sk-***"
1. 使用LLM
from langchain_openai import OpenAI
API Reference:OpenAI
llm = OpenAI(callbacks=[TrubricsCallbackHandler()])
[32m2023-09-26 11:30:02.149[0m | [1mINFO [0m | [36mtrubrics.platform.auth[0m:[36mget_trubrics_auth_token[0m:[36m61[0m - [1mUser jeff.kayne@trubrics.com has been authenticated.[0m
res = llm.generate(["Tell me a joke", "Write me a poem"])
[32m2023-09-26 11:30:07.760[0m | [1mINFO [0m | [36mtrubrics.platform[0m:[36mlog_prompt[0m:[36m102[0m - [1mUser prompt saved to Trubrics.[0m
[32m2023-09-26 11:30:08.042[0m | [1mINFO [0m | [36mtrubrics.platform[0m:[36mlog_prompt[0m:[36m102[0m - [1mUser prompt saved to Trubrics.[0m
print("--> GPT's joke: ", res.generations[0][0].text)
print()
print("--> GPT's poem: ", res.generations[1][0].text)
--> GPT's joke:
Q: What did the fish say when it hit the wall?
A: Dam!
--> GPT's poem:
A Poem of Reflection
I stand here in the night,
The stars above me filling my sight.
I feel such a deep connection,
To the world and all its perfection.
A moment of clarity,
The calmness in the air so serene.
My mind is filled with peace,
And I am released.
The past and the present,
My thoughts create a pleasant sentiment.
My heart is full of joy,
My soul soars like a toy.
I reflect on my life,
And the choices I have made.
My struggles and my strife,
The lessons I have paid.
The future is a mystery,
But I am ready to take the leap.
I am ready to take the lead,
And to create my own destiny.
2. 使用聊天模型
from langchain_core.messages import HumanMessage, SystemMessage
from langchain_openai import ChatOpenAI
chat_llm = ChatOpenAI(
callbacks=[
TrubricsCallbackHandler(
project="default",
tags=["chat model"],
user_id="user-id-1234",
some_metadata={"hello": [1, 2]},
)
]
)
chat_res = chat_llm.invoke(
[
SystemMessage(content="Every answer of yours must be about OpenAI."),
HumanMessage(content="Tell me a joke"),
]
)
[32m2023-09-26 11:30:10.550[0m | [1mINFO [0m | [36mtrubrics.platform[0m:[36mlog_prompt[0m:[36m102[0m - [1mUser prompt saved to Trubrics.[0m
print(chat_res.content)
Why did the OpenAI computer go to the party?
Because it wanted to meet its AI friends and have a byte of fun!