Skip to main content
Open In ColabOpen on GitHub

SambaStudio

SambaNova's Sambastudio 是一个平台，允许您训练、运行批量推理作业，并部署在线推理端点以运行您自己微调的开源模型。

caution

您当前所在的页面记录了SambaStudio模型作为文本完成模型的使用。我们建议您使用聊天完成模型。

你可能在寻找 SambaStudio Chat Models 。

概述​

集成详情​

类包本地可序列化JS支持包下载包最新
SambaStudiolangchain_community❌测试版❌PyPI - 下载量PyPI - 版本

本示例介绍了如何使用LangChain与SambaStudio模型进行交互

设置​

凭证​

部署模型需要一个SambaStudio环境。获取更多信息，请访问 sambanova.ai/products/enterprise-ai-platform-sambanova-suite

你需要部署一个端点并设置SAMBASTUDIO_URL和SAMBASTUDIO_API_KEY环境变量：

import getpass
import os

if "SAMBASTUDIO_URL" not in os.environ:
    os.environ["SAMBASTUDIO_URL"] = getpass.getpass()
if "SAMBASTUDIO_API_KEY" not in os.environ:
    os.environ["SAMBASTUDIO_API_KEY"] = getpass.getpass()

安装​

集成位于langchain-community包中。我们还需要安装sseclient-py包，这是运行流式预测所必需的。

%pip install --quiet -U langchain-community sseclient-py

实例化​

from langchain_community.llms.sambanova import SambaStudio

llm = SambaStudio(
    model_kwargs={
        "do_sample": True,
        "max_tokens": 1024,
        "temperature": 0.01,
        "process_prompt": True,  # set if using CoE endpoints
        "model": "Meta-Llama-3-70B-Instruct-4096",  # set if using CoE endpoints
        # "repetition_penalty":  1.0,
        # "top_k": 50,
        # "top_logprobs": 0,
        # "top_p": 1.0
    },
)
API Reference:SambaStudio

调用​

现在我们可以实例化我们的模型对象并生成聊天完成：

input_text = "Why should I use open source models?"

completion = llm.invoke(input_text)
completion
"Using open source models can have numerous benefits. Here are some reasons why you should consider using open source models:\n\n1. **Cost-effective**: Open source models are often free to use, modify, and distribute, which can significantly reduce costs compared to proprietary models.\n2. **Customizability**: Open source models can be modified to fit your specific needs, allowing you to tailor the model to your project's requirements.\n3. **Transparency**: Open source models provide complete transparency into the model's architecture, training data, and algorithms, which can be essential for understanding how the model works and identifying potential biases.\n4. **Community involvement**: Open source models are often maintained by a community of developers, researchers, and users, which can lead to faster bug fixes, new feature additions, and improved performance.\n5. **Flexibility**: Open source models can be used in a variety of applications, from research to production, and can be easily integrated into different workflows and systems.\n6. **Auditability**: With open source models, you can audit the model's performance, data, and algorithms, which is critical in regulated industries or when working with sensitive data.\n7. **No vendor lock-in**: By using open source models, you're not tied to a specific vendor or proprietary technology, giving you more freedom to switch or modify your approach as needed.\n8. **Improved security**: Open source models can be reviewed and audited by the community, which can help identify and fix security vulnerabilities more quickly.\n9. **Access to cutting-edge research**: Open source models can provide access to the latest research and advancements in AI and machine learning, allowing you to leverage the work of experts in the field.\n10. **Ethical considerations**: By using open source models, you can ensure that your AI systems are transparent, explainable, and fair, which is essential for building trust in AI applications.\n11. **Reduced risk of bias**: Open source models can help reduce the risk of bias by providing transparency into the model's development, training data, and algorithms.\n12. **Faster development**: Open source models can accelerate your development process by providing pre-trained models, datasets, and tools that can be easily integrated into your project.\n13. **Improved collaboration**: Open source models can facilitate collaboration among researchers, developers, and organizations, leading to faster progress and innovation in AI and machine learning.\n14. **Access to large datasets**: Open source models can provide access to large datasets, which can be essential for training and testing AI models.\n15. **Compliance with regulations**: In some cases, using open source models can help ensure compliance with regulations, such as GDPR, HIPAA, or CCPA, which require transparency and explainability in AI systems.\n\nOverall, using open source models can provide numerous benefits, from cost savings to improved transparency and customizability. By leveraging open source models, you can accelerate your AI and machine learning projects while ensuring that your systems are transparent, explainable, and fair."
# Streaming response
for chunk in llm.stream("Why should I use open source models?"):
    print(chunk, end="", flush=True)
Using open source models can have numerous benefits. Here are some reasons why you should consider using open source models:

1. **Cost-effective**: Open source models are often free to use, modify, and distribute, which can significantly reduce costs compared to proprietary models.
2. **Customizability**: Open source models can be modified to fit your specific needs, allowing you to tailor the model to your project's requirements.
3. **Transparency**: Open source models provide complete transparency into the model's architecture, training data, and algorithms, which can be essential for understanding how the model works and identifying potential biases.
4. **Community involvement**: Open source models are often maintained by a community of developers, researchers, and users, which can lead to faster bug fixes, new feature additions, and improved performance.
5. **Flexibility**: Open source models can be used in a variety of applications, from research to production, and can be easily integrated into different workflows and systems.
6. **Auditability**: With open source models, you can audit the model's performance, data, and algorithms, which is critical in regulated industries or when working with sensitive data.
7. **No vendor lock-in**: By using open source models, you're not tied to a specific vendor or proprietary technology, giving you more freedom to switch or modify your approach as needed.
8. **Improved security**: Open source models can be reviewed and audited by the community, which can help identify and fix security vulnerabilities more quickly.
9. **Access to cutting-edge research**: Open source models can provide access to the latest research and advancements in AI and machine learning, allowing you to leverage the work of experts in the field.
10. **Ethical considerations**: By using open source models, you can ensure that your AI systems are transparent, explainable, and fair, which is essential for building trust in AI applications.
11. **Reduced risk of bias**: Open source models can help reduce the risk of bias by providing transparency into the model's development, training data, and algorithms.
12. **Faster development**: Open source models can accelerate your development process by providing pre-trained models, datasets, and tools that can be easily integrated into your project.
13. **Improved collaboration**: Open source models can facilitate collaboration among researchers, developers, and organizations, leading to faster progress and innovation in AI and machine learning.
14. **Access to large datasets**: Open source models can provide access to large datasets, which can be essential for training and testing AI models.
15. **Compliance with regulations**: In some cases, using open source models can help ensure compliance with regulations, such as GDPR, HIPAA, or CCPA, which require transparency and explainability in AI systems.

Overall, using open source models can provide numerous benefits, from cost savings to improved transparency and customizability. By leveraging open source models, you can accelerate your AI and machine learning projects while ensuring that your systems are transparent, explainable, and fair.

链式调用​

from langchain_core.prompts import PromptTemplate

prompt = PromptTemplate.from_template("How to say {input} in {output_language}:\n")

chain = prompt | llm
chain.invoke(
    {
        "output_language": "German",
        "input": "I love programming.",
    }
)
API Reference:PromptTemplate
'In German, you can say:\n\n"Ich liebe das Programmieren."\n\nHere\'s a breakdown of the sentence:\n\n* "Ich" means "I"\n* "liebe" is the verb "to love" in the present tense, first person singular (I love)\n* "das" is the definite article "the"\n* "Programmieren" is the noun "programming"\n\nSo, "Ich liebe das Programmieren" literally means "I love the programming".\n\nIf you want to make it sound more casual, you can say:\n\n"Ich liebe\'s Programmieren."\n\nThe apostrophe in "liebe\'s" is a contraction of "liebe es", which is a more informal way of saying "I love it".\n\nAlternatively, you can also say:\n\n"Programmieren ist meine Leidenschaft."\n\nThis sentence means "Programming is my passion". Here, "Programmieren" is the subject, "ist" is the verb "to be" in the present tense, and "meine Leidenschaft" means "my passion".'

API 参考​

有关所有SambaStudio llm功能和配置的详细文档，请参阅API参考：https://python.langchain.com/api_reference/community/llms/langchain_community.llms.sambanova.SambaStudio.html

这个页面有帮助吗？