如何处理速率限制
当你反复调用 OpenAI API 时，可能会遇到 429: 'Too Many Requests' 或
RateLimitError 的错误信息。这些错误信息是由于超过 API 的速率限制所致。
本指南分享了一些避免和处理速率限制错误的技巧。
要查看一个用于限制并行请求以避免速率限制错误的示例脚本，请参见 api_request_parallel_processor.py。
速率限制存在的原因
速率限制是 API 的常见做法，它们的设置有几个不同的原因。
- 首先，它们有助于防止对 API 的滥用或误用。例如，恶意行为者可能会通过大量请求来淹没 API，试图使其过载或导致服务中断。通过设置速率限制，OpenAI 可以防止这种活动。
- 其次，速率限制有助于确保每个人都能公平地使用 API。如果某个人或某个组织发出了过多的请求，可能会拖慢整个 API 的速度。通过限制单个用户的请求数量，OpenAI 确保每个人都有机会在不遇到减速的情况下使用 API。
- 最后，速率限制可以帮助 OpenAI 管理其基础设施的整体负载。如果对 API 的请求量急剧增加，可能会给服务器带来负担并导致性能问题。通过设置速率限制，OpenAI 可以帮助保持所有用户的体验平稳和一致。
尽管遇到速率限制可能会令人沮丧，但速率限制的存在是为了保护 API 用户的可靠操作。
默认速率限制
您的速率限制和消费限制（配额）会根据多种因素自动调整。随着您对OpenAI API的使用量增加并成功支付账单，我们会自动提高您的使用等级。您可以使用以下资源找到有关速率限制的具体信息。
其他速率限制资源
了解更多关于 OpenAI 速率限制的信息，请参阅以下资源：
请求增加速率限制
如果您希望增加您所在组织的速率限制，请访问您的 速率限制设置页面 以了解如何提高您的使用层级。
import openai
import os
client = openai.OpenAI(api_key=os.environ.get("OPENAI_API_KEY", "<your OpenAI API key if not set as env var>"))
示例速率限制错误
当API请求发送得太快时，将会发生速率限制错误。如果使用OpenAI Python库，错 误信息看起来会像这样：
RateLimitError: Rate limit reached for default-codex in organization org-{id} on requests per min. Limit: 20.000000 / min. Current: 24.000000 / min. Contact support@openai.com if you continue to have issues or if you’d like to request an increase.
下面是触发速率限制错误的示例代码。
# 在循环中请求大量完成
for _ in range(100):
client.chat.completions.create(
model="gpt-3.5-turbo",
messages=[{"role": "user", "content": "Hello"}],
max_tokens=10,
)
如何避免速率限制错误
使用指数退避进行重试
避免速率限制错误的一种简单方法是使用随机指数退避自动重试请求。使用指数退避进行重试意味着在遇到速率限制错误时执行短暂休眠，然后重试未成功的请求。如果请求仍然不成功，则增加休眠时间并重复该过程。这将持续到请求成功为止，或者达到最大重试次数为止。
这种方法有许多好处：
- 自动重试意味着您可以在没有崩溃或丢失数据的情况下从速率限制错误中恢复
- 指数退避意味着您的第一次重试可以很快尝试，同时如果您的前几次重试失败，仍然可以从较长的延迟中受益
- 将随机抖动添加到延迟中有助于避免所有重试同时发生
请注意，未成功的请求会影响您的每分钟限制，因此持续重新发送请求是行不通的。
以下是一些示例解决方案。
示例＃1：使用Tenacity库
Tenacity是一个Apache 2.0许可的通用重试库，用Python编写，旨在简化向几乎任何内容添加重试行为的任务。
要为您的请求添加指数退避，您可以使用tenacity.retry
装饰器。以下示例使用tenacity.wait_random_exponential函数为请求添加随机指数退避。
请注意，Tenacity库是一个第三方工具，OpenAI不对其可靠性或安全性提供任何保证。
from tenacity import (
retry,
stop_after_attempt,
wait_random_exponential,
) # 指数退避
@retry(wait=wait_random_exponential(min=1, max=60), stop=stop_after_attempt(6))
def completion_with_backoff(**kwargs):
return client.chat.completions.create(**kwargs)
completion_with_backoff(model="gpt-3.5-turbo", messages=[{"role": "user", "content": "Once upon a time,"}])
ChatCompletion(id='chatcmpl-8PAu6anX2JxQdYmJRzps38R8u0ZBC', choices=[Choice(finish_reason='stop', index=0, message=ChatCompletionMessage(content='in a small village nestled among green fields and rolling hills, there lived a kind-hearted and curious young girl named Lily. Lily was known for her bright smile and infectious laughter, bringing joy to everyone around her.\n\nOne sunny morning, as Lily played in the meadows, she stumbled upon a mysterious book tucked away beneath a tall oak tree. Intrigued, she picked it up and dusted off its weathered cover to reveal intricate golden patterns. Without hesitation, she opened it, discovering that its pages were filled with magical tales and enchanting adventures.\n\nAmong the stories she found, one particularly caught her attention—a tale of a long-lost treasure hidden deep within a mysterious forest. Legend had it that whoever found this hidden treasure would be granted one wish, no matter how big or small. Excited by the prospect of finding such treasure and fulfilling her wildest dreams, Lily decided to embark on a thrilling journey to the forest.\n\nGathering her courage, Lily told her parents about the magical book and her quest to find the hidden treasure. Though concerned for their daughter\'s safety, they couldn\'t help but admire her spirit and determination. They hugged her tightly and blessed her with love and luck, promising to await her return.\n\nEquipped with a map she found within the book, Lily ventured into the depths of the thick forest. The trees whispered tales of forgotten secrets, and the enchanted creatures hidden within watched her every step. But Lily remained undeterred, driven by her desire to discover what lay ahead.\n\nDays turned into weeks as Lily traversed through dense foliage, crossed swift rivers, and climbed treacherous mountains. She encountered mystical beings who offered guidance and protection along her perilous journey. With their help, she overcame countless obstacles and grew braver with each passing day.\n\nFinally, after what felt like an eternity, Lily reached the heart of the forest. There, beneath a jeweled waterfall, she found the long-lost treasure—a magnificent chest adorned with sparkling gemstones. Overwhelmed with excitement, she gently opened the chest to reveal a brilliant light that illuminated the forest.\n\nWithin the glow, a wise voice echoed, "You have proven your courage and pure heart, young Lily. Make your wish, and it shall be granted."\n\nLily thought deeply about her wish, realizing that her true treasure was the love and happiness she felt in her heart. Instead of making a wish for herself, she asked for the wellbeing and prosperity of her village, spreading joy and harmony to everyone living there.\n\nAs the light faded, Lily knew her quest was complete. She retraced her steps through the forest, returning home to find her village flourishing. Fields bloomed with vibrant flowers, and laughter filled the air.\n\nThe villagers greeted Lily with open arms, recognizing her selflessness and the magic she had brought into their lives. From that day forward, they told the tale of Lily\'s journey, celebrating her as a heroine who embodied the power of love, kindness, and the belief that true treasure lies within oneself.\n\nAnd so, the story of Lily became an everlasting legend, inspiring generations to follow their dreams, be selfless, and find the true treasures that lie within their hearts.', role='assistant', function_call=None, tool_calls=None))], created=1701010806, model='gpt-3.5-turbo-0613', object='chat.completion', system_fingerprint=None, usage=CompletionUsage(completion_tokens=641, prompt_tokens=12, total_tokens=653))
示例 #2：使用backoff库
另一个提供退避和重试功能装饰器的库是backoff。
与Tenacity类似，backoff库也是一个第三方工具，OpenAI不对其可靠性或安全性提供任何保证。
import backoff # 指数退避
@backoff.on_exception(backoff.expo, openai.RateLimitError)
def completions_with_backoff(**kwargs):
return client.chat.completions.create(**kwargs)
completions_with_backoff(model="gpt-3.5-turbo", messages=[{"role": "user", "content": "Once upon a time,"}])
ChatCompletion(id='chatcmpl-8PAwkg7Q9pPeAkvVuAZ8AyA108WhR', choices=[Choice(finish_reason='stop', index=0, message=ChatCompletionMessage(content="in a small village, there lived a young girl named Lily. She had fiery red hair, lively green eyes, and a spirit as wild as the rushing river nearby. Lily was known for her curious nature and her desire to explore the world beyond the village boundaries.\n\nOne day, while playing near the river, Lily spotted an injured bird nested on a branch. Its wing was broken, and it seemed unable to fly away. Lily's heart filled with sadness, and she knew she couldn't leave the bird alone.\n\nCarefully, she climbed up the tree and gently placed the bird inside her pocket. Lily brought it home and made a cozy bed for it in a small wooden box. She named the bird Ruby, after its shimmering red feathers.\n\nDays turned into weeks, and Ruby's wing slowly healed under Lily's constant care and attention. As they spent time together, a deep bond grew between them. Ruby would chirp happily whenever Lily approached, and she would spend hours talking to the bird, sharing stories of her adventures, dreams, and fears.\n\nOne evening, as Lily was about to go to bed, a peculiar thing happened. Ruby hopped out of his box and fluttered onto the windowsill. He turned to face Lily with his bright eyes and began to sing a beautiful melody.\n\nLily was astonished. Never before had she heard Ruby sing. The tune was so captivating that it filled the room and made the quiet night come alive. The magical music seemed to touch Lily's soul, awakening a deep sense of wonder and wanderlust within her.\n\nFilled with an undeniable urge to explore, Lily decided it was time to go on an adventure with her newfound friend, Ruby. She packed a small bag and bid farewell to her family and friends, promising to return one day.\n\nTogether, Lily and Ruby embarked on a grand journey, soaring across expansive skies, diving into lush forests, and exploring hidden caves. They encountered magnificent landscapes, unique creatures, and encountered kind-hearted individuals who shared their wisdom and stories.\n\nThroughout their journey, Ruby's song continued to inspire and guide them. It became a symbol of hope, reminding them to embrace bravery, follow their dreams, and always remain true to themselves.\n\nAs the years passed, Lily and Ruby traversed the world, weaving their stories into the tapestry of time. They became renowned for their extraordinary bond and the magic they shared with everyone they encountered.\n\nEventually, it was time for Lily to return to her village, a place eagerly awaiting her return. She had grown wise, learned many lessons, and gained a deeper understanding of herself and the world around her.\n\nWith Ruby perched on her shoulder, they descended upon the village like a ray of sunshine, bringing joy and wonder to every heart. Lily shared the wisdom she had acquired and inspired others to embrace their own adventures, no matter how big or small.\n\nAnd so, the tale of Lily and Ruby became legend, passed down from generation to generation. Their story reminded people to cherish the connections they make, to nurture their dreams, and to believe in the magic that lies within them.", role='assistant', function_call=None, tool_calls=None))], created=1701010970, model='gpt-3.5-turbo-0613', object='chat.completion', system_fingerprint=None, usage=CompletionUsage(completion_tokens=621, prompt_tokens=12, total_tokens=633))
示例3：手动退避实现
如果你不想使用第三方库，你可以实现自己的退避逻辑。
# 导入
import random
import time
# 定义一个重试装饰器
def retry_with_exponential_backoff(
func,
initial_delay: float = 1,
exponential_base: float = 2,
jitter: bool = True,
max_retries: int = 10,
errors: tuple = (openai.RateLimitError,),
):
"""使用指数退避算法重试函数。"""
def wrapper(*args, **kwargs):
# 初始化变量
num_retries = 0
delay = initial_delay
# 循环直到收到成功 的响应、达到最大重试次数或引发异常为止。
while True:
try:
return func(*args, **kwargs)
# 在指定错误时重试
except errors as e:
# 增加重试次数
num_retries += 1
# 检查是否已达到最大重试次数
if num_retries > max_retries:
raise Exception(
f"Maximum number of retries ({max_retries}) exceeded."
)
# 增加延迟
delay *= exponential_base * (1 + jitter * random.random())
# 睡眠以应对延误
time.sleep(delay)
# 对未指定的任何错误抛出异常
except Exception as e:
raise e
return wrapper
@retry_with_exponential_backoff
def completions_with_backoff(**kwargs):
return client.chat.completions.create(**kwargs)
completions_with_backoff(model="gpt-3.5-turbo", messages=[{"role": "user", "content": "Once upon a time,"}])
ChatCompletion(id='chatcmpl-8PAxGvV3GbLpnOoKSvJ00XCUdOglM', choices=[Choice(finish_reason='stop', index=0, message=ChatCompletionMessage(content="in a faraway kingdom, there lived a young princess named Aurora. She was known for her beauty, grace, and kind heart. Aurora's kingdom was filled with lush green meadows, towering mountains, and sparkling rivers. The princess loved spending time exploring the enchanting forests surrounding her castle.\n\nOne day, while Aurora was wandering through the woods, she stumbled upon a hidden clearing. At the center stood a majestic oak tree, its branches reaching towards the sky. Aurora approached the tree with curiosity, and as she got closer, she noticed a small door at its base.\n\nIntrigued, she gently pushed open the door and was amazed to find herself in a magical realm. The forest transformed into a breathtaking wonderland, with colorful flowers blooming in every direction and woodland creatures frolicking joyously. Aurora's eyes widened with wonder as she explored this extraordinary world.\n\nAs she explored further, Aurora came across a small cottage in the distance. Curiosity overcame her, and she cautiously approached the cottage. To her surprise, an elderly woman with twinkling eyes and a warm smile stood in the doorway, welcoming her inside.\n\nThe woman revealed herself to be a fairy named Luna. Luna informed Aurora that she had been chosen to undertake a quest that would bring harmony to both her kingdom and the mystical realm. Aurora, eager to help, listened intently as Luna explained that a powerful enchantress had cast a spell on the kingdom, causing darkness and despair to loom over the land.\n\nTo break the curse, Aurora had to embark on a journey to retrieve a magical crystal hidden deep within the heart of an ancient cave. Without hesitation, the princess agreed and bid farewell to Luna, promising to return victorious.\n\nWith newfound determination, Aurora set off on her quest. Along the way, she encountered numerous challenges and obstacles but never lost hope. She often drew strength from the enchanting woodland creatures who accompanied her on this journey, reminding her that she was not alone.\n\nAfter a long and arduous journey, Aurora reached the entrance of the ancient cave. Inside, she faced a series of tests that pushed her physical and emotional limits. With sheer determination and unwavering courage, she overcame each trial, paving her way to the crystal's resting place.\n\nAs Aurora held the crystal in her hands, its warmth spread through her body. The artifact contained unimaginable power that could shatter the enchantress's curse and restore light to her kingdom. Brimming with joy and newfound strength, she made her way back to Luna's cottage.\n\nUpon her return, Aurora and Luna performed a powerful ritual, using the crystal's magic to break the curse. Waves of light and color spread across the kingdom, banishing darkness and despair. The once-gray skies turned blue, and laughter filled the air once again. The kingdom rejoiced, thanking Princess Aurora for her bravery and selflessness.\n\nFrom that day forward, Aurora was hailed as a hero, not only in her kingdom but also in the mystical realm. She continued to be a beacon of hope and kindness, reminding everyone that true courage lies within, waiting to be awakened.\n\nAnd so, Princess Aurora's tale lived on as a timeless reminder that even in the darkest of times, there is always light and hope to be found.", role='assistant', function_call=None, tool_calls=None))], created=1701011002, model='gpt-3.5-turbo-0613', object='chat.completion', system_fingerprint=None, usage=CompletionUsage(completion_tokens=657, prompt_tokens=12, total_tokens=669))
如何在给定速率限制的情况下最大化批处理的吞吐量
如果您正在处理用户的实时请求，退避和重试是一种很好的策略，可以最大限度地减少延迟，同时避免速率限制错误。
然而，如果您正在处理大量的批处理数据，其中吞吐量比延迟更重要，除了退避和重试之外，还有一些其他事项可以帮助您提高吞吐量。
主动在请求之间添加延迟
如果您不断地达到速率限制，然后退避，然后再次达到速率限制，然后再次退避，那么您的请求预算中可能有相当 大一部分会被“浪费”在需要重试的请求上。这会限制您的处理吞吐量，即使速率限制是固定的。
在这种情况下，一个潜在的解决方案是计算您的速率限制，并为每个请求添加一个等于其倒数的延迟（例如，如果您的速率限制是每分钟20个请求，则为每个请求添加3-6秒的延迟）。这可以帮助您在接近速率限制上限的情况下运行，而不会达到上限并产生浪费的请求。
添加延迟到请求的示例
# 导入
import time
# 定义一个函数，为Completion API调用添加延迟
def delayed_completion(delay_in_seconds: float = 1, **kwargs):
"""延迟指定时间量以完成操作。"""
# 因延误而休息
time.sleep(delay_in_seconds)
# 调用完成API并返回结果
return client.chat.completions.create(**kwargs)
# 根据您的速率限制计算延迟
rate_limit_per_minute = 20
delay = 60.0 / rate_limit_per_minute
delayed_completion(
delay_in_seconds=delay,
model="gpt-3.5-turbo",
messages=[{"role": "user", "content": "Once upon a time,"}]
)
ChatCompletion(id='chatcmpl-8PAyCR1axKsomV0e349XiCN1Z81pH', choices=[Choice(finish_reason='stop', index=0, message=ChatCompletionMessage(content="in a small village, there lived a young girl named Maya. Maya was known for her kindness and love for nature. She spent hours exploring the forests surrounding the village, admiring the vibrant flowers and talking to the animals.\n\nOne sunny day, as Maya was picking wildflowers, she stumbled upon a wounded blackbird with a broken wing. Feeling sorry for the bird, Maya gently picked it up and cradled it in her hands. She knew she had to help the bird, so she hurried back to her cottage.\n\nMaya set up a cozy nest for the blackbird and carefully splinted its wing. She fed it worms and berries, doing everything she could to nurse it back to health. Each day, she would sing lullabies and tell stories to keep the blackbird company. Slowly, the bird's wing healed, and before long, it was ready to fly again.\n\nOn a beautiful morning, Maya opened the window of her cottage and released the blackbird into the sky. As the bird soared into the air, Maya's heart filled with joy and gratitude. Little did she know, this act of kindness would change her life forever.\n\nThe following night, a mysterious glowing light illuminated Maya's room. Startled, she sat up and saw a magical creature standing before her. It was a fairy, tiny yet radiating warmth and light.\n\nThe fairy introduced herself as Luna, the Guardian of the Forest. She had witnessed Maya's kindness towards the blackbird and had been watching her ever since. Luna explained that she had come to reward Maya for her selflessness.\n\nWith a wave of her wand, Luna granted Maya the ability to communicate with animals. Maya's eyes widened with amazement as she realized she could now understand the language of nature. Birds chirped melodies, rabbits whispered secrets, and trees shared their ancient wisdom.\n\nOver time, Maya's ability made her beloved by both humans and animals. Farmers sought her advice on how to care for their crops, and children flocked to her for stories of her enchanting encounters with the forest creatures. Maya used her gift to teach others about the importance of living in harmony with nature.\n\nAs years passed, Maya became known as the Village Guardian. She dedicated herself to protecting the surrounding forests from harm and educating others on sustainable living. The village flourished under Maya's guidance, and animals and humans lived side by side peacefully.\n\nAnd so, Maya's story became a legend passed down through generations. Her kindness, love for nature, and her ability to communicate with animals inspired people to treat the world around them with compassion and care.", role='assistant', function_call=None, tool_calls=None))], created=1701011060, model='gpt-3.5-turbo-0613', object='chat.completion', system_fingerprint=None, usage=CompletionUsage(completion_tokens=524, prompt_tokens=12, total_tokens=536))
批量请求
OpenAI API对每分钟请求次数和每分钟令牌数有单独的限制。
如果你达到了每分钟请求次数的限制，但在每分钟令牌数上还有余地，你可以通过将多个任务合并到每个请求中来增加吞吐量。这将允许你每分钟处理更多的令牌，特别是对于较小的模型。
发送一批提示的请求与普通的API调用完全相同，只是将一组字符串传递给prompt参数，而不是单个字符串。
警告：
响应对象可能不会按照提示的顺序返回完成，因此请始终记得使用index字段将响应与提示匹配回来。
无批量处理示例
num_stories = 10
content = "Once upon a time,"
# 连续示例，每次请求完成一个故事情节
for _ in range(num_stories):
response = client.chat.completions.create(
model="gpt-3.5-turbo",
messages=[{"role": "user", "content": content}],
max_tokens=20,
)
# 打印故事
print(content + response.choices[0].message.content)
Once upon a time,in a small village nestled between rolling green hills, there lived a young girl named Lily. She had
Once upon a time,in a small village nestled in the heart of a lush forest, lived a young girl named Evelyn.
Once upon a time,in a faraway kingdom, there lived a young princess named Aurora. She was known for her kind
Once upon a time,in a faraway kingdom called Enchantia, there lived a young girl named Ella. Ella was
Once upon a time,in a small village nestled among the rolling hills, lived a young woman named Lucy. Lucy was known
Once upon a time,in a small village nestled between rolling hills, there lived a young girl named Ava. Ava was a
Once upon a time,in a faraway kingdom, there lived a wise and just king named Arthur. King Arthur ruled over
Once upon a time,in a small village nestled among towering mountains, lived a young girl named Lily. She was known for
Once upon a time,in a small village nestled in the heart of a lush forest, there lived a young girl named Lily
Once upon a time,in a far-off kingdom, there lived a kind and beloved queen named Isabella. She ruled with
批处理示例
num_stories = 10
prompts = ["Once upon a time,"] * num_stories
# 批量示例，每请求包含10个故事完成项
response = client.chat.completions.create(
model="curie",
prompt=prompts,
max_tokens=20,
)
# 按索引将完成项与提示匹配
stories = [""] * len(prompts)
for choice in response.choices:
stories[choice.index] = prompts[choice.index] + choice.text
# 打印故事
for story in stories:
print(story)
Once upon a time, I lived in hope. I convinced myself I knew best, because, naive as it might sound,
Once upon a time, Thierry Henry was invited to have a type of frosty exchange with English fans, in which
Once upon a time, and a long time ago as well, PV was passively cooled because coils cooled by use of metal driving
Once upon a time, there was a land called Texas. It was about the size of Wisconsin. It contained, however,
Once upon a time, there was an old carpenter who had three sons. The locksmith never learned to read or write
Once upon a time, there was a small farming town called Moonridge Village, far West across the great vast plains that lay
Once upon a time, California’s shorelines, lakes, and valleys were host to expanses of untamed wilderness
Once upon a time, she said. It started with a simple question: Why don’t we know any stories?
Once upon a time, when I was a young woman, there was a movie named Wuthering Heights. Stand by alleges
Once upon a time, a very long time I mean, in the year 1713, died a beautiful Duchess called the young
并行处理脚本示例
我们编写了一个示例脚本，用于并行处理大量的API请求：api_request_parallel_processor.py。
该脚本结合了一些方便的功能： - 从文件中流式传输请求，以避免在处理大型作业时耗尽内存 - 并发进行请求，以最大化吞吐量 - 对请求和令牌使用进行限速，以保持在速率限制之下 - 重试失败的请求，以避免丢失数据 - 记录错误，以诊断请求中的问题
请随意使用原样或根据您的需求进行修改。