- You can think of tokens as pieces of words, where 1,000 tokens is about 750 words.
- It depends on the model used, for example for text-davinci-003 requests can use up to
4097 tokens shared between prompt and completion. If your prompt is 4000 tokens, your completion can be 97 tokens at most. - The limit is currently a technical limitation
- Solution: there are often creative ways to solve problems within the limit, e.g. condensing the prompt, breaking the text into smaller pieces, etc.
- Techniques for improving reliability around prompts
Even with careful planning, it's important to be prepared for unexpected issues when using GPT-3 in your application. In some cases, the model may fail on a task, so it's helpful to consider what you can do to improve the reliability of your application. If your task involves logical reasoning or complexity, you may need to take additional steps to build more reliable prompts. For some helpful suggestions, consult our Techniques to improve reliability guide. Overall the recommendations revolve around: - Decomposing unreliable operations into smaller, more reliable operations (e.g., selection-inference prompting)
- Using multiple steps or multiple relationships to make the system's reliability greater than any individual component (e.g., maieutic prompting)
|