AWS introduced Chain-of-Draft (CoD), a new prompting technique on Amazon Bedrock. Developed by Zoom AI Research, this method enables AI models to work faster and cheaper.
Inference costs dominate 70-90% of large language model (LLM) operational expenses. Verbose prompting strategies can inflate token volume by 3-5x. Chain-of-Draft directly addresses this problem.
Why Chain-of-Thought Falls Short
Chain-of-Thought (CoT) prompting guides AI models to reason through problems step by step. It delivers effective results for complex logic puzzles and math problems.
However, CoT has serious drawbacks. Lengthy explanations bloat token usage and increase costs. Detailed responses extend latency times. Real-time applications suffer as a result.
For example, a simple math question might get this CoT response: “I start with 5 apples. I eat 2 apples. I subtract 2 from 5. 5 – 2 = 3 apples remaining.”
How Chain-of-Draft Works
Chain-of-Draft draws inspiration from how humans solve problems. Instead of verbose explanations, it uses brief mental notes. Each reasoning step is limited to a maximum of 5 words.
The same math question gets this CoD response: “Start: 5, End: 3, 5 – 2 = 3.”
This minimalist approach reaches the same logical conclusion with far fewer tokens.
According to Zoom AI’s original research paper, CoD achieved 91.4% accuracy on GSM8K tests. It came close to CoT’s 95.3% while reducing output tokens by 92.1%.
Impressive Results in AWS Testing
The AWS team tested Chain-of-Draft using Amazon Bedrock and AWS Lambda. They used the “Red, Blue, and Green Balls” puzzle as a benchmark.
Results proved remarkable:
In Model-1 testing, CoD reduced total token usage from 350 to 216. That represents a 39% reduction. Latency dropped from 3.28 seconds to 1.58 seconds. A 52% improvement was achieved.
Model-2 testing showed even higher gains. Token usage fell from 601 to 142. A 76% reduction was recorded. Latency dropped from 3.81 seconds to 0.79 seconds. A 79% improvement was achieved.
When Not to Use It
Chain-of-Draft isn’t ideal for every situation. AWS offers some important caveats.
CoD struggles in zero-shot scenarios. It performs best when paired with few-shot examples.
High-interpretability use cases favor CoT. Legal or medical documents requiring audit trails need detailed explanations.
Small models under 3 billion parameters show poor CoD performance. These models struggle to follow the minimalist prompt style.
Creative writing or open-ended tasks also favor CoT. Jobs requiring elaboration rather than summarization don’t suit CoD.
Conclusion
Chain-of-Draft offers a powerful tool for organizations looking to cut AI costs. It delivers significant advantages when token efficiency and speed are critical. However, it doesn’t fit every task. Proper model selection and prompt design remain keys to success.
For more information, check out AWS’s prompt engineering guide.
