Balancing LLM Token Consumption and Generation Quality

Determine methods that balance large language model token consumption and generation quality within GenDB’s code-generation pipeline, achieving reductions in prompt size, model reasoning effort, and compressed inter-agent communications, schema descriptions, and queries without degrading the correctness or performance of the synthesized query execution code.

Background

The authors discuss approaches to reduce token usage, including concise prompt templates, lower reasoning effort, and compression for inter-agent communication, database schemas, and queries. They note that these methods can degrade the quality of generated code. Hence, the central challenge is to find techniques that reduce tokens while preserving or improving generation quality.

This trade-off directly impacts GenDB’s cost and efficiency because LLM invocation fees and latency scale with token counts, yet overly aggressive compression or simplification can harm the quality and correctness of synthesized query execution code.