Small Models, Specific Jobs: ScaleDown on Building Efficient Voice AI

In this episode of the Convo AI World Podcast, Hermes Frangoudis speaks with Neal Patel, co-founder of ScaleDown, about choosing models for the specific jobs inside an AI application. Neal explains how his work forecasting data center demand led him to focus on token efficiency. At ScaleDown, that means building small language models for tasks such as classification, extraction, and summarization. He describes how the team combines existing model components with task-specific neural heads, and where these specialists can work alongside a larger model that handles planning and conversation. For voice AI builders, the discussion covers routing, parallel subagents, conversation history, and post-call analysis. Neal explains the difference between query-aware compression and a generic summary, and why preserving the information needed for an answer matters more than simply shrinking a prompt. He also explores using classifiers for evaluation, where consistency can matter as much as speed. The conversation closes with ScaleDown's integration tools and roadmap, plus Neal's view of how model selection could become more automated. Throughout, he makes a useful distinction: specialized models may reduce variability on a defined task, but they are not fully deterministic.

Key Topics Covered

• •The origins of ScaleDown: data center demand, token efficiency, and task-specific models
• •Model architecture: combining existing model components with specialized neural heads
• •Classification and extraction: defining labels, handling ambiguity, and returning structured outputs
• •Voice agent orchestration: pairing a general-purpose model with specialist subagents
• •Query-aware compression: retaining relevant information after retrieval and reranking
• •Conversation history and post-call analysis: managing context and analyzing transcripts at scale
• •Evaluations and consistency: using classifiers as judges without assuming deterministic results
• •Integration and roadmap: model selection, developer tooling, and future multimodal support