iCreatAI News
Skip to content
Trending storyUpdating

PacketSnitch uses small models for tool routing

1 reports1 sources15 hr agoUpdated

Understand this story

AI overview

PacketSnitch developer aestetix shared a small-model tool routing approach: using Qwen2.5-0.5B-Instruct as the initial routing model, it classifies user requests into intents and generates JSON actions; for complex requests, application logic escalates to the 1.5B model. The model does not memorize all APIs but retrieves a pre-generated documentation index on demand, with a single test case latency of 153ms. According to the developer, training took more than a week on an L40S, with compute costs of about $40; the resulting model can run locally on a laptop with 32GB of memory. The above latency and cost are test and training data self-reported by the developer.

AI overview based on source reports · Updated 14 hr ago

Coverage timeline

Follow the coverage to explore different perspectives.

Nov 20
  1. Hacker News AI
    Teaching PacketSnitch to Use Its Own Documentation: Build a Small AI Tool Router

    The PacketSnitch developer describes routing natural-language requests to application actions by having a small model classify intent and look up pregenerated documentation rather than memorizing every function. A 0.5B Qwen2.5 router handles simple requests, with application-controlled escalation to a 1.5B model when intent or JSON validation fails. The author reports roughly $40 in training compute and a 153ms example response for a TCP filter request.

Activity for this story

Current comparable activity
7
Peak comparable activity
10Nov 20 06:00
Change in the past 24 hours
–

Trends compare the same fully observed participants, which may cover fewer participants than current activity. Move the pointer or tap the chart to inspect hourly activity; use the left and right arrow keys with a keyboard.