We recently went through the agent framework decision at SportChartz. We build trading agents that need to monitor live sports games, detect technical signals, and post alerts to our community. When we started, the natural instinct was to reach for a framework. We evaluated a full-featured open-source agent platform against building directly on an LLM SDK. We ended up on the SDK, but the decision turned on the specific characteristics of our application, not on framework quality. The answer is application-dependent, and the questions worth asking before committing are the same regardless of what you’re building.
The context
During a busy night with 10+ concurrent games across multiple leagues, our system is monitoring game events in real time, evaluating signal conditions every 60-90 seconds and generating content when something fires. It looked like a multi-agent problem: one process watching games, another monitoring social mentions, another reading community chat, another generating content. The path to our decision mattered more than the outcome, because we learned that the right framework depends entirely on what your application actually does.
What’s out there
Agent frameworks sit between your application code and the LLM. They manage the orchestration loop: deciding what tool to call, handling the response, maintaining state, determining the next action. As of early 2026, they roughly fall into three tiers:
Full-featured platforms like OpenClaw provide a complete runtime. Gateway process, channel adapters (Slack, Discord, WhatsApp, Telegram), per-agent memory files, heartbeat schedulers, a plugin ecosystem. You define agents, bind them to channels, and the gateway handles routing. These are essentially operating systems for agents, and they’re powerful when your problem fits the model they were built for.
Orchestration frameworks like LangGraph and CrewAI sit one level down. LangGraph gives you graph-based execution with explicit state machines - define nodes and edges, the framework manages state transitions. CrewAI gives you role-based teams where agents have defined roles and the framework coordinates between them. Both are Python-native and have strong adoption for complex workflows.
LLM SDKs with agent capabilities like the Claude Agent SDK or OpenAI Agents SDK are the thinnest layer. They expose the model’s native tool-use in a loop: the model decides what to do, calls tools, gets results, decides again. Subagent delegation, session persistence, context management are built in, but the outer orchestration logic is yours.
Each tier solves a real problem.
The questions that matter
The decision breaks down along five questions. Your application’s position on each one pushes you toward different points on the framework spectrum.
Trigger type: internal vs. external
Internal trigger means your system decides when to act. A polling loop checks data on a cadence, evaluates conditions, maybe takes action. The timing is yours to control. A timer, a cron job, a scheduled function.
External trigger means something outside your system initiates the action. A user sends a message. A webhook fires. A WebSocket pushes an event. The system needs to be listening and ready to respond.
This is the first dimension because it determines whether you need always-on infrastructure. Full-featured agent platforms with channel adapters and WebSocket listeners are purpose-built for external triggers. They keep connections alive, route inbound events to the right agent, and handle the plumbing of being permanently available. If your application is genuinely event-driven across multiple channels, that plumbing is valuable and building it yourself is significant work.
If your application is internally triggered - running on a cycle, checking data, acting when conditions are met - the always-on infrastructure is overhead you’re paying for but not using. A timer and an API call handle it.
There’s a gray zone. Polling at 60-90 seconds feels like real-time to most users. If someone posts in a community chat and the agent notices on its next cycle 45 seconds later, that’s fast enough for most applications. The question is whether your latency requirement is measured in seconds (event-driven) or minutes (polling handles it).
Our trading agents are internally triggered. They poll on a cycle during game windows. If we wanted them to also respond to community chat messages, we’d add “read recent chat” as another input in the same cycle. The architecture doesn’t change.
Decision type: deterministic vs. judgment
If the trigger logic is “when value X exceeds threshold Y, take action Z” - that’s code. It’s a conditional. Routing that decision through an LLM agent loop means paying for a model to evaluate something a boolean check handles. The LLM adds value when you need it to write the alert, interpret context, or make a judgment call. It doesn’t add value for the threshold check itself.
Most agent frameworks route every decision through the model by design. That’s the ReAct loop: observe state, reason about what to do, act, repeat. For applications where the decisions genuinely require interpretation of unstructured input - reading a customer email and deciding which department to route it to, analyzing a document and determining next steps, interpreting ambiguous user intent - the ReAct loop is exactly right.
For applications where the trigger is deterministic and only the output requires generation, you’re paying the framework’s coordination overhead on every cycle for logic that could be an if statement. The research quantifies this: CrewAI consumes roughly 3x the tokens and runs 3x slower than a direct API call for simple single-tool tasks. That overhead is the framework’s coordination logic, not the model’s reasoning.
The practical split that worked for us: deterministic code for signal detection and threshold evaluation, LLM for content generation and contextual judgment. The two concerns don’t need to live in the same loop.
Cost structure at scale
Token economics at agent scale are different from chatbot scale and worth modeling before you commit to an architecture.
A chatbot handles one request, generates a response, moves on. An agent running a polling loop makes API calls every 60-90 seconds for hours. With subagent delegation, a single cycle can be 3-5 API calls. Over a 4-hour window, that’s 160-240 cycles.
Framework overhead multiplies the per-cycle cost. For the same simple task (single tool call and response), token consumption varies by framework:
Direct API calls: roughly 150-200 tokens
LangGraph: under 900 tokens (minimal overhead)
CrewAI: roughly 2,700 tokens (3x multiplier from coordination logic)
At low volume, the difference is negligible. At scale, it compounds. A million daily executions at the direct API rate costs roughly $300/day. The same million through CrewAI’s coordination layer costs roughly $1,200/day. That’s $27,000/month in framework overhead.
Two levers help regardless of framework choice. Model routing (cheap models for routine scanning, expensive models for generation) and prompt caching (reusing the same system prompt and tool definitions across cycles, cutting costs 50%+ on repeated context). Both favor simpler architectures where the context is predictable cycle to cycle. Our polling loop runs every 90 seconds for 4 hours. The overhead is predictable and avoidable.
Security surface
A full-featured agent framework handles your API keys, social credentials, OAuth tokens, and platform secrets. It’s another system in the chain with access to sensitive material. OpenClaw, the most popular open-source agent framework with over 350,000 GitHub stars, had a January 2026 security audit that found 512 vulnerabilities, 8 of them critical, including OAuth credentials stored in plaintext JSON files. The framework’s creator left weeks later to join OpenAI, and the project was handed to a newly formed non-profit foundation that is still establishing its governance processes. In March 2026, the Chinese government banned state agencies from using OpenClaw over security concerns.
Our agents handle social media tokens, financial API keys, and user authentication credentials. Minimizing the number of systems that touch those credentials is not optional. The thin SDK approach keeps credentials in our code, our infrastructure, our control.
Debugging depth
Every abstraction layer between your application code and the model is a debugging surface. A full-featured framework wraps LLM calls in its own agent loop (ReAct), which internally may use another orchestration library. That’s potentially two agent loops and three abstraction layers between your business logic and the model.
When something breaks, the number of layers determines how quickly you find the problem. Our agents need to post during live game windows. If they stop working at 9pm, we can’t debug until morning. Fewer layers means faster recovery.
How it played out for us
Our application landed clearly on the “simpler is better” end of every dimension. Internally triggered. Mostly deterministic decision logic. Predictable cost structure. Sensitive credentials in the loop. Time-sensitive operations.
We went with the thin SDK approach. Single process, direct MCP tool integration, a time-aware scheduler that adjusts behavior based on the game window. The outer loop is about 100 lines of code. It handles 30+ concurrent games, evaluates confluence across 5 technical indicators, and calls the LLM when it needs to generate content. It runs on a container. The costs are predictable.
The five dimensions, summarized
Trigger type. Internal (polling, scheduled) favors thin SDKs and custom loops. External (event-driven, multi-channel inbound) favors full-featured platforms with channel adapters.
Decision type. Deterministic (thresholds, rules, conditions) favors keeping the LLM out of the decision loop. Judgment-heavy (unstructured input, ambiguous intent, contextual reasoning) favors agent frameworks with ReAct orchestration.
Cost structure. Predictable, high-frequency workloads amplify framework overhead. Bursty, variable workloads make framework overhead proportionally smaller. Model routing and caching help in both cases.
Security surface. Sensitive credentials favor fewer dependencies and thinner frameworks. Sandboxed operations are less constrained.
Debugging depth. Time-sensitive operations favor fewer abstraction layers. Batch operations can absorb more layers in exchange for richer tooling.
Neal Foster is Co-Founder & CTO of SportChartz and Founder & Partner of Vybe Capital.