Agents Are Not an Architecture (and 'Long Context' Isn't Either)

Agents — a loop, not a wiring

An "AI agent" is an LLM placed inside a scaffolding loop — observe, think, act, observe again. Every framework you have heard of (LangChain, LangGraph, CrewAI, AutoGen, OpenAI Agents SDK, Claude Agent SDK) is a loop with state management around standard LLM calls. Each iteration is one normal model invocation with a carefully constructed prompt. The model has no awareness it is "in an agent" — it just sees a chat conversation.

The agent loop in pseudocode

If you can implement an agent in 30 lines of orchestration code, the agent is not an architecture. It is plumbing. The model is unchanged.

Long context is not a new architecture

Extending context length from 4K to 128K to 1M+ tokens involves:

RoPE frequency scaling (NTK-aware scaling, dynamic NTK, YaRN) — adjusting the rotary position encoding to extrapolate beyond the trained window.
Continued training on long sequences with the new RoPE configuration.
Better attention implementations for the longer KV-cache.

None of this is a new architecture. The backbone Transformer is unchanged; what changes is the position encoding and the training data length.

Effective context ≠ advertised context

A 1M-token context window does not mean the model uses 1M tokens well. Research like the "Needle in a Haystack" test and various MECW (Maximum Effective Context Window) evaluations show many models degrade significantly long before reaching their advertised limit. The advertised number is a capacity, not a capability.

The general lesson

Anything that wraps a fixed model with extra software — agents, RAG, long context tricks, guardrails, system prompts — is product or training, not architecture. The architectural skeleton is what is encoded in the weights file. Everything else is application code.

Code

An entire agent in 25 lines (no model change)·python

def agent_loop(goal, llm, tools, max_steps=20):
    history = [{"role": "system", "content": f"You are an agent. Goal: {goal}"}]
    for step in range(max_steps):
        response = llm.generate(history)
        history.append({"role": "assistant", "content": response})
        if "FINAL_ANSWER" in response:
            return response
        tool_call = parse_tool_call(response)
        if tool_call is None:
            continue
        result = tools[tool_call["name"]](**tool_call["args"])
        history.append({"role": "tool", "content": str(result)})
    return "STOPPED: max_steps reached"

RoPE frequency scaling — the long-context trick·python

# Standard RoPE (training-time)
def rope_freqs(d_model, base=10000.0):
    return base ** (-torch.arange(0, d_model, 2) / d_model)

# YaRN-style scaling for extension
def yarn_scaled_freqs(d_model, base=10000.0, scale=4.0):
    return base ** (-torch.arange(0, d_model, 2) / d_model) / scale
# Position encoding adjustment, no architectural change.

Agents Are Not an Architecture (and 'Long Context' Isn't Either)

Agents — a loop, not a wiring

The agent loop in pseudocode

Long context is not a new architecture

Effective context ≠ advertised context

The general lesson

Code

External links

Exercise

Progress

Comments 0