2 posts tagged “architecture”
software archeitecture
2026
Core of Pi—the while loop.
The core of the Pi is basically a while loop, in packages/agent/src/agent-loop.ts.
// Outer loop: continues when queued follow-up messages arrive after agent would stop
while (true) {
let hasMoreToolCalls = true;
let steeringAfterTools: AgentMessage[] | null = null;
// Inner loop: process tool calls and steering messages
while (hasMoreToolCalls || pendingMessages.length > 0) {
if (!firstTurn) {
stream.push({ type: "turn_start" });
} else {
firstTurn = false;
}
// Process pending messages (inject before next assistant response)
if (pendingMessages.length > 0) {
for (const message of pendingMessages) {
stream.push({ type: "message_start", message });
stream.push({ type: "message_end", message });
currentContext.messages.push(message);
newMessages.push(message);
}
pendingMessages = [];
}
// Stream assistant response
const message = await streamAssistantResponse(currentContext, config, signal, stream, streamFn);
newMessages.push(message);
if (message.stopReason === "error" || message.stopReason === "aborted") {
stream.push({ type: "turn_end", message, toolResults: [] });
stream.push({ type: "agent_end", messages: newMessages });
stream.end(newMessages);
return;
}
// Check for tool calls
const toolCalls = message.content.filter((c) => c.type === "toolCall");
hasMoreToolCalls = toolCalls.length > 0;
const toolResults: ToolResultMessage[] = [];
if (hasMoreToolCalls) {
const toolExecution = await executeToolCalls(
currentContext.tools,
message,
signal,
stream,
config.getSteeringMessages,
);
toolResults.push(...toolExecution.toolResults);
steeringAfterTools = toolExecution.steeringMessages ?? null;
for (const result of toolResults) {
currentContext.messages.push(result);
newMessages.push(result);
}
}
stream.push({ type: "turn_end", message, toolResults });
// Get steering messages after turn completes
if (steeringAfterTools && steeringAfterTools.length > 0) {
pendingMessages = steeringAfterTools;
steeringAfterTools = null;
} else {
pendingMessages = (await config.getSteeringMessages?.()) || [];
}
}
// Agent would stop here. Check for follow-up messages.
const followUpMessages = (await config.getFollowUpMessages?.()) || [];
if (followUpMessages.length > 0) {
// Set as pending so inner loop processes them
pendingMessages = followUpMessages;
continue;
}
// No more messages, exit
break;
}
This loop is conceptually simple:
- User sends messages to AI.
- AI decides it needs tool calls, executes them, and gets results.
- AI checks results; if it needs more tools, repeat.
- AI finishes and checks for follow-up messages; continue if present, otherwise stop.
Auth Problem Looked Bigger Than It Was
I spent most of this afternoon deep in the weeds designing an auth bridge between an existing cluster of servers and a new service used by the same clients base across the servers. The initial conversations went straight to the “big” answers—Cognito, full OAuth flows, external identity plumbing everywhere—and for a while it felt like the only responsible path was also the most complex one.
Then, after nearly two hours, I realized what we really needed was a trusted issuer and a trusted verifier. We can use the existing platform to issue JWT bearer tokens from our user/client model, sign them with private keys we control, and let the new service verify them with public keys while enforcing claims like issuer, audience, scope, subject, and expiry.
Suddenly the design felt natural: no per-request callback to the issuer, no unnecessary moving parts, and clean attribution of every service call to a known user and client for metering and audit.
A good reminder that “production-grade” doesn’t always mean “maximal complexity”—sometimes the strongest design is the one that makes trust boundaries explicit and keeps the system understandable.