Control Loop
The Control Loop is the core decision-making system of Cellium Agent. Each iteration contains 5 stages:
- Feature Extraction - The heuristic engine extracts current state features including stagnation iterations, progress trends, repetition scores, and context saturation.
- Rule Evaluation - Hard rules provide action candidate sets. For example, when a loop is detected, candidates are [redirect, compress].
- Bandit Tie-break - When there are multiple candidate actions, the Bandit intervenes using Thompson Sampling + Heuristic Bias to select the optimal action.
- Execute & Feedback - Execute the selected action (continue/retry/redirect/compress/terminate). FeedbackEvaluator provides segmented evaluation.
- Learning & Update - Use n-step return to accumulate rewards, update Bandit's Beta distribution parameters, and decay old data periodically.
Action Types
| Action | Description | Condition |
|---|---|---|
| continue | Continue current direction | Progress score > 0.5 or no stagnation |
| retry | Maintain direction, correct strategy | Mild stagnation or slow progress |
| redirect | Change direction/tool | High repetition or prolonged stagnation |
| compress | Compress context | High saturation or moderate stagnation |
| terminate | End session | Output loop detected |
State Features
The Agent perceives runtime state through FeatureExtractor:
- progress_score - Task completion estimate (0-1)
- stuck_iterations - Consecutive non-progress iterations
- progress_trend - EMA-smoothed progress trend (-1 to 1)
- context_saturation - Context fill level (0-1)
- repetition_score - Tool repetition call score
- error_rate - Tool call error rate
Hard Constraint Renderer
HardConstraintRenderer converts control decisions into natural language hard constraint prompts injected into the LLM system prompt. The constraint structure contains three sections: HARD CONSTRAINTS (non-negotiable criteria), CONTROL ACTION (specific behavior rules), and CONTEXT HINT (contextual guidance).
When the Agent gets stuck, the system injects a REDIRECT constraint, requiring the LLM to switch strategy and tools, forbidding repetition of the same path.
[HARD CONSTRAINTS]
You MUST obey control instruction. Violations = incorrect.
[CONTROL ACTION]
Action: REDIRECT
MUST: Change strategy, use different tool.
MUST NOT: Repeat same tool, continue same path.
[CONTEXT HINT]
Current approach may be stuck.