01 · Prompt compiler
Ground the request
Serializes the user goal, current entities and predicates, admissible action signatures, constraints, and a few formatted demonstrations into one model request.
Embodied AI · PlanningFrozen LLM · Zero-shot
LLM Planners for Embodied Agents
A planning interface that turns high-level language into typed, admissible environment actions—using prompt-conditioned task decomposition, semantic translation, symbolic validation, and targeted repair around a frozen language model.
Model
Frozen GPT / Codex-style LLM
Input
Goal + world state + action schema
Output
Executable symbolic plan
Role
ML / Planning Engineer
The execution gap
A plan can sound correct to a human and still be unusable by an agent.
Frozen LLMs are strong at commonsense ordering and high-level decomposition, but their native output is unconstrained text. They may invent verbs, omit required arguments, reference objects absent from the scene, or propose actions whose preconditions are false. Larger models often improve semantic correctness while leaving the interface problem unchanged: the output still must be grounded into the environment's finite action space.
Architecture
Language → intermediate plan → executable actions
The LLM supplies semantic priors. Deterministic components own parsing, grounding, state transitions, and executability.
→
02 · Frozen LLM
Decompose the task
Produces an ordered sequence of mid-level subgoals and candidate action phrases without model fine-tuning.
→
03 · Semantic translator
Compile to typed IR
Maps free-form phrases and entity mentions onto canonical operators, resolves arguments, normalizes aliases, and rejects out-of-vocabulary actions.
04 · Validator
Simulate symbolically
Checks types, preconditions, ordering constraints, object availability, and goal satisfaction by applying each operator's effects to a symbolic state.
→
05 · Repair loop
Patch only the failure
Returns the failing step, violated predicate, and allowed operators to the model; the valid prefix remains fixed while the suffix is regenerated.
→
06 · Executor adapter
Dispatch admissible calls
Converts the verified intermediate representation into the exact environment API calls consumed by the robot or virtual agent.
Data contracts
The planner is organized around typed data
Input · planning_request.json
{
"goal": "put the red mug in the cabinet",
"state": {
"entities": ["robot", "red_mug", "cabinet"],
"predicates": [
["on", "red_mug", "counter"],
["closed", "cabinet"],
["hand_empty", "robot"]
]
},
"action_catalog": [
"OPEN(container)",
"PICK(object, support)",
"PLACE_IN(object, container)"
]
}Output · executable_plan.json
{
"subgoals": [
"make cabinet accessible",
"acquire target object",
"place object at goal"
],
"steps": [
{"op":"OPEN", "args":["cabinet"]},
{"op":"PICK", "args":["red_mug","counter"]},
{"op":"PLACE_IN", "args":["red_mug","cabinet"]}
],
"validation": {
"executable": true,
"goal_satisfied": true
}
}Key techniques
What makes zero-shot planning usable
Task decomposition
Reason at the subgoal level
The model first converts the high-level instruction into an ordered subgoal graph. This separates semantic reasoning—what must become true—from the low-level operator sequence used to make it true.
Demonstration conditioning
Teach the interface in context
Few-shot prompt examples pair goals and world states with correctly serialized plans. The examples constrain structure and argument order without updating model weights.
Semantic translation
Map language onto operators
A canonicalizer resolves “grab,” “take,” and “pick up” to PICK; an entity linker binds “it” to a scene object; a type checker ensures each argument matches the operator signature.
Constraint-guided repair
Return machine-readable failures
Instead of asking the model to “try again,” the validator reports a structured error such as precondition_not_met: open(cabinet), enabling a small, local correction.
Planner dataset
What is recorded for evaluation
Each episode is stored as a structured trace rather than only a final success bit. This makes failures attributable to generation, translation, validation, or execution.
Task record
Instruction, initial symbolic state, entity inventory, action catalog, and target predicates.
Model trace
Prompt version, demonstration IDs, raw completion, parsed subgoals, token usage, and latency.
Translation trace
Normalized operators, resolved entities, rejected phrases, ambiguity scores, and parser errors.
Verification trace
First invalid step, violated precondition, state before failure, repair count, executability, and goal satisfaction.
Evaluation
Measure semantics and execution separately
Logical correctness
Does the plan satisfy the requested goal under the symbolic transition model?
Executability
What fraction of steps use admissible operators, valid entities, correct types, and satisfied preconditions?
Interface efficiency
Time to first valid plan, parser rejection rate, number of repair rounds, and valid-prefix preservation.
Design insight
The LLM is not the executor
The strongest models are useful because they decompose unfamiliar goals and reason over semantic relationships. Reliability comes from placing a typed action boundary after the model—not from pretending free-form text is a control policy.
Limitation
Symbolic validity is not physical feasibility
A plan can satisfy symbolic preconditions yet fail because of reachability, collision, perception error, or unstable grasp geometry. The next layer should connect operators to motion planners and propagate geometric failure evidence back into repair.