A cloud-ready, full-stack agentic memory platform for robotics teams. RoboMemory converts experiment telemetry, logs, notes, and artifacts into structured evidence that LLM agents can retrieve, cite, and use to diagnose failures, recommend follow-up experiments, and preserve engineering knowledge across sessions.
AI Systems / Robotics
January–July 2026
Full-stack AI engineer
Python, FastAPI, MERN, CockroachDB, Azure, AWS S3, Docker, Cloud
MVP implemented
Experiment evidence is scattered across logs, CSV files, videos, IMU traces, notebooks, cloud drives, and team conversations. When a failure repeats, researchers often reconstruct the same diagnosis from scratch because the previous run's context was never converted into durable, retrievable memory.
RoboMemory exposes typed ingestion APIs that normalize heterogeneous robot data, calculate quality signals, and persist artifacts, embeddings, summaries, and recommendations as linked records. An agentic orchestration layer gives LLMs controlled retrieval and memory-writing tools, so each response is grounded in cited experiment evidence and each decision becomes auditable application state.
The architecture separates the web client, typed API layer, agent orchestration, model providers, transactional memory, and object storage. Stable experiment and memory IDs connect every request, retrieved record, LLM response, recommendation, and artifact across local and cloud deployments.
React and Node.js capture task context, robot identity, outcomes, notes, metrics, and artifact manifests. MongoDB supports operational application documents and interactive session state.
Python services validate schemas, normalize heterogeneous telemetry, calculate explainable quality scores, and expose typed ingestion, retrieval, and chat endpoints. The agent service orchestrates retrieval, prompt assembly, provider calls, citations, and durable memory writes behind stable APIs.
CockroachDB stores transactional experiment state, vector memory, model outputs, citations, and agent recommendations. Azure and AWS cloud services support deployable application infrastructure; S3-compatible object storage keeps large logs, videos, traces, and other binary evidence outside the database.
Experiment ID, robot, task, outcome, failure mode, metrics, notes, timestamps, and S3 artifact references.
Quality score, data-quality flags, LLM summary, vector embedding, and links back to raw evidence.
Retrieved memory IDs, grounded diagnosis, proposed next experiment, and the recommendation timestamp.
{
"task_name": "drawer_opening",
"robot_name": "franka_panda",
"success": false,
"failure_mode": "gripper_slipped",
"notes": "Reached handle but failed to maintain grip.",
"metrics": {
"trajectory_smoothness": 0.62,
"hand_visibility": 0.41,
"imu_shake": 0.73,
"policy_confidence": 0.58
}
}Deterministic scores and data-quality notes expose weak visibility, excessive shake, low confidence, and other collection issues.
Each run becomes a compact memory containing task context, outcome, likely failure explanation, and suggested follow-up.
Metadata and quality filters narrow the candidate set; vector similarity then ranks semantically related failures such as gripper slip, camera occlusion, high IMU shake, and low policy confidence.
The chat API runs a retrieval-augmented agent loop: translate a question into structured filters, rank candidate memories by vector similarity, assemble evidence-bounded model context, return cited findings, and persist the recommendation for the next engineering session.
Model outputs are stored as versioned application records with source memory IDs, timestamps, and recommendation metadata, enabling auditability, reproducible debugging, and reuse across sessions.
Dockerized services, deterministic model adapters, and environment-based configuration keep the same API contracts testable locally and deployable to Azure or AWS with CockroachDB Cloud, object storage, and external LLM providers.
Why are our drawer-opening demonstrations failing?
Similar failed runs share high IMU shake, reduced hand visibility, and lower policy confidence. The strongest recurring failure mode is loss of gripper contact after reaching the handle.
Recommended next run: stabilize the camera mount, improve hand visibility, and repeat the task with a slower approach velocity before changing the policy.
Transactional experiment metadata, vector memory, and agent decisions remain connected and consistent instead of being split across separate operational databases.
Strategy-based LLM and embedding adapters support deterministic mocks, Azure OpenAI, Amazon Bedrock, or OpenAI without coupling business logic to one vendor. The orchestration layer owns tool schemas, retrieval context, retries, citations, and persistence.
Typed API payloads and stable identifiers make the data lifecycle reproducible across local Docker services and cloud deployments without changing the experiment-memory contract.
The next milestone is production validation across Azure and AWS: deploy the containerized services with CockroachDB Cloud, object storage, and a managed LLM endpoint; add observability and evaluation traces; then measure retrieval quality, grounded-answer accuracy, and recommendation usefulness on repeated robot failure modes.
Feature branch: feat/robomemory-mvp