From 21ff9091bbf08e062576fed4c1156bac60dd47ef Mon Sep 17 00:00:00 2001 From: Rob Colbert Date: Thu, 30 Apr 2026 04:58:27 -0400 Subject: [PATCH] doc updates --- README.md | 232 ++++-- docs/architecture-stats.md | 1358 +++++++++++++++++++----------------- 2 files changed, 890 insertions(+), 700 deletions(-) diff --git a/README.md b/README.md index a433d95..621e150 100644 --- a/README.md +++ b/README.md @@ -2,67 +2,209 @@ A self-hosted **Agentic Engineering Platform (AEP)** — an IDE that drives autonomous AI agents to perform software engineering work on your behalf, running in your own environment. +## Quick Start + +```bash +# Install dependencies +pnpm install + +# Start backend (Terminal 1) +cd gadget-code +pnpm dev:backend + +# Start frontend (Terminal 2) +cd gadget-code/frontend +pnpm dev + +# Open browser to https://localhost:5174 +``` + ## Projects | Package | Role | | -------------- | ------------------------------------------------------------------------ | | `gadget-code` | Web service — agentic IDE, browser UI, API server | | `gadget-drone` | Worker process — runs the agentic workflow loop in workspace directories | -| `@gadget/ai` | Shared AI API abstraction — Ollama and OpenAI, called by both | - -## AI API Abstraction (`@gadget/ai`) - -All AI API calls throughout the Gadget Code ecosystem route through `@gadget/ai`. No consumer code imports Ollama or OpenAI SDKs directly. No consumer code checks `provider.sdk` after the factory call. The shared module translates Gadget Code's internal API contract into whatever provider is configured, and translates responses back to Gadget Code's internal types. - -See `packages/ai/README.md` for the full API reference. - -## Setup - -```bash -pnpm install -``` - -## Build - -```bash -pnpm -r build # build all packages -pnpm --filter @gadget/ai build -pnpm --filter gadget-drone build -pnpm --filter gadget-code build:backend -``` - -## Run - -```bash -# Backend -pnpm --filter gadget-code dev - -# Drone worker (in a project workspace directory) -pnpm --filter gadget-drone dev -``` +| `@gadget/ai` | Shared AI API abstraction — Ollama and OpenAI | +| `@gadget/api` | Shared TypeScript interfaces — common types across all packages | ## Architecture -### @gadget/ai +``` +┌──────────────┐ ┌──────────────┐ ┌──────────────┐ +│ Browser IDE │────▶│ gadget-code │────▶│ gadget-drone │ +│ (React 19) │◀────│ (Express 5) │◀────│ (Worker) │ +└──────────────┘ └──────────────┘ └──────────────┘ + │ │ │ + │ Socket.IO │ MongoDB │ Files + │ JWT Auth │ Redis │ Git +``` -AI API calls are handled by `@gadget/ai`, which both projects depend on. This keeps all AI SDK knowledge in one place, and currently implements: +### How It Works -- [AiApi](./packages/ai/src/api.ts) - abstract base class for all AI APIs/SDKs -- [OllamaAiApi](./packages/ai/src/ollama.ts) - Ollama API implementation -- [OpenAiApi](./packages/ai/src/openai.ts) - OpenAI API implementation +1. **User creates a project** in the browser IDE +2. **User selects a drone** instance to work on the project +3. **User enters a prompt** in the Chat Session view +4. **Prompt flows** IDE → Web → Drone via Socket.IO +5. **Drone executes** the Agentic Workflow Loop (AWL) +6. **Streaming response** flows back: thinking → response → tool calls +7. **Results persist** in MongoDB as ChatTurn records -### gadget-drone +## AI Provider Setup -gadget-drone is a headless process runs on end-user machines, connecting via Socket.IO to gadget-code, to receive and execute work orders for the agentic workflow loop. +Before using AI features, add a provider via CLI: -At startup, gadget-drone examines `process.cwd()` to determine if it's a workspace directory, and if so, starts a worker process that connects to gadget-code and waits for work orders. It processes work orders in project directories in the gadget-drone workspace directory, and communicates events, status, and results to the IDE via gadget-code's web services and Socket.IO. +```bash +# Add Ollama provider (no API key needed) +cd gadget-code +pnpm cli provider add "Local Ollama" ollama http://localhost:11434 -gadget-drone never connects directly to MongoDB or Redis — it communicates entirely through the Gadget Code API. +# Add OpenAI provider +pnpm cli provider add "OpenAI" openai https://api.openai.com $OPENAI_API_KEY -### gadget-code +# List providers +pnpm cli provider ls -gadget-code runs on server infrastructure (MongoDB, Redis, etc.) and serves the browser-based IDE. The IDE connects to gadget-code via Socket.IO to send and receive commands in chat sessions to gadget-drone. +# Probe provider for models (discovers available models) +pnpm cli provider probe +``` -gadget-code can be stacked on a single host for local development, and can achieve significant scale on a single host. It can also be deployed in tiers, potentially made of clusters, and the web tier can be horizontally scaled for production use with high availability. +## User Management -Libraries such as [redis-adapter](https://github.com/socketio/socket.io-redis-adapter) and [redis-emitter](https://github.com/socketio/socket.io-redis-emitter) are used for message routing and distribution in the Socket.IO library, handling the real-time message routing among gadget-code, gadget-drone, and the IDE running in the browser. +```bash +# Add a user +pnpm cli user add user@example.com password123 "Display Name" + +# Grant admin rights +pnpm cli admin grant user@example.com + +# Reset password +pnpm cli user password user@example.com newpassword +``` + +## Development + +### Prerequisites + +- Node.js 22+ +- pnpm 10+ +- MongoDB on `localhost:27017` +- Redis on `localhost:6379` +- SSL certificates in `ssl/` directory (for dev servers) + +### Build + +```bash +# Build all packages +pnpm -r build + +# Build specific package +pnpm --filter @gadget/ai build +pnpm --filter gadget-drone build +pnpm --filter gadget-code build +``` + +### Run Development Servers + +```bash +# Backend (gadget-code directory) +cd gadget-code +pnpm dev:backend +# Runs on https://localhost:3443 + +# Frontend (gadget-code/frontend directory) +cd gadget-code/frontend +pnpm dev +# Runs on https://localhost:5174 + +# Drone worker (in a workspace directory) +cd ~/my-gadget-workspace +pnpm --filter @gadget/drone dev +``` + +### Testing + +```bash +# Unit tests (gadget-code directory) +cd gadget-code +pnpm test + +# E2E tests with Playwright (requires running dev servers) +npx playwright test tests/e2e/project-manager.test.ts +npx playwright test tests/e2e/chat-session.test.ts + +# Seed test data for E2E tests +npx tsx scripts/seed-test-drones.ts +``` + +## Key Features + +### Project Manager + +- Create and manage projects +- Select available drones (filters out offline) +- View chat session history +- Create new chat sessions with provider/model selection + +### Chat Session View + +- Real-time streaming responses +- Collapsible thinking content +- Tool call summaries +- Model/mode selection per session +- Session statistics (tool calls, file ops, subagents) + +### Workspace Management + +- Each drone manages a workspace directory +- Projects cloned into workspace by slug +- Crash recovery via `.gadget/` directory +- Workspace persistence across restarts + +## AI API Abstraction + +All AI calls route through `@gadget/ai`. No consumer code imports Ollama or OpenAI SDKs directly. + +```typescript +// Correct: Use the factory +import { createAiApi } from '@gadget/ai'; +const ai = createAiApi(providerConfig); + +// Incorrect: Don't import SDKs directly +import { Ollama } from 'ollama'; // ❌ +``` + +See [`packages/ai/README.md`](./packages/ai/README.md) for the full API reference. + +## Documentation + +- [Architecture Overview](./docs/architecture-stats.md) +- [Socket Protocol](./docs/socket-protocol.md) +- [Workspace Management](./docs/gadget-workspace.md) +- [Drone Documentation](./gadget-drone/docs/gadget-drone.md) +- [UI Design Guide](./gadget-code/docs/ui-design-guide.md) + +## Monorepo Structure + +``` +gadget/ +├── packages/ai/ # @gadget/ai — AI API abstraction +├── packages/api/ # @gadget/api — Shared interfaces +├── gadget-code/ # Web service + browser IDE +│ ├── src/ # Backend (Express, Socket.IO, Mongoose) +│ ├── frontend/ # Frontend (React, Vite, Tailwind) +│ └── tests/ # Unit + E2E tests +├── gadget-drone/ # Worker process +│ ├── src/ # Drone implementation +│ └── docs/ # Drone documentation +└── docs/ # Architecture & protocol docs +``` + +## License + +Apache 2.0 — See [LICENSE](./LICENSE) for details. + +--- + +**Status:** Production-ready foundation with complete Chat Session UI +**Last Updated:** April 29, 2026 diff --git a/docs/architecture-stats.md b/docs/architecture-stats.md index abc099a..ed4a7f1 100644 --- a/docs/architecture-stats.md +++ b/docs/architecture-stats.md @@ -1,22 +1,566 @@ -# Gadget Code Architecture Review +# Gadget Code Architecture **Date:** April 29, 2026 -**Scope:** Socket.IO Communication System for Agentic Workflow Loop -**Status:** ✅ **FOUNDATION COMPLETE** - Ready for UI Implementation +**Status:** ✅ **CHAT SESSION UI COMPLETE** - Full end-to-end prompt flow operational ## Executive Summary -The Gadget Code architecture foundation is **100% complete** with all critical gaps filled. The Socket.IO infrastructure is fully implemented with message handlers, data models are consistent, and the agentic workflow loop can execute end-to-end. +Gadget Code is a fully functional Agentic Engineering Platform with complete Chat Session UI, AI provider management, and end-to-end prompt processing. Users can create projects, select drones, configure AI providers/models, and submit prompts that flow through the complete agentic workflow loop. -**Primary Blocker:** ✅ **RESOLVED** - Prompts now flow IDE→Web→Drone→Web→IDE with full event routing and persistence. +**Key Capabilities:** -**Completion Date:** April 29, 2026 -**Commits:** 5 commits on `feature/socket-protocol` branch -**Tests:** 21 unit tests passing (CodeSession + DroneSession) +- ✅ Complete Socket.IO communication (IDE↔Web↔Drone) +- ✅ AI Provider CLI management (add/list/probe/remove) +- ✅ Chat Session REST API with full CRUD +- ✅ Project Manager with drone selection & chat session lists +- ✅ Chat Session View with streaming responses +- ✅ Workspace persistence for crash recovery +- ✅ 21 unit tests passing -### Architectural Decision: Socket.IO Only (No Bull Queue) +--- -**Decision:** Bull queue will **not** be used. All message routing uses Socket.IO with directed delivery. +## System Architecture + +``` +┌─────────────────────────────────────────────────────────────────┐ +│ Gadget Code Ecosystem │ +├─────────────────────────────────────────────────────────────────┤ +│ │ +│ ┌──────────────┐ ┌──────────────┐ ┌──────────────┐ │ +│ │ Browser IDE │────▶│ gadget-code │────▶│ gadget-drone │ │ +│ │ (React 19) │◀────│ (Express 5) │◀────│ (Worker) │ │ +│ └──────────────┘ └──────────────┘ └──────────────┘ │ +│ │ │ │ │ +│ │ Socket.IO │ MongoDB │ Files │ +│ │ JWT Auth │ Redis Sessions │ Git │ +│ │ │ │ │ +│ ┌──────────────┐ ┌──────────────┐ ┌──────────────┐ │ +│ │ Chat View │ │ REST API │ │ AWL Loop │ │ +│ │ Streaming │ │ /projects │ │ Tool Calls │ │ +│ │ Tool Calls │ │ /sessions │ │ File Ops │ │ +│ └──────────────┘ └──────────────┘ └──────────────┘ │ +│ │ +└─────────────────────────────────────────────────────────────────┘ +``` + +### Core Components + +| Component | Technology | Role | +| ------------------- | -------------------------------- | ----------------------------------------------- | +| **Browser IDE** | React 19 + Vite 8 + Tailwind 4 | Agentic development environment | +| **gadget-code:web** | Express 5 + Socket.IO + Mongoose | API server, session management, routing | +| **gadget-code:ide** | TypeScript + React Router | Chat sessions, project management, file editing | +| **gadget-drone** | TypeScript + Socket.IO client | Agentic Workflow Loop executor | +| **@gadget/ai** | TypeScript abstraction | Unified AI API (Ollama/OpenAI) | +| **@gadget/api** | TypeScript interfaces | Shared type definitions | + +--- + +## Message Flow + +### Complete Prompt Journey + +``` +User Types Prompt + │ + ▼ +┌──────────────────┐ +│ Chat Session View│ +│ - Prompt Input │ +│ - submitPrompt │ +└────────┬─────────┘ + │ Socket: submitPrompt(content) + ▼ +┌──────────────────┐ +│ CodeSession │ +│ - Create ChatTurn│ +│ - Track drone │ +└────────┬─────────┘ + │ Socket: processWorkOrder(project, session, turn) + ▼ +┌──────────────────┐ +│ DroneSession │ +│ - Route to drone│ +└────────┬─────────┘ + │ Socket: processWorkOrder(...) + ▼ +┌──────────────────┐ +│ gadget-drone │ +│ - AgentService │ +│ - AWL Loop │ +└────────┬─────────┘ + │ Processing... + │ + ├─► thinking(content) ────┐ + ├─► response(content) ────┤ + ├─► toolCall(...) ────────┤ + └─► workOrderComplete() ──┤ + │ + ▼ + ┌──────────────────┐ + │ DroneSession │ + │ - Update Turn │ + │ - Route to IDE │ + └────────┬─────────┘ + │ Socket: thinking/response/toolCall + ▼ + ┌──────────────────┐ + │ Chat Session View│ + │ - Stream display│ + │ - Update UI │ + └──────────────────┘ +``` + +--- + +## AI Provider Architecture + +### Provider Management Flow + +``` +Admin CLI Database Runtime + │ │ │ + ├─ provider add ───────▶│ │ + │ (name, sdk, url) │ │ + │ │ │ + ├─ provider probe ─────▶│ AiProvider model │ + │ │ - Fetch models │ + │ │ - Discover caps │ + │ │ - Update in-place │ + │ │ │ + │ ├────────────────────▶│ createAiApi() + │ │ │ - SDK factory + │ │ │ - Ollama/OpenAI + │ │ │ + │ │◀────────────────────┤ + │ │ Runtime config │ + │ │ - _id, name, sdk │ + │ │ - baseUrl, apiKey │ + │ │ │ + └─ provider ls ────────▶│ List providers │ + │ (hide apiKey) │ +``` + +### Provider Data Model + +```typescript +interface IAiProvider { + _id: ObjectId; + name: string; + apiType: "ollama" | "openai"; + baseUrl: string; + apiKey: string; // Encrypted, select: false + enabled: boolean; + models: IAiModel[]; + lastModelRefresh: Date; +} + +interface IAiModel { + id: string; + name: string; + parameterCount?: number; + parameterLabel?: string; // "7b", "70b" + contextWindow?: number; + capabilities: { + canCallTools: boolean; + hasVision: boolean; + hasEmbedding: boolean; + hasThinking: boolean; + isInstructTuned: boolean; + }; + settings?: { + temperature?: number; + topP?: number; + topK?: number; + numCtx?: number; + }; +} +``` + +--- + +## Chat Session Architecture + +### Session Lifecycle + +``` +User Creates Session + │ + ▼ +┌────────────────────────────┐ +│ POST /api/v1/chat-sessions │ +│ - projectId │ +│ - providerId │ +│ - selectedModel │ +│ - mode (plan/build/..) │ +└───────────┬────────────────┘ + │ + ▼ +┌────────────────────────────┐ +│ ChatSession Document │ +│ - _id │ +│ - user, project │ +│ - provider, model │ +│ - mode, stats │ +│ - pins[] │ +└───────────┬────────────────┘ + │ + ▼ +┌────────────────────────────┐ +│ Chat Session View │ +│ - Messages list │ +│ - Prompt input │ +│ - Session sidebar │ +│ - Model selector │ +└───────────┬────────────────┘ + │ + ▼ +┌────────────────────────────┐ +│ User Submits Prompt │ +│ - submitPrompt event │ +│ - Creates ChatTurn │ +│ - Routes to drone │ +└───────────┬────────────────┘ + │ + ▼ +┌────────────────────────────┐ +│ Streaming Response │ +│ - thinking (collapsible) │ +│ - response (streaming) │ +│ - toolCall (summaries) │ +│ - workOrderComplete │ +└────────────────────────────┘ +``` + +### Chat Turn Structure + +```typescript +interface IChatTurn { + _id: ObjectId; + createdAt: Date; + user: IUser | ObjectId; + project: IProject | ObjectId; + session: IChatSession | ObjectId; + provider: IAiProvider | ObjectId; + llm: string; // Model ID + mode: ChatSessionMode; + status: "processing" | "finished" | "error"; + prompts: { + user: string; + system?: string; + }; + thinking?: string; + response?: string; + toolCalls: IChatToolCall[]; + subagents: IChatSubagentProcess[]; + stats: IChatTurnStats; +} + +interface IChatToolCall { + callId: string; + name: string; + parameters?: string; + response?: string; +} + +interface IChatTurnStats { + toolCallCount: number; + inputTokens: number; + thinkingTokenCount: number; + responseTokens: number; + durationMs: number; + durationLabel: string; // "hh:mm:ss" +} +``` + +--- + +## Workspace Architecture + +### Project Manager Layout + +``` +┌──────────────────────────────────────────────────────────────┐ +│ Header Bar │ +├──────────┬───────────────────────┬───────────────────────────┤ +│ │ │ │ +│ Projects │ Project Inspector │ Right Sidebar │ +│ │ │ │ +│ List │ - Name, Slug │ ┌─────────────────────┐ │ +│ │ - Git URL │ │ Available Drones │ │ +│ [proj-1] │ - Status, Created │ │ (40% of space) │ │ +│ [proj-2] │ - Delete button │ │ │ │ +│ │ │ │ drone-alpha ● │ │ +│ │ │ │ [Select] │ │ +│ │ │ │ │ │ +│ │ │ │ drone-beta ○ │ │ +│ │ │ │ [Select] │ │ +│ │ │ │ │ │ +│ │ │ └─────────────────────┘ │ +│ │ │ ┌─────────────────────┐ │ +│ │ │ │ Chat Sessions │ │ +│ │ │ │ (60% of space) │ │ +│ │ │ │ │ │ +│ │ │ │ Session 1 │ │ +│ │ │ │ [Open] │ │ +│ │ │ │ │ │ +│ │ │ │ Session 2 │ │ +│ │ │ │ [Open] │ │ +│ │ │ │ │ │ +│ │ │ └─────────────────────┘ │ +│ │ │ ┌─────────────────────┐ │ +│ │ │ │ [New Chat Session] │ │ +│ │ │ │ (fixed at bottom) │ │ +│ │ │ └─────────────────────┘ │ +├──────────┴───────────────────────┴───────────────────────────┤ +│ Status Bar │ +└──────────────────────────────────────────────────────────────┘ +``` + +### Drone Filtering Rules + +- **Shown in list:** `status !== 'offline'` (available + busy only) +- **Selectable:** `status === 'available'` only +- **Busy drones:** Visible but Select button disabled +- **Offline drones:** Completely filtered out + +### Chat Session Creation Flow + +1. User selects project → loads Project Manager +2. System fetches available drones (filters offline) +3. User clicks "Select" on a drone → stores in state +4. User clicks "[New Chat Session]" → opens modal +5. Modal shows: + - Provider dropdown (from `/api/v1/providers`) + - Model dropdown (from selected provider's models) + - Mode selector (Plan/Build/Test/Ship/Dev) + - Optional session name +6. User submits → `POST /api/v1/chat-sessions` +7. System creates session → navigates to `/projects/:projectId/chat-session/:sessionId` + +--- + +## Chat Session View Design + +### Layout Specification + +``` +┌─────────────────────────────────────────────────────────────┐ +│ Work Area (flex-1) │ Session Sidebar (w-80) │ +├──────────────────────────────┤ │ +│ │ Chat: Session Name │ +│ Chat Messages │ ID: abc123... │ +│ - User prompts (right) │ Model: llama3.2 (Ollama) ▼ │ +│ - Assistant responses │ Mode: BUILD ▼ │ +│ - Thinking (collapsible) │ │ +│ - Tool calls (1-line) ├──────────────────────────────┤ +│ │ TC │ FO │ SA │ +│ │ 12 │ 0 │ 0 │ +├──────────────────────────────┤ │ +│ [Prompt input ][Send] │ Project: my-project │ +└──────────────────────────────┴──────────────────────────────┘ +``` + +### Message Display Rules + +- **User messages:** Right-aligned, brand color background +- **Assistant responses:** Left-aligned, dark background +- **Thinking content:** Collapsible with toggle, shows char count +- **Tool calls:** One-line summary in chat (`⚡ toolName ✓`) +- **Tool call details:** Click to open inspector (future) +- **Auto-scroll:** Always scroll to bottom on new message + +### Session Sidebar Features + +- **Model selector:** Dropdown showing `Model Name (Provider Name)` +- **Mode selector:** Plan/Build/Test/Ship/Dev +- **Stats panel:** + - TC = Tool Calls + - FO = File Operations (future) + - SA = Subagents (future) +- **Session settings:** Cog icon → edit name, provider, mode + +--- + +## CLI Commands + +### Provider Management + +```bash +# Add a provider (auto-probes for models) +pnpm cli provider add "My Ollama" ollama http://localhost:11434 +pnpm cli provider add "OpenAI" openai https://api.openai.com $OPENAI_API_KEY + +# List all providers +pnpm cli provider ls + +# Enable/disable a provider +pnpm cli provider status active +pnpm cli provider status inactive + +# Remove a provider +pnpm cli provider remove + +# Probe provider for models (updates in-place) +pnpm cli provider probe +``` + +### User Management + +```bash +# Add user +pnpm cli user add user@example.com password123 "Display Name" + +# Reset password +pnpm cli user password user@example.com newpassword + +# Ban/unban user +pnpm cli user ban user@example.com +pnpm cli user unban user@example.com + +# Remove user +pnpm cli user remove user@example.com +``` + +### Admin Commands + +```bash +# Grant admin rights +pnpm cli admin grant user@example.com + +# Revoke admin rights +pnpm cli admin revoke user@example.com +``` + +--- + +## Testing Strategy + +### Unit Tests + +**Location:** `gadget-code/tests/**/*.test.ts` + +**Coverage:** + +- `code-session.test.ts` - Prompt submission flow (9 tests) +- `drone-session.test.ts` - Event routing (12 tests) +- Total: 21 tests passing + +**Run:** + +```bash +cd gadget-code +pnpm test +``` + +### E2E Tests (Playwright) + +**Location:** `gadget-code/tests/e2e/**/*.test.ts` + +**Test Scenarios:** + +1. **Project Manager Layout** - Three-column layout verification +2. **Drone Filtering** - Only available/busy drones shown +3. **Drone Selection** - Select buttons functional +4. **New Session Button** - Visible, disabled until drone selected +5. **Chat Session Creation** - Modal flow with provider/model selection +6. **Prompt Submission** - End-to-end prompt flow +7. **Response Streaming** - Thinking, response, tool calls displayed + +**Run:** + +```bash +# Prerequisites: MongoDB, Redis, dev servers running +cd gadget-code + +# Start backend (in gadget-code directory) +pnpm dev:backend + +# Start frontend (in gadget-code/frontend directory) +pnpm dev + +# Run tests (in separate terminal) +npx playwright test tests/e2e/project-manager.test.ts +npx playwright test tests/e2e/chat-session.test.ts +``` + +**Test Data Seeding:** + +```bash +# Seed mock drones for testing +cd gadget-code +npx tsx scripts/seed-test-drones.ts +``` + +This creates: + +- 2 available drones +- 1 busy drone +- 2 offline drones (filtered from UI) + +--- + +## Development Setup + +### Prerequisites + +- Node.js 22+ +- pnpm 10+ +- MongoDB on `localhost:27017` +- Redis on `localhost:6379` +- SSL certificates in `ssl/` directory + +### Installation + +```bash +# Install all workspace packages +pnpm install + +# Build all packages +pnpm -r build +``` + +### Running Development Servers + +```bash +# Terminal 1: Backend (gadget-code directory) +cd gadget-code +pnpm dev:backend +# Runs on https://localhost:3443 + +# Terminal 2: Frontend (gadget-code/frontend directory) +cd gadget-code/frontend +pnpm dev +# Runs on https://localhost:5174 + +# Terminal 3: Drone (separate workspace directory) +cd ~/my-gadget-workspace +pnpm --filter @gadget/drone dev +# Registers with platform, waits for work orders +``` + +### Building for Production + +```bash +# Build all packages +pnpm -r build + +# Build backend only +cd gadget-code +pnpm build:backend # Outputs to dist/ + +# Build frontend only +cd gadget-code +pnpm build:frontend # Outputs to frontend/dist/ +``` + +--- + +## Key Architectural Decisions + +### 1. Socket.IO Only (No Bull Queue) + +**Decision:** All message routing uses Socket.IO with directed delivery. **Rationale:** @@ -24,659 +568,163 @@ The Gadget Code architecture foundation is **100% complete** with all critical g - Eliminates Redis dependency for end users - Simpler deployment model -**Recovery Strategy:** Workspace persistence via `.gadget/` directory (see Section 7: Workspace Persistence Architecture). +**Recovery:** Workspace persistence via `.gadget/` directory + +### 2. CLI-Only Administration + +**Decision:** No admin UI - all administrative functions via CLI. + +**Rationale:** + +- Reduces attack surface +- Simpler codebase +- Admins comfortable with CLI + +**Commands:** User management, provider management, admin rights + +### 3. AI API Abstraction + +**Decision:** All AI calls route through `@gadget/ai`. No direct SDK imports. + +**Rationale:** + +- Single source of truth for AI interfaces +- Easy to add new providers +- Consistent error handling + +### 4. Workspace Persistence + +**Decision:** Drones maintain `.gadget/` directory for crash recovery. + +**Rationale:** + +- No external dependencies for drones +- Survives restarts +- Enables retry routing + +### 5. Model Selection in UI + +**Decision:** Users select provider + model per chat session, changeable anytime. + +**Rationale:** + +- Flexibility for different tasks +- Cost optimization +- Model experimentation --- -## 1. Architecture Soundness - -### ✅ What's Working Well - -1. **Socket.IO Server Setup** (`gadget-code/src/services/socket.ts`) ✅ - - Proper authentication middleware distinguishing Code (IDE) vs Drone sessions - - Session management via `CodeSession` and `DroneSession` classes - - Clean separation of concerns with session types - -2. **Event Interface Definitions** (`packages/api/src/messages/*.ts`) ✅ - - `ClientToServerEvents` and `ServerToClientEvents` properly typed - - Message signatures match between IDE↔Web↔Drone - - Callback-based request/response pattern is sound - -3. **Data Model Foundation** (`packages/api/src/interfaces/*.ts`) ✅ - - `IChatTurn`, `IChatSession`, `IChatToolCall` capture AWL state - - `WorkspaceMode` enum correctly models mutual exclusion - - Socket routing architecture is correct - -4. **Message Handlers** ✅ **NEW** - - `CodeSession.onSubmitPrompt()` creates ChatTurn and sends work orders - - `DroneSession` routes thinking, response, toolCall, workOrderComplete - - SocketService maintains chat session reverse index - -5. **AWL Event Emissions** ✅ **NEW** - - AgentService.process() emits streaming events - - workOrderComplete signals turn completion - -6. **Workspace Persistence** ✅ **NEW** - - `.gadget/workspace.json` for crash recovery - - Work order cache for retry routing - - Crash recovery socket events implemented - -### ❌ Critical Design Issues - ALL RESOLVED ✅ - -#### Issue 1: Duplicate `DroneStatus` Enum ✅ **FIXED** - -**Location:** `packages/api/src/interfaces/drone-registration.ts` vs `gadget-drone/src/services/platform.ts` - -**Resolution:** Removed local enum from `gadget-drone/src/services/platform.ts`, now imports from `@gadget/api`. - -#### Issue 2: Conflicting `IAiProvider` Interfaces ✅ **FIXED** - -**Location:** `gadget-drone/src/services/ai.ts` - -**Resolution:** Created `mapDbProviderToConfig()` mapper function that converts `IAiProvider | ObjectId` → runtime config before calling `createAiApi()`. - -#### Issue 3: Missing `callId` in Tool Call Message ✅ **FIXED** - -**Location:** `packages/api/src/messages/drone.ts:26-30` - -**Resolution:** Added `callId: string` as first parameter to `ToolCallMessage`. Also added `callId` field to `ChatToolCallSchema` in `gadget-code/src/models/chat-turn.ts`. - -#### Additional Issues Fixed: - -- **ChatTurnStats Schema Mismatch** ✅ - Standardized on `thinkingTokenCount` in schema and interface -- **Missing User Reference** ✅ - Added type guard in `buildSessionContext()` to handle ObjectId vs populated session - ---- - -## 2. Completeness Analysis - -### 2.1 Socket.IO Message Flow - ALL OPERATIONAL ✅ - -| Message | IDE→Web | Web→Drone | Drone→Web | Web→IDE | Status | -| ---------------------- | ------- | -------------- | ----------- | ------------------ | ------------- | -| `requestSessionLock` | ✅ Sent | ✅ Routed | ✅ Received | ✅ Implemented | ✅ Complete | -| `requestWorkspaceMode` | ✅ Sent | ✅ Routed | ✅ Received | ⚠️ Deferred | ⚠️ Deferred | -| `submitPrompt` | ✅ Sent | ✅ Handled | ✅ Sent | ✅ Implemented | ✅ Complete | -| `processWorkOrder` | N/A | ✅ Sent | ✅ Received | ✅ Implemented | ✅ Complete | -| `thinking` | N/A | ✅ Routed | ✅ Sent | ✅ Emitted | ✅ Complete | -| `response` | N/A | ✅ Routed | ✅ Sent | ✅ Emitted | ✅ Complete | -| `toolCall` | N/A | ✅ Routed | ✅ Sent | ✅ Emitted | ✅ Complete | -| `workOrderComplete` | N/A | ✅ Routed | ✅ Sent | ✅ Emitted | ✅ Complete | -| `requestCrashRecovery` | N/A | ✅ Sent | ✅ Received | ✅ Implemented | ✅ Complete | - -**Assessment:** ✅ **End-to-end flow operational**. Forward path (IDE→Drone) and return path (Drone→IDE) fully implemented with crash recovery. - -### 2.2 gadget-code:web Implementation Gaps - ALL FILLED ✅ - -#### `submitPrompt` Handler ✅ **IMPLEMENTED** - -**File:** `gadget-code/src/lib/code-session.ts:95-167` - -**Implementation:** -- Creates `ChatTurn` document with status `Processing` -- Tracks selected drone, chat session, and project -- Emits `processWorkOrder` to drone with full context -- Updates ChatTurn on drone acknowledgment/rejection -- Sets current turn ID on drone session for event routing - -#### Drone→IDE Event Routing ✅ **IMPLEMENTED** - -**File:** `gadget-code/src/lib/drone-session.ts:21-240` - -**Implementation:** -- `onThinking()` - routes thinking content to IDE, updates ChatTurn -- `onResponse()` - routes response content to IDE, updates ChatTurn -- `onToolCall()` - routes tool calls to IDE, updates ChatTurn with call details -- `onWorkOrderComplete()` - finalizes ChatTurn status, emits to IDE -- `onRequestCrashRecovery()` - handles drone crash recovery requests - -#### ChatTurn Persistence Updates ✅ **IMPLEMENTED** - -**File:** `gadget-code/src/lib/drone-session.ts` (inline in event handlers) - -**Implementation:** -- Each event handler updates ChatTurn incrementally -- `ChatTurn.findByIdAndUpdate()` for thinking/response -- Direct model manipulation for tool calls (pushes to array, updates stats) -- Final status update on workOrderComplete - -### 2.3 gadget-drone Implementation Gaps - ALL FILLED ✅ - -#### Work Order Acknowledgment Flow ✅ **IMPLEMENTED** - -**File:** `gadget-drone/src/gadget-drone.ts:209-257` - -**Implementation:** -- Validates socket connection before processing -- Writes work order cache BEFORE processing (crash recovery) -- Accepts work order with `cb(true)` -- Removes cache AFTER successful completion -- Leaves cache in place on error for recovery - -#### Socket Event Emissions ✅ **IMPLEMENTED** - -**File:** `gadget-drone/src/services/agent.ts:46-107` - -**Implementation:** -- `AgentService.process()` accepts `socket: DroneSocket` parameter -- Emits `thinking` when response.thinking is present -- Emits `response` when response.response is present -- Emits `toolCall` with callId, name, arguments, result for each tool call -- Emits `workOrderComplete` when AWL loop exits - -#### Workspace Mode Management ⚠️ **DEFERRED** - -**File:** `gadget-drone/src/gadget-drone.ts:168-188` - -**Status:** Deferred to integration testing phase. Current implementation sets workspace mode but doesn't emit transitions. Can be added during UI integration when mode indicators are needed. - -### 2.4 Data Model Inconsistencies - ALL RESOLVED ✅ - -#### ChatTurn Schema Mismatch ✅ **FIXED** - -**File:** `gadget-code/src/models/chat-turn.ts:22-29` - -**Resolution:** Standardized on `thinkingTokenCount` in both schema and interface. - -#### Missing User Reference ✅ **FIXED** - -**File:** `gadget-drone/src/services/agent.ts:101-120` - -**Resolution:** Added type guard check: -```typescript -const session = workOrder.turn.session; -if (session instanceof Types.ObjectId || !session.user) { - throw new Error("ChatSession must be populated with user data"); -} -``` - -#### Additional Data Model Updates ✅ - -- Added `provider` and `selectedModel` fields to `IChatSession` and `ChatSessionSchema` -- Added `workspaceId` field to `IDroneRegistration` for crash recovery routing -- Added `callId` field to `ChatToolCallSchema` to match `IChatToolCall` interface - ---- - -## 3. Conflicts and Redundancies - -### 3.1 Documentation Conflicts - -**Work Order Interface Discrepancy:** - -- `gadget-code/docs/agentic-workflow-loop.md:21-52` defines `IWorkOrder` with `provider.apiKey` and `context: IChatMessage[]` -- `gadget-drone/docs/agentic-workflow-loop.md:15-62` defines `IWorkOrder` with `provider.sdk` and `chatSession.context` -- Actual implementation in `gadget-drone/src/services/agent.ts:24-28` uses `IAgentWorkOrder` with `turn: IChatTurn` and `context: IChatTurn[]` - -**Resolution:** Delete both markdown docs' interface definitions. Reference `@gadget/api` interfaces only. Update docs to match `IAgentWorkOrder`. - -### 3.2 Bull Queue vs Socket.IO (Resolved) - -**Documentation states:** - -- `gadget-drone/docs/agentic-workflow-loop.md:10-12`: "Each Gadget Drone registered by the User implements a named Bull job queue" -- `gadget-drone/AGENTS.md`: "Queue: Bull queue named `gadget-drone`, job type `prompt`" - -**Reality:** `gadget-drone/src/gadget-drone.ts` uses **Socket.IO** for work order delivery, not Bull. There's no Bull queue setup in the drone. - -**Decision:** ✅ **Option A (Socket.IO only)** — Bull references are legacy and must be removed from all documentation. - -**Recovery from Drone Crash:** Handled via workspace persistence in `.gadget/` directory (see Section 7). When a drone restarts: - -1. It validates/creates `.gadget/workspace.json` with workspace UUID -2. Web service reads workspace state to route retry to same directory -3. Agent can resume from last persisted ChatTurn state - -### 3.3 Redundant Service Layers - -**Observation:** `gadget-code/src/services/api-client.ts` exists alongside direct Mongoose model usage. - -**Check:** `gadget-code/src/controllers/api/v1/drone.ts` likely duplicates `DroneService` methods. - -**Action:** Audit API controllers — if they just proxy service methods, remove and call services directly from Socket handlers. - ---- - -## 4. Implementation Roadmap - ✅ COMPLETE - -### Phase 1: Fix Type Errors ✅ **COMPLETE** -- ✅ Resolved `IAiProvider` conflict with mapper function -- ✅ Fixed `DroneStatus` duplication -- ✅ Fixed `ChatTurnStats` field names -- ✅ Added `callId` to ToolCallMessage and ChatToolCallSchema - -### Phase 2: Implement Prompt Submission ✅ **COMPLETE** -- ✅ Implemented `CodeSession.onSubmitPrompt()` -- ✅ Added drone/chat session tracking to CodeSession -- ✅ Added `provider` and `selectedModel` to ChatSession - -### Phase 3: Implement Event Routing ✅ **COMPLETE** -- ✅ Added DroneSession event handlers (thinking, response, toolCall, workOrderComplete) -- ✅ Implemented routing logic with ChatTurn updates -- ✅ Added `getCodeSessionByChatSessionId()` to SocketService -- ✅ Added crash recovery handler (`onRequestCrashRecovery`) - -### Phase 4: Emit Events from AWL ✅ **COMPLETE** -- ✅ Pass socket into `AgentService.process()` -- ✅ Added emissions for thinking, response, toolCall -- ✅ Emit workOrderComplete on finish - -### Phase 5: Workspace Persistence ✅ **COMPLETE** -- ✅ Created `WorkspaceService` with `.gadget/` directory management -- ✅ Implemented `workspace.json` for persistent identity -- ✅ Write work order cache during processing -- ✅ Update drone registration with `workspaceId` -- ✅ Implement crash recovery socket events - -### Phase 6: End-to-End Test ⏳ **READY FOR INTEGRATION** -- Backend foundation complete -- Unit tests passing (21 tests) -- Ready for UI integration testing - ---- - -## 5. Risk Assessment - -### High Risk - -1. **No Streaming in @gadget/ai** - - `AiApi.chat()` returns `Promise`, not async iterable - - Cannot stream tokens in real-time without refactoring - - **Mitigation:** Add `streamCallback` parameter (already exists in signature) but implement it in Ollama/OpenAI clients - -2. **No Error Propagation** - - If drone crashes mid-turn, IDE hangs forever - - **Mitigation:** Add timeout + heartbeat mechanism - -3. **No Workspace Persistence Layer** ⚠️ **CRITICAL** - - Drone restart loses all context: which workspace, which projects, which chat session - - Cannot retry work orders without knowing original workspace directory - - **Mitigation:** Implement `.gadget/` directory persistence (see Section 7) - -### Medium Risk - -1. **Session State Not Persisted** - - `CodeSession` and `DroneSession` are in-memory - - Server restart loses all active sessions - - **Mitigation:** Store session state in Redis - -2. **No Concurrency Control** - - Multiple prompts can queue for same drone - - Drone processes one at a time but doesn't reject extras - - **Mitigation:** Check `DroneStatus.Busy` before accepting work - -### Low Risk - -1. **TypeScript Strict Mode Violations** - - Several `any` and missing null checks - - Build passes but runtime errors possible - - **Mitigation:** Enable `noUncheckedIndexedAccess` in drone - ---- - -## 6. Recommended Next Steps - -1. **Fix TypeScript errors** in `gadget-drone/src/services/agent.ts` (Phase 1) -2. **Implement `submitPrompt` handler** (Phase 2, Task 2.1) -3. **Add basic event routing** (Phase 3, minimal viable path) -4. **Test end-to-end** with stubbed tool calls -5. **Iterate** on streaming, error handling, and persistence - ---- - -## Appendix A: File Inventory - -### Core Socket Implementation - -- `gadget-code/src/services/socket.ts` — Socket.IO server setup ✅ -- `gadget-code/src/lib/socket-session.ts` — Base session class ✅ -- `gadget-code/src/lib/code-session.ts` — IDE session (partial) -- `gadget-code/src/lib/drone-session.ts` — Drone session (minimal) -- `gadget-drone/src/gadget-drone.ts` — Drone client ✅ - -### Data Models - -- `packages/api/src/interfaces/*.ts` — TypeScript interfaces ✅ -- `gadget-code/src/models/*.ts` — Mongoose schemas ✅ -- `gadget-drone/src/models/` — None (drone is stateless) - -### Message Definitions - -- `packages/api/src/messages/socket.ts` — Event map ✅ -- `packages/api/src/messages/ide.ts` — IDE→Web messages ✅ -- `packages/api/src/messages/drone.ts` — Drone messages (incomplete) - -### AI Integration - -- `packages/ai/src/api.ts` — AI interface ✅ -- `packages/ai/src/ollama.ts` — Ollama client ✅ -- `packages/ai/src/openai.ts` — OpenAI client ✅ -- `gadget-drone/src/services/ai.ts` — AI service wrapper ✅ -- `gadget-drone/src/services/agent.ts` — AWL implementation (partial) - ---- - -## Appendix B: Build Status - ✅ ALL PASS - -| Package | Build Status | Notes | -| -------------- | ------------ | -------------------------------- | -| `@gadget/api` | ✅ Passes | Type definitions only | -| `@gadget/ai` | ✅ Passes | AI SDK abstraction | -| `gadget-code` | ✅ Passes | Web server + frontend builds | -| `gadget-drone` | ✅ Passes | All type errors resolved | - -**Build Command:** `pnpm -r build` - All packages build successfully - ---- - -## 7. Workspace Persistence Architecture - -### 7.1 Design Goals - -1. **No External Dependencies:** End users should not need to run Redis, MongoDB, or other infrastructure just to run `gadget-drone` -2. **Crash Recovery:** When a drone crashes mid-work-order, it must be able to resume in the same workspace with the same project state -3. **Workspace Identity:** Each workspace directory needs a persistent, unique identifier that survives drone restarts -4. **State Visibility:** Both the drone and web service must be able to inspect workspace state at any time - -### 7.2 Directory Structure +## File Structure ``` -/ -├── .gadget/ -│ ├── workspace.json # Persistent workspace identity & state -│ ├── work-order.json # Active work order cache (deleted when complete) -│ └── logs/ -│ └── drone.log # Drone execution logs -├── / # Project directories managed by this workspace -├── / -└── ... +gadget/ +├── packages/ +│ ├── ai/ # @gadget/ai - AI API abstraction +│ │ ├── src/ +│ │ │ ├── api.ts # Core interfaces +│ │ │ ├── ollama.ts # Ollama implementation +│ │ │ ├── openai.ts # OpenAI implementation +│ │ │ └── index.ts # Factory function +│ │ └── README.md +│ └── api/ # @gadget/api - Shared types +│ └── src/ +│ └── interfaces/ # TypeScript interfaces +│ +├── gadget-code/ # Web service + IDE +│ ├── src/ +│ │ ├── web-app.ts # Express app entry +│ │ ├── web-cli.ts # CLI entry +│ │ ├── controllers/ +│ │ │ └── api/v1/ +│ │ │ ├── chat-session.ts +│ │ │ ├── provider.ts +│ │ │ ├── project.ts +│ │ │ └── drone.ts +│ │ ├── services/ +│ │ │ ├── chat-session.ts +│ │ │ ├── socket.ts +│ │ │ └── drone.ts +│ │ ├── models/ +│ │ │ ├── chat-session.ts +│ │ │ ├── chat-turn.ts +│ │ │ └── ai-provider.ts +│ │ └── lib/ +│ │ ├── code-session.ts +│ │ └── drone-session.ts +│ ├── frontend/ +│ │ └── src/ +│ │ ├── App.tsx +│ │ ├── pages/ +│ │ │ ├── ProjectManager.tsx +│ │ │ ├── ChatSessionView.tsx +│ │ │ └── Home.tsx +│ │ ├── components/ +│ │ │ ├── Header.tsx +│ │ │ └── StatusBar.tsx +│ │ └── lib/ +│ │ ├── api.ts +│ │ └── socket.ts +│ ├── tests/ +│ │ ├── code-session.test.ts +│ │ ├── drone-session.test.ts +│ │ └── e2e/ +│ │ ├── project-manager.test.ts +│ │ └── chat-session.test.ts +│ └── scripts/ +│ └── seed-test-drones.ts +│ +├── gadget-drone/ # Worker process +│ ├── src/ +│ │ ├── gadget-drone.ts +│ │ └── services/ +│ │ ├── agent.ts # AWL implementation +│ │ ├── ai.ts # AI service +│ │ ├── platform.ts # Platform registration +│ │ └── workspace.ts # Workspace persistence +│ └── docs/ +│ └── gadget-drone.md +│ +└── docs/ + ├── architecture-stats.md # This document + ├── socket-protocol.md + └── gadget-workspace.md ``` -### 7.3 File Specifications - -#### `.gadget/workspace.json` - -**Created:** When drone starts in a directory (new or existing workspace) -**Updated:** When chat session lock acquired/released, projects added/removed -**Deleted:** Never (only if user manually deletes workspace) - -```typescript -interface WorkspaceData { - workspaceId: string; // UUID v4, immutable once created - createdAt: string; // ISO 8601 timestamp - hostname: string; // Machine hostname where drone runs - workspaceDir: string; // Absolute path to workspace directory - - // Active session state (null when idle) - chatSession: { - _id: string; // MongoDB ChatSession._id - name: string; // Session name for display - lockedAt: string; // ISO 8601 timestamp - } | null; - - // Project currently being worked on (null when idle) - lockedProject: { - _id: string; // MongoDB Project._id - slug: string; // Project slug (directory name) - gitUrl: string; // Remote git URL - lockedAt: string; // ISO 8601 timestamp - } | null; - - // All projects cloned into this workspace - projects: Array<{ - _id: string; - slug: string; - gitUrl: string; - clonedAt: string; - lastSyncAt: string; - }>; - - // Drone registration (updated each startup) - registration: { - _id: string; // MongoDB DroneRegistration._id - status: string; // Current drone status - registeredAt: string; // ISO 8601 timestamp - } | null; -} -``` - -**Example:** - -```json -{ - "workspaceId": "550e8400-e29b-41d4-a716-446655440000", - "createdAt": "2026-04-29T19:30:00.000Z", - "hostname": "mysterymachine", - "workspaceDir": "/home/rob/projects/my-gadget-workspace", - "chatSession": { - "_id": "65f8a9b2c3d4e5f6a7b8c9d0", - "name": "Fix authentication bug", - "lockedAt": "2026-04-29T20:15:00.000Z" - }, - "lockedProject": { - "_id": "65f8a9b2c3d4e5f6a7b8c9d1", - "slug": "auth-service", - "gitUrl": "https://github.com/user/auth-service.git", - "lockedAt": "2026-04-29T20:15:00.000Z" - }, - "projects": [ - { - "_id": "65f8a9b2c3d4e5f6a7b8c9d1", - "slug": "auth-service", - "gitUrl": "https://github.com/user/auth-service.git", - "clonedAt": "2026-04-29T19:30:00.000Z", - "lastSyncAt": "2026-04-29T20:15:00.000Z" - } - ], - "registration": { - "_id": "65f8a9b2c3d4e5f6a7b8c9d2", - "status": "busy", - "registeredAt": "2026-04-29T19:30:00.000Z" - } -} -``` - -#### `.gadget/work-order.json` - -**Created:** When `processWorkOrder` message received -**Updated:** Not updated (immutable cache) -**Deleted:** When work order completes (success or error) - -```typescript -interface WorkOrderCache { - turnId: string; // ChatTurn._id for persistence updates - chatSessionId: string; // For routing events back to IDE - projectId: string; // For file operations - workOrderId: string; // Unique ID for this work order instance - receivedAt: string; // ISO 8601 timestamp - prompt: string; // User's prompt (for retry context) - status: "processing" | "completed" | "error"; - error?: string; // Error message if status === 'error' -} -``` - -**Purpose:** If drone crashes while this file exists, the web service knows: - -- Which ChatTurn was being processed -- Which workspace to route the retry to -- What prompt needs to be re-processed - -### 7.4 Drone Startup Sequence - -```typescript -// Pseudocode for gadget-drone.ts startup -async start(): Promise { - // Step 1: Validate/create workspace (BEFORE anything else) - await this.validateWorkspace(); - - // Step 2: Get user credentials - const credentials = await this.getUserCredentials(); - - // Step 3: Register with platform (includes workspaceId) - this.registration = await PlatformService.register( - credentials.email, - credentials.password, - process.cwd(), - this.workspaceData.workspaceId, // NEW parameter - ); - - // Step 4: Update workspace.json with registration - this.workspaceData.registration = { - _id: this.registration._id.toHexString(), - status: 'starting', - registeredAt: new Date().toISOString(), - }; - await this.writeWorkspaceData(); - - // Step 5: Connect Socket.IO - await this.connectSocket(); - - // Step 6: Check for incomplete work order (crash recovery) - await this.checkCrashRecovery(); - - // Step 7: Mark as available - await PlatformService.setStatus(DroneStatus.Available); - this.workspaceData.registration!.status = 'available'; - await this.writeWorkspaceData(); -} - -async checkCrashRecovery(): Promise { - const workOrderFile = path.join(this.gadgetDir, 'work-order.json'); - - if (fs.existsSync(workOrderFile)) { - const cache = JSON.parse(await fs.promises.readFile(workOrderFile, 'utf-8')); - - this.log.warn('incomplete work order found - crash recovery needed', { - turnId: cache.turnId, - prompt: cache.prompt, - }); - - // Notify web service that this workspace has pending recovery - this.socket.emit('requestCrashRecovery', { - workspaceId: this.workspaceData.workspaceId, - turnId: cache.turnId, - chatSessionId: cache.chatSessionId, - }); - - // DO NOT delete work-order.json yet - wait for web service instruction - } -} -``` - -### 7.5 Web Service: Crash Recovery Flow - -When web service receives `requestCrashRecovery`: - -1. **Fetch ChatTurn** by `turnId` -2. **Check Turn Status:** - - If `status === 'finished'`: Acknowledge, tell drone to delete cache (turn completed before crash notification) - - If `status === 'processing'`: Queue retry for this workspace -3. **Route Retry:** When retrying, filter drones by `workspaceId` to ensure same workspace handles it -4. **Acknowledge:** Tell drone it can delete `work-order.json` - -```typescript -// In gadget-code/src/lib/drone-session.ts -async onRequestCrashRecovery(data: { - workspaceId: string; - turnId: string; - chatSessionId: string; -}): Promise { - const turn = await ChatTurn.findById(data.turnId); - - if (!turn) { - this.socket.emit('crashRecoveryResponse', { - turnId: data.turnId, - action: 'discard', // Turn doesn't exist, delete cache - }); - return; - } - - if (turn.status === ChatTurnStatus.Finished) { - this.socket.emit('crashRecoveryResponse', { - turnId: data.turnId, - action: 'discard', // Already done, delete cache - }); - return; - } - - // Turn is still processing - mark for retry - turn.status = ChatTurnStatus.Error; - turn.response = 'Drone crashed during processing - retrying'; - await turn.save(); - - this.socket.emit('crashRecoveryResponse', { - turnId: data.turnId, - action: 'retry', - retryDelay: 5000, // Wait 5 seconds before retry - }); - - // Schedule retry (will route to same workspaceId) - setTimeout(() => { - this.retryWorkOrder(turn); - }, 5000); -} -``` - -### 7.6 Workspace-Aware Drone Selection - -When selecting a drone for a work order: - -```typescript -// In gadget-code/src/lib/code-session.ts -async onSubmitPrompt(content: string): Promise { - // ... create ChatTurn ... - - // Prefer drone in same workspace (for continuity) - let targetDrone: DroneSession; - - if (this.chatSession.workspaceId) { - // Try to find drone in same workspace - targetDrone = SocketService.getDroneSessionByWorkspaceId( - this.chatSession.workspaceId - ); - - if (!targetDrone) { - this.log.warn('workspace drone unavailable, selecting alternative'); - // Fall through to any available drone - } - } - - if (!targetDrone) { - // Select any available drone for this user - targetDrone = SocketService.getAvailableDroneForUser(this.user); - } - - // Include workspaceId in work order for persistence - targetDrone.socket.emit('processWorkOrder', { - // ... existing fields ... - workspaceId: this.chatSession.workspaceId, - }); -} -``` - -### 7.7 Implementation Checklist - ✅ ALL COMPLETE - -- [x] Create `WorkspaceService` in `gadget-drone/src/services/workspace.ts` -- [x] Implement `validateWorkspace()` and `writeWorkspaceData()` -- [x] Update `PlatformService.register()` to accept `workspaceId` -- [x] Add `workspaceId` field to `IDroneRegistration` interface and model -- [x] Add `workspaceId` field to `IChatSession` interface and model (deferred - not needed for basic recovery) -- [x] Implement `work-order.json` cache write/remove in `onProcessWorkOrder()` -- [x] Implement `requestCrashRecovery` socket handler in drone -- [x] Implement `crashRecoveryResponse` socket handler in web service -- [x] Add workspace tracking in CodeSession (selectedDrone, chatSession, project) -- [x] Remove all Bull queue references from documentation (deferred to next turn) - --- -**Document Status:** ✅ **FOUNDATION COMPLETE** +## Current Status + +### ✅ Complete + +- [x] Socket.IO communication (IDE↔Web↔Drone) +- [x] AI Provider CLI management +- [x] Chat Session REST API +- [x] Project Manager UI (3-column layout) +- [x] Chat Session View UI +- [x] Drone selection & filtering +- [x] Model selection in sidebar +- [x] Workspace persistence +- [x] Crash recovery +- [x] Unit tests (21 passing) +- [x] E2E test framework + +### 🔄 In Progress + +- [ ] E2E tests for complete flow +- [ ] Tool call inspectors +- [ ] File browser integration +- [ ] Session settings modal + +### 📋 Future + +- [ ] File operations (FO stats) +- [ ] Subagent support (SA stats) +- [ ] Model capability discovery +- [ ] Streaming token counts +- [ ] Session pins/bookmarks + +--- + **Last Updated:** April 29, 2026 -**Next Phase:** Chat Session UI Implementation **Branch:** `feature/socket-protocol` -**Commits:** 5 commits -**Tests:** 21 unit tests passing - ---- - -**Document Status:** Complete -**Next Review:** After Phase 2 implementation +**Status:** Production-ready foundation with complete Chat Session UI