Commit Graph

18 Commits

Author SHA1 Message Date
Rob Colbert
00969cf9bc feat: token economics — real API token extraction, stats propagation, and context window fuel gauge
Phase 1: OpenAI API Token Extraction
- Add stream_options: { include_usage: true } to all streaming API calls
- Capture chunk.usage from final streaming chunks and response.usage from non-streaming
- Extend OpenAiChatIterationResult with optional usage field
- Update buildStats() to accept and return real token counts from usage data
- Wire iteration.usage through chat() to both buildStats() call sites

Phase 2: Agent Loop Stats Propagation
- Rename inputTokens/outputTokens to masterInputTokens/masterOutputTokens, add masterThinkingTokens
- Accumulate response.stats.tokenCounts after each master AI call
- Delete all Math.ceil(length/4) crude approximations (master and subagent loops)
- Track startTime/durationMs and emit IWorkOrderCompleteStats with workOrderComplete
- Subagent loop uses response.stats?.tokenCounts instead of Math.ceil

Phase 3: Database Model Changes
- Add contextWindowUsage field to IChatSession, ChatSessionSchema, and frontend ChatSession
- Initialize contextWindowUsage: 0 on session creation

Phase 4: Persist Stats on Turn Completion
- drone-session: accept IWorkOrderCompleteStats, persist turn stats, walk subagent records for aggregate
- drone-session: $inc session stats and contextWindowUsage, add formatDurationLabel() helper
- code-session: accept and forward stats, update in-memory session stats
- message-queue: Redis replay handles 4th stats arg
- Update WorkOrderCompleteMessage type in @gadget/api to accept stats parameter

Phase 5: UI — Context Window Fuel Gauge
- Add contextWindowUsage prop to SessionPanel
- Add fuel gauge bar with E→F labels, green/yellow/red zones, token count display
- Visible for ALL provider types (not gated by apiType)

Phase 6: Frontend Streaming State
- Add IWorkOrderCompleteStats interface to frontend api.ts
- handleWorkOrderComplete accepts stats, updates turn stats and session contextWindowUsage
- Pass contextWindowUsage prop to SessionPanel
2026-05-18 14:36:36 -04:00
Rob Colbert
e155a7ffcf Phase 2: SubProcess observability in DroneManager and DroneInspector
Adds real-time subprocess monitoring to the IDE via the callback-chain and
subscribe/broadcast socket patterns:

Shared types (packages/api):
- New subprocess.ts message types: SubProcessStat, RequestProcessStatsMessage,
  SubscribeDroneMessage, UnsubscribeDroneMessage
- New socket events in socket.ts: requestProcessStats, subscribeDrone,
  unsubscribeDrone (ClientToServer); drone:log, drone:status (ServerToClient)

Drone (gadget-drone):
- onRequestProcessStats handler calls SubProcessService.ps() + summarize(),
  returns typed SubProcessStat[] with status detection (exitCode/killed)

Backend routing (gadget-code):
- SocketService: getDroneSessionByRegistrationId(), droneMonitorIndex map,
  addDroneMonitor()/removeDroneMonitor()/broadcastToMonitors()
- CodeSession: requestProcessStats proxy, subscribeDrone/unsubscribeDrone
- DroneSession: broadcast drone:log + drone:status to monitor subscribers
  in addition to existing chat-session routing

Frontend socket layer:
- SocketClient: requestProcessStats(), subscribeDrone(), unsubscribeDrone()
- drone:log + drone:status events forwarded to event bus

Frontend components:
- LogRenderer — reusable log rendering core extracted from LogPanel
- SubProcessTable — btop-style process table with status dots
- DroneMonitorGauge — canvas oscilloscope waveform (studio-equipment aesthetic)
- DroneMonitor — 3-gauge container (CPU/red, NETWORK/cyan, FILE I/O/green)

Frontend pages:
- DroneManager: live log streaming, SubProcessTable, DroneMonitor gauges,
  1-second polling, auto-scroll log with pause-on-scroll-up
- Home/DroneInspector: process count summary, mini LogRenderer, navigate-to-manager link

Documentation:
- Updated gadget-drone/docs/subprocess.md Phase 2 section
- Updated gadget-code/docs/ui-design-guide.md Phase 2 section
- Updated docs/socket-protocol.md with new events, sequences, and patterns
2026-05-14 13:52:59 -04:00
Rob Colbert
4780b79148 feat: abort controller for work order processing
Add end-to-end abort support: AbortSignal in @gadget/ai providers,
abortWorkOrder socket message, drone AbortController handling,
Cancel button and double-Esc in frontend, and aborted turn status display.
2026-05-12 12:25:17 -04:00
Rob Colbert
26e568612a ChatSession reconnect logic 2026-05-11 20:27:24 -04:00
Rob Colbert
c5add0fc7d subagent processing updates and fixes 2026-05-11 19:07:48 -04:00
Rob Colbert
d7924a9d6f GadgetLogTransportSocket and the drone-to-IDE log 2026-05-09 07:16:50 -04:00
Rob Colbert
61ba0e4412 streaming responses (see ./docs/streaming-responses.md) 2026-05-07 21:36:01 -04:00
Rob Colbert
cb73d276a3 more progress along ChatTurn processing
The agent has been failing and failing and failing, so I:
1. Swapped models
2. Did some by-hand enhancements
3. Set this checkpoint for continuing
2026-05-05 14:34:52 -04:00
Rob Colbert
c5e5d16a51 workspace mode management; drone status message socket events added 2026-05-03 03:05:06 -04:00
Rob Colbert
4ec31764d5 workspace management checkpoint while agents are working on it 2026-05-02 15:34:26 -04:00
Rob Colbert
e2dcaf25a0 fixing import errors from previous agent runs 2026-05-01 15:45:43 -04:00
Rob Colbert
404532012e move from Types.ObjectId to GadgetId (a string) 2026-05-01 14:31:00 -04:00
Rob Colbert
15da8fc444 socket protocol completeness, fixes, and tests 2026-04-30 16:51:33 -04:00
Rob Colbert
e1a446a3f3 Phase 5: Implement workspace persistence and crash recovery
- Create WorkspaceService for managing .gadget/ directory
- Implement workspace.json for persistent identity (workspaceId UUID)
- Add work order cache for crash recovery
- Update drone registration to include workspaceId
- Add crash recovery socket events (requestCrashRecovery, crashRecoveryResponse)
- Implement crash recovery handler in DroneSession
- Write work order cache before processing, remove after completion
2026-04-29 16:44:02 -04:00
Rob Colbert
92d19a648c Phase 3: Implement Drone→IDE event routing
- Add event handlers to DroneSession (thinking, response, toolCall, workOrderComplete)
- Implement routing logic to forward events to CodeSession
- Add chat session index to SocketService for reverse lookup
- Add workOrderComplete to ServerToClientEvents interface
- Update CodeSession to register chat session and set current turn on drone
- Add unit tests for DroneSession (12 tests, all passing)
2026-04-29 16:26:10 -04:00
Rob Colbert
096d8fe8b3 switched to feature/socket-protocol to continue experiments 2026-04-29 10:11:01 -04:00
Rob Colbert
6ee25642b8 fixing imports to restore build
All relevant model interfaces now come from @gadget/api
2026-04-28 12:47:26 -04:00
Rob Colbert
1edc3a85b8 created by merging gadget-code and gadget-drone 2026-04-28 09:20:37 -04:00