Commit Graph

21 Commits

Author SHA1 Message Date
Rob Colbert
00969cf9bc feat: token economics — real API token extraction, stats propagation, and context window fuel gauge
Phase 1: OpenAI API Token Extraction
- Add stream_options: { include_usage: true } to all streaming API calls
- Capture chunk.usage from final streaming chunks and response.usage from non-streaming
- Extend OpenAiChatIterationResult with optional usage field
- Update buildStats() to accept and return real token counts from usage data
- Wire iteration.usage through chat() to both buildStats() call sites

Phase 2: Agent Loop Stats Propagation
- Rename inputTokens/outputTokens to masterInputTokens/masterOutputTokens, add masterThinkingTokens
- Accumulate response.stats.tokenCounts after each master AI call
- Delete all Math.ceil(length/4) crude approximations (master and subagent loops)
- Track startTime/durationMs and emit IWorkOrderCompleteStats with workOrderComplete
- Subagent loop uses response.stats?.tokenCounts instead of Math.ceil

Phase 3: Database Model Changes
- Add contextWindowUsage field to IChatSession, ChatSessionSchema, and frontend ChatSession
- Initialize contextWindowUsage: 0 on session creation

Phase 4: Persist Stats on Turn Completion
- drone-session: accept IWorkOrderCompleteStats, persist turn stats, walk subagent records for aggregate
- drone-session: $inc session stats and contextWindowUsage, add formatDurationLabel() helper
- code-session: accept and forward stats, update in-memory session stats
- message-queue: Redis replay handles 4th stats arg
- Update WorkOrderCompleteMessage type in @gadget/api to accept stats parameter

Phase 5: UI — Context Window Fuel Gauge
- Add contextWindowUsage prop to SessionPanel
- Add fuel gauge bar with E→F labels, green/yellow/red zones, token count display
- Visible for ALL provider types (not gated by apiType)

Phase 6: Frontend Streaming State
- Add IWorkOrderCompleteStats interface to frontend api.ts
- handleWorkOrderComplete accepts stats, updates turn stats and session contextWindowUsage
- Pass contextWindowUsage prop to SessionPanel
2026-05-18 14:36:36 -04:00
Rob Colbert
c90e8ce0e8 plan tool updates
Plan tools extended to all modes; recent chat sessions added to
Authenticated Home view; drone locking and chat session startup factored
out of Project Manager into Chat Session view and made universal; update
for AGENTS.md.
2026-05-17 14:09:24 -04:00
Rob Colbert
72102e7fdb project manager upgrades Phases 1-4 2026-05-16 10:48:42 -04:00
Rob Colbert
cc7cdee60b Pull Project initial implementation 2026-05-15 19:34:08 -04:00
Rob Colbert
140c8a75ee add numCtx (Context Window) setting to SESSION panel
- IChatSession/IChatTurn interfaces: add optional numCtx field
- Mongoose schemas: add numCtx field to ChatSession and ChatTurn
- API controller: validate numCtx (>=16384, multiple of 8192)
- ChatSessionService: persist numCtx on session, pass through to turn
- SessionPanel: add range slider for Context Window (hidden for OpenAI)
- Drone buildDroneModelConfig: prefer turn.numCtx over model defaults
2026-05-15 16:02:22 -04:00
Rob Colbert
e155a7ffcf Phase 2: SubProcess observability in DroneManager and DroneInspector
Adds real-time subprocess monitoring to the IDE via the callback-chain and
subscribe/broadcast socket patterns:

Shared types (packages/api):
- New subprocess.ts message types: SubProcessStat, RequestProcessStatsMessage,
  SubscribeDroneMessage, UnsubscribeDroneMessage
- New socket events in socket.ts: requestProcessStats, subscribeDrone,
  unsubscribeDrone (ClientToServer); drone:log, drone:status (ServerToClient)

Drone (gadget-drone):
- onRequestProcessStats handler calls SubProcessService.ps() + summarize(),
  returns typed SubProcessStat[] with status detection (exitCode/killed)

Backend routing (gadget-code):
- SocketService: getDroneSessionByRegistrationId(), droneMonitorIndex map,
  addDroneMonitor()/removeDroneMonitor()/broadcastToMonitors()
- CodeSession: requestProcessStats proxy, subscribeDrone/unsubscribeDrone
- DroneSession: broadcast drone:log + drone:status to monitor subscribers
  in addition to existing chat-session routing

Frontend socket layer:
- SocketClient: requestProcessStats(), subscribeDrone(), unsubscribeDrone()
- drone:log + drone:status events forwarded to event bus

Frontend components:
- LogRenderer — reusable log rendering core extracted from LogPanel
- SubProcessTable — btop-style process table with status dots
- DroneMonitorGauge — canvas oscilloscope waveform (studio-equipment aesthetic)
- DroneMonitor — 3-gauge container (CPU/red, NETWORK/cyan, FILE I/O/green)

Frontend pages:
- DroneManager: live log streaming, SubProcessTable, DroneMonitor gauges,
  1-second polling, auto-scroll log with pause-on-scroll-up
- Home/DroneInspector: process count summary, mini LogRenderer, navigate-to-manager link

Documentation:
- Updated gadget-drone/docs/subprocess.md Phase 2 section
- Updated gadget-code/docs/ui-design-guide.md Phase 2 section
- Updated docs/socket-protocol.md with new events, sequences, and patterns
2026-05-14 13:52:59 -04:00
Rob Colbert
4780b79148 feat: abort controller for work order processing
Add end-to-end abort support: AbortSignal in @gadget/ai providers,
abortWorkOrder socket message, drone AbortController handling,
Cancel button and double-Esc in frontend, and aborted turn status display.
2026-05-12 12:25:17 -04:00
Rob Colbert
24975b58c4 Define missing socket event types and enforce typed events in frontend build
Adds type definitions + forwarding for status, reconnect_attempt, reconnect_failed, reconnect events.
Frontend build now runs tsc --noEmit before vite build so undefined socket events cause failures.
Fixes pre-existing type errors exposed by strict mode in the frontend.
2026-05-12 10:42:31 -04:00
Rob Colbert
c5add0fc7d subagent processing updates and fixes 2026-05-11 19:07:48 -04:00
Rob Colbert
009863cf2b fix: resolve drone heartbeat timeouts and JWT expiration bugs
This commit addresses two interrelated issues causing drones to
de-register and users to be forcibly signed out:

## Heartbeat Timeout Fixes

1. Move heartbeat interval to a Web Worker (not subject to browser
   tab throttling). Chrome throttles setInterval in background tabs
   to ~1/min, which causes the 19s heartbeat to miss the drone's
   timeout timer. The Web Worker fires reliably regardless of tab
   visibility.

2. Add visibilitychange handler: when the tab becomes visible again,
   send an immediate heartbeat to reset the drone's timer after any
   throttling that may have occurred.

3. Fix onReleaseSessionLock to clear the heartbeat timer. Previously,
   releasing the lock left the 60s timer running, causing a spurious
   timeout and status emit after the lock was already released.

4. Increase drone heartbeat timeout from 60s to 120s. With the Web
   Worker fix, heartbeats should be reliable, but doubling the timeout
   provides a generous safety margin.

5. Add socket disconnect/reconnect handlers on the drone side. On
   disconnect, clear the heartbeat timer. On reconnect, re-emit drone
   status so the platform knows the drone is alive.

6. Configure Socket.IO pingInterval/pingTimeout explicitly (25s/60s)
   instead of relying on defaults.

## JWT Expiration Fixes

1. Increase WebToken DB record expiration from 1 hour to 7 days. The
   1-hour expiration was the real session lifetime gate (the JWT crypto
   exp was already 24h), and it was far too aggressive for a dev tool.

2. Fix web /auth/renew-token endpoint to use req.user from the session
   cookie instead of verifyJsonWebToken(req.body.token). This eliminates
   the catch-22 where an expired token cannot be used to request its
   own renewal.

3. Fix token refresh response parsing. The API v1 renew-token endpoint
   returns { success: true, token } at the top level, but the frontend
   was looking for json.data?.token, causing every refresh to fail.

4. Add proactive token refresh: check the JWT exp claim before each
   request and refresh if expiring within 5 minutes. This avoids
   unnecessary 401 errors and the resulting socket disconnections.

5. Update socket JWT on token renewal via a callback registered in
   App.tsx. This ensures that future socket reconnections use the new
   token instead of the expired one.

## Files Modified

- gadget-code/frontend/src/workers/heartbeat.worker.ts (NEW)
- gadget-code/frontend/src/lib/socket.ts
- gadget-code/frontend/src/lib/api.ts
- gadget-code/frontend/src/App.tsx
- gadget-code/src/services/session.ts
- gadget-code/src/controllers/auth.ts
- gadget-code/src/services/socket.ts
- gadget-drone/src/gadget-drone.ts
2026-05-11 17:46:09 -04:00
Rob Colbert
8eff66dcec user settings 2026-05-08 14:08:49 -04:00
Rob Colbert
11bdd5e3b0 make reasoning effort configurable; remove sign up concept
- Implemented reasoning effort setting in SESSION panel of Chat Sessio
View
- Removed all ability to "sign up" for an account
2026-05-08 11:40:30 -04:00
Rob Colbert
61ba0e4412 streaming responses (see ./docs/streaming-responses.md) 2026-05-07 21:36:01 -04:00
Rob Colbert
cb73d276a3 more progress along ChatTurn processing
The agent has been failing and failing and failing, so I:
1. Swapped models
2. Did some by-hand enhancements
3. Set this checkpoint for continuing
2026-05-05 14:34:52 -04:00
Rob Colbert
bab0b1810f many fixes
- JWT refresh logic to prevent dead sessions
- drone status messages now arrive in IDE for display
- WorkspaceService.deployProject method added to clone into a repo or
create the directory (new project not yet in git)
2026-05-03 04:15:48 -04:00
Rob Colbert
4642609d06 socket message and drone workspace cleanup 2026-05-02 02:46:45 -04:00
Rob Colbert
c6d0c66563 drone manager created (wip) 2026-04-30 09:40:27 -04:00
Rob Colbert
089a5b5fab checkpoint 2026-04-29 18:59:42 -04:00
Rob Colbert
db0d1586d6 front-end progress; Socket.IO messaging API; inquirer for credentials 2026-04-29 00:24:38 -04:00
Rob Colbert
2e9571a74e Implement dark industrial theme, Project Manager, and JWT authentication
- Add dark industrial theme with brand color #c20600 and CSS variables
- Create Header component with version display and user dropdown menu
- Create StatusBar with connection indicator and project/session display
- Create ProjectManager page with CRUD, list view, and inspector
- Add JWT Bearer token to API requests for authenticated endpoints
- Add project API endpoints (GET/POST/PUT/DELETE /api/v1/projects)
- Add ProjectService methods: findById, findBySlug, delete
- Add unit tests for project API endpoints
- Add Playwright E2E tests for projects flow
- Update UI design guide with implementation details
- Fix: empty JSON body parsing in web-app.ts middleware
2026-04-28 17:13:49 -04:00
Rob Colbert
1edc3a85b8 created by merging gadget-code and gadget-drone 2026-04-28 09:20:37 -04:00