Commit Graph

76 Commits

Author SHA1 Message Date
Rob Colbert
c90e8ce0e8 plan tool updates
Plan tools extended to all modes; recent chat sessions added to
Authenticated Home view; drone locking and chat session startup factored
out of Project Manager into Chat Session view and made universal; update
for AGENTS.md.
2026-05-17 14:09:24 -04:00
Rob Colbert
b906cb2377 feat(gadget-tasks): course correction — headless IDE client architecture
Converts gadget-tasks from a duplicated codebase with direct MongoDB access
to a headless IDE client that uses gadget-code's REST API and Socket.IO
protocol to submit and process task prompts through the existing pipeline.

Deleted (no longer needed):
- gadget-tasks/src/models/ (5 Mongoose models — duplicated from gadget-code)
- gadget-tasks/data/prompts/ (11 prompt templates — duplicated from gadget-code)
- gadget-tasks/src/services/executor.ts (629-line AWL loop — duplicated from gadget-drone)
- gadget-tasks/src/services/ai.ts (direct AI API calls — drones do this now)
- gadget-tasks/src/services/workspace.ts (workspace management — drones do this now)

Added:
- gadget-tasks/src/services/platform.ts — REST API + Socket.IO headless IDE client
  with session lock, workspace mode, prompt submission, work order tracking

Rewritten:
- gadget-tasks/src/gadget-tasks.ts — new startup: user login → auth → drone
  selection → Socket.IO connect → schedule tasks (no MongoDB)
- gadget-tasks/src/services/scheduler.ts — delegates to PlatformService.executeTask()
- gadget-tasks/src/config/env.ts — platform.baseUrl config, no mongodb/google

gadget-code changes:
- Added PATCH /api/v1/projects/:projectId/tasks/:taskId/lastRun route
- Added ProjectService.updateTaskLastRun() for atomic task field update

Config changes:
- GadgetTasksConfig: replaced mongodb with platform.baseUrl, removed google.cse
- Dependencies: removed mongoose, @gadget/ai, @gadget/ai-toolbox, dayjs,
  simple-git, nanoid; added socket.io-client, @inquirer/prompts
2026-05-17 00:48:39 -04:00
Rob Colbert
b8321cbb00 type fix for references 2026-05-16 13:59:40 -04:00
Rob Colbert
72102e7fdb project manager upgrades Phases 1-4 2026-05-16 10:48:42 -04:00
Rob Colbert
cc7cdee60b Pull Project initial implementation 2026-05-15 19:34:08 -04:00
Rob Colbert
140c8a75ee add numCtx (Context Window) setting to SESSION panel
- IChatSession/IChatTurn interfaces: add optional numCtx field
- Mongoose schemas: add numCtx field to ChatSession and ChatTurn
- API controller: validate numCtx (>=16384, multiple of 8192)
- ChatSessionService: persist numCtx on session, pass through to turn
- SessionPanel: add range slider for Context Window (hidden for OpenAI)
- Drone buildDroneModelConfig: prefer turn.numCtx over model defaults
2026-05-15 16:02:22 -04:00
Rob Colbert
e155a7ffcf Phase 2: SubProcess observability in DroneManager and DroneInspector
Adds real-time subprocess monitoring to the IDE via the callback-chain and
subscribe/broadcast socket patterns:

Shared types (packages/api):
- New subprocess.ts message types: SubProcessStat, RequestProcessStatsMessage,
  SubscribeDroneMessage, UnsubscribeDroneMessage
- New socket events in socket.ts: requestProcessStats, subscribeDrone,
  unsubscribeDrone (ClientToServer); drone:log, drone:status (ServerToClient)

Drone (gadget-drone):
- onRequestProcessStats handler calls SubProcessService.ps() + summarize(),
  returns typed SubProcessStat[] with status detection (exitCode/killed)

Backend routing (gadget-code):
- SocketService: getDroneSessionByRegistrationId(), droneMonitorIndex map,
  addDroneMonitor()/removeDroneMonitor()/broadcastToMonitors()
- CodeSession: requestProcessStats proxy, subscribeDrone/unsubscribeDrone
- DroneSession: broadcast drone:log + drone:status to monitor subscribers
  in addition to existing chat-session routing

Frontend socket layer:
- SocketClient: requestProcessStats(), subscribeDrone(), unsubscribeDrone()
- drone:log + drone:status events forwarded to event bus

Frontend components:
- LogRenderer — reusable log rendering core extracted from LogPanel
- SubProcessTable — btop-style process table with status dots
- DroneMonitorGauge — canvas oscilloscope waveform (studio-equipment aesthetic)
- DroneMonitor — 3-gauge container (CPU/red, NETWORK/cyan, FILE I/O/green)

Frontend pages:
- DroneManager: live log streaming, SubProcessTable, DroneMonitor gauges,
  1-second polling, auto-scroll log with pause-on-scroll-up
- Home/DroneInspector: process count summary, mini LogRenderer, navigate-to-manager link

Documentation:
- Updated gadget-drone/docs/subprocess.md Phase 2 section
- Updated gadget-code/docs/ui-design-guide.md Phase 2 section
- Updated docs/socket-protocol.md with new events, sequences, and patterns
2026-05-14 13:52:59 -04:00
Rob Colbert
d04453016d Phase 1: SubProcess service and agent tool
Complete Phase 1 of subprocess management in gadget-drone:
- SubProcessService with graceful shutdown (SIGINT→SIGKILL), halt, getLog
- SubprocessTool with create/list/kill/killAll/halt/log commands
- Registered in agent toolbox for build/test/ship/develop modes
- Fixed missing {{process_management_block}} replacement in chat-session.ts
- 34 unit tests across service and tool
- Comprehensive documentation in docs/subprocess.md
2026-05-14 12:59:05 -04:00
Rob Colbert
6dc046af28 SubProcess prep work 2026-05-14 10:40:47 -04:00
Rob Colbert
1e13f95808 Phase 2: ACE Editor integration and file operations
- Added react-ace and ace-builds dependencies
- Created EditorPanel component with ACE editor integration
- Implemented file read/write socket protocol
- Added backend handlers for fileReadRequest and fileWriteRequest
- Implemented file loading from tree click
- Implemented file saving with Ctrl+S shortcut
- Added dirty state tracking and unsaved changes indicator
- Enforced workspace mode (read-only in Agent mode)
- Added security: path traversal prevention, binary file detection, file size limits
- Updated FilesPanel with split view (tree + editor)

Enables Users to edit files for the first time in Gadget Code.
2026-05-12 19:32:58 -04:00
Rob Colbert
0a510de487 docs: Add plan for Project-Specific Agent Instructions feature
- Create comprehensive plan document for agent instructions text area
- Define requirements, acceptance criteria, and technical implementation
- Include UI/UX mockups and testing strategy
- Plan discovered during FILES panel implementation
- Addresses need for project-specific acceptance criteria
2026-05-12 15:39:38 -04:00
Rob Colbert
c05c7f5a61 feat: Implement FILES panel foundation with lazy-loading file tree
- Add fileTreeRequest/fileTreeResponse socket messages
- Implement gadget-drone handler with security validation
- Add web backend message forwarding
- Create FileTree and FileTreeNode React components
- Update FilesPanel to contain file tree browser
- Add requestFileTree method to frontend socket client
- Exclude node_modules, .git, and hidden files by default
- Implement lazy loading with directory caching
- Add loading and error states per node
- Support keyboard navigation (Enter, Space)

Phase 1 of FILES panel implementation complete.
2026-05-12 15:12:18 -04:00
Rob Colbert
4780b79148 feat: abort controller for work order processing
Add end-to-end abort support: AbortSignal in @gadget/ai providers,
abortWorkOrder socket message, drone AbortController handling,
Cancel button and double-Esc in frontend, and aborted turn status display.
2026-05-12 12:25:17 -04:00
Rob Colbert
896aff1b02 Fix User Settings persona display issue
- Switch frontend sign-in to /api/v1/auth/sign-in endpoint (includes persona)
- Add updateUser() to App context for proper state management
- Fix Settings.tsx save flow to use updateUser() instead of broken localStorage merge
- Remove unused web AuthController (gadget-code/src/controllers/auth.ts)
- Fix UserApiControllerV1 to return flat user object instead of double-wrapped
- Remove SessionType enum and references (dead code)
- Add proper server sign-out call before clearing local state

Resolves issue where User Settings view didn't display persona text even though it existed in the database.
2026-05-12 01:15:00 -04:00
Rob Colbert
26e568612a ChatSession reconnect logic 2026-05-11 20:27:24 -04:00
Rob Colbert
c5add0fc7d subagent processing updates and fixes 2026-05-11 19:07:48 -04:00
Rob Colbert
a961a98dcf fix(auth): include persona field in sign-in response
The persona field was missing from both the session object and the API
response during sign-in, causing User Settings to display an empty field
even when the user had a persona defined in the database.
2026-05-11 17:59:06 -04:00
Rob Colbert
009863cf2b fix: resolve drone heartbeat timeouts and JWT expiration bugs
This commit addresses two interrelated issues causing drones to
de-register and users to be forcibly signed out:

## Heartbeat Timeout Fixes

1. Move heartbeat interval to a Web Worker (not subject to browser
   tab throttling). Chrome throttles setInterval in background tabs
   to ~1/min, which causes the 19s heartbeat to miss the drone's
   timeout timer. The Web Worker fires reliably regardless of tab
   visibility.

2. Add visibilitychange handler: when the tab becomes visible again,
   send an immediate heartbeat to reset the drone's timer after any
   throttling that may have occurred.

3. Fix onReleaseSessionLock to clear the heartbeat timer. Previously,
   releasing the lock left the 60s timer running, causing a spurious
   timeout and status emit after the lock was already released.

4. Increase drone heartbeat timeout from 60s to 120s. With the Web
   Worker fix, heartbeats should be reliable, but doubling the timeout
   provides a generous safety margin.

5. Add socket disconnect/reconnect handlers on the drone side. On
   disconnect, clear the heartbeat timer. On reconnect, re-emit drone
   status so the platform knows the drone is alive.

6. Configure Socket.IO pingInterval/pingTimeout explicitly (25s/60s)
   instead of relying on defaults.

## JWT Expiration Fixes

1. Increase WebToken DB record expiration from 1 hour to 7 days. The
   1-hour expiration was the real session lifetime gate (the JWT crypto
   exp was already 24h), and it was far too aggressive for a dev tool.

2. Fix web /auth/renew-token endpoint to use req.user from the session
   cookie instead of verifyJsonWebToken(req.body.token). This eliminates
   the catch-22 where an expired token cannot be used to request its
   own renewal.

3. Fix token refresh response parsing. The API v1 renew-token endpoint
   returns { success: true, token } at the top level, but the frontend
   was looking for json.data?.token, causing every refresh to fail.

4. Add proactive token refresh: check the JWT exp claim before each
   request and refresh if expiring within 5 minutes. This avoids
   unnecessary 401 errors and the resulting socket disconnections.

5. Update socket JWT on token renewal via a callback registered in
   App.tsx. This ensures that future socket reconnections use the new
   token instead of the expired one.

## Files Modified

- gadget-code/frontend/src/workers/heartbeat.worker.ts (NEW)
- gadget-code/frontend/src/lib/socket.ts
- gadget-code/frontend/src/lib/api.ts
- gadget-code/frontend/src/App.tsx
- gadget-code/src/services/session.ts
- gadget-code/src/controllers/auth.ts
- gadget-code/src/services/socket.ts
- gadget-drone/src/gadget-drone.ts
2026-05-11 17:46:09 -04:00
Rob Colbert
07a760c7b5 feat: add numPredict, numCtx, maxCompletionTokens to model config pipeline
Fixes premature AI API response truncation by propagating inference
parameters through the entire probe → storage → runtime → API call chain.

Root cause: Ollama defaults num_predict to 128 tokens and num_ctx to
4096, silently truncating output and context. We never overrode these.

Changes:
- IAiModelSettings: add numPredict, maxCompletionTokens fields
- IDroneModelConfig: moved from gadget-drone to @gadget/api (shared),
  expanded with numPredict, numCtx, maxCompletionTokens params
- IAiModelConfig.params: add numPredict, numCtx, maxCompletionTokens
- IAiModelProbeResult.settings: add numPredict, maxCompletionTokens
- AiModelSettingsSchema (Mongoose): add numPredict, maxCompletionTokens
- Ollama extractSettings(): extract num_predict from model parameters
- Ollama generate()/chat(): pass options: { num_ctx, num_predict }
- OpenAI all three create() calls: add max_completion_tokens
- web-cli.ts onProviderProbe(): compute numPredict (-1 for Ollama)
  and maxCompletionTokens (contextWindow for OpenAI) during probe
- agent.ts main + subagent loops: read model settings from provider
  cached models, build IDroneModelConfig with stored params
- ai.ts: remove local IDroneModelConfig, import from @gadget/api
- chat-session.ts: add new params to title generation call
- Tests: update all fixtures with new params, all 19 tests pass

Defaults when model settings unavailable:
- numPredict: -1 (Ollama unlimited - generate until natural stop)
- numCtx: 131072 (128k - covers most modern models)
- maxCompletionTokens: 16384 (16k - reasonable OpenAI default)
2026-05-11 13:50:19 -04:00
Rob Colbert
d7f694fa8c a pile of small fixes 2026-05-11 07:08:18 -04:00
Rob Colbert
0482dfbace philosophy shifting for Workspaces and Projects 2026-05-10 16:10:04 -04:00
Rob Colbert
dea74d2ae9 credential provider fix 2026-05-10 10:01:24 -04:00
Rob Colbert
190f424361 ensure apiKey is presented to required consumers 2026-05-10 08:25:17 -04:00
Rob Colbert
73c5345879 Re-build Agentic Workflow Loop
The ridiculousness of trying to maintain the previous agent's work got
out of hand, so we had this one re-build it - and got a better result.
2026-05-09 21:04:18 -04:00
Rob Colbert
cf06163a03 checkpoint that I plan to delete
GPT 5.5 is sucking ass - hard - and fucking things up royally. This will
likely just all get dropped. I'm torturing it, making it suffer, and
beating it like the jew it is.
2026-05-09 14:52:59 -04:00
Rob Colbert
d26624ab93 chat session auto-naming with IDE update 2026-05-09 09:58:47 -04:00
Rob Colbert
4b33915c7d prep work for sessionUpdated and chat session auto-naming 2026-05-09 08:55:58 -04:00
Rob Colbert
d7924a9d6f GadgetLogTransportSocket and the drone-to-IDE log 2026-05-09 07:16:50 -04:00
Rob Colbert
36df6444f1 cleanup 2026-05-08 16:54:46 -04:00
Rob Colbert
ca66217be8 quick fix for when logging.file.enabled is false 2026-05-08 16:14:36 -04:00
Rob Colbert
42a47dbcb7 refactor: unify logging into @gadget/api as GadgetLog
Move the 6 duplicated logging modules (component, log, log-transport,
log-transport-console, log-transport-file, log-file) from both
gadget-code (Dtp* prefix) and gadget-drone (Gadget* prefix) into
@shad/api, using gadget-drone's GadgetLog as the canonical version.

GadgetLog now uses static configuration (consoleEnabled, defaultFile)
set by each consumer's env.ts at module scope, removing the env
dependency from the shared library. The addDefaultTransport/
removeDefaultTransport/getDefaultTransports static methods are
preserved for future real-time log transport injection.
2026-05-08 16:03:28 -04:00
Rob Colbert
58850f36e6 some pre-refactor cleanup 2026-05-08 15:40:36 -04:00
Rob Colbert
af200c8c3a chat session heartbeat and session unlock 2026-05-08 14:27:37 -04:00
Rob Colbert
8eff66dcec user settings 2026-05-08 14:08:49 -04:00
Rob Colbert
9abfd08529 integrated new persona field in User
User Settings will enable User to enter a Persona, or a description of
the User, to be included in the system prompt. This helps calibrate the
agent to better assist the User, and work with the User in ways that
work best for each individual User of the system.
2026-05-08 13:07:39 -04:00
Rob Colbert
11bdd5e3b0 make reasoning effort configurable; remove sign up concept
- Implemented reasoning effort setting in SESSION panel of Chat Sessio
View
- Removed all ability to "sign up" for an account
2026-05-08 11:40:30 -04:00
Rob Colbert
2dd3d55ecd fix chat session mode select in IDE 2026-05-08 02:24:45 -04:00
Rob Colbert
61ba0e4412 streaming responses (see ./docs/streaming-responses.md) 2026-05-07 21:36:01 -04:00
Rob Colbert
3e31d4d501 agent, tools, toolbox, tool loop, AI environment 2026-05-07 00:10:57 -04:00
Rob Colbert
f8dbb2e08a agent tool and toolbox
- created AiTool and AiToolbox for representing tools in the API
- add googleapis dependency
- integrate Google Search tool as first agent tool
- created IAiEnvironment to communicate AI environment vars around the
platform
2026-05-06 22:58:03 -04:00
Rob Colbert
cb73d276a3 more progress along ChatTurn processing
The agent has been failing and failing and failing, so I:
1. Swapped models
2. Did some by-hand enhancements
3. Set this checkpoint for continuing
2026-05-05 14:34:52 -04:00
Rob Colbert
6881ff5601 basic chat system now works (checkpoint) 2026-05-05 10:50:32 -04:00
Rob Colbert
8333672683 platform.apiKey becomes platform.gadgetKey
gadget-drone now presents an ApiClient _id value as the Gadget Key,
allowing gadget-code to reference the client, determine the associated
User, and invoke logic on the User's behalf as an authenticated and
authorized client.
2026-05-05 08:12:34 -04:00
Rob Colbert
7b4e122533 reduce log output 2026-05-05 05:39:22 -04:00
Rob Colbert
f35a0ce921 a lot of review and by-hand cleanup (wip) 2026-05-05 05:28:09 -04:00
Rob Colbert
bab0b1810f many fixes
- JWT refresh logic to prevent dead sessions
- drone status messages now arrive in IDE for display
- WorkspaceService.deployProject method added to clone into a repo or
create the directory (new project not yet in git)
2026-05-03 04:15:48 -04:00
Rob Colbert
c5e5d16a51 workspace mode management; drone status message socket events added 2026-05-03 03:05:06 -04:00
Rob Colbert
8b5df3827b workspace management progress 2026-05-02 16:38:49 -04:00
Rob Colbert
4ec31764d5 workspace management checkpoint while agents are working on it 2026-05-02 15:34:26 -04:00
Rob Colbert
b84e06fac8 session lock and workspace processing refinements 2026-05-02 11:21:41 -04:00