Commit Graph

90 Commits

Author SHA1 Message Date
Rob Colbert
3c076fc01a Merge branch 'develop' of git.digitaltelepresence.com:rob/gadget into develop 2026-05-19 15:46:16 -04:00
Rob Colbert
66774d94f9 search input and backend error fixes 2026-05-19 15:43:24 -04:00
Rob Colbert
70ad578425 Merge branch 'develop' of git.digitaltelepresence.com:rob/gadget into develop 2026-05-19 14:30:04 -04:00
Rob Colbert
9aaacead42 quick auto-scroll fix 2026-05-19 14:29:58 -04:00
Rob Colbert
56f35a2fdf feat: Qdrant semantic search over chat history
Adds vector-based semantic search across all chat sessions using Qdrant.
When a ChatTurn finishes, its content is chunked, embedded, and upserted
to a Qdrant collection. A search API and UI components enable searching
at user, project, and session scope.

Phase 1 — Configuration & Dependencies
- Add port/apiKey to GadgetCodeConfig.qdrant type
- Uncomment and update qdrant section in YAML config example
- Add qdrant config passthrough in env.ts
- Add @qdrant/js-client-rest dependency

Phase 2 — AI Embedding API (@gadget/ai)
- Add IAiEmbeddingResponse interface and abstract embeddings() to AiApi
- Implement embeddings() in OllamaAiApi (client.embeddings)
- Implement embeddings() in OpenAiApi (client.embeddings.create)
- Export IAiEmbeddingResponse from package index

Phase 3 — Backend Vector Store Service
- Create VectorStoreService (ingestTurn, search, removeTurnPoints)
- Hook fire-and-forget ingest after turn.save() in drone-session
- Register VectorStoreService in service startup/shutdown

Phase 4 — Backend Search API
- Create POST /api/v1/search controller with userId enforcement
- Batch-hydrate results from MongoDB (user, project, session, turn)
- Register search route in v1 API router

Phase 5 — Frontend Search Components
- SearchInput: debounced input with lucide-react icons
- ChatSearchResults: modal with score badges, metadata, loading states
- DroneSelectionModal: drone picker for sessions without a drone
- Add searchApi and ISearchResult to API client
- Add search to Home (global), ProjectManager (project), ChatSessionView (session)
- Add id=turn-{turnId} to ChatTurn for scroll targeting
- Scroll-to-turn from search result selection and router state
- Show DroneSelectionModal when no drone available
- Add Select Drone button in ChatSessionView sidebar
2026-05-19 14:28:30 -04:00
Rob Colbert
6696d79a0d repairs for OpenAI token stats/economics 2026-05-18 16:40:19 -04:00
Rob Colbert
2483161663 context window fuel gauge auto update fix 2026-05-18 15:00:40 -04:00
Rob Colbert
00969cf9bc feat: token economics — real API token extraction, stats propagation, and context window fuel gauge
Phase 1: OpenAI API Token Extraction
- Add stream_options: { include_usage: true } to all streaming API calls
- Capture chunk.usage from final streaming chunks and response.usage from non-streaming
- Extend OpenAiChatIterationResult with optional usage field
- Update buildStats() to accept and return real token counts from usage data
- Wire iteration.usage through chat() to both buildStats() call sites

Phase 2: Agent Loop Stats Propagation
- Rename inputTokens/outputTokens to masterInputTokens/masterOutputTokens, add masterThinkingTokens
- Accumulate response.stats.tokenCounts after each master AI call
- Delete all Math.ceil(length/4) crude approximations (master and subagent loops)
- Track startTime/durationMs and emit IWorkOrderCompleteStats with workOrderComplete
- Subagent loop uses response.stats?.tokenCounts instead of Math.ceil

Phase 3: Database Model Changes
- Add contextWindowUsage field to IChatSession, ChatSessionSchema, and frontend ChatSession
- Initialize contextWindowUsage: 0 on session creation

Phase 4: Persist Stats on Turn Completion
- drone-session: accept IWorkOrderCompleteStats, persist turn stats, walk subagent records for aggregate
- drone-session: $inc session stats and contextWindowUsage, add formatDurationLabel() helper
- code-session: accept and forward stats, update in-memory session stats
- message-queue: Redis replay handles 4th stats arg
- Update WorkOrderCompleteMessage type in @gadget/api to accept stats parameter

Phase 5: UI — Context Window Fuel Gauge
- Add contextWindowUsage prop to SessionPanel
- Add fuel gauge bar with E→F labels, green/yellow/red zones, token count display
- Visible for ALL provider types (not gated by apiType)

Phase 6: Frontend Streaming State
- Add IWorkOrderCompleteStats interface to frontend api.ts
- handleWorkOrderComplete accepts stats, updates turn stats and session contextWindowUsage
- Pass contextWindowUsage prop to SessionPanel
2026-05-18 14:36:36 -04:00
Rob Colbert
802184a487 fix: home sidebar flex column layout with independent scrolling sections
- Convert sidebar <aside> to flex column container (flex flex-col overflow-hidden min-h-0)
- Remove sidebar-level scroll (overflow-y-auto → overflow-hidden)
- Add min-h-0 to parent flex containers in both layout paths for proper height containment
- Reorder sidebar sections: Clock → Projects → Drones → Recent Chats
- Add shrink-0 to Clock component to keep it at intrinsic content size
- Convert Projects, Drones, and Recent Chats sections to flex-1 min-h-0 flex-col with scrollable inner content
- Pin section headers with shrink-0; list content scrolls independently via overflow-y-auto
- Pin Recent Chats hint/footer with shrink-0 below its scrollable list
- No logic changes — purely CSS/flexbox layout correction
2026-05-18 09:04:25 -04:00
Rob Colbert
a23224279e prompt draft save/restore in Chat Session view 2026-05-17 17:39:34 -04:00
Rob Colbert
c90e8ce0e8 plan tool updates
Plan tools extended to all modes; recent chat sessions added to
Authenticated Home view; drone locking and chat session startup factored
out of Project Manager into Chat Session view and made universal; update
for AGENTS.md.
2026-05-17 14:09:24 -04:00
Rob Colbert
4fa4f29f0b ex[amded prompt editor 2026-05-17 02:00:53 -04:00
Rob Colbert
72102e7fdb project manager upgrades Phases 1-4 2026-05-16 10:48:42 -04:00
Rob Colbert
cc7cdee60b Pull Project initial implementation 2026-05-15 19:34:08 -04:00
Rob Colbert
69f45867b2 tool call observability 2026-05-15 18:24:00 -04:00
Rob Colbert
d1e77f52f1 new file create 2026-05-15 17:50:24 -04:00
Rob Colbert
140c8a75ee add numCtx (Context Window) setting to SESSION panel
- IChatSession/IChatTurn interfaces: add optional numCtx field
- Mongoose schemas: add numCtx field to ChatSession and ChatTurn
- API controller: validate numCtx (>=16384, multiple of 8192)
- ChatSessionService: persist numCtx on session, pass through to turn
- SessionPanel: add range slider for Context Window (hidden for OpenAI)
- Drone buildDroneModelConfig: prefer turn.numCtx over model defaults
2026-05-15 16:02:22 -04:00
Rob Colbert
8650533697 Merge remote-tracking branch 'origin/develop' into develop 2026-05-15 15:56:31 -04:00
Rob Colbert
4c628a25a2 ChatSessionView refactor; and a scripts directory 2026-05-15 15:52:14 -04:00
Rob Colbert
d990d0dd71 added Gab AI affiliate link; changed Ollama abort procedure 2026-05-15 14:11:45 -04:00
Rob Colbert
e155a7ffcf Phase 2: SubProcess observability in DroneManager and DroneInspector
Adds real-time subprocess monitoring to the IDE via the callback-chain and
subscribe/broadcast socket patterns:

Shared types (packages/api):
- New subprocess.ts message types: SubProcessStat, RequestProcessStatsMessage,
  SubscribeDroneMessage, UnsubscribeDroneMessage
- New socket events in socket.ts: requestProcessStats, subscribeDrone,
  unsubscribeDrone (ClientToServer); drone:log, drone:status (ServerToClient)

Drone (gadget-drone):
- onRequestProcessStats handler calls SubProcessService.ps() + summarize(),
  returns typed SubProcessStat[] with status detection (exitCode/killed)

Backend routing (gadget-code):
- SocketService: getDroneSessionByRegistrationId(), droneMonitorIndex map,
  addDroneMonitor()/removeDroneMonitor()/broadcastToMonitors()
- CodeSession: requestProcessStats proxy, subscribeDrone/unsubscribeDrone
- DroneSession: broadcast drone:log + drone:status to monitor subscribers
  in addition to existing chat-session routing

Frontend socket layer:
- SocketClient: requestProcessStats(), subscribeDrone(), unsubscribeDrone()
- drone:log + drone:status events forwarded to event bus

Frontend components:
- LogRenderer — reusable log rendering core extracted from LogPanel
- SubProcessTable — btop-style process table with status dots
- DroneMonitorGauge — canvas oscilloscope waveform (studio-equipment aesthetic)
- DroneMonitor — 3-gauge container (CPU/red, NETWORK/cyan, FILE I/O/green)

Frontend pages:
- DroneManager: live log streaming, SubProcessTable, DroneMonitor gauges,
  1-second polling, auto-scroll log with pause-on-scroll-up
- Home/DroneInspector: process count summary, mini LogRenderer, navigate-to-manager link

Documentation:
- Updated gadget-drone/docs/subprocess.md Phase 2 section
- Updated gadget-code/docs/ui-design-guide.md Phase 2 section
- Updated docs/socket-protocol.md with new events, sequences, and patterns
2026-05-14 13:52:59 -04:00
Rob Colbert
5dac708117 fix: properly constrain CodeMirror editor height in flex layout
The editor was overflowing its container because @uiw/react-codemirror
renders a wrapper div that doesn't inherit flex constraints by default.

Fixed by adding CSS rules that:
1. Set overflow:hidden on .cm-editor-container
2. Target the wrapper div (> div) with flex:1, min-height:0, overflow:hidden
3. Set .cm-editor and .cm-scroller to overflow appropriately

This ensures the flex height constraint propagates through every level
of the CodeMirror DOM tree, and only .cm-scroller scrolls.

Layout chain:
  .cm-editor-container (flex-1, min-h-0)
    └─ wrapper div (flex:1, min-h-0, overflow:hidden)
      └─ .cm-editor (flex:1, min-h-0, overflow:hidden)
        └─ .cm-scroller (min-h-0, overflow:auto) ← only this scrolls
2026-05-13 06:49:07 -04:00
Rob Colbert
5228034a5f fixes 2026-05-13 06:38:29 -04:00
Rob Colbert
c705cbd3d9 feat: replace react-ace with @uiw/react-codemirror
react-ace v14 is CommonJS-only with no ESM entry point, making it
fundamentally incompatible with Vite's ESM-first dev server. Every
CJS interop workaround failed. Switched to @uiw/react-codemirror v4.25
which ships proper dual ESM+CJS and works with Vite out of the box.

Changes:
- Remove ace-builds and react-ace dependencies
- Add @uiw/react-codemirror + 16 @codemirror/lang-* packages
- Add @uiw/codemirror-theme-tomorrow-night-blue (closest to ACE's 'tomorrow')
- Add @replit/codemirror-lang-csharp for C# support
- Rewrite EditorPanel.tsx: delete 108 lines of ACE boilerplate
  (?url imports, setModuleUrl, CJS interop hack), replace with
  ~30 lines of clean CodeMirror language extension setup
- Delete vite.d.ts (only needed for ACE ?url import types)
- Remove optimizeDeps.include from vite.config.ts (not needed for CM)
- Add CodeMirror flex layout CSS to index.css

Supported languages: JavaScript/JSX, TypeScript/TSX, Python, JSON,
HTML, CSS, Less, YAML, Markdown, SQL, Java, Go, Rust, C/C++, C#,
PHP, XML. Unsupported types fall back to plain text.

Verified: tsc clean, vite build passes, heartbeat worker intact.
2026-05-12 22:41:31 -04:00
Rob Colbert
dc54ea3dec fix: handle CJS default export interop for react-ace v14 in Vite dev server
react-ace v14 ships CommonJS only ('main': 'lib/index.js'). Vite's dev
server pre-bundling wraps CJS modules as namespace objects where the
default export is nested under .default. This caused 'Element type is
invalid: got object' because import Ace from 'react-ace' resolved to
the module namespace instead of the React component.

Fix: import * as ReactAceModule and extract default with fallback:
  const Ace = ReactAceModule.default || ReactAceModule

Same treatment for ace-builds import. Production build (Rolldown) was
unaffected due to different CJS interop handling.
2026-05-12 21:49:55 -04:00
Rob Colbert
37907ef098 fix 2026-05-12 21:40:43 -04:00
Rob Colbert
c14c3a235a fix: ACE editor integration crash — React 19 compat, Vite ?url pattern, Error Boundary
Root causes:
- react-ace v12 doesn't support React 19 (project uses ^19.2.5)
- Dynamic import(`ace-builds/src-noconflict/mode-${language}`) broken with Vite
  (template-literal dynamic imports can't be statically analyzed)
- No React Error Boundaries — ACE render crash whitescreens entire app
- ace-builds/react-ace duplicated in backend package.json

Fixes:
1. Upgrade react-ace ^12.0.0 → ^14.0.1 (React 19 support)
2. Upgrade ace-builds ^1.36.0 → ^1.43.6
3. Remove ACE deps from backend package.json (not used by Express)
4. Replace broken dynamic imports with Vite ?url + ace.config.setModuleUrl()
   pattern (canonical Vite solution per ace#4597)
5. Add ErrorBoundary component wrapping EditorPanel
6. Add vite.d.ts type declarations for ?url/?raw/?worker imports
7. Fix worker-typescript import (doesn't exist — TS uses worker-javascript)
8. Register 24 language modes, 4 workers, 1 theme, 2 extensions

Verified: TypeScript clean, production build passes, heartbeat worker intact.
2026-05-12 21:36:22 -04:00
Rob Colbert
fb8f15aeec Fix react-ace import: add optimizeDeps to Vite config
- Added react-ace and ace-builds to optimizeDeps.include
- This forces Vite to properly bundle the CommonJS react-ace module
- Resolves 'Element type is invalid' error when opening files

The react-ace package uses CommonJS exports, and Vite needs to
pre-bundle it to properly handle the default export in ESM context.
2026-05-12 20:02:13 -04:00
Rob Colbert
fb8b73e290 Fix editor integration: move EditorPanel to ChatSessionView
- Fixed Ace import (default import, not named)
- Moved EditorPanel from FilesPanel to ChatSessionView
- Editor now replaces chat view when file is selected (per UI design guide)
- FILES panel now contains only the file tree
- Added editorFilePath state to track open file
- File selection in tree opens editor in content area
- Close button returns to chat view

This matches the UI design guide specification where the File Editor
replaces the Chat View in the content area when a file is selected.
2026-05-12 19:45:50 -04:00
Rob Colbert
1e13f95808 Phase 2: ACE Editor integration and file operations
- Added react-ace and ace-builds dependencies
- Created EditorPanel component with ACE editor integration
- Implemented file read/write socket protocol
- Added backend handlers for fileReadRequest and fileWriteRequest
- Implemented file loading from tree click
- Implemented file saving with Ctrl+S shortcut
- Added dirty state tracking and unsaved changes indicator
- Enforced workspace mode (read-only in Agent mode)
- Added security: path traversal prevention, binary file detection, file size limits
- Updated FilesPanel with split view (tree + editor)

Enables Users to edit files for the first time in Gadget Code.
2026-05-12 19:32:58 -04:00
Rob Colbert
ec7b83d610 Fix: Proper flexbox scrolling for FILES panel sidebar
Root Cause: CSS flex items default to min-height: auto, which prevents
flex-1 elements from shrinking below their content size. This caused
the FilesPanel to grow beyond its allocated space when the file tree
had many entries, pushing content off-screen.

Changes:
- FilesPanel.tsx: Added min-h-0 to root div to override min-height: auto
  This allows flex-1 to actually constrain height and enable scrolling
- FileTree.tsx: Removed redundant overflow-auto and h-full from root div
  The parent (FilesPanel flex-1 overflow-auto) already handles scrolling.
  Nested scroll containers create height resolution issues.

The sidebar layout is now a clean single-scroll-container hierarchy:
  Sidebar (flex flex-col, no overflow)
  ↳ SESSION (shrink-0, natural height)
  ↳ PROJECT (shrink-0, natural height)
  ↳ FilesPanel (flex-1 min-h-0) — constrained to available space
    ↳ Header (natural height)
    ↳ Scroll area (flex-1 overflow-auto) — THE scroll container
      ↳ FileTree (p-2 only) — no overflow, no h-full
    ↳ Footer (natural height)
2026-05-12 17:40:11 -04:00
Rob Colbert
6dba16a8c5 Fix: Add shrink-0 to SESSION and PROJECT panels for proper flex layout
- SESSION and PROJECT panels now have shrink-0 to prevent them from
  growing unbounded and pushing FILES panel off-screen
- FILES panel with flex-1 now properly fills remaining vertical space
- File tree scrolls in-place while SESSION/PROJECT stay fixed
2026-05-12 17:26:20 -04:00
Rob Colbert
b0b94c2b7d Fix: FileTree scrolling - use h-full instead of flex-1
- FileTree had flex-1 inside an overflow container which didn't work
- Changed to h-full to properly fill the parent container height
- Scrolling now works correctly in the file tree area
2026-05-12 17:22:29 -04:00
Rob Colbert
869dd919c8 Fix: Remove overflow-y-auto from sidebar to prevent entire sidebar from scrolling
- The sidebar container had overflow-y-auto which caused the entire sidebar
  to scroll when FILES panel content grew
- FILES panel already has proper flex-1 and internal overflow-auto handling
- Now only the file tree scrolls in place while SESSION and PROJECT panels
  remain fixed as expected
- Fixes UX issue where entire sidebar would scroll vertically
2026-05-12 17:19:15 -04:00
Rob Colbert
a2a74c6da3 fix: FILES panel should expand to fill sidebar height
- Add flex-1 to FilesPanel root div
- Sidebar is flex flex-col, FILES panel now grows to fill space
- SESSION and PROJECT panels remain natural height
- FILES panel expands into all remaining vertical space

This eliminates the gap below the FILES panel footer by making
the panel itself grow rather than just making the tree bigger.
2026-05-12 16:58:35 -04:00
Rob Colbert
91d02e1468 fix: FILES panel should fill entire sidebar height
- Remove max-h-80 constraint from FileTree
- FileTree uses flex-1 to expand and fill available space
- Panel now properly fills sidebar from top to bottom
- File tree scrolls internally when content overflows

The FILES panel header, file tree, and footer now form a proper
flex column that fills all remaining vertical space in the sidebar.
2026-05-12 16:45:26 -04:00
Rob Colbert
a7c20d6105 fix: FileTree scrolling and text selection issues
- Change FilesPanel overflow-hidden to overflow-auto for scrolling
- Add max-h-80 to FileTree to limit height while allowing scroll
- Add select-none to FileTreeNode to prevent text selection
- Cursor-pointer already present for clickable indication

Now the file tree scrolls independently within the FILES panel,
and text cannot be selected during rapid clicking.
2026-05-12 16:38:27 -04:00
Rob Colbert
71f213d8d1 fix: FileTree duplication and corruption on expand
- Replace recursive buildTree with buildVisibleNodes approach
- Build flat list of visible nodes based on expanded state
- Use processNode helper to traverse tree structure correctly
- Each node appears exactly once in the rendered output
- Eliminates duplication when expanding directories
2026-05-12 16:11:20 -04:00
Rob Colbert
3062420e99 fix: FileTree expand/collapse not rendering children
- Change buildTree from .map() to for-loop to properly render children
- Append expanded directory children to nodes array immediately
- Fix TypeScript error by using ReactElement instead of JSX.Element
- Children now render correctly when directories are expanded
2026-05-12 16:03:18 -04:00
Rob Colbert
0a510de487 docs: Add plan for Project-Specific Agent Instructions feature
- Create comprehensive plan document for agent instructions text area
- Define requirements, acceptance criteria, and technical implementation
- Include UI/UX mockups and testing strategy
- Plan discovered during FILES panel implementation
- Addresses need for project-specific acceptance criteria
2026-05-12 15:39:38 -04:00
Rob Colbert
c05c7f5a61 feat: Implement FILES panel foundation with lazy-loading file tree
- Add fileTreeRequest/fileTreeResponse socket messages
- Implement gadget-drone handler with security validation
- Add web backend message forwarding
- Create FileTree and FileTreeNode React components
- Update FilesPanel to contain file tree browser
- Add requestFileTree method to frontend socket client
- Exclude node_modules, .git, and hidden files by default
- Implement lazy loading with directory caching
- Add loading and error states per node
- Support keyboard navigation (Enter, Space)

Phase 1 of FILES panel implementation complete.
2026-05-12 15:12:18 -04:00
Rob Colbert
4780b79148 feat: abort controller for work order processing
Add end-to-end abort support: AbortSignal in @gadget/ai providers,
abortWorkOrder socket message, drone AbortController handling,
Cancel button and double-Esc in frontend, and aborted turn status display.
2026-05-12 12:25:17 -04:00
Rob Colbert
c635209201 sidebar: navigate to project in Project Manager using slug
The Home icon in the PROJECT panel now navigates to /projects/{slug}
instead of /projects/{id}. This opens Project Manager with the
project already selected and displayed.
2026-05-12 11:14:53 -04:00
Rob Colbert
24975b58c4 Define missing socket event types and enforce typed events in frontend build
Adds type definitions + forwarding for status, reconnect_attempt, reconnect_failed, reconnect events.
Frontend build now runs tsc --noEmit before vite build so undefined socket events cause failures.
Fixes pre-existing type errors exposed by strict mode in the frontend.
2026-05-12 10:42:31 -04:00
Rob Colbert
896aff1b02 Fix User Settings persona display issue
- Switch frontend sign-in to /api/v1/auth/sign-in endpoint (includes persona)
- Add updateUser() to App context for proper state management
- Fix Settings.tsx save flow to use updateUser() instead of broken localStorage merge
- Remove unused web AuthController (gadget-code/src/controllers/auth.ts)
- Fix UserApiControllerV1 to return flat user object instead of double-wrapped
- Remove SessionType enum and references (dead code)
- Add proper server sign-out call before clearing local state

Resolves issue where User Settings view didn't display persona text even though it existed in the database.
2026-05-12 01:15:00 -04:00
Rob Colbert
b3c9579890 style and display fixes 2026-05-12 01:02:09 -04:00
Rob Colbert
dd47212471 fix: crush whitespace in subagent markdown output using .subagent-compact.gadget-markdown overrides 2026-05-11 21:28:02 -04:00
Rob Colbert
d118497d83 CSS and style fixes for subagent display 2026-05-11 21:09:17 -04:00
Rob Colbert
26e568612a ChatSession reconnect logic 2026-05-11 20:27:24 -04:00
Rob Colbert
01c84ba5a0 crash fix 2026-05-11 19:15:02 -04:00