Compare commits
29 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
8ca6881661 | ||
|
|
4fb0191460 | ||
|
|
47dd61f6d3 | ||
|
|
e0fd6ece54 | ||
|
|
a7a6a91a13 | ||
|
|
3c076fc01a | ||
|
|
66774d94f9 | ||
|
|
00e2fdfa4f | ||
|
|
70ad578425 | ||
|
|
9aaacead42 | ||
|
|
56f35a2fdf | ||
|
|
d47ef96916 | ||
|
|
1758bb2db7 | ||
|
|
aeaefc53dd | ||
|
|
faf815d66d | ||
|
|
e2cefc2617 | ||
|
|
f539f91018 | ||
|
|
cf31aff8db | ||
|
|
6696d79a0d | ||
|
|
ef43fa2a00 | ||
|
|
241f3b5b69 | ||
|
|
73b0b3149d | ||
|
|
51b3013659 | ||
|
|
2483161663 | ||
|
|
00969cf9bc | ||
|
|
9c1e65785d | ||
|
|
0437ca66d8 | ||
|
|
ca0cbf8e71 | ||
|
|
802184a487 |
31
CHANGE.md
31
CHANGE.md
@ -4,6 +4,37 @@ This document tracks all significant changes to Gadget Code, organized by date.
|
|||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
|
## 2026-05-19 (Monday)
|
||||||
|
|
||||||
|
**Qdrant Semantic Search and Vector Embeddings**
|
||||||
|
|
||||||
|
Implemented Qdrant-powered semantic search over chat history with full embedding pipeline, including Qdrant and LangChain configuration, vector dimension passthrough for both Ollama and OpenAI, and a new search API endpoint with frontend search input component. Also fixed Mongoose syncIndexes errors across all models and improved drone-session test isolation.
|
||||||
|
|
||||||
|
### Changes:
|
||||||
|
- **Semantic Search**: Implemented Qdrant vector search over chat history with embedding pipeline and search API endpoint.
|
||||||
|
- **Embeddings**: Added Qdrant and embedding model configuration; added `@langchain/textsplitters` for text vector processing; fixed vector dimension passthrough for Ollama and OpenAI.
|
||||||
|
- **Frontend**: Added SearchInput and ChatSearchResults components; integrated search into ChatSessionView.
|
||||||
|
- **Models**: Removed `syncIndexes` from all Mongoose models (caused duplicate index errors); mocked DB in drone-session tests.
|
||||||
|
- **Fixes**: Fixed auto-scroll, search input errors, and backend error handling; logging and formatting cleanup.
|
||||||
|
- **Build**: Added clean script.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 2026-05-18 (Sunday)
|
||||||
|
|
||||||
|
**Token Economics, Chat Export, and UI Layout Fixes**
|
||||||
|
|
||||||
|
Implemented token economics with real API token extraction from OpenAI responses, stats propagation through the session pipeline, and a context window fuel gauge. Added chat_export agent tool for markdown/JSON session export. Fixed home sidebar layout for independent scrolling sections.
|
||||||
|
|
||||||
|
### Changes:
|
||||||
|
- **Token Economics**: Implemented real API token extraction from OpenAI responses, stats propagation, and context window fuel gauge with auto-update.
|
||||||
|
- **Logging**: Added DtpLog logging of all OpenAI API responses with raw response object capture.
|
||||||
|
- **Chat Export**: Added `chat_export` agent tool for session export in markdown and JSON formats.
|
||||||
|
- **UI**: Fixed home sidebar flex column layout with independent scrolling sections; added `lucide-react` icons.
|
||||||
|
- **Config**: Added Qdrant and embedding model config types.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
## 2026-05-17 (Sunday)
|
## 2026-05-17 (Sunday)
|
||||||
|
|
||||||
**Release Hardening, Subagent Improvements, and Workspace Intelligence**
|
**Release Hardening, Subagent Improvements, and Workspace Intelligence**
|
||||||
|
|||||||
24
README.md
24
README.md
@ -41,6 +41,8 @@ pnpm dev
|
|||||||
│ │ │
|
│ │ │
|
||||||
│ Socket.IO │ MongoDB │ Files
|
│ Socket.IO │ MongoDB │ Files
|
||||||
│ JWT Auth │ Redis │ Git
|
│ JWT Auth │ Redis │ Git
|
||||||
|
│ │ Qdrant │
|
||||||
|
│ │ │
|
||||||
```
|
```
|
||||||
|
|
||||||
### How It Works
|
### How It Works
|
||||||
@ -117,6 +119,7 @@ pnpm cli user password user@example.com newpassword
|
|||||||
- pnpm 10+
|
- pnpm 10+
|
||||||
- MongoDB on `localhost:27017`
|
- MongoDB on `localhost:27017`
|
||||||
- Redis on `localhost:6379`
|
- Redis on `localhost:6379`
|
||||||
|
- Qdrant on `localhost:6333` (for semantic search)
|
||||||
- SSL certificates in `ssl/` directory (for dev servers)
|
- SSL certificates in `ssl/` directory (for dev servers)
|
||||||
|
|
||||||
### Build
|
### Build
|
||||||
@ -196,6 +199,23 @@ npx tsx scripts/seed-test-drones.ts
|
|||||||
- Model/mode selection per session
|
- Model/mode selection per session
|
||||||
- Session statistics (tool calls, file ops, subagents)
|
- Session statistics (tool calls, file ops, subagents)
|
||||||
|
|
||||||
|
### Token Economics
|
||||||
|
|
||||||
|
- Real API token extraction from OpenAI and Ollama responses
|
||||||
|
- Token usage stats propagated through session pipeline
|
||||||
|
- Context window fuel gauge with auto-update
|
||||||
|
|
||||||
|
### Semantic Search
|
||||||
|
|
||||||
|
- Qdrant-powered vector search over chat history
|
||||||
|
- Embedding pipeline with Ollama and OpenAI support
|
||||||
|
- Configurable embedding model and vector dimensions
|
||||||
|
|
||||||
|
### Chat Export
|
||||||
|
|
||||||
|
- Export chat sessions as markdown or JSON
|
||||||
|
- Agent tool for automated session export
|
||||||
|
|
||||||
### Workspace Management
|
### Workspace Management
|
||||||
|
|
||||||
- Each drone manages a workspace directory
|
- Each drone manages a workspace directory
|
||||||
@ -253,5 +273,5 @@ Apache 2.0 — See [LICENSE](./LICENSE) for details.
|
|||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
**Status:** v1.0.1 — First release. Self-hosted agentic engineering platform with local/self-hosted AI model support, complete Chat Session UI, subagent processing, and workspace management.
|
**Status:** v1.0.1 — Self-hosted agentic engineering platform with local/self-hosted AI model support, complete Chat Session UI, subagent processing, workspace management, token economics, and Qdrant semantic search.
|
||||||
**Last Updated:** May 17, 2026
|
**Last Updated:** May 19, 2026
|
||||||
|
|||||||
150
RELEASE.md
150
RELEASE.md
@ -1,111 +1,75 @@
|
|||||||
GADGET CODE v1.0.1
|
GADGET CODE v1.0.2
|
||||||
NAME: Liberation
|
NAME: Recall
|
||||||
HASH: ff46c35d38e29be64d9eca0ccc18e8a7f5c3d8ab
|
HASH: 47dd61f6d36a3287b67025e15caa03eb6f830195
|
||||||
TAG: v1.0.1
|
TAG: v1.0.2
|
||||||
|
|
||||||
SUMMARY:
|
SUMMARY:
|
||||||
|
|
||||||
Today marks the Liberation of Gadget Code — the first public release of a self-hosted agentic engineering platform that puts you in control of your code, your models, and your infrastructure.
|
Recall is the first patch release of Gadget Code — and the name is deliberate. This release introduces **semantic search over your chat history** using Qdrant vector embeddings, so your past engineering conversations are always just a query away. It also delivers real **token economics**: actual API token counts extracted from provider responses, propagated through the session stack, and surfaced in a live context window fuel gauge. Combined with chat export, streaming buffer persistence, and a round of model/test cleanup, Recall makes Gadget Code more observable, more searchable, and more reliable.
|
||||||
|
|
||||||
Over 200 commits spanning 21 days of intensive development, Gadget Code has grown from an initial commit into a complete platform for autonomous software engineering. And every line of it is Apache-2.0 licensed, fully open source, and designed to run on *your* terms.
|
## Semantic Search with Qdrant
|
||||||
|
|
||||||
## What Is Gadget Code?
|
The headline feature. Gadget Code now ships with an integrated **Qdrant vector search** pipeline that indexes every completed chat turn and makes it full-text+semantic searchable from the IDE. The architecture:
|
||||||
|
|
||||||
Gadget Code is an **Agentic Engineering Platform (AEP)** — a browser-based IDE that drives autonomous AI agents to perform real software engineering work on your behalf. Unlike cloud-locked alternatives, Gadget Code runs entirely in your environment: your servers, your data, your rules.
|
- **VectorStoreService** — A singleton service in `gadget-code` that manages Qdrant collection lifecycle, dimension mismatch detection, and embedding ingestion. It validates collection dimensions against the configured vector size and the model's actual output, surfacing clear error messages when they disagree.
|
||||||
|
- **Embedding pipeline** — Uses your configured AI provider to generate embeddings. Supports both Ollama and OpenAI-compatible endpoints. Configuration lives in `gadget-code.yaml` under the new `qdrant` section.
|
||||||
|
- **Search API endpoint** — `POST /api/v1/search` accepts a query string, generates its embedding, and returns ranked results from Qdrant.
|
||||||
|
- **Frontend components** — `SearchInput` and `ChatSearchResults` components in the React IDE provide a clean search-as-you-type experience with ranked results linked back to their source chat sessions.
|
||||||
|
|
||||||
The architecture is straightforward and powerful:
|
**New dependency**: Qdrant must be running on `localhost:6333` to enable semantic search. If `qdrant.providerId` is not configured, the service gracefully skips initialization — nothing breaks, you just don't get search.
|
||||||
|
|
||||||
```
|
**New package dependency**: `@langchain/textsplitters` for chunking turn content before embedding.
|
||||||
┌──────────────┐ ┌──────────────┐ ┌──────────────┐
|
|
||||||
│ Browser IDE │────▶│ gadget-code │────▶│ gadget-drone │
|
## Token Economics
|
||||||
│ (React 19) │◀────│ (Express 5) │◀────│ (Worker) │
|
|
||||||
└──────────────┘ └──────────────┘ └──────────────┘
|
Token counts are no longer estimates. This release extracts real usage data from OpenAI API responses and propagates it through the entire session stack:
|
||||||
│ │ │
|
|
||||||
│ Socket.IO │ MongoDB │ Files
|
- **Real token extraction** — OpenAI responses now return `usage.prompt_tokens`, `usage.completion_tokens`, and `usage.completion_tokens_details.reasoning_tokens` from the API. These are captured and stored on each `ChatTurn` and aggregated on the `ChatSession`.
|
||||||
│ JWT Auth │ Redis │ Git
|
- **Context window fuel gauge** — The session UI now displays a live context window usage indicator. As token counts accumulate across turns, the gauge updates automatically so you know when you're approaching your model's context limit.
|
||||||
|
- **Stats propagation** — `IWorkOrderCompleteStats` now carries `masterInputTokens`, `masterOutputTokens`, and `masterThinkingTokens`. The `onWorkOrderComplete` handler aggregates master + subagent tokens for the session-level totals.
|
||||||
|
- **DtpLog API response logging** — Added `logApiResponse` to the `DtpLog` transport for structured logging of AI provider responses with full token metadata.
|
||||||
|
|
||||||
|
## Chat Export
|
||||||
|
|
||||||
|
New `chat_export` agent tool lets the drone export chat sessions as markdown or JSON. Useful for documentation generation, session archival, and sharing engineering conversations outside the IDE.
|
||||||
|
|
||||||
|
## Streaming Buffer Persistence
|
||||||
|
|
||||||
|
Drone sessions now use an in-memory streaming buffer that aggregates thinking and response tokens by mode, flushing to the database at mode transitions (e.g., thinking → tool call → responding). This reduces database writes during streaming and ensures turn blocks are persisted in the correct order with accurate timestamps.
|
||||||
|
|
||||||
|
## Model and Test Cleanup
|
||||||
|
|
||||||
|
- **Removed `syncIndexes()` from all Mongoose models** — The auto-invoked `(async () => { await Model.syncIndexes(); })()` blocks at module level caused Mongoose connection timeout errors during tests when no database was available. These calls are no longer needed and have been removed from all 11 model files.
|
||||||
|
- **Mocked database in drone-session tests** — `ChatSession.updateOne()`, `VectorStoreService.ingestTurn()`, and `MessageQueue` are now properly mocked so tests run without any database dependency.
|
||||||
|
|
||||||
|
## Configuration Changes
|
||||||
|
|
||||||
|
New `qdrant` section in `gadget-code.yaml`:
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
qdrant:
|
||||||
|
host: localhost
|
||||||
|
port: 6333
|
||||||
|
providerId: <your-ai-provider-id>
|
||||||
|
embeddingModel: <model-name>
|
||||||
|
vectorSize: 1024
|
||||||
|
collection: gadget-code
|
||||||
```
|
```
|
||||||
|
|
||||||
A user creates a project in the browser IDE, selects a drone instance, enters a prompt, and the drone executes the Agentic Workflow Loop — thinking, responding, calling tools, spawning subagents — all streamed back in real time.
|
New types added to `@gadget/config`: `IQdrantConfig` and `IEmbeddingModelConfig`.
|
||||||
|
|
||||||
## Your Models, Your Way
|
## Other Changes
|
||||||
|
|
||||||
This is what Liberation means. Gadget Code ships with first-class support for **Ollama** and any **OpenAI-compatible API**. Run local models on your own GPU. Self-host an inference endpoint behind your firewall. Use a commercial provider when it suits you. Mix and match per session. The `@gadget/ai` package provides a unified abstraction — no consumer code ever imports an SDK directly. Swap providers without changing a line of application logic.
|
- **Auto-scroll fix** — Chat session view now properly auto-scrolls during streaming responses.
|
||||||
|
- **Home sidebar layout** — Fixed flex column layout for independent scrolling in the home sidebar.
|
||||||
The model configuration pipeline supports `numCtx` (context window), `numPredict`, and `maxCompletionTokens` settings, giving you fine-grained control over how your models behave.
|
- **Clean script** — Added `clean` script to `gadget-code/package.json` for easy build artifact removal.
|
||||||
|
- **UI** — Added `lucide-react` icon library for the search interface and other UI components.
|
||||||
## Complete Agentic Engineering Platform
|
|
||||||
|
|
||||||
Gadget Code isn't a chatbot wrapper. It's a full-stack engineering environment:
|
|
||||||
|
|
||||||
- **Project Manager** — Create projects, select available drone instances, manage chat session history.
|
|
||||||
- **Chat Session View** — Real-time streaming responses with collapsible thinking content, tool call summaries, session statistics, and per-session model/mode selection.
|
|
||||||
- **Agentic Workflow Loop** — The drone executes a multi-turn reasoning loop: think, respond, call tools, observe results, and iterate. Work orders are abortable, and the loop supports checkpointing for crash recovery.
|
|
||||||
- **Subagent Processing** — Agents can spawn specialized subagents for decomposition tasks. Subagent context is managed independently and streamed back to the IDE in real time.
|
|
||||||
- **FILE Panel** — Lazy-loading file tree with full workspace navigation and CodeMirror editor integration (React 19 compatible).
|
|
||||||
- **Workspace Management** — Each drone manages a workspace directory with crash recovery via the `.gadget/` directory and intelligent startup that detects the workspace by walking up the file hierarchy.
|
|
||||||
- **Scheduled Tasks (gadget-tasks)** — A headless IDE client that automates the browser flow on a cron schedule. It drives the existing gadget-code platform via the same REST API and Socket.IO protocol the browser uses — no duplicated logic.
|
|
||||||
- **Pull Project** — Clone remote repositories directly into your workspace.
|
|
||||||
- **Abort Controller** — Cancel in-progress work orders with proper state cleanup.
|
|
||||||
|
|
||||||
## Configuration and Dependencies
|
|
||||||
|
|
||||||
Gadget Code requires:
|
|
||||||
|
|
||||||
- **Node.js 22+** and **pnpm 10+**
|
|
||||||
- **MongoDB** on `localhost:27017`
|
|
||||||
- **Redis** on `localhost:6379`
|
|
||||||
- **SSL certificates** in the `ssl/` directory for dev servers
|
|
||||||
- An **Ollama** instance or **OpenAI-compatible API** endpoint for AI features
|
|
||||||
|
|
||||||
The monorepo is organized as follows:
|
|
||||||
|
|
||||||
| Package | Role |
|
|
||||||
| -------------------- | --------------------------------------------------------------- |
|
|
||||||
| `gadget-code` | Web service — agentic IDE, browser UI, API server |
|
|
||||||
| `gadget-drone` | Worker process — runs the agentic workflow loop |
|
|
||||||
| `gadget-tasks` | Scheduled task worker — headless IDE client for cron-driven tasks |
|
|
||||||
| `@gadget/ai` | Shared AI API abstraction — Ollama and OpenAI |
|
|
||||||
| `@gadget/ai-toolbox` | Shared AI tool implementations — search, file, plan, subagent |
|
|
||||||
| `@gadget/api` | Shared TypeScript interfaces — common types across all packages |
|
|
||||||
| `@gadget/config` | Shared YAML config loader — per-package configuration |
|
|
||||||
|
|
||||||
AI providers are managed via CLI:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# Add Ollama (no API key needed)
|
|
||||||
pnpm cli provider add "Local Ollama" ollama http://localhost:11434
|
|
||||||
|
|
||||||
# Add OpenAI-compatible provider
|
|
||||||
pnpm cli provider add "OpenAI" openai https://api.openai.com $OPENAI_API_KEY
|
|
||||||
|
|
||||||
# Discover available models
|
|
||||||
pnpm cli provider probe <provider-id>
|
|
||||||
```
|
|
||||||
|
|
||||||
## The Journey to v1.0.1
|
|
||||||
|
|
||||||
This release represents 21 days of development from April 27 through May 17, 2026:
|
|
||||||
|
|
||||||
- **Days 1–2** (Apr 27–28): Project initialization, dark industrial theme, JWT authentication, Project Manager, drone-to-IDE event routing, workspace persistence and crash recovery.
|
|
||||||
- **Days 3–4** (Apr 29–30): Socket protocol completeness, DroneManager, GadgetId migration from ObjectId to nanoid-based string IDs, "Welcome to The Grid" home view.
|
|
||||||
- **Days 5–6** (May 1–3): Workspace mode switching with validation, session locks, basic chat system milestone — the first end-to-end working demo.
|
|
||||||
- **Days 7–8** (May 5–6): Agent tool and toolbox architecture, tool registration patterns, AI environment foundation.
|
|
||||||
- **Day 9** (May 7): Streaming responses implementation — real-time token delivery from Ollama.
|
|
||||||
- **Day 10** (May 8): Chat session heartbeat, user settings, logging unification into `@gadget/api` as GadgetLog, redesigned sign-in form.
|
|
||||||
- **Day 9 continued** (May 9): Agentic Workflow Loop rebuild, chat session auto-naming, drone-to-IDE log transport.
|
|
||||||
- **Day 11** (May 10): Workspace/project philosophy refinement, agent toolbox refactor, credential provider fixes.
|
|
||||||
- **Day 12** (May 11): Subagent processing, authentication fixes, deployment documentation with systemd service files.
|
|
||||||
- **Day 13** (May 12): FILES panel with lazy-loading file tree, CodeMirror editor integration (replacing react-ace for React 19 compat), abort controller.
|
|
||||||
- **Day 14** (May 13): Editor height fixes, User Mode documentation.
|
|
||||||
- **Day 15** (May 14): SubProcess observability — tracking spawned processes in the DroneManager and DroneInspector.
|
|
||||||
- **Day 16** (May 15): Pull Project feature, tool call observability, context window configuration, Gab AI affiliate link.
|
|
||||||
- **Days 17–18** (May 16–17): AI toolbox extraction into `@gadget/ai-toolbox`, Project Manager upgrades, gadget-tasks headless client architecture, release hardening.
|
|
||||||
|
|
||||||
## Project Direction
|
## Project Direction
|
||||||
|
|
||||||
Gadget Code is built on a simple conviction: **your code, your models, your infrastructure**. The cloud AI ecosystem has moved toward vendor lock-in at every layer — proprietary models, closed platforms, data harvesting. Liberation is the alternative. Every component of Gadget Code is self-hosted. Every AI call goes through your providers. Every file stays on your disk. Every model runs on your hardware if you want it to.
|
Recall reinforces Gadget Code's core promise: **your code, your models, your infrastructure**. The semantic search feature runs on *your* Qdrant instance, using *your* embedding model, on *your* hardware. The token economics give you real observability into what your models are consuming — no vendor telemetry, no cloud analytics, just numbers on your screen from your API calls.
|
||||||
|
|
||||||
This is also the first release where the release document itself is authored by Gadget — the AI agent — as part of the automated release workflow. The agent that writes code for you also writes the release notes. We think that's fitting.
|
The removal of `syncIndexes()` from models and proper test mocking represents our ongoing commitment to clean, isolated unit tests that don't require standing up external services. Tests should be fast, deterministic, and runnable anywhere.
|
||||||
|
|
||||||
## Get Started
|
## Get Started
|
||||||
|
|
||||||
@ -113,7 +77,7 @@ Get the latest release at **https://g4dge7.com/** — the official home of Gadge
|
|||||||
|
|
||||||
Apache-2.0 licensed. Fully open source. No vendor lock-in. No data harvesting. No cloud dependency. Just you, your models, and your code.
|
Apache-2.0 licensed. Fully open source. No vendor lock-in. No data harvesting. No cloud dependency. Just you, your models, and your code.
|
||||||
|
|
||||||
That's Liberation.
|
That's Recall.
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
|
|||||||
@ -132,12 +132,14 @@ minio:
|
|||||||
videos: "gadget-videos"
|
videos: "gadget-videos"
|
||||||
audios: "gadget-audios"
|
audios: "gadget-audios"
|
||||||
|
|
||||||
# ── Qdrant Vector Database (COMING SOON) ────────────────────────────────────
|
# ── Qdrant Vector Database ──────────────────────────────────────────────────────
|
||||||
# Used for semantic search over chat history and conversation context.
|
# Used for semantic search over chat history and conversation context.
|
||||||
# Enable when the Qdrant integration is released.
|
|
||||||
#
|
#
|
||||||
# qdrant:
|
qdrant:
|
||||||
# host: "localhost"
|
host: "localhost"
|
||||||
# port: 6333
|
port: 6333
|
||||||
# apiKey: "${QDRANT_API_KEY}"
|
apiKey: "${QDRANT_API_KEY}"
|
||||||
# collection: "gadget-chat-embeddings"
|
collection: "gadget-chat-embeddings"
|
||||||
|
providerId: "${QDRANT_PROVIDER_ID}" # AiProvider._id from the database
|
||||||
|
embeddingModel: "nomic-embed-text" # model name on the provider
|
||||||
|
vectorSize: 768 # must match the embedding model output
|
||||||
|
|||||||
129
gadget-code/frontend/src/components/ChatSearchResults.tsx
Normal file
129
gadget-code/frontend/src/components/ChatSearchResults.tsx
Normal file
@ -0,0 +1,129 @@
|
|||||||
|
// src/components/ChatSearchResults.tsx
|
||||||
|
// Copyright (C) 2026 Robert Colbert <rob.colbert@openplatform.us>
|
||||||
|
// All Rights Reserved
|
||||||
|
|
||||||
|
import { useState, useEffect, useCallback } from "react";
|
||||||
|
import { Search, X, Loader2 } from "lucide-react";
|
||||||
|
import { searchApi, ISearchResult } from "../lib/api";
|
||||||
|
|
||||||
|
interface ChatSearchResultsProps {
|
||||||
|
query: string;
|
||||||
|
filters?: { projectId?: string; sessionId?: string; turnId?: string };
|
||||||
|
onSelect: (result: ISearchResult) => void;
|
||||||
|
onClose: () => void;
|
||||||
|
}
|
||||||
|
|
||||||
|
export default function ChatSearchResults({
|
||||||
|
query,
|
||||||
|
filters,
|
||||||
|
onSelect,
|
||||||
|
onClose,
|
||||||
|
}: ChatSearchResultsProps) {
|
||||||
|
const [editQuery, setEditQuery] = useState(query);
|
||||||
|
const [results, setResults] = useState<ISearchResult[]>([]);
|
||||||
|
const [loading, setLoading] = useState(false);
|
||||||
|
const [error, setError] = useState<string | null>(null);
|
||||||
|
|
||||||
|
const performSearch = useCallback(async () => {
|
||||||
|
if (!editQuery.trim()) return;
|
||||||
|
setLoading(true);
|
||||||
|
setError(null);
|
||||||
|
try {
|
||||||
|
const data = await searchApi.search(editQuery, filters, 20);
|
||||||
|
setResults(data);
|
||||||
|
} catch (err) {
|
||||||
|
setError((err as Error).message || "Search failed");
|
||||||
|
} finally {
|
||||||
|
setLoading(false);
|
||||||
|
}
|
||||||
|
}, [editQuery, filters]);
|
||||||
|
|
||||||
|
useEffect(() => { performSearch(); }, [performSearch]);
|
||||||
|
|
||||||
|
const handleSearchKeyDown = (e: React.KeyboardEvent<HTMLInputElement>) => {
|
||||||
|
if (e.key === "Enter") {
|
||||||
|
e.preventDefault();
|
||||||
|
performSearch();
|
||||||
|
}
|
||||||
|
};
|
||||||
|
|
||||||
|
const formatScore = (score: number): string => `${Math.round(score * 100)}%`;
|
||||||
|
|
||||||
|
const formatTimestamp = (iso: string): string => {
|
||||||
|
if (!iso) return "";
|
||||||
|
try { return new Date(iso).toLocaleDateString(undefined, { month: "short", day: "numeric", year: "numeric", hour: "2-digit", minute: "2-digit" }); }
|
||||||
|
catch { return iso; }
|
||||||
|
};
|
||||||
|
|
||||||
|
return (
|
||||||
|
<div className="fixed inset-0 bg-black/80 flex items-center justify-center z-50"
|
||||||
|
onClick={(e) => { if (e.target === e.currentTarget) onClose(); }}
|
||||||
|
onKeyDown={(e) => { if (e.key === "Escape") onClose(); }}
|
||||||
|
>
|
||||||
|
<div className="bg-bg-secondary border border-border-default rounded-lg w-full max-w-3xl max-h-[80vh] flex flex-col mx-4">
|
||||||
|
{/* Header with editable search input */}
|
||||||
|
<div className="flex items-center gap-3 px-6 py-4 border-b border-border-default">
|
||||||
|
<Search className="text-text-tertiary shrink-0" size={16} />
|
||||||
|
<input
|
||||||
|
type="text"
|
||||||
|
value={editQuery}
|
||||||
|
onChange={(e) => setEditQuery(e.target.value)}
|
||||||
|
onKeyDown={handleSearchKeyDown}
|
||||||
|
className="flex-1 bg-transparent text-text-primary text-sm focus:outline-none placeholder:text-text-tertiary"
|
||||||
|
placeholder="Edit search query…"
|
||||||
|
autoFocus
|
||||||
|
/>
|
||||||
|
<button
|
||||||
|
onClick={performSearch}
|
||||||
|
className="text-text-tertiary hover:text-brand transition-colors shrink-0"
|
||||||
|
aria-label="Submit search"
|
||||||
|
>
|
||||||
|
<Search size={16} />
|
||||||
|
</button>
|
||||||
|
<button onClick={onClose} className="text-text-tertiary hover:text-text-primary transition-colors p-1 shrink-0" aria-label="Close">
|
||||||
|
<X size={18} />
|
||||||
|
</button>
|
||||||
|
</div>
|
||||||
|
|
||||||
|
{/* Result count bar */}
|
||||||
|
{!loading && (
|
||||||
|
<div className="px-6 py-2 text-sm text-text-tertiary border-b border-border-default">
|
||||||
|
{results.length} result{results.length !== 1 ? "s" : ""} for "{query === editQuery ? query : editQuery}"
|
||||||
|
</div>
|
||||||
|
)}
|
||||||
|
|
||||||
|
<div className="overflow-y-auto flex-1 p-4">
|
||||||
|
{loading && (
|
||||||
|
<div className="flex items-center justify-center py-12">
|
||||||
|
<Loader2 className="text-brand text-2xl animate-spin" />
|
||||||
|
<span className="ml-3 text-text-secondary">Searching…</span>
|
||||||
|
</div>
|
||||||
|
)}
|
||||||
|
{error && <div className="text-center py-12"><p className="text-red-400">{error}</p></div>}
|
||||||
|
{!loading && !error && results.length === 0 && <div className="text-center py-12"><p className="text-text-tertiary">No results found</p></div>}
|
||||||
|
{!loading && !error && results.length > 0 && (
|
||||||
|
<div className="space-y-2">
|
||||||
|
{results.map((result) => (
|
||||||
|
<button key={result.id} onClick={() => onSelect(result)}
|
||||||
|
className="w-full text-left p-4 rounded-lg bg-bg-primary hover:bg-bg-tertiary border border-border-default hover:border-brand transition-colors group"
|
||||||
|
>
|
||||||
|
<div className="flex items-start justify-between gap-3">
|
||||||
|
<span className="shrink-0 inline-flex items-center px-2 py-0.5 rounded text-xs font-mono bg-brand/20 text-brand">{formatScore(result.score)}</span>
|
||||||
|
<div className="flex-1 min-w-0"><p className="text-sm text-text-primary line-clamp-3 whitespace-pre-wrap">{result.content}</p></div>
|
||||||
|
</div>
|
||||||
|
<div className="flex items-center gap-2 mt-2 text-xs text-text-tertiary">
|
||||||
|
{result.project?.name && <span>{result.project.name}</span>}
|
||||||
|
{result.session?.name && <><span>›</span><span>{result.session.name}</span></>}
|
||||||
|
<span>›</span>
|
||||||
|
<span className="font-mono">{result.turnId.slice(0, 8)}…</span>
|
||||||
|
{result.createdAt && <><span className="mx-1">·</span><span>{formatTimestamp(result.createdAt)}</span></>}
|
||||||
|
</div>
|
||||||
|
</button>
|
||||||
|
))}
|
||||||
|
</div>
|
||||||
|
)}
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
);
|
||||||
|
}
|
||||||
@ -34,7 +34,7 @@ const ChatTurn = memo(function ChatTurn({ turn }: ChatTurnProps) {
|
|||||||
const startedAt = new Date(turn.createdAt);
|
const startedAt = new Date(turn.createdAt);
|
||||||
|
|
||||||
return (
|
return (
|
||||||
<div className="border-b border-border-subtle pb-4 last:border-b-0">
|
<div id={`turn-${turn._id}`} className="border-b border-border-subtle pb-4 last:border-b-0">
|
||||||
{/* Turn Header */}
|
{/* Turn Header */}
|
||||||
<div className="flex items-center gap-4 text-xs text-text-muted mb-3 px-4">
|
<div className="flex items-center gap-4 text-xs text-text-muted mb-3 px-4">
|
||||||
<div>{startedAt.toLocaleTimeString()}</div>
|
<div>{startedAt.toLocaleTimeString()}</div>
|
||||||
|
|||||||
@ -28,7 +28,7 @@ export default function Clock() {
|
|||||||
}, []);
|
}, []);
|
||||||
|
|
||||||
return (
|
return (
|
||||||
<div className="p-3">
|
<div className="p-3 shrink-0">
|
||||||
<div className="text-xs font-semibold text-text-muted uppercase tracking-wider mb-2">
|
<div className="text-xs font-semibold text-text-muted uppercase tracking-wider mb-2">
|
||||||
Clock
|
Clock
|
||||||
</div>
|
</div>
|
||||||
|
|||||||
84
gadget-code/frontend/src/components/DroneSelectionModal.tsx
Normal file
84
gadget-code/frontend/src/components/DroneSelectionModal.tsx
Normal file
@ -0,0 +1,84 @@
|
|||||||
|
// src/components/DroneSelectionModal.tsx
|
||||||
|
// Copyright (C) 2026 Robert Colbert <rob.colbert@openplatform.us>
|
||||||
|
// All Rights Reserved
|
||||||
|
|
||||||
|
import { useState, useEffect } from "react";
|
||||||
|
import { X, Loader2 } from "lucide-react";
|
||||||
|
import { droneApi, type DroneRegistration } from "../lib/api";
|
||||||
|
|
||||||
|
interface DroneSelectionModalProps {
|
||||||
|
onSelect: (drone: DroneRegistration) => void;
|
||||||
|
onClose: () => void;
|
||||||
|
}
|
||||||
|
|
||||||
|
export default function DroneSelectionModal({ onSelect, onClose }: DroneSelectionModalProps) {
|
||||||
|
const [drones, setDrones] = useState<DroneRegistration[]>([]);
|
||||||
|
const [loading, setLoading] = useState(true);
|
||||||
|
const [error, setError] = useState<string | null>(null);
|
||||||
|
|
||||||
|
useEffect(() => {
|
||||||
|
const loadDrones = async () => {
|
||||||
|
try { setDrones(await droneApi.getAll()); }
|
||||||
|
catch (err) { setError((err as Error).message || "Failed to load drones"); }
|
||||||
|
finally { setLoading(false); }
|
||||||
|
};
|
||||||
|
loadDrones();
|
||||||
|
}, []);
|
||||||
|
|
||||||
|
const statusColor = (status: string) => {
|
||||||
|
switch (status) {
|
||||||
|
case "available": return "bg-green-500";
|
||||||
|
case "busy": return "bg-yellow-500";
|
||||||
|
default: return "bg-gray-500";
|
||||||
|
}
|
||||||
|
};
|
||||||
|
|
||||||
|
const availableDrones = drones.filter(d => d.status === "available" || d.status === "busy");
|
||||||
|
|
||||||
|
return (
|
||||||
|
<div className="fixed inset-0 bg-black/80 flex items-center justify-center z-50"
|
||||||
|
onClick={(e) => { if (e.target === e.currentTarget) onClose(); }}
|
||||||
|
onKeyDown={(e) => { if (e.key === "Escape") onClose(); }}
|
||||||
|
>
|
||||||
|
<div className="bg-bg-secondary border border-border-default rounded-lg w-full max-w-xl mx-4">
|
||||||
|
<div className="flex items-center justify-between px-6 py-4 border-b border-border-default">
|
||||||
|
<h2 className="text-lg font-semibold text-text-primary">Select a Drone</h2>
|
||||||
|
<button onClick={onClose} className="text-text-tertiary hover:text-text-primary transition-colors p-1" aria-label="Close"><X size={18} /></button>
|
||||||
|
</div>
|
||||||
|
<div className="p-4 max-h-[60vh] overflow-y-auto">
|
||||||
|
{loading && (
|
||||||
|
<div className="flex items-center justify-center py-8">
|
||||||
|
<Loader2 className="text-brand text-xl animate-spin" />
|
||||||
|
<span className="ml-3 text-text-secondary">Loading drones…</span>
|
||||||
|
</div>
|
||||||
|
)}
|
||||||
|
{error && <div className="text-center py-8"><p className="text-red-400">{error}</p></div>}
|
||||||
|
{!loading && !error && availableDrones.length === 0 && (
|
||||||
|
<div className="text-center py-8">
|
||||||
|
<p className="text-text-tertiary">No drones are currently available.</p>
|
||||||
|
<p className="text-text-muted text-sm mt-2">Start a drone agent to begin working.</p>
|
||||||
|
</div>
|
||||||
|
)}
|
||||||
|
{!loading && !error && availableDrones.length > 0 && (
|
||||||
|
<div className="space-y-2">
|
||||||
|
{availableDrones.map((drone) => (
|
||||||
|
<button key={drone._id} onClick={() => onSelect(drone)}
|
||||||
|
className="w-full text-left p-3 rounded-lg bg-bg-primary hover:bg-bg-tertiary border border-border-default hover:border-brand transition-colors group"
|
||||||
|
>
|
||||||
|
<div className="flex items-center gap-3">
|
||||||
|
<span className={`shrink-0 w-2.5 h-2.5 rounded-full ${statusColor(drone.status)}`} />
|
||||||
|
<div className="flex-1 min-w-0">
|
||||||
|
<div className="font-mono text-sm text-text-primary group-hover:text-brand transition-colors">{drone.hostname}</div>
|
||||||
|
<div className="text-xs text-text-tertiary truncate">{drone.workspaceDir}</div>
|
||||||
|
</div>
|
||||||
|
<span className="text-xs text-text-muted capitalize">{drone.status}</span>
|
||||||
|
</div>
|
||||||
|
</button>
|
||||||
|
))}
|
||||||
|
</div>
|
||||||
|
)}
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
);
|
||||||
|
}
|
||||||
68
gadget-code/frontend/src/components/SearchInput.tsx
Normal file
68
gadget-code/frontend/src/components/SearchInput.tsx
Normal file
@ -0,0 +1,68 @@
|
|||||||
|
// src/components/SearchInput.tsx
|
||||||
|
// Copyright (C) 2026 Robert Colbert <rob.colbert@openplatform.us>
|
||||||
|
// All Rights Reserved
|
||||||
|
|
||||||
|
import { useState } from "react";
|
||||||
|
import { Search, X } from "lucide-react";
|
||||||
|
|
||||||
|
interface SearchInputProps {
|
||||||
|
placeholder?: string;
|
||||||
|
onSearch: (query: string) => void;
|
||||||
|
onClear?: () => void;
|
||||||
|
className?: string;
|
||||||
|
}
|
||||||
|
|
||||||
|
export default function SearchInput({
|
||||||
|
placeholder = "Search…",
|
||||||
|
onSearch,
|
||||||
|
onClear,
|
||||||
|
className = "",
|
||||||
|
}: SearchInputProps) {
|
||||||
|
const [value, setValue] = useState("");
|
||||||
|
|
||||||
|
const handleChange = (e: React.ChangeEvent<HTMLInputElement>) => {
|
||||||
|
const v = e.target.value;
|
||||||
|
setValue(v);
|
||||||
|
if (!v.trim()) {
|
||||||
|
onClear?.();
|
||||||
|
}
|
||||||
|
};
|
||||||
|
|
||||||
|
const handleClear = () => {
|
||||||
|
setValue("");
|
||||||
|
onClear?.();
|
||||||
|
};
|
||||||
|
|
||||||
|
const handleSubmit = () => {
|
||||||
|
if (value.trim()) onSearch(value.trim());
|
||||||
|
};
|
||||||
|
|
||||||
|
const handleKeyDown = (e: React.KeyboardEvent<HTMLInputElement>) => {
|
||||||
|
if (e.key === "Enter") handleSubmit();
|
||||||
|
if (e.key === "Escape") handleClear();
|
||||||
|
};
|
||||||
|
|
||||||
|
return (
|
||||||
|
<div className={`relative flex items-center ${className}`}>
|
||||||
|
<Search className="absolute left-3 text-text-tertiary pointer-events-none" size={14} />
|
||||||
|
<input
|
||||||
|
type="text"
|
||||||
|
value={value}
|
||||||
|
onChange={handleChange}
|
||||||
|
onKeyDown={handleKeyDown}
|
||||||
|
placeholder={placeholder}
|
||||||
|
className="w-full bg-bg-secondary border border-border-default rounded pl-8 pr-16 py-2 text-sm text-text-primary placeholder:text-text-tertiary focus:outline-none focus:border-brand transition-colors"
|
||||||
|
/>
|
||||||
|
{value && (
|
||||||
|
<>
|
||||||
|
<button onClick={handleClear} className="absolute right-8 text-text-tertiary hover:text-text-primary transition-colors" aria-label="Clear search">
|
||||||
|
<X size={14} />
|
||||||
|
</button>
|
||||||
|
<button onClick={handleSubmit} className="absolute right-2 text-text-tertiary hover:text-brand transition-colors" aria-label="Submit search">
|
||||||
|
<Search size={14} />
|
||||||
|
</button>
|
||||||
|
</>
|
||||||
|
)}
|
||||||
|
</div>
|
||||||
|
);
|
||||||
|
}
|
||||||
@ -41,6 +41,7 @@ interface SessionPanelProps {
|
|||||||
truncateModelName: (name: string, maxLength?: number) => string;
|
truncateModelName: (name: string, maxLength?: number) => string;
|
||||||
getSelectedModelCapabilities: () => { hasThinking: boolean } | null;
|
getSelectedModelCapabilities: () => { hasThinking: boolean } | null;
|
||||||
getSelectedModel: () => AiModel | null;
|
getSelectedModel: () => AiModel | null;
|
||||||
|
contextWindowUsage: number;
|
||||||
}
|
}
|
||||||
|
|
||||||
export default function SessionPanel({
|
export default function SessionPanel({
|
||||||
@ -82,7 +83,9 @@ export default function SessionPanel({
|
|||||||
truncateModelName,
|
truncateModelName,
|
||||||
getSelectedModelCapabilities,
|
getSelectedModelCapabilities,
|
||||||
getSelectedModel,
|
getSelectedModel,
|
||||||
|
contextWindowUsage,
|
||||||
}: SessionPanelProps) {
|
}: SessionPanelProps) {
|
||||||
|
const usagePercent = sessionNumCtx > 0 ? (contextWindowUsage / sessionNumCtx) * 100 : 0;
|
||||||
return (
|
return (
|
||||||
<div className="border-b border-border-subtle shrink-0">
|
<div className="border-b border-border-subtle shrink-0">
|
||||||
<div className="flex items-center justify-between px-4 py-2 bg-bg-tertiary">
|
<div className="flex items-center justify-between px-4 py-2 bg-bg-tertiary">
|
||||||
@ -292,6 +295,33 @@ export default function SessionPanel({
|
|||||||
</div>
|
</div>
|
||||||
</div>
|
</div>
|
||||||
)}
|
)}
|
||||||
|
<div>
|
||||||
|
<div className="text-xs text-text-muted">Context Window Usage</div>
|
||||||
|
<div className="mt-1">
|
||||||
|
<div className="flex items-center gap-1.5">
|
||||||
|
<span className="text-[10px] font-mono text-text-muted w-3">E</span>
|
||||||
|
<div className="flex-1 h-3 bg-bg-tertiary rounded-full overflow-hidden border border-border-subtle">
|
||||||
|
<div
|
||||||
|
className={`h-full rounded-full transition-all duration-300 ${
|
||||||
|
usagePercent >= 80 ? 'bg-red-500' :
|
||||||
|
usagePercent >= 50 ? 'bg-yellow-500' :
|
||||||
|
'bg-green-500'
|
||||||
|
}`}
|
||||||
|
style={{ width: `${Math.min(usagePercent, 100)}%` }}
|
||||||
|
/>
|
||||||
|
</div>
|
||||||
|
<span className="text-[10px] font-mono text-text-muted w-3">F</span>
|
||||||
|
</div>
|
||||||
|
<div className="flex justify-between mt-0.5">
|
||||||
|
<span className="text-[10px] text-text-muted tabular-nums">
|
||||||
|
{contextWindowUsage.toLocaleString()} tokens
|
||||||
|
</span>
|
||||||
|
<span className="text-[10px] text-text-muted tabular-nums">
|
||||||
|
{sessionNumCtx.toLocaleString()} max
|
||||||
|
</span>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
</div>
|
</div>
|
||||||
</div>
|
</div>
|
||||||
);
|
);
|
||||||
|
|||||||
@ -388,6 +388,7 @@ export interface ChatSession {
|
|||||||
selectedModel: string;
|
selectedModel: string;
|
||||||
reasoningEffort?: string;
|
reasoningEffort?: string;
|
||||||
numCtx?: number;
|
numCtx?: number;
|
||||||
|
contextWindowUsage: number;
|
||||||
stats: ChatSessionStats;
|
stats: ChatSessionStats;
|
||||||
pins: Array<{ _id?: string; content: string }>;
|
pins: Array<{ _id?: string; content: string }>;
|
||||||
}
|
}
|
||||||
@ -401,6 +402,18 @@ export interface ChatTurnStats {
|
|||||||
durationLabel: string;
|
durationLabel: string;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
export interface IWorkOrderCompleteStats {
|
||||||
|
masterInputTokens: number;
|
||||||
|
masterOutputTokens: number;
|
||||||
|
masterThinkingTokens: number;
|
||||||
|
toolCallCount: number;
|
||||||
|
durationMs: number;
|
||||||
|
/** Aggregate (master + subagent) totals for session-level accounting */
|
||||||
|
totalInputTokens?: number;
|
||||||
|
totalOutputTokens?: number;
|
||||||
|
totalToolCallCount?: number;
|
||||||
|
}
|
||||||
|
|
||||||
export interface ChatTurnPrompts {
|
export interface ChatTurnPrompts {
|
||||||
user: string;
|
user: string;
|
||||||
system?: string;
|
system?: string;
|
||||||
@ -487,3 +500,32 @@ export const chatSessionApi = {
|
|||||||
getTurns: (id: string) =>
|
getTurns: (id: string) =>
|
||||||
api.get<ChatTurn[]>(`/api/v1/chat-sessions/${id}/turns`),
|
api.get<ChatTurn[]>(`/api/v1/chat-sessions/${id}/turns`),
|
||||||
};
|
};
|
||||||
|
|
||||||
|
export interface ISearchResult {
|
||||||
|
id: string;
|
||||||
|
content: string;
|
||||||
|
score: number;
|
||||||
|
userId: string;
|
||||||
|
projectId: string;
|
||||||
|
sessionId: string;
|
||||||
|
turnId: string;
|
||||||
|
role: string;
|
||||||
|
createdAt: string;
|
||||||
|
user?: { _id: string; displayName: string; username: string };
|
||||||
|
project?: { _id: string; name: string; slug: string };
|
||||||
|
session?: { _id: string; name: string };
|
||||||
|
turn?: { _id: string; status: string; mode: string; llm: string };
|
||||||
|
}
|
||||||
|
|
||||||
|
export const searchApi = {
|
||||||
|
search: (
|
||||||
|
query: string,
|
||||||
|
filters?: { projectId?: string; sessionId?: string; turnId?: string },
|
||||||
|
topK?: number,
|
||||||
|
) =>
|
||||||
|
api.post<ISearchResult[]>("/api/v1/search", {
|
||||||
|
query,
|
||||||
|
...filters,
|
||||||
|
topK,
|
||||||
|
}),
|
||||||
|
};
|
||||||
|
|||||||
@ -1,6 +1,6 @@
|
|||||||
import { createContext } from "react";
|
import { createContext } from "react";
|
||||||
import { io, Socket } from "socket.io-client";
|
import { io, Socket } from "socket.io-client";
|
||||||
import type { ChatSession } from "./api";
|
import type { ChatSession, IWorkOrderCompleteStats } from "./api";
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* Web Worker for heartbeat timing — avoids browser tab throttling.
|
* Web Worker for heartbeat timing — avoids browser tab throttling.
|
||||||
@ -30,6 +30,7 @@ export interface ServerToClientEvents {
|
|||||||
turnId: string,
|
turnId: string,
|
||||||
success: boolean,
|
success: boolean,
|
||||||
message?: string,
|
message?: string,
|
||||||
|
stats?: IWorkOrderCompleteStats,
|
||||||
) => void;
|
) => void;
|
||||||
log: (
|
log: (
|
||||||
timestamp: string,
|
timestamp: string,
|
||||||
@ -90,6 +91,7 @@ export interface SocketEvents {
|
|||||||
turnId: string,
|
turnId: string,
|
||||||
success: boolean,
|
success: boolean,
|
||||||
message?: string,
|
message?: string,
|
||||||
|
stats?: IWorkOrderCompleteStats,
|
||||||
) => void;
|
) => void;
|
||||||
"agent:thinking": (data: { agentId: string; thinking: string }) => void;
|
"agent:thinking": (data: { agentId: string; thinking: string }) => void;
|
||||||
"agent:response": (data: { agentId: string; chunk: string }) => void;
|
"agent:response": (data: { agentId: string; chunk: string }) => void;
|
||||||
@ -228,8 +230,8 @@ class SocketClient {
|
|||||||
|
|
||||||
this.socket.on(
|
this.socket.on(
|
||||||
"workOrderComplete",
|
"workOrderComplete",
|
||||||
(turnId: string, success: boolean, message?: string) => {
|
(turnId: string, success: boolean, message?: string, stats?: IWorkOrderCompleteStats) => {
|
||||||
this.emit("workOrderComplete", turnId, success, message);
|
this.emit("workOrderComplete", turnId, success, message, stats);
|
||||||
},
|
},
|
||||||
);
|
);
|
||||||
|
|
||||||
|
|||||||
@ -2,7 +2,7 @@ import { useState, useEffect, useRef, useContext, useCallback } from 'react';
|
|||||||
import { useParams, useNavigate, useLocation } from 'react-router-dom';
|
import { useParams, useNavigate, useLocation } from 'react-router-dom';
|
||||||
import { BrushCleaning, Maximize2 } from 'lucide-react';
|
import { BrushCleaning, Maximize2 } from 'lucide-react';
|
||||||
import { socketClient } from '../lib/socket';
|
import { socketClient } from '../lib/socket';
|
||||||
import { chatSessionApi, projectApi, providerApi, type ChatSession, type ChatTurn, type ChatTurnBlock, ChatTurnStats, ChatSessionMode, type AiProvider, type Project } from '../lib/api';
|
import { chatSessionApi, projectApi, providerApi, type ChatSession, type ChatTurn, type ChatTurnBlock, ChatTurnStats, ChatSessionMode, type AiProvider, type Project, type IWorkOrderCompleteStats, type ISearchResult } from '../lib/api';
|
||||||
import { WorkspaceMode } from '../lib/types';
|
import { WorkspaceMode } from '../lib/types';
|
||||||
import SessionPanel from '../components/SessionPanel';
|
import SessionPanel from '../components/SessionPanel';
|
||||||
import ProjectPanel from '../components/ProjectPanel';
|
import ProjectPanel from '../components/ProjectPanel';
|
||||||
@ -12,6 +12,9 @@ import ChatTurnComponent from '../components/ChatTurn';
|
|||||||
import EditorPanel from '../components/EditorPanel';
|
import EditorPanel from '../components/EditorPanel';
|
||||||
import ErrorBoundary from '../components/ErrorBoundary';
|
import ErrorBoundary from '../components/ErrorBoundary';
|
||||||
import ExpandedPromptEditor from '../components/ExpandedPromptEditor';
|
import ExpandedPromptEditor from '../components/ExpandedPromptEditor';
|
||||||
|
import SearchInput from '../components/SearchInput';
|
||||||
|
import ChatSearchResults from '../components/ChatSearchResults';
|
||||||
|
import DroneSelectionModal from '../components/DroneSelectionModal';
|
||||||
import { AppContext } from '../App';
|
import { AppContext } from '../App';
|
||||||
|
|
||||||
interface LogEntry {
|
interface LogEntry {
|
||||||
@ -56,6 +59,7 @@ enum SessionStartupState {
|
|||||||
interface RouterState {
|
interface RouterState {
|
||||||
drone?: any; // DroneRegistration passed via router state
|
drone?: any; // DroneRegistration passed via router state
|
||||||
project?: Project; // Project passed via router state (optional — can be fetched)
|
project?: Project; // Project passed via router state (optional — can be fetched)
|
||||||
|
scrollToTurnId?: string; // Turn to scroll to after search result selection
|
||||||
}
|
}
|
||||||
|
|
||||||
export default function ChatSessionView() {
|
export default function ChatSessionView() {
|
||||||
@ -104,6 +108,9 @@ export default function ChatSessionView() {
|
|||||||
const [reconnectAttempts, setReconnectAttempts] = useState(0);
|
const [reconnectAttempts, setReconnectAttempts] = useState(0);
|
||||||
const [editorFilePath, setEditorFilePath] = useState<string | undefined>(undefined);
|
const [editorFilePath, setEditorFilePath] = useState<string | undefined>(undefined);
|
||||||
const [isExpandedEditor, setIsExpandedEditor] = useState(false);
|
const [isExpandedEditor, setIsExpandedEditor] = useState(false);
|
||||||
|
const [searchQuery, setSearchQuery] = useState("");
|
||||||
|
const [showSearchResults, setShowSearchResults] = useState(false);
|
||||||
|
const [showDroneModal, setShowDroneModal] = useState(false);
|
||||||
|
|
||||||
const messagesEndRef = useRef<HTMLDivElement>(null);
|
const messagesEndRef = useRef<HTMLDivElement>(null);
|
||||||
const inputRef = useRef<HTMLTextAreaElement>(null);
|
const inputRef = useRef<HTMLTextAreaElement>(null);
|
||||||
@ -123,6 +130,23 @@ export default function ChatSessionView() {
|
|||||||
loadSessionData();
|
loadSessionData();
|
||||||
}, [projectId, sessionId]);
|
}, [projectId, sessionId]);
|
||||||
|
|
||||||
|
// Scroll to turn if navigated from search results
|
||||||
|
useEffect(() => {
|
||||||
|
const routerState = location.state as RouterState | null;
|
||||||
|
const targetTurnId = routerState?.scrollToTurnId;
|
||||||
|
if (targetTurnId && turns.length > 0) {
|
||||||
|
// Clear the router state so we don't re-scroll on re-renders
|
||||||
|
window.history.replaceState({}, '');
|
||||||
|
// Use requestAnimationFrame to ensure DOM is painted
|
||||||
|
requestAnimationFrame(() => {
|
||||||
|
const el = document.getElementById(`turn-${targetTurnId}`);
|
||||||
|
if (el) {
|
||||||
|
el.scrollIntoView({ behavior: 'smooth', block: 'center' });
|
||||||
|
}
|
||||||
|
});
|
||||||
|
}
|
||||||
|
}, [turns, location.state]);
|
||||||
|
|
||||||
useEffect(() => {
|
useEffect(() => {
|
||||||
setupSocketListeners();
|
setupSocketListeners();
|
||||||
return () => cleanupSocketListeners();
|
return () => cleanupSocketListeners();
|
||||||
@ -310,9 +334,9 @@ export default function ChatSessionView() {
|
|||||||
return; // Don't proceed to ready state
|
return; // Don't proceed to ready state
|
||||||
}
|
}
|
||||||
} else {
|
} else {
|
||||||
// No drone provided — skip lock step, go directly to Ready
|
// No drone provided — show drone selection modal
|
||||||
// (This handles the case where the view is loaded directly via URL without a drone,
|
// The user must select a drone to interact with the session
|
||||||
// e.g., for viewing completed sessions or task runs)
|
setShowDroneModal(true);
|
||||||
setStartupState(SessionStartupState.Ready);
|
setStartupState(SessionStartupState.Ready);
|
||||||
}
|
}
|
||||||
|
|
||||||
@ -686,7 +710,7 @@ export default function ChatSessionView() {
|
|||||||
}, 4000);
|
}, 4000);
|
||||||
}, []);
|
}, []);
|
||||||
|
|
||||||
const handleWorkOrderComplete = useCallback((turnId: string, success: boolean, message?: string) => {
|
const handleWorkOrderComplete = useCallback((turnId: string, success: boolean, message?: string, stats?: IWorkOrderCompleteStats) => {
|
||||||
// Backend has already flushed and persisted all streaming content
|
// Backend has already flushed and persisted all streaming content
|
||||||
// Just clean up frontend streaming state and update status
|
// Just clean up frontend streaming state and update status
|
||||||
if (streamingStateRef.current.has(turnId)) {
|
if (streamingStateRef.current.has(turnId)) {
|
||||||
@ -734,6 +758,40 @@ export default function ChatSessionView() {
|
|||||||
setIsAborting(false);
|
setIsAborting(false);
|
||||||
setIsProcessing(false);
|
setIsProcessing(false);
|
||||||
currentTurnIdRef.current = null;
|
currentTurnIdRef.current = null;
|
||||||
|
|
||||||
|
// Update turn stats from completion event
|
||||||
|
if (stats) {
|
||||||
|
setTurns(prev => prev.map(t => {
|
||||||
|
if (t._id !== turnId) return t;
|
||||||
|
return {
|
||||||
|
...t,
|
||||||
|
stats: {
|
||||||
|
...t.stats,
|
||||||
|
inputTokens: stats.masterInputTokens,
|
||||||
|
thinkingTokenCount: stats.masterThinkingTokens,
|
||||||
|
responseTokens: stats.masterOutputTokens,
|
||||||
|
toolCallCount: stats.toolCallCount,
|
||||||
|
durationMs: stats.durationMs,
|
||||||
|
durationLabel: stats.durationMs < 1000
|
||||||
|
? `${stats.durationMs}ms`
|
||||||
|
: `${(stats.durationMs / 1000).toFixed(1)}s`,
|
||||||
|
},
|
||||||
|
};
|
||||||
|
}));
|
||||||
|
|
||||||
|
// Update session contextWindowUsage for fuel gauge
|
||||||
|
// Uses aggregate totals (master+subagent) when available, falls back to master-only
|
||||||
|
setSession(prev => prev ? {
|
||||||
|
...prev,
|
||||||
|
contextWindowUsage: (prev.contextWindowUsage ?? 0) + stats.masterInputTokens,
|
||||||
|
stats: {
|
||||||
|
...prev.stats,
|
||||||
|
inputTokens: prev.stats.inputTokens + (stats.totalInputTokens ?? stats.masterInputTokens),
|
||||||
|
outputTokens: prev.stats.outputTokens + (stats.totalOutputTokens ?? stats.masterOutputTokens),
|
||||||
|
toolCallCount: prev.stats.toolCallCount + (stats.totalToolCallCount ?? stats.toolCallCount),
|
||||||
|
},
|
||||||
|
} : prev);
|
||||||
|
}
|
||||||
}, [showToast]);
|
}, [showToast]);
|
||||||
|
|
||||||
const handleWorkspaceModeChanged = useCallback((mode: string) => {
|
const handleWorkspaceModeChanged = useCallback((mode: string) => {
|
||||||
@ -1186,9 +1244,57 @@ export default function ChatSessionView() {
|
|||||||
}, [isProcessing, showToast, handleCancel]);
|
}, [isProcessing, showToast, handleCancel]);
|
||||||
|
|
||||||
const scrollToBottom = () => {
|
const scrollToBottom = () => {
|
||||||
messagesEndRef.current?.scrollIntoView({ behavior: 'smooth' });
|
messagesEndRef.current?.scrollIntoView({ behavior: 'instant' });
|
||||||
};
|
};
|
||||||
|
|
||||||
|
// ── Search handlers ──
|
||||||
|
const handleSearch = useCallback((query: string) => {
|
||||||
|
setSearchQuery(query);
|
||||||
|
setShowSearchResults(true);
|
||||||
|
}, []);
|
||||||
|
|
||||||
|
const handleSearchSelect = useCallback((result: ISearchResult) => {
|
||||||
|
setShowSearchResults(false);
|
||||||
|
setSearchQuery("");
|
||||||
|
// Scroll to the turn within this session view
|
||||||
|
const el = document.getElementById(`turn-${result.turnId}`);
|
||||||
|
if (el) {
|
||||||
|
el.scrollIntoView({ behavior: 'smooth', block: 'center' });
|
||||||
|
}
|
||||||
|
}, []);
|
||||||
|
|
||||||
|
const handleSearchClear = useCallback(() => {
|
||||||
|
setSearchQuery("");
|
||||||
|
setShowSearchResults(false);
|
||||||
|
}, []);
|
||||||
|
|
||||||
|
// ── Drone selection handler ──
|
||||||
|
const handleDroneSelect = useCallback(async (drone: any) => {
|
||||||
|
setDroneRegistration(drone);
|
||||||
|
droneRegistrationRef.current = drone;
|
||||||
|
setShowDroneModal(false);
|
||||||
|
|
||||||
|
// Persist and request lock
|
||||||
|
localStorage.setItem('dtp_drone_registration', JSON.stringify(drone));
|
||||||
|
try {
|
||||||
|
const success = await socketClient.requestSessionLock(
|
||||||
|
drone,
|
||||||
|
projectRef.current,
|
||||||
|
sessionRef.current,
|
||||||
|
);
|
||||||
|
if (success) {
|
||||||
|
setStartupState(SessionStartupState.Ready);
|
||||||
|
appContext?.setStatusMessage('Session ready');
|
||||||
|
} else {
|
||||||
|
setStartupState(SessionStartupState.LockFailed);
|
||||||
|
appContext?.setStatusMessage('Failed to lock drone — drone may be busy');
|
||||||
|
}
|
||||||
|
} catch (err) {
|
||||||
|
setStartupState(SessionStartupState.LockFailed);
|
||||||
|
appContext?.setStatusMessage('Failed to lock drone');
|
||||||
|
}
|
||||||
|
}, [session, appContext]);
|
||||||
|
|
||||||
const sessionLocked = startupState === SessionStartupState.Ready;
|
const sessionLocked = startupState === SessionStartupState.Ready;
|
||||||
|
|
||||||
const promptDisabled = isProcessing || workspaceMode !== WorkspaceMode.Agent || !sessionLocked || !session?.selectedModel || isEditingProvider;
|
const promptDisabled = isProcessing || workspaceMode !== WorkspaceMode.Agent || !sessionLocked || !session?.selectedModel || isEditingProvider;
|
||||||
@ -1332,7 +1438,16 @@ export default function ChatSessionView() {
|
|||||||
</div>
|
</div>
|
||||||
|
|
||||||
{/* Prompt Input */}
|
{/* Prompt Input */}
|
||||||
<div className="border-t border-border-subtle p-4 bg-bg-secondary shrink-0">
|
<div className="border-t border-border-subtle bg-bg-secondary shrink-0">
|
||||||
|
{/* Search bar */}
|
||||||
|
<div className="px-4 pt-3 pb-1">
|
||||||
|
<SearchInput
|
||||||
|
placeholder="Search in session"
|
||||||
|
onSearch={handleSearch}
|
||||||
|
onClear={handleSearchClear}
|
||||||
|
/>
|
||||||
|
</div>
|
||||||
|
<div className="p-4 pt-2">
|
||||||
<form onSubmit={handleSubmitPrompt} className="flex gap-2">
|
<form onSubmit={handleSubmitPrompt} className="flex gap-2">
|
||||||
<div className="flex flex-col self-end">
|
<div className="flex flex-col self-end">
|
||||||
<button
|
<button
|
||||||
@ -1390,6 +1505,7 @@ export default function ChatSessionView() {
|
|||||||
</form>
|
</form>
|
||||||
</div>
|
</div>
|
||||||
</div>
|
</div>
|
||||||
|
</div>
|
||||||
)}
|
)}
|
||||||
|
|
||||||
{/* Log Panel (25%) */}
|
{/* Log Panel (25%) */}
|
||||||
@ -1402,6 +1518,20 @@ export default function ChatSessionView() {
|
|||||||
|
|
||||||
{/* Sidebar */}
|
{/* Sidebar */}
|
||||||
<div className="w-80 border-l border-border-subtle bg-bg-secondary flex flex-col">
|
<div className="w-80 border-l border-border-subtle bg-bg-secondary flex flex-col">
|
||||||
|
{/* Drone Selection Button (when no drone is registered) */}
|
||||||
|
{!droneRegistration && (
|
||||||
|
<div className="px-4 py-3 border-b border-border-subtle">
|
||||||
|
<button
|
||||||
|
onClick={() => setShowDroneModal(true)}
|
||||||
|
className="w-full px-3 py-2 bg-brand text-white rounded hover:bg-red-700 transition-colors text-sm font-medium"
|
||||||
|
>
|
||||||
|
Select Drone
|
||||||
|
</button>
|
||||||
|
<p className="text-xs text-text-muted mt-1.5">
|
||||||
|
A drone is required to send prompts and interact with the session.
|
||||||
|
</p>
|
||||||
|
</div>
|
||||||
|
)}
|
||||||
<SessionPanel
|
<SessionPanel
|
||||||
session={session}
|
session={session}
|
||||||
workspaceMode={workspaceMode}
|
workspaceMode={workspaceMode}
|
||||||
@ -1441,6 +1571,7 @@ export default function ChatSessionView() {
|
|||||||
truncateModelName={truncateModelName}
|
truncateModelName={truncateModelName}
|
||||||
getSelectedModelCapabilities={getSelectedModelCapabilities}
|
getSelectedModelCapabilities={getSelectedModelCapabilities}
|
||||||
getSelectedModel={getSelectedModel}
|
getSelectedModel={getSelectedModel}
|
||||||
|
contextWindowUsage={session?.contextWindowUsage ?? 0}
|
||||||
/>
|
/>
|
||||||
|
|
||||||
<ProjectPanel
|
<ProjectPanel
|
||||||
@ -1455,6 +1586,20 @@ export default function ChatSessionView() {
|
|||||||
}}
|
}}
|
||||||
/>
|
/>
|
||||||
</div>
|
</div>
|
||||||
|
{showSearchResults && searchQuery && session && (
|
||||||
|
<ChatSearchResults
|
||||||
|
query={searchQuery}
|
||||||
|
filters={{ sessionId: session._id }}
|
||||||
|
onSelect={handleSearchSelect}
|
||||||
|
onClose={handleSearchClear}
|
||||||
|
/>
|
||||||
|
)}
|
||||||
|
{showDroneModal && (
|
||||||
|
<DroneSelectionModal
|
||||||
|
onSelect={handleDroneSelect}
|
||||||
|
onClose={() => setShowDroneModal(false)}
|
||||||
|
/>
|
||||||
|
)}
|
||||||
</div>
|
</div>
|
||||||
);
|
);
|
||||||
}
|
}
|
||||||
|
|||||||
@ -1,12 +1,14 @@
|
|||||||
import { useState, useEffect, useRef } from "react";
|
import { useState, useEffect, useRef } from "react";
|
||||||
import { Link, useNavigate } from "react-router-dom";
|
import { Link, useNavigate } from "react-router-dom";
|
||||||
import type { User, Project, DroneRegistration, ChatSession, SubProcessStat } from "../lib/api";
|
import type { User, Project, DroneRegistration, ChatSession, SubProcessStat, ISearchResult } from "../lib/api";
|
||||||
import { projectApi, droneApi, chatSessionApi } from "../lib/api";
|
import { projectApi, droneApi, chatSessionApi } from "../lib/api";
|
||||||
import { socketClient } from "../lib/socket";
|
import { socketClient } from "../lib/socket";
|
||||||
import Clock from "../components/Clock";
|
import Clock from "../components/Clock";
|
||||||
import GadgetGrid from "../components/GadgetGrid";
|
import GadgetGrid from "../components/GadgetGrid";
|
||||||
import LogRenderer from "../components/LogRenderer";
|
import LogRenderer from "../components/LogRenderer";
|
||||||
import type { LogRendererEntry } from "../components/LogRenderer";
|
import type { LogRendererEntry } from "../components/LogRenderer";
|
||||||
|
import SearchInput from "../components/SearchInput";
|
||||||
|
import ChatSearchResults from "../components/ChatSearchResults";
|
||||||
|
|
||||||
function SystemReady() {
|
function SystemReady() {
|
||||||
return (
|
return (
|
||||||
@ -269,53 +271,12 @@ function DashboardSidebar({
|
|||||||
};
|
};
|
||||||
|
|
||||||
return (
|
return (
|
||||||
<aside className="w-64 border-l border-border-subtle bg-bg-secondary overflow-y-auto">
|
<aside className="w-64 border-l border-border-subtle bg-bg-secondary flex flex-col overflow-hidden min-h-0">
|
||||||
<Clock />
|
<Clock />
|
||||||
|
|
||||||
<div className="p-3 border-t border-border-subtle">
|
{/* Projects */}
|
||||||
<div className="text-xs font-semibold text-text-muted uppercase tracking-wider mb-2">
|
<div className="flex-1 min-h-0 flex flex-col border-t border-border-subtle">
|
||||||
Recent Chats
|
<div className="shrink-0 px-3 pt-3 pb-2 flex items-center justify-between text-xs font-semibold text-text-muted uppercase tracking-wider">
|
||||||
</div>
|
|
||||||
{loading ? (
|
|
||||||
<p className="text-sm text-text-muted px-2">Loading...</p>
|
|
||||||
) : recentChats.length === 0 ? (
|
|
||||||
<p className="text-sm text-text-muted px-2">No recent chats.</p>
|
|
||||||
) : (
|
|
||||||
<div className="space-y-1">
|
|
||||||
{recentChats.map((session) => (
|
|
||||||
<button
|
|
||||||
key={session._id}
|
|
||||||
onClick={() => handleOpenChat(session)}
|
|
||||||
className="w-full text-left px-2 py-1.5 text-sm text-text-secondary hover:bg-bg-tertiary hover:text-text-primary rounded transition-colors"
|
|
||||||
>
|
|
||||||
<div className="flex items-center justify-between gap-1">
|
|
||||||
<span className="truncate font-medium text-xs">{session.name || 'Untitled'}</span>
|
|
||||||
<span className={`text-[10px] px-1.5 py-0.5 rounded-full font-medium shrink-0 ${modeBadgeColors[session.mode] || 'bg-gray-500/20 text-gray-400'}`}>
|
|
||||||
{session.mode}
|
|
||||||
</span>
|
|
||||||
</div>
|
|
||||||
<div className="flex items-center justify-between gap-1 mt-0.5">
|
|
||||||
<span className="text-[11px] text-text-muted truncate">{getProjectName(session)}</span>
|
|
||||||
<span className="text-[10px] text-text-muted shrink-0">
|
|
||||||
{formatRelativeTime(session.lastMessageAt || session.createdAt)}
|
|
||||||
</span>
|
|
||||||
</div>
|
|
||||||
</button>
|
|
||||||
))}
|
|
||||||
</div>
|
|
||||||
)}
|
|
||||||
{noDroneMessage && (
|
|
||||||
<p className="text-xs text-yellow-400 mt-2 px-2">{noDroneMessage}</p>
|
|
||||||
)}
|
|
||||||
{selectedDrone && (
|
|
||||||
<p className="text-[10px] text-text-muted mt-2 px-2 italic">
|
|
||||||
Select a chat to resume with {selectedDrone.hostname}
|
|
||||||
</p>
|
|
||||||
)}
|
|
||||||
</div>
|
|
||||||
|
|
||||||
<div className="p-3 border-t border-border-subtle">
|
|
||||||
<div className="flex items-center justify-between text-xs font-semibold text-text-muted uppercase tracking-wider mb-2">
|
|
||||||
<span>Projects</span>
|
<span>Projects</span>
|
||||||
<Link
|
<Link
|
||||||
to="/projects"
|
to="/projects"
|
||||||
@ -324,6 +285,7 @@ function DashboardSidebar({
|
|||||||
[+]
|
[+]
|
||||||
</Link>
|
</Link>
|
||||||
</div>
|
</div>
|
||||||
|
<div className="flex-1 min-h-0 overflow-y-auto px-3 pb-3">
|
||||||
{loading ? (
|
{loading ? (
|
||||||
<p className="text-sm text-text-muted px-2">Loading...</p>
|
<p className="text-sm text-text-muted px-2">Loading...</p>
|
||||||
) : projects.length === 0 ? (
|
) : projects.length === 0 ? (
|
||||||
@ -342,9 +304,11 @@ function DashboardSidebar({
|
|||||||
</div>
|
</div>
|
||||||
)}
|
)}
|
||||||
</div>
|
</div>
|
||||||
|
</div>
|
||||||
|
|
||||||
<div className="p-3 border-t border-border-subtle">
|
{/* Drones */}
|
||||||
<div className="flex items-center justify-between text-xs font-semibold text-text-muted uppercase tracking-wider mb-2">
|
<div className="flex-1 min-h-0 flex flex-col border-t border-border-subtle">
|
||||||
|
<div className="shrink-0 px-3 pt-3 pb-2 flex items-center justify-between text-xs font-semibold text-text-muted uppercase tracking-wider">
|
||||||
<span>Drones</span>
|
<span>Drones</span>
|
||||||
<Link
|
<Link
|
||||||
to="/drones"
|
to="/drones"
|
||||||
@ -372,6 +336,7 @@ function DashboardSidebar({
|
|||||||
</svg>
|
</svg>
|
||||||
</Link>
|
</Link>
|
||||||
</div>
|
</div>
|
||||||
|
<div className="flex-1 min-h-0 overflow-y-auto px-3 pb-3">
|
||||||
{loading ? (
|
{loading ? (
|
||||||
<p className="text-sm text-text-muted px-2">Loading...</p>
|
<p className="text-sm text-text-muted px-2">Loading...</p>
|
||||||
) : drones.length === 0 ? (
|
) : drones.length === 0 ? (
|
||||||
@ -418,6 +383,54 @@ function DashboardSidebar({
|
|||||||
</div>
|
</div>
|
||||||
)}
|
)}
|
||||||
</div>
|
</div>
|
||||||
|
</div>
|
||||||
|
|
||||||
|
{/* Recent Chats */}
|
||||||
|
<div className="flex-1 min-h-0 flex flex-col border-t border-border-subtle">
|
||||||
|
<div className="shrink-0 px-3 pt-3 pb-2 text-xs font-semibold text-text-muted uppercase tracking-wider">
|
||||||
|
Recent Chats
|
||||||
|
</div>
|
||||||
|
<div className="flex-1 min-h-0 overflow-y-auto px-3 pb-3">
|
||||||
|
{loading ? (
|
||||||
|
<p className="text-sm text-text-muted px-2">Loading...</p>
|
||||||
|
) : recentChats.length === 0 ? (
|
||||||
|
<p className="text-sm text-text-muted px-2">No recent chats.</p>
|
||||||
|
) : (
|
||||||
|
<div className="space-y-1">
|
||||||
|
{recentChats.map((session) => (
|
||||||
|
<button
|
||||||
|
key={session._id}
|
||||||
|
onClick={() => handleOpenChat(session)}
|
||||||
|
className="w-full text-left px-2 py-1.5 text-sm text-text-secondary hover:bg-bg-tertiary hover:text-text-primary rounded transition-colors"
|
||||||
|
>
|
||||||
|
<div className="flex items-center justify-between gap-1">
|
||||||
|
<span className="truncate font-medium text-xs">{session.name || 'Untitled'}</span>
|
||||||
|
<span className={`text-[10px] px-1.5 py-0.5 rounded-full font-medium shrink-0 ${modeBadgeColors[session.mode] || 'bg-gray-500/20 text-gray-400'}`}>
|
||||||
|
{session.mode}
|
||||||
|
</span>
|
||||||
|
</div>
|
||||||
|
<div className="flex items-center justify-between gap-1 mt-0.5">
|
||||||
|
<span className="text-[11px] text-text-muted truncate">{getProjectName(session)}</span>
|
||||||
|
<span className="text-[10px] text-text-muted shrink-0">
|
||||||
|
{formatRelativeTime(session.lastMessageAt || session.createdAt)}
|
||||||
|
</span>
|
||||||
|
</div>
|
||||||
|
</button>
|
||||||
|
))}
|
||||||
|
</div>
|
||||||
|
)}
|
||||||
|
</div>
|
||||||
|
<div className="shrink-0 px-3 pb-3">
|
||||||
|
{noDroneMessage && (
|
||||||
|
<p className="text-xs text-yellow-400 mt-2 px-2">{noDroneMessage}</p>
|
||||||
|
)}
|
||||||
|
{selectedDrone && (
|
||||||
|
<p className="text-[10px] text-text-muted mt-2 px-2 italic">
|
||||||
|
Select a chat to resume with {selectedDrone.hostname}
|
||||||
|
</p>
|
||||||
|
)}
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
|
||||||
</aside>
|
</aside>
|
||||||
);
|
);
|
||||||
@ -432,6 +445,27 @@ export default function Home({ user }: HomeProps) {
|
|||||||
const [selectedDrone, setSelectedDrone] = useState<DroneRegistration | null>(
|
const [selectedDrone, setSelectedDrone] = useState<DroneRegistration | null>(
|
||||||
null,
|
null,
|
||||||
);
|
);
|
||||||
|
const [searchQuery, setSearchQuery] = useState("");
|
||||||
|
const [showSearchResults, setShowSearchResults] = useState(false);
|
||||||
|
|
||||||
|
const handleSearch = (query: string) => {
|
||||||
|
setSearchQuery(query);
|
||||||
|
setShowSearchResults(true);
|
||||||
|
};
|
||||||
|
|
||||||
|
const handleSearchSelect = (result: ISearchResult) => {
|
||||||
|
setShowSearchResults(false);
|
||||||
|
setSearchQuery("");
|
||||||
|
// Navigate to the chat session — the turn will be scrolled to via id attr
|
||||||
|
navigate(`/projects/${result.projectId}/chat-session/${result.sessionId}`, {
|
||||||
|
state: { scrollToTurnId: result.turnId },
|
||||||
|
});
|
||||||
|
};
|
||||||
|
|
||||||
|
const handleSearchClear = () => {
|
||||||
|
setSearchQuery("");
|
||||||
|
setShowSearchResults(false);
|
||||||
|
};
|
||||||
|
|
||||||
if (!user) {
|
if (!user) {
|
||||||
return (
|
return (
|
||||||
@ -447,7 +481,7 @@ export default function Home({ user }: HomeProps) {
|
|||||||
}
|
}
|
||||||
|
|
||||||
const mainContent = selectedDrone ? (
|
const mainContent = selectedDrone ? (
|
||||||
<div className="relative z-10 flex-1 flex">
|
<div className="relative z-10 flex-1 flex min-h-0">
|
||||||
<DroneInspector
|
<DroneInspector
|
||||||
drone={selectedDrone}
|
drone={selectedDrone}
|
||||||
onClose={() => setSelectedDrone(null)}
|
onClose={() => setSelectedDrone(null)}
|
||||||
@ -463,10 +497,17 @@ export default function Home({ user }: HomeProps) {
|
|||||||
/>
|
/>
|
||||||
</div>
|
</div>
|
||||||
) : (
|
) : (
|
||||||
<div className="relative z-10 flex-1 flex flex-col">
|
<div className="relative z-10 flex-1 flex flex-col min-h-0">
|
||||||
<div className="flex-1 flex">
|
<div className="flex-1 flex min-h-0">
|
||||||
<div className="flex-1 flex items-center justify-center p-8">
|
<div className="flex-1 flex items-center justify-center p-8">
|
||||||
<div className="text-center">
|
<div className="text-center w-full max-w-lg">
|
||||||
|
<div className="mb-6">
|
||||||
|
<SearchInput
|
||||||
|
placeholder="Search your work"
|
||||||
|
onSearch={handleSearch}
|
||||||
|
onClear={handleSearchClear}
|
||||||
|
/>
|
||||||
|
</div>
|
||||||
<h1 className="text-2xl font-semibold mb-4">
|
<h1 className="text-2xl font-semibold mb-4">
|
||||||
Welcome, {user.displayName}!
|
Welcome, {user.displayName}!
|
||||||
</h1>
|
</h1>
|
||||||
@ -505,6 +546,13 @@ export default function Home({ user }: HomeProps) {
|
|||||||
<GadgetGrid />
|
<GadgetGrid />
|
||||||
</div>
|
</div>
|
||||||
{mainContent}
|
{mainContent}
|
||||||
|
{showSearchResults && searchQuery && (
|
||||||
|
<ChatSearchResults
|
||||||
|
query={searchQuery}
|
||||||
|
onSelect={handleSearchSelect}
|
||||||
|
onClose={handleSearchClear}
|
||||||
|
/>
|
||||||
|
)}
|
||||||
</div>
|
</div>
|
||||||
);
|
);
|
||||||
}
|
}
|
||||||
|
|||||||
@ -1,7 +1,7 @@
|
|||||||
import { useState, useEffect } from "react";
|
import { useState, useEffect } from "react";
|
||||||
import { useNavigate, useParams } from "react-router-dom";
|
import { useNavigate, useParams } from "react-router-dom";
|
||||||
import slug from "slug";
|
import slug from "slug";
|
||||||
import type { User, Project, ProjectSkill, ProjectTask } from "../lib/api";
|
import type { User, Project, ProjectSkill, ProjectTask, ISearchResult } from "../lib/api";
|
||||||
import {
|
import {
|
||||||
projectApi,
|
projectApi,
|
||||||
droneApi,
|
droneApi,
|
||||||
@ -11,6 +11,8 @@ import {
|
|||||||
type AiProvider,
|
type AiProvider,
|
||||||
providerApi,
|
providerApi,
|
||||||
} from "../lib/api";
|
} from "../lib/api";
|
||||||
|
import SearchInput from "../components/SearchInput";
|
||||||
|
import ChatSearchResults from "../components/ChatSearchResults";
|
||||||
import gabAiImage from "./assets/gab-ai.jpeg";
|
import gabAiImage from "./assets/gab-ai.jpeg";
|
||||||
|
|
||||||
interface ProjectManagerProps {
|
interface ProjectManagerProps {
|
||||||
@ -1501,6 +1503,26 @@ export default function ProjectManager({ user }: ProjectManagerProps) {
|
|||||||
const [loading, setLoading] = useState(true);
|
const [loading, setLoading] = useState(true);
|
||||||
const [showNewForm, setShowNewForm] = useState(false);
|
const [showNewForm, setShowNewForm] = useState(false);
|
||||||
const [showPullForm, setShowPullForm] = useState(false);
|
const [showPullForm, setShowPullForm] = useState(false);
|
||||||
|
const [searchQuery, setSearchQuery] = useState("");
|
||||||
|
const [showSearchResults, setShowSearchResults] = useState(false);
|
||||||
|
|
||||||
|
const handleSearch = (query: string) => {
|
||||||
|
setSearchQuery(query);
|
||||||
|
setShowSearchResults(true);
|
||||||
|
};
|
||||||
|
|
||||||
|
const handleSearchSelect = (result: ISearchResult) => {
|
||||||
|
setShowSearchResults(false);
|
||||||
|
setSearchQuery("");
|
||||||
|
navigate(`/projects/${result.projectId}/chat-session/${result.sessionId}`, {
|
||||||
|
state: { scrollToTurnId: result.turnId },
|
||||||
|
});
|
||||||
|
};
|
||||||
|
|
||||||
|
const handleSearchClear = () => {
|
||||||
|
setSearchQuery("");
|
||||||
|
setShowSearchResults(false);
|
||||||
|
};
|
||||||
|
|
||||||
useEffect(() => {
|
useEffect(() => {
|
||||||
loadProjects();
|
loadProjects();
|
||||||
@ -1688,12 +1710,21 @@ export default function ProjectManager({ user }: ProjectManagerProps) {
|
|||||||
) : selectedProject ? (
|
) : selectedProject ? (
|
||||||
<>
|
<>
|
||||||
{/* Center - Project Inspector */}
|
{/* Center - Project Inspector */}
|
||||||
|
<div className="flex-1 flex flex-col overflow-hidden">
|
||||||
|
<div className="px-6 pt-4 pb-2 border-b border-border-subtle">
|
||||||
|
<SearchInput
|
||||||
|
placeholder="Search in project"
|
||||||
|
onSearch={handleSearch}
|
||||||
|
onClear={handleSearchClear}
|
||||||
|
/>
|
||||||
|
</div>
|
||||||
<ProjectInspector
|
<ProjectInspector
|
||||||
project={selectedProject}
|
project={selectedProject}
|
||||||
providers={providers}
|
providers={providers}
|
||||||
onDelete={handleProjectDeleted}
|
onDelete={handleProjectDeleted}
|
||||||
onUpdate={handleProjectUpdated}
|
onUpdate={handleProjectUpdated}
|
||||||
/>
|
/>
|
||||||
|
</div>
|
||||||
|
|
||||||
{/* Right Sidebar - Drones & Chat Sessions */}
|
{/* Right Sidebar - Drones & Chat Sessions */}
|
||||||
<RightSidebar
|
<RightSidebar
|
||||||
@ -1712,6 +1743,14 @@ export default function ProjectManager({ user }: ProjectManagerProps) {
|
|||||||
</div>
|
</div>
|
||||||
)}
|
)}
|
||||||
</div>
|
</div>
|
||||||
|
{showSearchResults && searchQuery && (
|
||||||
|
<ChatSearchResults
|
||||||
|
query={searchQuery}
|
||||||
|
filters={selectedProject ? { projectId: selectedProject._id } : undefined}
|
||||||
|
onSelect={handleSearchSelect}
|
||||||
|
onClose={handleSearchClear}
|
||||||
|
/>
|
||||||
|
)}
|
||||||
</div>
|
</div>
|
||||||
);
|
);
|
||||||
}
|
}
|
||||||
|
|||||||
@ -25,6 +25,8 @@
|
|||||||
"@gadget/ai": "workspace:*",
|
"@gadget/ai": "workspace:*",
|
||||||
"@gadget/api": "workspace:*",
|
"@gadget/api": "workspace:*",
|
||||||
"@gadget/config": "workspace:*",
|
"@gadget/config": "workspace:*",
|
||||||
|
"@langchain/textsplitters": "^1.0.1",
|
||||||
|
"@qdrant/js-client-rest": "^1.12.0",
|
||||||
"@react-three/drei": "^10.7.7",
|
"@react-three/drei": "^10.7.7",
|
||||||
"@react-three/fiber": "^9.6.1",
|
"@react-three/fiber": "^9.6.1",
|
||||||
"ansicolor": "^2.0.3",
|
"ansicolor": "^2.0.3",
|
||||||
|
|||||||
@ -105,6 +105,15 @@ export default {
|
|||||||
host: yamlConfig.mongodb?.host || "localhost",
|
host: yamlConfig.mongodb?.host || "localhost",
|
||||||
database: yamlConfig.mongodb?.database || "",
|
database: yamlConfig.mongodb?.database || "",
|
||||||
},
|
},
|
||||||
|
qdrant: {
|
||||||
|
host: yamlConfig.qdrant?.host || "localhost",
|
||||||
|
port: yamlConfig.qdrant?.port || 6333,
|
||||||
|
apiKey: yamlConfig.qdrant?.apiKey,
|
||||||
|
collection: yamlConfig.qdrant?.collection || "gadget-chat-embeddings",
|
||||||
|
providerId: yamlConfig.qdrant?.providerId || "",
|
||||||
|
embeddingModel: yamlConfig.qdrant?.embeddingModel || "nomic-embed-text",
|
||||||
|
vectorSize: yamlConfig.qdrant?.vectorSize || 768,
|
||||||
|
},
|
||||||
redis: {
|
redis: {
|
||||||
host: yamlConfig.redis?.host || "localhost",
|
host: yamlConfig.redis?.host || "localhost",
|
||||||
port: yamlConfig.redis?.port || 6379,
|
port: yamlConfig.redis?.port || 6379,
|
||||||
|
|||||||
@ -35,6 +35,7 @@ export class ApiControllerV1 extends DtpController {
|
|||||||
await this.loadChild(path.join(basePath, "drone.js"));
|
await this.loadChild(path.join(basePath, "drone.js"));
|
||||||
await this.loadChild(path.join(basePath, "project.js"));
|
await this.loadChild(path.join(basePath, "project.js"));
|
||||||
await this.loadChild(path.join(basePath, "provider.js"));
|
await this.loadChild(path.join(basePath, "provider.js"));
|
||||||
|
await this.loadChild(path.join(basePath, "search.js"));
|
||||||
await this.loadChild(path.join(basePath, "user.js"));
|
await this.loadChild(path.join(basePath, "user.js"));
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|||||||
170
gadget-code/src/controllers/api/v1/search.ts
Normal file
170
gadget-code/src/controllers/api/v1/search.ts
Normal file
@ -0,0 +1,170 @@
|
|||||||
|
// src/controllers/api/v1/search.ts
|
||||||
|
// Copyright (C) 2026 Robert Colbert <rob.colbert@openplatform.us>
|
||||||
|
// All Rights Reserved
|
||||||
|
|
||||||
|
import { Request, Response } from "express";
|
||||||
|
|
||||||
|
import { DtpController } from "../../../lib/controller.js";
|
||||||
|
import VectorStoreService, { ISearchFilters, ISearchResult } from "../../../services/vector-store.js";
|
||||||
|
import User from "../../../models/user.js";
|
||||||
|
import Project from "../../../models/project.js";
|
||||||
|
import ChatSession from "../../../models/chat-session.js";
|
||||||
|
import ChatTurn from "../../../models/chat-turn.js";
|
||||||
|
|
||||||
|
interface IHydratedSearchResult extends ISearchResult {
|
||||||
|
user?: { _id: string; displayName: string; username: string };
|
||||||
|
project?: { _id: string; name: string; slug: string };
|
||||||
|
session?: { _id: string; name: string };
|
||||||
|
turn?: { _id: string; status: string; mode: string; llm: string };
|
||||||
|
}
|
||||||
|
|
||||||
|
interface ISearchRequestBody {
|
||||||
|
query: string;
|
||||||
|
userId?: string;
|
||||||
|
projectId?: string;
|
||||||
|
sessionId?: string;
|
||||||
|
turnId?: string;
|
||||||
|
topK?: number;
|
||||||
|
}
|
||||||
|
|
||||||
|
class SearchController extends DtpController {
|
||||||
|
get name(): string {
|
||||||
|
return "SearchController";
|
||||||
|
}
|
||||||
|
get slug(): string {
|
||||||
|
return "ctrl:api:v1:search";
|
||||||
|
}
|
||||||
|
get route(): string {
|
||||||
|
return "/search";
|
||||||
|
}
|
||||||
|
|
||||||
|
async start(): Promise<void> {
|
||||||
|
this.router.post("/", this.requireUser(), this.search.bind(this));
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* POST /api/v1/search
|
||||||
|
* Semantic search across chat history.
|
||||||
|
*
|
||||||
|
* Enforces userId filter: users can only search their own corpus.
|
||||||
|
*/
|
||||||
|
private async search(req: Request, res: Response): Promise<void> {
|
||||||
|
try {
|
||||||
|
const body = req.body as ISearchRequestBody;
|
||||||
|
|
||||||
|
if (!body.query?.trim()) {
|
||||||
|
res.status(400).json({
|
||||||
|
success: false,
|
||||||
|
message: "query is required",
|
||||||
|
});
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
// Enforce that the user can only search their own corpus
|
||||||
|
const userId = (req.user as any)?._id;
|
||||||
|
if (!userId) {
|
||||||
|
res.status(403).json({
|
||||||
|
success: false,
|
||||||
|
message: "Authentication required",
|
||||||
|
});
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
const filters: ISearchFilters = {
|
||||||
|
userId,
|
||||||
|
projectId: body.projectId,
|
||||||
|
sessionId: body.sessionId,
|
||||||
|
turnId: body.turnId,
|
||||||
|
};
|
||||||
|
|
||||||
|
const topK = body.topK && body.topK > 0 && body.topK <= 50
|
||||||
|
? body.topK
|
||||||
|
: 10;
|
||||||
|
|
||||||
|
// Check if vector store is initialized
|
||||||
|
if (!VectorStoreService.isReady) {
|
||||||
|
res.status(503).json({
|
||||||
|
success: false,
|
||||||
|
message: "Vector store is not configured. Set qdrant.providerId in gadget-code.yaml to enable semantic search.",
|
||||||
|
});
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
const results = await VectorStoreService.search(
|
||||||
|
body.query.trim(),
|
||||||
|
filters,
|
||||||
|
topK,
|
||||||
|
);
|
||||||
|
|
||||||
|
// Hydrate results with MongoDB documents
|
||||||
|
const hydratedResults = await this.hydrateResults(results);
|
||||||
|
|
||||||
|
res.json({
|
||||||
|
success: true,
|
||||||
|
data: hydratedResults,
|
||||||
|
});
|
||||||
|
} catch (error) {
|
||||||
|
this.log.error("search failed", { error });
|
||||||
|
res.status(500).json({
|
||||||
|
success: false,
|
||||||
|
message: (error as Error).message || "Search failed",
|
||||||
|
});
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Hydrate search results with user, project, session, and turn data from MongoDB.
|
||||||
|
* Collects unique IDs to batch-load documents efficiently.
|
||||||
|
*/
|
||||||
|
private async hydrateResults(
|
||||||
|
results: ISearchResult[],
|
||||||
|
): Promise<IHydratedSearchResult[]> {
|
||||||
|
if (results.length === 0) return [];
|
||||||
|
|
||||||
|
// Collect unique IDs for batch loading
|
||||||
|
const userIds = [...new Set(results.map((r) => r.userId).filter(Boolean))];
|
||||||
|
const projectIds = [...new Set(results.map((r) => r.projectId).filter(Boolean))];
|
||||||
|
const sessionIds = [...new Set(results.map((r) => r.sessionId).filter(Boolean))];
|
||||||
|
const turnIds = [...new Set(results.map((r) => r.turnId).filter(Boolean))];
|
||||||
|
|
||||||
|
// Batch load from MongoDB
|
||||||
|
const [users, projects, sessions, turns] = await Promise.all([
|
||||||
|
userIds.length > 0
|
||||||
|
? User.find({ _id: { $in: userIds } })
|
||||||
|
.select("_id displayName username")
|
||||||
|
.lean()
|
||||||
|
: [],
|
||||||
|
projectIds.length > 0
|
||||||
|
? Project.find({ _id: { $in: projectIds } })
|
||||||
|
.select("_id name slug")
|
||||||
|
.lean()
|
||||||
|
: [],
|
||||||
|
sessionIds.length > 0
|
||||||
|
? ChatSession.find({ _id: { $in: sessionIds } })
|
||||||
|
.select("_id name")
|
||||||
|
.lean()
|
||||||
|
: [],
|
||||||
|
turnIds.length > 0
|
||||||
|
? ChatTurn.find({ _id: { $in: turnIds } })
|
||||||
|
.select("_id status mode llm")
|
||||||
|
.lean()
|
||||||
|
: [],
|
||||||
|
]);
|
||||||
|
|
||||||
|
// Build lookup maps
|
||||||
|
const userMap = new Map(users.map((u) => [u._id, u]));
|
||||||
|
const projectMap = new Map(projects.map((p) => [p._id, p]));
|
||||||
|
const sessionMap = new Map(sessions.map((s) => [s._id, s]));
|
||||||
|
const turnMap = new Map(turns.map((t) => [t._id, t]));
|
||||||
|
|
||||||
|
return results.map((result) => ({
|
||||||
|
...result,
|
||||||
|
user: userMap.get(result.userId) as any || undefined,
|
||||||
|
project: projectMap.get(result.projectId) as any || undefined,
|
||||||
|
session: sessionMap.get(result.sessionId) as any || undefined,
|
||||||
|
turn: turnMap.get(result.turnId) as any || undefined,
|
||||||
|
}));
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
export default SearchController;
|
||||||
@ -23,6 +23,7 @@ import {
|
|||||||
SubmitPromptCallback,
|
SubmitPromptCallback,
|
||||||
AbortWorkOrderCallback,
|
AbortWorkOrderCallback,
|
||||||
SubProcessStat,
|
SubProcessStat,
|
||||||
|
IWorkOrderCompleteStats,
|
||||||
} from "@gadget/api";
|
} from "@gadget/api";
|
||||||
|
|
||||||
import ChatSession from "../models/chat-session.ts";
|
import ChatSession from "../models/chat-session.ts";
|
||||||
@ -669,7 +670,18 @@ export class CodeSession extends SocketSession {
|
|||||||
turnId: string,
|
turnId: string,
|
||||||
success: boolean,
|
success: boolean,
|
||||||
message?: string,
|
message?: string,
|
||||||
|
stats?: IWorkOrderCompleteStats,
|
||||||
): void {
|
): void {
|
||||||
this.socket.emit("workOrderComplete", turnId, success, message);
|
this.socket.emit("workOrderComplete", turnId, success, message, stats);
|
||||||
|
|
||||||
|
// Update in-memory session stats for UI reactivity
|
||||||
|
// Uses aggregate totals (master + subagent) when available, falls back to master-only
|
||||||
|
if (stats && this.chatSession) {
|
||||||
|
this.chatSession.stats.inputTokens += stats.totalInputTokens ?? stats.masterInputTokens;
|
||||||
|
this.chatSession.stats.outputTokens += stats.totalOutputTokens ?? stats.masterOutputTokens;
|
||||||
|
this.chatSession.stats.toolCallCount += stats.totalToolCallCount ?? stats.toolCallCount;
|
||||||
|
// contextWindowUsage is intentionally master-only (fuel gauge tracks master context)
|
||||||
|
this.chatSession.contextWindowUsage = (this.chatSession.contextWindowUsage ?? 0) + stats.masterInputTokens;
|
||||||
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|||||||
@ -12,6 +12,7 @@ import {
|
|||||||
WorkspaceMode,
|
WorkspaceMode,
|
||||||
IChatToolCall,
|
IChatToolCall,
|
||||||
IChatTurnBlock,
|
IChatTurnBlock,
|
||||||
|
IWorkOrderCompleteStats,
|
||||||
} from "@gadget/api";
|
} from "@gadget/api";
|
||||||
import {
|
import {
|
||||||
GadgetSocket,
|
GadgetSocket,
|
||||||
@ -19,7 +20,9 @@ import {
|
|||||||
SocketSessionType,
|
SocketSessionType,
|
||||||
} from "./socket-session.js";
|
} from "./socket-session.js";
|
||||||
import { SocketService } from "../services/index.js";
|
import { SocketService } from "../services/index.js";
|
||||||
|
import { VectorStoreService } from "../services/index.js";
|
||||||
import { ChatTurn } from "../models/chat-turn.js";
|
import { ChatTurn } from "../models/chat-turn.js";
|
||||||
|
import ChatSession from "../models/chat-session.js";
|
||||||
import MessageQueue, { type QueuedMessage } from "./message-queue.js";
|
import MessageQueue, { type QueuedMessage } from "./message-queue.js";
|
||||||
|
|
||||||
interface IStreamingBuffer {
|
interface IStreamingBuffer {
|
||||||
@ -292,6 +295,7 @@ export class DroneSession extends SocketSession {
|
|||||||
turnId: string,
|
turnId: string,
|
||||||
success: boolean,
|
success: boolean,
|
||||||
message?: string,
|
message?: string,
|
||||||
|
stats?: IWorkOrderCompleteStats,
|
||||||
): Promise<void> {
|
): Promise<void> {
|
||||||
if (!this.chatSessionId) {
|
if (!this.chatSessionId) {
|
||||||
this.log.warn("workOrderComplete event received but no chat session is active");
|
this.log.warn("workOrderComplete event received but no chat session is active");
|
||||||
@ -303,6 +307,8 @@ export class DroneSession extends SocketSession {
|
|||||||
await this.flushBuffer(turnId);
|
await this.flushBuffer(turnId);
|
||||||
this.streamingBuffers.delete(turnId);
|
this.streamingBuffers.delete(turnId);
|
||||||
|
|
||||||
|
let enrichedStats: IWorkOrderCompleteStats | undefined;
|
||||||
|
|
||||||
const turn = await ChatTurn.findById(turnId);
|
const turn = await ChatTurn.findById(turnId);
|
||||||
if (turn) {
|
if (turn) {
|
||||||
if (success && message === "aborted") {
|
if (success && message === "aborted") {
|
||||||
@ -314,20 +320,73 @@ export class DroneSession extends SocketSession {
|
|||||||
turn.errorMessage = message;
|
turn.errorMessage = message;
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// UPDATE TURN STATS — master agent tokens only
|
||||||
|
if (stats) {
|
||||||
|
turn.stats.inputTokens = stats.masterInputTokens;
|
||||||
|
turn.stats.thinkingTokenCount = stats.masterThinkingTokens;
|
||||||
|
turn.stats.responseTokens = stats.masterOutputTokens;
|
||||||
|
turn.stats.toolCallCount = stats.toolCallCount;
|
||||||
|
turn.stats.durationMs = stats.durationMs;
|
||||||
|
turn.stats.durationLabel = this.formatDurationLabel(stats.durationMs);
|
||||||
|
}
|
||||||
|
|
||||||
await turn.save();
|
await turn.save();
|
||||||
|
|
||||||
|
// Ingest to vector store (fire-and-forget for finished turns)
|
||||||
|
if (turn.status === ChatTurnStatus.Finished) {
|
||||||
|
VectorStoreService.ingestTurn(turnId).catch((err) => {
|
||||||
|
this.log.error("Failed to ingest turn to vector store", {
|
||||||
|
turnId,
|
||||||
|
error: err.message,
|
||||||
|
});
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
|
// COMPUTE AGGREGATE: walk the turn's subagent records for aggregate totals
|
||||||
|
let totalInputTokens = stats?.masterInputTokens ?? 0;
|
||||||
|
let totalOutputTokens = stats?.masterOutputTokens ?? 0;
|
||||||
|
let totalToolCallCount = stats?.toolCallCount ?? 0;
|
||||||
|
if (stats) {
|
||||||
|
for (const tc of turn.toolCalls) {
|
||||||
|
if ((tc as any).subagent?.stats) {
|
||||||
|
const subStats = (tc as any).subagent.stats;
|
||||||
|
totalInputTokens += subStats.inputTokens;
|
||||||
|
totalOutputTokens += subStats.responseTokens + subStats.thinkingTokenCount;
|
||||||
|
totalToolCallCount += subStats.toolCallCount;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Build enriched stats with aggregate values for in-memory consumers
|
||||||
|
enrichedStats = stats
|
||||||
|
? { ...stats, totalInputTokens, totalOutputTokens, totalToolCallCount }
|
||||||
|
: undefined;
|
||||||
|
|
||||||
|
// INCREMENT SESSION STATS — aggregate totals (master + subagents)
|
||||||
|
await ChatSession.updateOne(
|
||||||
|
{ _id: this.chatSessionId },
|
||||||
|
{ $inc: {
|
||||||
|
"stats.inputTokens": totalInputTokens,
|
||||||
|
"stats.outputTokens": totalOutputTokens,
|
||||||
|
"stats.toolCallCount": totalToolCallCount,
|
||||||
|
// Master-only field for the fuel gauge
|
||||||
|
"contextWindowUsage": stats?.masterInputTokens ?? 0,
|
||||||
|
}},
|
||||||
|
);
|
||||||
}
|
}
|
||||||
|
|
||||||
const codeSession = SocketService.getCodeSessionByChatSessionId(
|
const codeSession = SocketService.getCodeSessionByChatSessionId(
|
||||||
this.chatSessionId,
|
this.chatSessionId,
|
||||||
);
|
);
|
||||||
codeSession.onWorkOrderComplete(turnId, success, message);
|
codeSession.onWorkOrderComplete(turnId, success, message, enrichedStats);
|
||||||
|
|
||||||
this.currentTurnId = undefined;
|
this.currentTurnId = undefined;
|
||||||
} catch (error) {
|
} catch (error) {
|
||||||
// Routing failed - queue to Redis
|
// Routing failed - queue to Redis
|
||||||
await MessageQueue.enqueue(this.chatSessionId, {
|
await MessageQueue.enqueue(this.chatSessionId, {
|
||||||
type: 'workOrderComplete',
|
type: 'workOrderComplete',
|
||||||
args: [turnId, success, message],
|
args: [turnId, success, message, stats],
|
||||||
timestamp: Date.now(),
|
timestamp: Date.now(),
|
||||||
});
|
});
|
||||||
this.log.debug("queued workOrderComplete message", { chatSessionId: this.chatSessionId });
|
this.log.debug("queued workOrderComplete message", { chatSessionId: this.chatSessionId });
|
||||||
@ -343,6 +402,17 @@ export class DroneSession extends SocketSession {
|
|||||||
this.streamingBuffers.clear();
|
this.streamingBuffers.clear();
|
||||||
}
|
}
|
||||||
|
|
||||||
|
private formatDurationLabel(durationMs: number): string {
|
||||||
|
if (durationMs < 1000) return `${durationMs}ms`;
|
||||||
|
const seconds = Math.floor(durationMs / 1000);
|
||||||
|
const minutes = Math.floor(seconds / 60);
|
||||||
|
const secs = seconds % 60;
|
||||||
|
if (minutes > 0) {
|
||||||
|
return `${minutes}m ${secs}s`;
|
||||||
|
}
|
||||||
|
return `${seconds}.${Math.floor((durationMs % 1000) / 100)}s`;
|
||||||
|
}
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* Gets or creates a streaming buffer for a turn.
|
* Gets or creates a streaming buffer for a turn.
|
||||||
*/
|
*/
|
||||||
@ -669,7 +739,8 @@ export class DroneSession extends SocketSession {
|
|||||||
codeSession.onWorkOrderComplete(
|
codeSession.onWorkOrderComplete(
|
||||||
msg.args[0] as string,
|
msg.args[0] as string,
|
||||||
msg.args[1] as boolean,
|
msg.args[1] as boolean,
|
||||||
msg.args[2] as string,
|
msg.args[2] as string | undefined,
|
||||||
|
msg.args[3] as IWorkOrderCompleteStats | undefined,
|
||||||
);
|
);
|
||||||
break;
|
break;
|
||||||
case 'log':
|
case 'log':
|
||||||
|
|||||||
@ -35,6 +35,3 @@ export const ApiClientLog = model<IApiClientLog>(
|
|||||||
);
|
);
|
||||||
export default ApiClientLog;
|
export default ApiClientLog;
|
||||||
|
|
||||||
(async () => {
|
|
||||||
await ApiClientLog.syncIndexes();
|
|
||||||
})();
|
|
||||||
|
|||||||
@ -55,6 +55,3 @@ const ApiClientSchema = new Schema<IApiClient>({
|
|||||||
export const ApiClient = model<IApiClient>("ApiClient", ApiClientSchema);
|
export const ApiClient = model<IApiClient>("ApiClient", ApiClientSchema);
|
||||||
export default ApiClient;
|
export default ApiClient;
|
||||||
|
|
||||||
(async () => {
|
|
||||||
await ApiClient.syncIndexes();
|
|
||||||
})();
|
|
||||||
|
|||||||
@ -31,6 +31,7 @@ export const ChatSessionSchema = new Schema<IChatSession>({
|
|||||||
default: "off",
|
default: "off",
|
||||||
},
|
},
|
||||||
numCtx: { type: Number },
|
numCtx: { type: Number },
|
||||||
|
contextWindowUsage: { type: Number, default: 0, required: true },
|
||||||
stats: {
|
stats: {
|
||||||
turnCount: { type: Number, default: 0, required: true },
|
turnCount: { type: Number, default: 0, required: true },
|
||||||
toolCallCount: { type: Number, default: 0, required: true },
|
toolCallCount: { type: Number, default: 0, required: true },
|
||||||
|
|||||||
@ -41,6 +41,3 @@ const CsrfTokenSchema = new Schema<ICsrfToken>({
|
|||||||
export const CsrfToken = model<ICsrfToken>("CsrfToken", CsrfTokenSchema);
|
export const CsrfToken = model<ICsrfToken>("CsrfToken", CsrfTokenSchema);
|
||||||
export default CsrfToken;
|
export default CsrfToken;
|
||||||
|
|
||||||
(async () => {
|
|
||||||
await CsrfToken.syncIndexes();
|
|
||||||
})();
|
|
||||||
|
|||||||
@ -41,6 +41,3 @@ export const DroneMonitor = model<IDroneMonitor>(
|
|||||||
);
|
);
|
||||||
export default DroneMonitor;
|
export default DroneMonitor;
|
||||||
|
|
||||||
(async () => {
|
|
||||||
await DroneMonitor.syncIndexes();
|
|
||||||
})();
|
|
||||||
|
|||||||
@ -34,6 +34,3 @@ export const DroneRegistration = model<IDroneRegistration>(
|
|||||||
);
|
);
|
||||||
export default DroneRegistration;
|
export default DroneRegistration;
|
||||||
|
|
||||||
(async () => {
|
|
||||||
await DroneRegistration.syncIndexes();
|
|
||||||
})();
|
|
||||||
|
|||||||
@ -29,6 +29,3 @@ export const EmailLogSchema = new Schema<IEmailLog>({
|
|||||||
export const EmailLog = model<IEmailLog>("EmailLog", EmailLogSchema);
|
export const EmailLog = model<IEmailLog>("EmailLog", EmailLogSchema);
|
||||||
export default EmailLog;
|
export default EmailLog;
|
||||||
|
|
||||||
(async () => {
|
|
||||||
await EmailLog.syncIndexes();
|
|
||||||
})();
|
|
||||||
|
|||||||
@ -38,6 +38,3 @@ export const EmailVerification = model<IEmailVerification>(
|
|||||||
);
|
);
|
||||||
export default EmailVerification;
|
export default EmailVerification;
|
||||||
|
|
||||||
(async () => {
|
|
||||||
await EmailVerification.syncIndexes();
|
|
||||||
})();
|
|
||||||
|
|||||||
@ -21,6 +21,3 @@ IdeSessionSchema.index({
|
|||||||
export const IdeSession = model<IIdeSession>("IdeSession", IdeSessionSchema);
|
export const IdeSession = model<IIdeSession>("IdeSession", IdeSessionSchema);
|
||||||
export default IdeSession;
|
export default IdeSession;
|
||||||
|
|
||||||
(async () => {
|
|
||||||
await IdeSession.syncIndexes();
|
|
||||||
})();
|
|
||||||
|
|||||||
@ -31,6 +31,3 @@ export const UserSchema = new Schema<IUser>({
|
|||||||
export const User = model<IUser>("User", UserSchema);
|
export const User = model<IUser>("User", UserSchema);
|
||||||
export default User;
|
export default User;
|
||||||
|
|
||||||
(async () => {
|
|
||||||
await User.syncIndexes();
|
|
||||||
})();
|
|
||||||
|
|||||||
@ -26,6 +26,3 @@ export const WebTokenSchema = new Schema<IWebToken>({
|
|||||||
export const WebToken = model<IWebToken>("WebToken", WebTokenSchema);
|
export const WebToken = model<IWebToken>("WebToken", WebTokenSchema);
|
||||||
export default WebToken;
|
export default WebToken;
|
||||||
|
|
||||||
(async () => {
|
|
||||||
await WebToken.syncIndexes();
|
|
||||||
})();
|
|
||||||
|
|||||||
@ -43,6 +43,3 @@ export const WebVisitSchema = new Schema<IWebVisit>({
|
|||||||
});
|
});
|
||||||
export const WebVisit = model<IWebVisit>("WebVisit", WebVisitSchema);
|
export const WebVisit = model<IWebVisit>("WebVisit", WebVisitSchema);
|
||||||
|
|
||||||
(async () => {
|
|
||||||
await WebVisit.syncIndexes();
|
|
||||||
})();
|
|
||||||
|
|||||||
@ -131,6 +131,7 @@ class ChatSessionService extends DtpService {
|
|||||||
inputTokens: 0,
|
inputTokens: 0,
|
||||||
outputTokens: 0,
|
outputTokens: 0,
|
||||||
},
|
},
|
||||||
|
contextWindowUsage: 0,
|
||||||
pins: [],
|
pins: [],
|
||||||
});
|
});
|
||||||
|
|
||||||
|
|||||||
@ -13,6 +13,7 @@ import SessionService from "./session.js";
|
|||||||
import SocketService from "./socket.js";
|
import SocketService from "./socket.js";
|
||||||
import StorageService from "./storage.js";
|
import StorageService from "./storage.js";
|
||||||
import UserService from "./user.js";
|
import UserService from "./user.js";
|
||||||
|
import VectorStoreService from "./vector-store.js";
|
||||||
|
|
||||||
export async function startServices(): Promise<void> {
|
export async function startServices(): Promise<void> {
|
||||||
await ApiClientService.start();
|
await ApiClientService.start();
|
||||||
@ -26,6 +27,7 @@ export async function startServices(): Promise<void> {
|
|||||||
await SocketService.start();
|
await SocketService.start();
|
||||||
await StorageService.start();
|
await StorageService.start();
|
||||||
await UserService.start();
|
await UserService.start();
|
||||||
|
await VectorStoreService.start();
|
||||||
}
|
}
|
||||||
|
|
||||||
export async function stopServices(): Promise<void> {
|
export async function stopServices(): Promise<void> {
|
||||||
@ -40,6 +42,7 @@ export async function stopServices(): Promise<void> {
|
|||||||
await SocketService.stop();
|
await SocketService.stop();
|
||||||
await StorageService.stop();
|
await StorageService.stop();
|
||||||
await UserService.stop();
|
await UserService.stop();
|
||||||
|
await VectorStoreService.stop();
|
||||||
}
|
}
|
||||||
|
|
||||||
export {
|
export {
|
||||||
@ -54,4 +57,5 @@ export {
|
|||||||
SocketService,
|
SocketService,
|
||||||
StorageService,
|
StorageService,
|
||||||
UserService,
|
UserService,
|
||||||
|
VectorStoreService,
|
||||||
};
|
};
|
||||||
|
|||||||
495
gadget-code/src/services/vector-store.ts
Normal file
495
gadget-code/src/services/vector-store.ts
Normal file
@ -0,0 +1,495 @@
|
|||||||
|
// src/services/vector-store.ts
|
||||||
|
// Copyright (C) 2026 Robert Colbert <rob.colbert@openplatform.us>
|
||||||
|
// All Rights Reserved
|
||||||
|
|
||||||
|
import { QdrantClient } from "@qdrant/js-client-rest";
|
||||||
|
import { RecursiveCharacterTextSplitter } from "@langchain/textsplitters";
|
||||||
|
|
||||||
|
import env from "../config/env.js";
|
||||||
|
import { DtpService } from "../lib/service.js";
|
||||||
|
import { ChatTurnStatus } from "@gadget/api";
|
||||||
|
import {
|
||||||
|
createAiApi,
|
||||||
|
IAiEnvironment,
|
||||||
|
IAiProvider as IAiApiProvider,
|
||||||
|
AiApi,
|
||||||
|
} from "@gadget/ai";
|
||||||
|
import ChatTurn from "../models/chat-turn.js";
|
||||||
|
import AiProvider from "../models/ai-provider.js";
|
||||||
|
import { ChatSessionService } from "./index.js";
|
||||||
|
|
||||||
|
export interface ISearchFilters {
|
||||||
|
userId?: string;
|
||||||
|
projectId?: string;
|
||||||
|
sessionId?: string;
|
||||||
|
turnId?: string;
|
||||||
|
}
|
||||||
|
|
||||||
|
export interface ISearchResult {
|
||||||
|
id: string;
|
||||||
|
content: string;
|
||||||
|
score: number;
|
||||||
|
userId: string;
|
||||||
|
projectId: string;
|
||||||
|
sessionId: string;
|
||||||
|
turnId: string;
|
||||||
|
role: string;
|
||||||
|
createdAt: string;
|
||||||
|
}
|
||||||
|
|
||||||
|
const aiEnv: IAiEnvironment = {
|
||||||
|
NODE_ENV: env.NODE_ENV || "develop",
|
||||||
|
services: {
|
||||||
|
google: {
|
||||||
|
cse: {
|
||||||
|
apiKey: env.google.cse.apiKey,
|
||||||
|
engineId: env.google.cse.engineId,
|
||||||
|
},
|
||||||
|
},
|
||||||
|
},
|
||||||
|
};
|
||||||
|
|
||||||
|
class VectorStoreService extends DtpService {
|
||||||
|
private client!: QdrantClient;
|
||||||
|
private aiApi!: AiApi;
|
||||||
|
private splitter!: RecursiveCharacterTextSplitter;
|
||||||
|
private _initialized = false;
|
||||||
|
private _dimensionMismatch = false;
|
||||||
|
|
||||||
|
get name(): string {
|
||||||
|
return "VectorStoreService";
|
||||||
|
}
|
||||||
|
get slug(): string {
|
||||||
|
return "svc:vector-store";
|
||||||
|
}
|
||||||
|
|
||||||
|
async start(): Promise<void> {
|
||||||
|
if (!env.qdrant.providerId) {
|
||||||
|
this.log.warn(
|
||||||
|
"qdrant.providerId is not configured — vector store service will not start. " +
|
||||||
|
"Set providerId in gadget-code.yaml under qdrant to enable semantic search.",
|
||||||
|
);
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
this.log.info("initializing Qdrant client", {
|
||||||
|
host: env.qdrant.host,
|
||||||
|
port: env.qdrant.port,
|
||||||
|
collection: env.qdrant.collection,
|
||||||
|
providerId: env.qdrant.providerId,
|
||||||
|
embeddingModel: env.qdrant.embeddingModel,
|
||||||
|
vectorSize: env.qdrant.vectorSize,
|
||||||
|
});
|
||||||
|
|
||||||
|
// Initialize Qdrant client
|
||||||
|
this.client = new QdrantClient({
|
||||||
|
url: `http://${env.qdrant.host}:${env.qdrant.port}`,
|
||||||
|
apiKey: env.qdrant.apiKey || undefined,
|
||||||
|
});
|
||||||
|
|
||||||
|
// Load the configured AI provider for embeddings
|
||||||
|
const providerDoc = await AiProvider.findById(env.qdrant.providerId);
|
||||||
|
if (!providerDoc) {
|
||||||
|
this.log.error(
|
||||||
|
`Qdrant providerId "${env.qdrant.providerId}" not found in database — ` +
|
||||||
|
"vector store service will not start.",
|
||||||
|
);
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
const aiProvider: IAiApiProvider = {
|
||||||
|
_id: providerDoc._id,
|
||||||
|
name: providerDoc.name,
|
||||||
|
sdk: providerDoc.apiType, // map apiType → sdk
|
||||||
|
baseUrl: providerDoc.baseUrl,
|
||||||
|
apiKey: providerDoc.apiKey,
|
||||||
|
};
|
||||||
|
this.aiApi = createAiApi(aiEnv, aiProvider, this.log);
|
||||||
|
|
||||||
|
// Initialize text splitter
|
||||||
|
this.splitter = new RecursiveCharacterTextSplitter({
|
||||||
|
chunkSize: 1000,
|
||||||
|
chunkOverlap: 200,
|
||||||
|
});
|
||||||
|
|
||||||
|
// Ensure the Qdrant collection exists and validate dimensions
|
||||||
|
await this.ensureCollection();
|
||||||
|
|
||||||
|
// Validate that the embedding model produces vectors of the configured size
|
||||||
|
await this.validateEmbeddingDimensions();
|
||||||
|
|
||||||
|
if (this._dimensionMismatch) {
|
||||||
|
this.log.error(
|
||||||
|
"VectorStoreService started with DIMENSION MISMATCH — searches and ingestion will fail. " +
|
||||||
|
"See previous error logs for fix instructions.",
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
this._initialized = true;
|
||||||
|
this.log.info("started", {
|
||||||
|
host: env.qdrant.host,
|
||||||
|
port: env.qdrant.port,
|
||||||
|
collection: env.qdrant.collection,
|
||||||
|
providerId: env.qdrant.providerId,
|
||||||
|
embeddingModel: env.qdrant.embeddingModel,
|
||||||
|
vectorSize: env.qdrant.vectorSize,
|
||||||
|
dimensionMismatch: this._dimensionMismatch,
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
|
async stop(): Promise<void> {
|
||||||
|
this._initialized = false;
|
||||||
|
this.log.info("stopped");
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Check if the service is initialized and ready.
|
||||||
|
*/
|
||||||
|
get isReady(): boolean {
|
||||||
|
return this._initialized;
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Create the Qdrant collection if it doesn't already exist.
|
||||||
|
* Validates existing collection dimensions against the configured vectorSize.
|
||||||
|
*/
|
||||||
|
private async ensureCollection(): Promise<void> {
|
||||||
|
const collections = await this.client.getCollections();
|
||||||
|
const exists = collections.collections.some(
|
||||||
|
(c) => c.name === env.qdrant.collection,
|
||||||
|
);
|
||||||
|
|
||||||
|
if (!exists) {
|
||||||
|
await this.client.createCollection(env.qdrant.collection, {
|
||||||
|
vectors: {
|
||||||
|
size: env.qdrant.vectorSize,
|
||||||
|
distance: "Cosine" as const,
|
||||||
|
},
|
||||||
|
});
|
||||||
|
this.log.info(`created Qdrant collection "${env.qdrant.collection}"`, {
|
||||||
|
vectorSize: env.qdrant.vectorSize,
|
||||||
|
distance: "Cosine",
|
||||||
|
});
|
||||||
|
} else {
|
||||||
|
// Validate existing collection dimensions against config
|
||||||
|
try {
|
||||||
|
const collectionInfo = await this.client.getCollection(
|
||||||
|
env.qdrant.collection,
|
||||||
|
);
|
||||||
|
const vectorsConfig = collectionInfo.config?.params?.vectors;
|
||||||
|
// Handle both named and unnamed vector configurations
|
||||||
|
// Unnamed: { size: N, distance: "Cosine" } — Named: { "default": { size: N, distance: "Cosine" } }
|
||||||
|
const actualSize =
|
||||||
|
(vectorsConfig as Record<string, any>)?.size ??
|
||||||
|
(
|
||||||
|
Object.values(
|
||||||
|
(vectorsConfig as Record<string, any>) || {},
|
||||||
|
)[0] as Record<string, any>
|
||||||
|
)?.size;
|
||||||
|
|
||||||
|
if (actualSize && actualSize !== env.qdrant.vectorSize) {
|
||||||
|
this._dimensionMismatch = true;
|
||||||
|
this.log.error(
|
||||||
|
"QDRANT COLLECTION DIMENSION MISMATCH — searches and ingestion will fail!",
|
||||||
|
{
|
||||||
|
collectionName: env.qdrant.collection,
|
||||||
|
configuredVectorSize: env.qdrant.vectorSize,
|
||||||
|
actualCollectionVectorSize: actualSize,
|
||||||
|
fix: `Either (1) delete the collection and restart to recreate it, or (2) update qdrant.vectorSize in your config to ${actualSize}`,
|
||||||
|
},
|
||||||
|
);
|
||||||
|
} else {
|
||||||
|
this.log.info(
|
||||||
|
`Qdrant collection "${env.qdrant.collection}" already exists`,
|
||||||
|
{
|
||||||
|
vectorSize: actualSize || env.qdrant.vectorSize,
|
||||||
|
},
|
||||||
|
);
|
||||||
|
}
|
||||||
|
} catch (err) {
|
||||||
|
this.log.warn("Could not validate Qdrant collection dimensions", {
|
||||||
|
error: (err as Error).message,
|
||||||
|
});
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Validate that the embedding model produces vectors matching the configured vectorSize.
|
||||||
|
* Sends a test embedding and compares its length against env.qdrant.vectorSize.
|
||||||
|
*/
|
||||||
|
private async validateEmbeddingDimensions(): Promise<void> {
|
||||||
|
try {
|
||||||
|
const testEmbedding = await this.getEmbedding("test");
|
||||||
|
if (testEmbedding.length !== env.qdrant.vectorSize) {
|
||||||
|
this._dimensionMismatch = true;
|
||||||
|
this.log.error(
|
||||||
|
"EMBEDDING MODEL DIMENSION MISMATCH — the configured vectorSize does not match the model output!",
|
||||||
|
{
|
||||||
|
configuredVectorSize: env.qdrant.vectorSize,
|
||||||
|
actualModelDimensions: testEmbedding.length,
|
||||||
|
embeddingModel: env.qdrant.embeddingModel,
|
||||||
|
fix: `Update qdrant.vectorSize in your config to ${testEmbedding.length}`,
|
||||||
|
},
|
||||||
|
);
|
||||||
|
} else {
|
||||||
|
this.log.info("embedding dimension validation passed", {
|
||||||
|
vectorSize: testEmbedding.length,
|
||||||
|
model: env.qdrant.embeddingModel,
|
||||||
|
});
|
||||||
|
}
|
||||||
|
} catch (err) {
|
||||||
|
this.log.warn("Could not validate embedding dimensions at startup", {
|
||||||
|
error: (err as Error).message,
|
||||||
|
});
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Generate an embedding vector for the given text.
|
||||||
|
*/
|
||||||
|
private async getEmbedding(text: string): Promise<number[]> {
|
||||||
|
const response = await this.aiApi.embeddings(
|
||||||
|
env.qdrant.embeddingModel,
|
||||||
|
text,
|
||||||
|
env.qdrant.vectorSize,
|
||||||
|
);
|
||||||
|
return response.embedding;
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Ingest a chat turn into the vector store.
|
||||||
|
* Fetches the turn by ID, extracts content, chunks, embeds, and upserts to Qdrant.
|
||||||
|
* Fire-and-forget safe — logs errors but does not throw.
|
||||||
|
*/
|
||||||
|
async ingestTurn(turnId: string): Promise<void> {
|
||||||
|
if (!this._initialized) {
|
||||||
|
this.log.warn(
|
||||||
|
"ingestTurn called but service is not initialized — skipping",
|
||||||
|
{ turnId },
|
||||||
|
);
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
if (this._dimensionMismatch) {
|
||||||
|
this.log.error(
|
||||||
|
"ingestTurn skipped — vector dimension mismatch detected at startup. Fix config and restart.",
|
||||||
|
{
|
||||||
|
turnId,
|
||||||
|
configuredVectorSize: env.qdrant.vectorSize,
|
||||||
|
},
|
||||||
|
);
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
try {
|
||||||
|
// Fetch and populate the turn
|
||||||
|
let turn = await ChatTurn.findById(turnId);
|
||||||
|
if (!turn) {
|
||||||
|
this.log.warn("ingestTurn: turn not found", { turnId });
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
// Only ingest finished turns
|
||||||
|
if (turn.status !== ChatTurnStatus.Finished) {
|
||||||
|
this.log.debug("ingestTurn: skipping non-finished turn", {
|
||||||
|
turnId,
|
||||||
|
status: turn.status,
|
||||||
|
});
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
// Populate references for metadata
|
||||||
|
turn = await ChatTurn.populate(turn, ChatSessionService.populateChatTurn);
|
||||||
|
|
||||||
|
// Extract content
|
||||||
|
const userPrompt = turn.prompts.user || "";
|
||||||
|
|
||||||
|
// Extract agent response: concatenate all 'responding' mode blocks
|
||||||
|
let agentResponse = "";
|
||||||
|
for (const block of turn.blocks) {
|
||||||
|
if (block.mode === "responding" && typeof block.content === "string") {
|
||||||
|
agentResponse += block.content + "\n";
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Combine user + agent text for chunking
|
||||||
|
const combinedText = [
|
||||||
|
userPrompt ? `User: ${userPrompt}` : "",
|
||||||
|
agentResponse ? `Agent: ${agentResponse.trim()}` : "",
|
||||||
|
]
|
||||||
|
.filter(Boolean)
|
||||||
|
.join("\n\n");
|
||||||
|
|
||||||
|
if (!combinedText.trim()) {
|
||||||
|
this.log.debug("ingestTurn: no content to ingest", { turnId });
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
// Chunk the text
|
||||||
|
const chunks = await this.splitter.splitText(combinedText);
|
||||||
|
|
||||||
|
// Determine role for each chunk based on content
|
||||||
|
const points = [];
|
||||||
|
for (let i = 0; i < chunks.length; i++) {
|
||||||
|
const chunk = chunks[i]!;
|
||||||
|
let role: string;
|
||||||
|
if (chunk.startsWith("User:") && !chunk.includes("Agent:")) {
|
||||||
|
role = "user";
|
||||||
|
} else if (chunk.startsWith("Agent:") && !chunk.includes("User:")) {
|
||||||
|
role = "agent";
|
||||||
|
} else {
|
||||||
|
role = "both";
|
||||||
|
}
|
||||||
|
|
||||||
|
const embedding = await this.getEmbedding(chunk);
|
||||||
|
|
||||||
|
// Validate embedding dimensions before upsert
|
||||||
|
if (embedding.length !== env.qdrant.vectorSize) {
|
||||||
|
this.log.error(
|
||||||
|
"embedding dimension mismatch during ingest — skipping chunk",
|
||||||
|
{
|
||||||
|
expected: env.qdrant.vectorSize,
|
||||||
|
actual: embedding.length,
|
||||||
|
turnId,
|
||||||
|
chunkIndex: i,
|
||||||
|
fix: `Update qdrant.vectorSize in your config to ${embedding.length}`,
|
||||||
|
},
|
||||||
|
);
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
|
||||||
|
points.push({
|
||||||
|
id: `${turnId}:${i}`,
|
||||||
|
vector: embedding,
|
||||||
|
payload: {
|
||||||
|
content: chunk,
|
||||||
|
userId: String(turn.user),
|
||||||
|
projectId: turn.project ? String(turn.project) : "",
|
||||||
|
sessionId: String(turn.session),
|
||||||
|
turnId: turn._id,
|
||||||
|
role,
|
||||||
|
createdAt: turn.createdAt.toISOString(),
|
||||||
|
},
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
|
// Upsert all points
|
||||||
|
await this.client.upsert(env.qdrant.collection, {
|
||||||
|
wait: false,
|
||||||
|
points,
|
||||||
|
});
|
||||||
|
|
||||||
|
this.log.info("ingested turn to vector store", {
|
||||||
|
turnId,
|
||||||
|
chunkCount: points.length,
|
||||||
|
});
|
||||||
|
} catch (error) {
|
||||||
|
this.log.error("ingestTurn failed", {
|
||||||
|
turnId,
|
||||||
|
error,
|
||||||
|
});
|
||||||
|
// Do not rethrow — this is designed to be fire-and-forget
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Search the vector store for relevant chunks.
|
||||||
|
*/
|
||||||
|
async search(
|
||||||
|
query: string,
|
||||||
|
filters?: ISearchFilters,
|
||||||
|
topK: number = 10,
|
||||||
|
): Promise<ISearchResult[]> {
|
||||||
|
if (!this._initialized) {
|
||||||
|
throw new Error("VectorStoreService is not initialized");
|
||||||
|
}
|
||||||
|
|
||||||
|
if (this._dimensionMismatch) {
|
||||||
|
throw new Error(
|
||||||
|
`Vector dimension mismatch: the Qdrant collection dimensions do not match the ` +
|
||||||
|
`configured vectorSize (${env.qdrant.vectorSize}). Either delete the collection ` +
|
||||||
|
`and restart, or update qdrant.vectorSize in your config.`,
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
const queryVector = await this.getEmbedding(query);
|
||||||
|
|
||||||
|
// Validate embedding dimensions before searching
|
||||||
|
if (queryVector.length !== env.qdrant.vectorSize) {
|
||||||
|
throw new Error(
|
||||||
|
`Embedding dimension mismatch: model produced ${queryVector.length} dimensions, ` +
|
||||||
|
`but collection expects ${env.qdrant.vectorSize}. ` +
|
||||||
|
`Update qdrant.vectorSize in your config to ${queryVector.length}.`,
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
// Build Qdrant filter from provided filters
|
||||||
|
const must: Array<{
|
||||||
|
key: string;
|
||||||
|
match: { value: string };
|
||||||
|
}> = [];
|
||||||
|
|
||||||
|
if (filters?.userId) {
|
||||||
|
must.push({ key: "userId", match: { value: filters.userId } });
|
||||||
|
}
|
||||||
|
if (filters?.projectId) {
|
||||||
|
must.push({ key: "projectId", match: { value: filters.projectId } });
|
||||||
|
}
|
||||||
|
if (filters?.sessionId) {
|
||||||
|
must.push({ key: "sessionId", match: { value: filters.sessionId } });
|
||||||
|
}
|
||||||
|
if (filters?.turnId) {
|
||||||
|
must.push({ key: "turnId", match: { value: filters.turnId } });
|
||||||
|
}
|
||||||
|
|
||||||
|
const searchResults = await this.client.search(env.qdrant.collection, {
|
||||||
|
vector: queryVector,
|
||||||
|
limit: topK,
|
||||||
|
filter: must.length > 0 ? { must } : undefined,
|
||||||
|
with_payload: true,
|
||||||
|
});
|
||||||
|
|
||||||
|
return searchResults.map((point) => ({
|
||||||
|
id: point.id as string,
|
||||||
|
content: (point.payload as any)?.content || "",
|
||||||
|
score: point.score ?? 0,
|
||||||
|
userId: (point.payload as any)?.userId || "",
|
||||||
|
projectId: (point.payload as any)?.projectId || "",
|
||||||
|
sessionId: (point.payload as any)?.sessionId || "",
|
||||||
|
turnId: (point.payload as any)?.turnId || "",
|
||||||
|
role: (point.payload as any)?.role || "",
|
||||||
|
createdAt: (point.payload as any)?.createdAt || "",
|
||||||
|
}));
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Remove all vector points for a given turn.
|
||||||
|
* Used for cleanup when a turn is deleted.
|
||||||
|
*/
|
||||||
|
async removeTurnPoints(turnId: string): Promise<void> {
|
||||||
|
if (!this._initialized) {
|
||||||
|
this.log.warn(
|
||||||
|
"removeTurnPoints called but service is not initialized — skipping",
|
||||||
|
{ turnId },
|
||||||
|
);
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
try {
|
||||||
|
await this.client.delete(env.qdrant.collection, {
|
||||||
|
filter: {
|
||||||
|
must: [{ key: "turnId", match: { value: turnId } }],
|
||||||
|
},
|
||||||
|
});
|
||||||
|
this.log.info("removed vector points for turn", { turnId });
|
||||||
|
} catch (error) {
|
||||||
|
this.log.error("removeTurnPoints failed", {
|
||||||
|
turnId,
|
||||||
|
error,
|
||||||
|
});
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
export default new VectorStoreService();
|
||||||
@ -1,5 +1,4 @@
|
|||||||
import { describe, it, expect, beforeEach, vi } from "vitest";
|
import { describe, it, expect, beforeEach, vi } from "vitest";
|
||||||
import { Types } from "mongoose";
|
|
||||||
import { DroneSession } from "../src/lib/drone-session";
|
import { DroneSession } from "../src/lib/drone-session";
|
||||||
import {
|
import {
|
||||||
IDroneRegistration,
|
IDroneRegistration,
|
||||||
@ -14,6 +13,27 @@ import { nanoid } from "nanoid";
|
|||||||
// Mock dependencies
|
// Mock dependencies
|
||||||
vi.mock("../src/services/socket");
|
vi.mock("../src/services/socket");
|
||||||
vi.mock("../src/models/chat-turn");
|
vi.mock("../src/models/chat-turn");
|
||||||
|
vi.mock("../src/models/chat-session", () => ({
|
||||||
|
default: {
|
||||||
|
updateOne: vi.fn().mockResolvedValue({ acknowledged: true, modifiedCount: 1 }),
|
||||||
|
},
|
||||||
|
}));
|
||||||
|
vi.mock("../src/services/index.js", async (importOriginal) => {
|
||||||
|
const actual = await importOriginal() as any;
|
||||||
|
return {
|
||||||
|
...actual,
|
||||||
|
SocketService: actual.SocketService,
|
||||||
|
VectorStoreService: {
|
||||||
|
ingestTurn: vi.fn().mockResolvedValue(undefined),
|
||||||
|
},
|
||||||
|
};
|
||||||
|
});
|
||||||
|
vi.mock("../src/lib/message-queue.js", () => ({
|
||||||
|
default: {
|
||||||
|
enqueue: vi.fn().mockResolvedValue(undefined),
|
||||||
|
drain: vi.fn().mockResolvedValue([]),
|
||||||
|
},
|
||||||
|
}));
|
||||||
|
|
||||||
describe("DroneSession", () => {
|
describe("DroneSession", () => {
|
||||||
let mockSocket: any;
|
let mockSocket: any;
|
||||||
@ -220,6 +240,7 @@ describe("DroneSession", () => {
|
|||||||
};
|
};
|
||||||
const mockTurn = {
|
const mockTurn = {
|
||||||
status: ChatTurnStatus.Processing,
|
status: ChatTurnStatus.Processing,
|
||||||
|
toolCalls: [],
|
||||||
save: vi.fn().mockResolvedValue(undefined),
|
save: vi.fn().mockResolvedValue(undefined),
|
||||||
};
|
};
|
||||||
vi.mocked(SocketService.getCodeSessionByChatSessionId).mockReturnValue(
|
vi.mocked(SocketService.getCodeSessionByChatSessionId).mockReturnValue(
|
||||||
@ -238,6 +259,7 @@ describe("DroneSession", () => {
|
|||||||
mockTurnId,
|
mockTurnId,
|
||||||
true,
|
true,
|
||||||
undefined,
|
undefined,
|
||||||
|
undefined,
|
||||||
);
|
);
|
||||||
expect(droneSession.currentTurnId).toBeUndefined();
|
expect(droneSession.currentTurnId).toBeUndefined();
|
||||||
});
|
});
|
||||||
@ -250,6 +272,7 @@ describe("DroneSession", () => {
|
|||||||
const mockTurn = {
|
const mockTurn = {
|
||||||
status: ChatTurnStatus.Processing,
|
status: ChatTurnStatus.Processing,
|
||||||
errorMessage: "",
|
errorMessage: "",
|
||||||
|
toolCalls: [],
|
||||||
save: vi.fn().mockResolvedValue(undefined),
|
save: vi.fn().mockResolvedValue(undefined),
|
||||||
};
|
};
|
||||||
vi.mocked(SocketService.getCodeSessionByChatSessionId).mockReturnValue(
|
vi.mocked(SocketService.getCodeSessionByChatSessionId).mockReturnValue(
|
||||||
|
|||||||
320
gadget-code/tests/vector-store.test.ts
Normal file
320
gadget-code/tests/vector-store.test.ts
Normal file
@ -0,0 +1,320 @@
|
|||||||
|
import { describe, it, expect, vi, beforeEach } from "vitest";
|
||||||
|
|
||||||
|
const mockEmbeddings = vi.hoisted(() => vi.fn());
|
||||||
|
const mockFindById = vi.hoisted(() =>
|
||||||
|
vi.fn().mockResolvedValue({
|
||||||
|
_id: "test-provider-id",
|
||||||
|
name: "Test Provider",
|
||||||
|
apiType: "ollama",
|
||||||
|
baseUrl: "http://test:11434",
|
||||||
|
apiKey: "",
|
||||||
|
}),
|
||||||
|
);
|
||||||
|
|
||||||
|
// Create mock methods shared between QdrantClient mock and test code
|
||||||
|
const mockQdrantMethods = vi.hoisted(() => ({
|
||||||
|
getCollections: vi.fn(),
|
||||||
|
createCollection: vi.fn(),
|
||||||
|
getCollection: vi.fn(),
|
||||||
|
search: vi.fn(),
|
||||||
|
upsert: vi.fn(),
|
||||||
|
delete: vi.fn(),
|
||||||
|
}));
|
||||||
|
|
||||||
|
vi.mock("../src/config/env.js", () => ({
|
||||||
|
default: {
|
||||||
|
NODE_ENV: "test",
|
||||||
|
INSTALL_DIR: "/tmp",
|
||||||
|
timezone: "UTC",
|
||||||
|
pkg: { name: "test", version: "0.0.0" },
|
||||||
|
site: {},
|
||||||
|
ai: { ollama: { apiUrl: "http://test:11434", apiKey: "" } },
|
||||||
|
auth: { jwtSecret: "test-secret" },
|
||||||
|
session: {
|
||||||
|
secret: "test-secret",
|
||||||
|
trustProxy: false,
|
||||||
|
cookie: { secure: false, sameSite: false },
|
||||||
|
},
|
||||||
|
google: { cse: { apiKey: "", engineId: "" } },
|
||||||
|
mongodb: { host: "localhost:27017", database: "test" },
|
||||||
|
qdrant: {
|
||||||
|
host: "localhost",
|
||||||
|
port: 6333,
|
||||||
|
collection: "test-collection",
|
||||||
|
providerId: "test-provider-id",
|
||||||
|
embeddingModel: "test-model",
|
||||||
|
vectorSize: 1024,
|
||||||
|
},
|
||||||
|
redis: {
|
||||||
|
host: "localhost",
|
||||||
|
port: 6379,
|
||||||
|
password: "",
|
||||||
|
keyPrefix: "test:",
|
||||||
|
lazyConnect: true,
|
||||||
|
},
|
||||||
|
minio: {
|
||||||
|
endpoint: "localhost",
|
||||||
|
port: 9000,
|
||||||
|
useSsl: false,
|
||||||
|
accessKey: "",
|
||||||
|
secretKey: "",
|
||||||
|
buckets: { uploads: "", images: "", videos: "", audios: "" },
|
||||||
|
},
|
||||||
|
user: { passwordSalt: "test-salt" },
|
||||||
|
https: { enabled: false, address: "localhost", port: 3443 },
|
||||||
|
socket: { maxHttpBufferSize: 1048576 },
|
||||||
|
frontend: { port: 5173 },
|
||||||
|
email: { enabled: false, smtp: {}, contact: {} },
|
||||||
|
log: {
|
||||||
|
https: { enabled: false },
|
||||||
|
console: { enabled: false },
|
||||||
|
file: { enabled: false },
|
||||||
|
},
|
||||||
|
},
|
||||||
|
}));
|
||||||
|
|
||||||
|
vi.mock("../src/models/ai-provider.js", () => ({
|
||||||
|
default: { findById: mockFindById },
|
||||||
|
}));
|
||||||
|
|
||||||
|
vi.mock("@gadget/ai", () => ({
|
||||||
|
createAiApi: vi.fn().mockReturnValue({
|
||||||
|
embeddings: mockEmbeddings,
|
||||||
|
}),
|
||||||
|
}));
|
||||||
|
|
||||||
|
vi.mock("@langchain/textsplitters", () => ({
|
||||||
|
RecursiveCharacterTextSplitter: vi.fn(function () {
|
||||||
|
return {
|
||||||
|
splitText: vi.fn().mockResolvedValue(["chunk1"]),
|
||||||
|
};
|
||||||
|
}),
|
||||||
|
}));
|
||||||
|
|
||||||
|
// QdrantClient mock constructor that returns the shared mock methods
|
||||||
|
vi.mock("@qdrant/js-client-rest", () => ({
|
||||||
|
QdrantClient: vi.fn(function () {
|
||||||
|
return mockQdrantMethods;
|
||||||
|
}),
|
||||||
|
}));
|
||||||
|
|
||||||
|
import VectorStoreService from "../src/services/vector-store.js";
|
||||||
|
|
||||||
|
const svc = VectorStoreService as unknown as {
|
||||||
|
_initialized: boolean;
|
||||||
|
_dimensionMismatch: boolean;
|
||||||
|
client: typeof mockQdrantMethods;
|
||||||
|
aiApi: {
|
||||||
|
embeddings: ReturnType<typeof vi.fn>;
|
||||||
|
};
|
||||||
|
};
|
||||||
|
|
||||||
|
function setMockDefaults() {
|
||||||
|
mockQdrantMethods.getCollections.mockResolvedValue({ collections: [] });
|
||||||
|
mockQdrantMethods.createCollection.mockResolvedValue(undefined);
|
||||||
|
mockQdrantMethods.getCollection.mockResolvedValue({});
|
||||||
|
mockQdrantMethods.search.mockResolvedValue([]);
|
||||||
|
mockQdrantMethods.upsert.mockResolvedValue(undefined);
|
||||||
|
mockQdrantMethods.delete.mockResolvedValue(undefined);
|
||||||
|
}
|
||||||
|
|
||||||
|
function ensureInstanceSetup() {
|
||||||
|
// For tests that don't call start(), manually assign client/aiApi
|
||||||
|
// since start() normally does this via new QdrantClient() + createAiApi()
|
||||||
|
if (!svc.client) {
|
||||||
|
(svc as Record<string, unknown>).client = mockQdrantMethods;
|
||||||
|
}
|
||||||
|
if (!svc.aiApi) {
|
||||||
|
(svc as Record<string, unknown>).aiApi = { embeddings: mockEmbeddings };
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
describe("VectorStoreService", () => {
|
||||||
|
beforeEach(() => {
|
||||||
|
vi.clearAllMocks();
|
||||||
|
setMockDefaults();
|
||||||
|
|
||||||
|
// Reset internal state on the instance
|
||||||
|
svc._initialized = false;
|
||||||
|
svc._dimensionMismatch = false;
|
||||||
|
});
|
||||||
|
|
||||||
|
describe("search", () => {
|
||||||
|
beforeEach(() => {
|
||||||
|
ensureInstanceSetup();
|
||||||
|
});
|
||||||
|
|
||||||
|
it("throws when service is not initialized", async () => {
|
||||||
|
svc._initialized = false;
|
||||||
|
svc._dimensionMismatch = false;
|
||||||
|
|
||||||
|
await expect(VectorStoreService.search("test query")).rejects.toThrow(
|
||||||
|
"VectorStoreService is not initialized",
|
||||||
|
);
|
||||||
|
});
|
||||||
|
|
||||||
|
it("throws when dimension mismatch flag is set", async () => {
|
||||||
|
svc._initialized = true;
|
||||||
|
svc._dimensionMismatch = true;
|
||||||
|
|
||||||
|
await expect(VectorStoreService.search("test query")).rejects.toThrow(
|
||||||
|
"Vector dimension mismatch: the Qdrant collection dimensions do not match the configured vectorSize (1024)",
|
||||||
|
);
|
||||||
|
});
|
||||||
|
|
||||||
|
it("throws when query embedding dimensions mismatch config", async () => {
|
||||||
|
svc._initialized = true;
|
||||||
|
svc._dimensionMismatch = false;
|
||||||
|
|
||||||
|
mockEmbeddings.mockResolvedValue({
|
||||||
|
embedding: new Array(512).fill(0.1),
|
||||||
|
model: "test-model",
|
||||||
|
});
|
||||||
|
|
||||||
|
await expect(VectorStoreService.search("test query")).rejects.toThrow(
|
||||||
|
"Embedding dimension mismatch: model produced 512 dimensions, but collection expects 1024",
|
||||||
|
);
|
||||||
|
});
|
||||||
|
|
||||||
|
it("passes vectorSize dimensions to the AI API when embedding", async () => {
|
||||||
|
svc._initialized = true;
|
||||||
|
svc._dimensionMismatch = false;
|
||||||
|
|
||||||
|
mockQdrantMethods.search.mockResolvedValue([]);
|
||||||
|
mockEmbeddings.mockResolvedValue({
|
||||||
|
embedding: new Array(1024).fill(0.1),
|
||||||
|
model: "test-model",
|
||||||
|
});
|
||||||
|
|
||||||
|
await VectorStoreService.search("test query");
|
||||||
|
|
||||||
|
expect(mockEmbeddings).toHaveBeenCalledWith(
|
||||||
|
"test-model",
|
||||||
|
"test query",
|
||||||
|
1024,
|
||||||
|
);
|
||||||
|
});
|
||||||
|
|
||||||
|
it("returns hydrated search results", async () => {
|
||||||
|
svc._initialized = true;
|
||||||
|
svc._dimensionMismatch = false;
|
||||||
|
|
||||||
|
mockEmbeddings.mockResolvedValue({
|
||||||
|
embedding: new Array(1024).fill(0.1),
|
||||||
|
model: "test-model",
|
||||||
|
});
|
||||||
|
|
||||||
|
mockQdrantMethods.search.mockResolvedValue([
|
||||||
|
{
|
||||||
|
id: "point-1",
|
||||||
|
score: 0.95,
|
||||||
|
payload: {
|
||||||
|
content: "some content",
|
||||||
|
userId: "user-1",
|
||||||
|
projectId: "proj-1",
|
||||||
|
sessionId: "session-1",
|
||||||
|
turnId: "turn-1",
|
||||||
|
role: "user",
|
||||||
|
createdAt: "2026-01-01T00:00:00.000Z",
|
||||||
|
},
|
||||||
|
},
|
||||||
|
]);
|
||||||
|
|
||||||
|
const results = await VectorStoreService.search("test query", undefined, 5);
|
||||||
|
expect(results).toHaveLength(1);
|
||||||
|
expect(results[0]).toMatchObject({
|
||||||
|
id: "point-1",
|
||||||
|
content: "some content",
|
||||||
|
score: 0.95,
|
||||||
|
userId: "user-1",
|
||||||
|
});
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
describe("start / ensureCollection", () => {
|
||||||
|
it("creates collection when it does not exist", async () => {
|
||||||
|
mockQdrantMethods.getCollections.mockResolvedValue({
|
||||||
|
collections: [],
|
||||||
|
});
|
||||||
|
mockQdrantMethods.createCollection.mockResolvedValue(undefined);
|
||||||
|
mockEmbeddings.mockResolvedValue({
|
||||||
|
embedding: new Array(1024).fill(0.1),
|
||||||
|
model: "test-model",
|
||||||
|
});
|
||||||
|
|
||||||
|
await VectorStoreService.start();
|
||||||
|
|
||||||
|
expect(mockQdrantMethods.createCollection).toHaveBeenCalledWith(
|
||||||
|
"test-collection",
|
||||||
|
{ vectors: { size: 1024, distance: "Cosine" } },
|
||||||
|
);
|
||||||
|
expect(svc._initialized).toBe(true);
|
||||||
|
expect(svc._dimensionMismatch).toBe(false);
|
||||||
|
});
|
||||||
|
|
||||||
|
it("detects dimension mismatch when existing collection has wrong size", async () => {
|
||||||
|
mockQdrantMethods.getCollections.mockResolvedValue({
|
||||||
|
collections: [{ name: "test-collection" }],
|
||||||
|
});
|
||||||
|
mockQdrantMethods.getCollection.mockResolvedValue({
|
||||||
|
config: {
|
||||||
|
params: {
|
||||||
|
vectors: { size: 768, distance: "Cosine" },
|
||||||
|
},
|
||||||
|
},
|
||||||
|
});
|
||||||
|
mockEmbeddings.mockResolvedValue({
|
||||||
|
embedding: new Array(1024).fill(0.1),
|
||||||
|
model: "test-model",
|
||||||
|
});
|
||||||
|
|
||||||
|
await VectorStoreService.start();
|
||||||
|
|
||||||
|
expect(svc._initialized).toBe(true);
|
||||||
|
expect(svc._dimensionMismatch).toBe(true);
|
||||||
|
});
|
||||||
|
|
||||||
|
it("sets dimension mismatch when embedding model produces wrong dimensions", async () => {
|
||||||
|
mockQdrantMethods.getCollections.mockResolvedValue({
|
||||||
|
collections: [],
|
||||||
|
});
|
||||||
|
mockQdrantMethods.createCollection.mockResolvedValue(undefined);
|
||||||
|
mockEmbeddings.mockResolvedValue({
|
||||||
|
embedding: new Array(512).fill(0.1),
|
||||||
|
model: "test-model",
|
||||||
|
});
|
||||||
|
|
||||||
|
await VectorStoreService.start();
|
||||||
|
|
||||||
|
expect(svc._initialized).toBe(true);
|
||||||
|
expect(svc._dimensionMismatch).toBe(true);
|
||||||
|
});
|
||||||
|
|
||||||
|
it("starts cleanly when everything matches", async () => {
|
||||||
|
mockQdrantMethods.getCollections.mockResolvedValue({
|
||||||
|
collections: [],
|
||||||
|
});
|
||||||
|
mockQdrantMethods.createCollection.mockResolvedValue(undefined);
|
||||||
|
mockEmbeddings.mockResolvedValue({
|
||||||
|
embedding: new Array(1024).fill(0.1),
|
||||||
|
model: "test-model",
|
||||||
|
});
|
||||||
|
|
||||||
|
await VectorStoreService.start();
|
||||||
|
|
||||||
|
expect(svc._dimensionMismatch).toBe(false);
|
||||||
|
expect(svc._initialized).toBe(true);
|
||||||
|
});
|
||||||
|
|
||||||
|
it("skips start when no providerId is configured", async () => {
|
||||||
|
const env = await import("../src/config/env.js");
|
||||||
|
const originalProviderId = env.default.qdrant.providerId;
|
||||||
|
env.default.qdrant.providerId = "";
|
||||||
|
|
||||||
|
await VectorStoreService.start();
|
||||||
|
expect(svc._initialized).toBe(false);
|
||||||
|
|
||||||
|
env.default.qdrant.providerId = originalProviderId;
|
||||||
|
});
|
||||||
|
});
|
||||||
|
});
|
||||||
@ -39,6 +39,7 @@ describe("AgentService", () => {
|
|||||||
inputTokens: 0,
|
inputTokens: 0,
|
||||||
outputTokens: 0,
|
outputTokens: 0,
|
||||||
},
|
},
|
||||||
|
contextWindowUsage: 0,
|
||||||
pins: [],
|
pins: [],
|
||||||
};
|
};
|
||||||
const workOrder: IAgentWorkOrder = {
|
const workOrder: IAgentWorkOrder = {
|
||||||
@ -150,6 +151,7 @@ describe("AgentService", () => {
|
|||||||
inputTokens: 0,
|
inputTokens: 0,
|
||||||
outputTokens: 0,
|
outputTokens: 0,
|
||||||
},
|
},
|
||||||
|
contextWindowUsage: 0,
|
||||||
pins: [],
|
pins: [],
|
||||||
};
|
};
|
||||||
|
|
||||||
@ -314,6 +316,7 @@ describe("AgentService", () => {
|
|||||||
inputTokens: 0,
|
inputTokens: 0,
|
||||||
outputTokens: 0,
|
outputTokens: 0,
|
||||||
},
|
},
|
||||||
|
contextWindowUsage: 0,
|
||||||
pins: [],
|
pins: [],
|
||||||
};
|
};
|
||||||
const rawResponse = "Subagent completed the task successfully.";
|
const rawResponse = "Subagent completed the task successfully.";
|
||||||
|
|||||||
@ -28,7 +28,9 @@ import {
|
|||||||
type ServerToClientEvents,
|
type ServerToClientEvents,
|
||||||
type ClientToServerEvents,
|
type ClientToServerEvents,
|
||||||
ChatSessionMode,
|
ChatSessionMode,
|
||||||
|
ChatTurnStatus,
|
||||||
type IProject,
|
type IProject,
|
||||||
|
type IWorkOrderCompleteStats,
|
||||||
} from "@gadget/api";
|
} from "@gadget/api";
|
||||||
|
|
||||||
import AiService from "./ai.ts";
|
import AiService from "./ai.ts";
|
||||||
@ -37,6 +39,7 @@ import WorkspaceService from "./workspace.ts";
|
|||||||
import { GadgetService } from "../lib/service.ts";
|
import { GadgetService } from "../lib/service.ts";
|
||||||
import {
|
import {
|
||||||
AiToolbox,
|
AiToolbox,
|
||||||
|
ExportTool,
|
||||||
FileEditTool,
|
FileEditTool,
|
||||||
FileReadTool,
|
FileReadTool,
|
||||||
FileWriteTool,
|
FileWriteTool,
|
||||||
@ -57,6 +60,8 @@ import {
|
|||||||
type DroneToolboxEnvironment,
|
type DroneToolboxEnvironment,
|
||||||
} from "../tools/index.ts";
|
} from "../tools/index.ts";
|
||||||
|
|
||||||
|
import type { IChatExportData } from "@gadget/ai-toolbox";
|
||||||
|
|
||||||
export interface IAgentWorkOrder {
|
export interface IAgentWorkOrder {
|
||||||
createdAt: Date;
|
createdAt: Date;
|
||||||
turn: IChatTurn;
|
turn: IChatTurn;
|
||||||
@ -133,6 +138,11 @@ class AgentService extends GadgetService {
|
|||||||
subagentTool.setSpawner((agentType: string, prompt: string) => this.spawnSubagent(agentType, prompt));
|
subagentTool.setSpawner((agentType: string, prompt: string) => this.spawnSubagent(agentType, prompt));
|
||||||
this.toolbox.register(subagentTool, readOnlyModes);
|
this.toolbox.register(subagentTool, readOnlyModes);
|
||||||
|
|
||||||
|
// Chat tools — export: available in all modes
|
||||||
|
const exportTool = new ExportTool(this.toolbox);
|
||||||
|
exportTool.setSessionDataProvider(() => this.getExportData());
|
||||||
|
this.toolbox.register(exportTool, readOnlyModes);
|
||||||
|
|
||||||
// Project tools — skill discovery: available in all modes
|
// Project tools — skill discovery: available in all modes
|
||||||
this.toolbox.register(new ListSkillsTool(this.toolbox), readOnlyModes);
|
this.toolbox.register(new ListSkillsTool(this.toolbox), readOnlyModes);
|
||||||
this.toolbox.register(new ReadSkillTool(this.toolbox), readOnlyModes);
|
this.toolbox.register(new ReadSkillTool(this.toolbox), readOnlyModes);
|
||||||
@ -156,8 +166,10 @@ class AgentService extends GadgetService {
|
|||||||
|
|
||||||
const { turn } = workOrder;
|
const { turn } = workOrder;
|
||||||
let toolCallCount = 0;
|
let toolCallCount = 0;
|
||||||
let inputTokens = 0;
|
let masterInputTokens = 0;
|
||||||
let outputTokens = 0;
|
let masterOutputTokens = 0;
|
||||||
|
let masterThinkingTokens = 0;
|
||||||
|
const startTime = Date.now();
|
||||||
|
|
||||||
// Build the full message array that grows with each iteration
|
// Build the full message array that grows with each iteration
|
||||||
const messages: IContextChatMessage[] = [];
|
const messages: IContextChatMessage[] = [];
|
||||||
@ -250,6 +262,11 @@ class AgentService extends GadgetService {
|
|||||||
throw new Error("Model failed to respond (still loading or error)");
|
throw new Error("Model failed to respond (still loading or error)");
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// Accumulate real token counts from the AI API response
|
||||||
|
masterInputTokens += response.stats?.tokenCounts?.input ?? 0;
|
||||||
|
masterOutputTokens += response.stats?.tokenCounts?.response ?? 0;
|
||||||
|
masterThinkingTokens += response.stats?.tokenCounts?.thinking ?? 0;
|
||||||
|
|
||||||
// Process tool calls if present
|
// Process tool calls if present
|
||||||
if (response.toolCalls && response.toolCalls.length > 0) {
|
if (response.toolCalls && response.toolCalls.length > 0) {
|
||||||
continueLoop = true;
|
continueLoop = true;
|
||||||
@ -296,6 +313,7 @@ class AgentService extends GadgetService {
|
|||||||
agentId: toolCall.callId,
|
agentId: toolCall.callId,
|
||||||
response: responseText,
|
response: responseText,
|
||||||
subagent: subagentData,
|
subagent: subagentData,
|
||||||
|
stats: subagentData?.stats as Record<string, unknown> | undefined,
|
||||||
});
|
});
|
||||||
} else {
|
} else {
|
||||||
socket.emit("toolCall", toolCall.callId, toolCall.function.name, toolCall.function.arguments, result);
|
socket.emit("toolCall", toolCall.callId, toolCall.function.name, toolCall.function.arguments, result);
|
||||||
@ -330,17 +348,30 @@ class AgentService extends GadgetService {
|
|||||||
toolName: toolCall.function.name,
|
toolName: toolCall.function.name,
|
||||||
});
|
});
|
||||||
|
|
||||||
inputTokens += Math.ceil(toolCall.function.arguments.length / 4);
|
|
||||||
outputTokens += Math.ceil(result.length / 4);
|
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
socket.emit("workOrderComplete", turn._id, true);
|
const durationMs = Date.now() - startTime;
|
||||||
|
const stats: IWorkOrderCompleteStats = {
|
||||||
|
masterInputTokens,
|
||||||
|
masterOutputTokens,
|
||||||
|
masterThinkingTokens,
|
||||||
|
toolCallCount,
|
||||||
|
durationMs,
|
||||||
|
};
|
||||||
|
socket.emit("workOrderComplete", turn._id, true, undefined, stats);
|
||||||
} catch (cause) {
|
} catch (cause) {
|
||||||
if (cause instanceof Error && cause.name === "AbortError") {
|
if (cause instanceof Error && cause.name === "AbortError") {
|
||||||
this.log.info("work order aborted by user", { turnId: turn._id });
|
this.log.info("work order aborted by user", { turnId: turn._id });
|
||||||
socket.emit("workOrderComplete", turn._id, true, "aborted");
|
const abortStats: IWorkOrderCompleteStats = {
|
||||||
|
masterInputTokens,
|
||||||
|
masterOutputTokens,
|
||||||
|
masterThinkingTokens,
|
||||||
|
toolCallCount,
|
||||||
|
durationMs: Date.now() - startTime,
|
||||||
|
};
|
||||||
|
socket.emit("workOrderComplete", turn._id, true, "aborted", abortStats);
|
||||||
return;
|
return;
|
||||||
}
|
}
|
||||||
const msg = cause instanceof Error ? cause.message : String(cause);
|
const msg = cause instanceof Error ? cause.message : String(cause);
|
||||||
@ -348,7 +379,14 @@ class AgentService extends GadgetService {
|
|||||||
turnId: turn._id,
|
turnId: turn._id,
|
||||||
error: msg,
|
error: msg,
|
||||||
});
|
});
|
||||||
socket.emit("workOrderComplete", turn._id, false, msg);
|
const errorStats: IWorkOrderCompleteStats = {
|
||||||
|
masterInputTokens,
|
||||||
|
masterOutputTokens,
|
||||||
|
masterThinkingTokens,
|
||||||
|
toolCallCount,
|
||||||
|
durationMs: Date.now() - startTime,
|
||||||
|
};
|
||||||
|
socket.emit("workOrderComplete", turn._id, false, msg, errorStats);
|
||||||
throw cause;
|
throw cause;
|
||||||
} finally {
|
} finally {
|
||||||
this.abortController = null;
|
this.abortController = null;
|
||||||
@ -540,6 +578,64 @@ class AgentService extends GadgetService {
|
|||||||
return modelRecord?.settings;
|
return modelRecord?.settings;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Assembles all data needed for a chat session export, applying security
|
||||||
|
* scrubbing to remove PII, credentials, and infrastructure details.
|
||||||
|
* Only includes turns with finished/aborted/error status (excludes processing).
|
||||||
|
*/
|
||||||
|
private async getExportData(): Promise<IChatExportData> {
|
||||||
|
const workOrder = this.currentWorkOrder;
|
||||||
|
if (!workOrder) {
|
||||||
|
throw new Error("No active work order — cannot export");
|
||||||
|
}
|
||||||
|
|
||||||
|
const session = workOrder.turn.session as IChatSession;
|
||||||
|
if (!session) {
|
||||||
|
throw new Error("Chat session is not available on the current turn");
|
||||||
|
}
|
||||||
|
|
||||||
|
const user = session.user as IUser;
|
||||||
|
const provider = session.provider as IAiProvider;
|
||||||
|
const project = workOrder.turn.project as IProject;
|
||||||
|
|
||||||
|
// Combine context turns (already completed) with current turn if finished
|
||||||
|
const allTurns = [...workOrder.context];
|
||||||
|
if (workOrder.turn.status !== ChatTurnStatus.Processing) {
|
||||||
|
allTurns.push(workOrder.turn);
|
||||||
|
}
|
||||||
|
|
||||||
|
// Filter to only finished/aborted/error turns
|
||||||
|
const exportableTurns = allTurns.filter(
|
||||||
|
(t) => t.status !== ChatTurnStatus.Processing,
|
||||||
|
);
|
||||||
|
|
||||||
|
// Build the model config (excluding the provider reference — we send it separately scrubbed)
|
||||||
|
const reasoning = workOrder.turn.reasoningEffort === "off"
|
||||||
|
? false
|
||||||
|
: workOrder.turn.reasoningEffort || false;
|
||||||
|
const modelConfig = this.buildDroneModelConfig(
|
||||||
|
provider,
|
||||||
|
session.selectedModel,
|
||||||
|
reasoning as boolean | "low" | "medium" | "high",
|
||||||
|
session.numCtx,
|
||||||
|
);
|
||||||
|
|
||||||
|
return {
|
||||||
|
session,
|
||||||
|
turns: exportableTurns,
|
||||||
|
user,
|
||||||
|
provider,
|
||||||
|
project: {
|
||||||
|
name: project.name,
|
||||||
|
slug: project.slug,
|
||||||
|
description: project.description,
|
||||||
|
},
|
||||||
|
modelConfig,
|
||||||
|
gadgetCodeVersion: env.pkg.version,
|
||||||
|
exportedAt: new Date(),
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
private async executeTool(name: string, argsJson: string): Promise<string> {
|
private async executeTool(name: string, argsJson: string): Promise<string> {
|
||||||
const tool = this.toolbox.getTool(name);
|
const tool = this.toolbox.getTool(name);
|
||||||
if (!tool) {
|
if (!tool) {
|
||||||
@ -674,11 +770,11 @@ class AgentService extends GadgetService {
|
|||||||
|
|
||||||
inputTokens += response.stats?.tokenCounts?.input ?? 0;
|
inputTokens += response.stats?.tokenCounts?.input ?? 0;
|
||||||
if (response.thinking) {
|
if (response.thinking) {
|
||||||
thinkingTokenCount += Math.ceil(response.thinking.length / 4);
|
thinkingTokenCount += response.stats?.tokenCounts?.thinking ?? 0;
|
||||||
}
|
}
|
||||||
if (response.response) {
|
if (response.response) {
|
||||||
fullResponse += response.response;
|
fullResponse += response.response;
|
||||||
outputTokens += Math.ceil(response.response.length / 4);
|
outputTokens += response.stats?.tokenCounts?.response ?? 0;
|
||||||
}
|
}
|
||||||
if (response.thinking) {
|
if (response.thinking) {
|
||||||
fullThinking = (fullThinking + response.thinking).trim();
|
fullThinking = (fullThinking + response.thinking).trim();
|
||||||
@ -734,8 +830,6 @@ class AgentService extends GadgetService {
|
|||||||
toolName: toolCall.function.name,
|
toolName: toolCall.function.name,
|
||||||
});
|
});
|
||||||
|
|
||||||
inputTokens += Math.ceil(toolArgsRaw.length / 4);
|
|
||||||
outputTokens += Math.ceil(result.length / 4);
|
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|||||||
@ -18,6 +18,7 @@ export {
|
|||||||
PlanFileWriteTool,
|
PlanFileWriteTool,
|
||||||
PlanFileEditTool,
|
PlanFileEditTool,
|
||||||
PlanListTool,
|
PlanListTool,
|
||||||
|
ExportTool,
|
||||||
SubagentTool,
|
SubagentTool,
|
||||||
ListSkillsTool,
|
ListSkillsTool,
|
||||||
ReadSkillTool,
|
ReadSkillTool,
|
||||||
|
|||||||
@ -1,11 +1,12 @@
|
|||||||
{
|
{
|
||||||
"name": "gadget-workspace",
|
"name": "gadget-workspace",
|
||||||
"version": "1.0.1",
|
"version": "1.0.2",
|
||||||
"private": true,
|
"private": true,
|
||||||
"description": "Gadget Code monorepo workspace root",
|
"description": "Gadget Code monorepo workspace root",
|
||||||
"packageManager": "pnpm@10.33.2",
|
"packageManager": "pnpm@10.33.2",
|
||||||
"license": "Apache-2.0",
|
"license": "Apache-2.0",
|
||||||
"scripts": {
|
"scripts": {
|
||||||
|
"clean": "rm -rf packages/ai/dist packages/api/dist packages/config/dist gadget-code/dist gadget-drone/dist",
|
||||||
"link:global": "pnpm --filter gadget-drone exec -- pnpm link --global && pnpm --filter gadget-tasks exec -- pnpm link --global"
|
"link:global": "pnpm --filter gadget-drone exec -- pnpm link --global && pnpm --filter gadget-tasks exec -- pnpm link --global"
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|||||||
728
packages/ai-toolbox/src/chat/export.ts
Normal file
728
packages/ai-toolbox/src/chat/export.ts
Normal file
@ -0,0 +1,728 @@
|
|||||||
|
// src/chat/export.ts
|
||||||
|
// Copyright (C) 2026 Rob Colbert <rob.colbert@openplatform.us>
|
||||||
|
// Licensed under the Apache License, Version 2.0
|
||||||
|
|
||||||
|
import fs from "node:fs/promises";
|
||||||
|
import path from "node:path";
|
||||||
|
|
||||||
|
import type { IAiLogger, IToolArguments, IToolDefinition } from "@gadget/ai";
|
||||||
|
import { formatError } from "@gadget/ai";
|
||||||
|
import type {
|
||||||
|
IChatSession,
|
||||||
|
IChatTurn,
|
||||||
|
IChatTurnBlock,
|
||||||
|
IChatTurnBlockThinking,
|
||||||
|
IChatTurnBlockResponding,
|
||||||
|
IChatTurnBlockTool,
|
||||||
|
IChatToolCall,
|
||||||
|
IChatSubagentProcess,
|
||||||
|
IChatTurnStats,
|
||||||
|
IAiProvider,
|
||||||
|
IDroneModelConfig,
|
||||||
|
IUser,
|
||||||
|
} from "@gadget/api";
|
||||||
|
import { ChatTurnStatus } from "@gadget/api";
|
||||||
|
|
||||||
|
import { GadgetTool } from "../tool.ts";
|
||||||
|
|
||||||
|
const VALID_FORMATS = ["markdown", "json"] as const;
|
||||||
|
type ExportFormat = (typeof VALID_FORMATS)[number];
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Safe user representation — PII stripped.
|
||||||
|
* Only _id and displayName are included. The persona is included because
|
||||||
|
* it is user-authored and non-sensitive.
|
||||||
|
*/
|
||||||
|
export interface ISafeUser {
|
||||||
|
_id: string;
|
||||||
|
displayName: string;
|
||||||
|
persona?: string;
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Safe AI provider representation — secrets and infrastructure stripped.
|
||||||
|
* baseUrl is excluded (internal network info). apiKey is excluded (secret).
|
||||||
|
*/
|
||||||
|
export interface ISafeAiProvider {
|
||||||
|
_id: string;
|
||||||
|
name: string;
|
||||||
|
apiType: string;
|
||||||
|
enabled: boolean;
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Safe project representation — only non-sensitive metadata.
|
||||||
|
* gitUrl is excluded (could contain auth tokens in practice).
|
||||||
|
* system is excluded (may contain infrastructure details).
|
||||||
|
*/
|
||||||
|
export interface ISafeProject {
|
||||||
|
_id: string;
|
||||||
|
name: string;
|
||||||
|
slug: string;
|
||||||
|
description?: string;
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* A single turn scrubbed for export. `prompts.system` is always excluded.
|
||||||
|
*/
|
||||||
|
export interface IExportTurn {
|
||||||
|
_id: string;
|
||||||
|
createdAt: string;
|
||||||
|
mode: string;
|
||||||
|
status: string;
|
||||||
|
prompts: {
|
||||||
|
user: string;
|
||||||
|
};
|
||||||
|
blocks: IChatTurnBlock[];
|
||||||
|
toolCalls: IChatToolCall[];
|
||||||
|
subagents: IChatSubagentProcess[];
|
||||||
|
stats: IChatTurnStats;
|
||||||
|
errorMessage?: string;
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* The full export payload delivered by the AgentService data provider.
|
||||||
|
*/
|
||||||
|
export interface IChatExportData {
|
||||||
|
session: IChatSession;
|
||||||
|
turns: IChatTurn[];
|
||||||
|
user: IUser;
|
||||||
|
provider: IAiProvider;
|
||||||
|
project: { name: string; slug: string; description?: string };
|
||||||
|
modelConfig: Omit<IDroneModelConfig, "provider">;
|
||||||
|
gadgetCodeVersion: string;
|
||||||
|
exportedAt: Date;
|
||||||
|
}
|
||||||
|
|
||||||
|
// ── Utility helpers ────────────────────────────────────────────────────
|
||||||
|
|
||||||
|
function sanitizeSessionName(name: string): string {
|
||||||
|
return name
|
||||||
|
.toLowerCase()
|
||||||
|
.replace(/[^a-z0-9]+/g, "-")
|
||||||
|
.replace(/^-+|-+$/g, "")
|
||||||
|
.slice(0, 50) || "session";
|
||||||
|
}
|
||||||
|
|
||||||
|
function formatTimestamp(date: Date): string {
|
||||||
|
const d = date instanceof Date ? date : new Date(date);
|
||||||
|
const pad = (n: number) => String(n).padStart(2, "0");
|
||||||
|
return `${d.getFullYear()}-${pad(d.getMonth() + 1)}-${pad(d.getDate())}-${pad(d.getHours())}${pad(d.getMinutes())}${pad(d.getSeconds())}`;
|
||||||
|
}
|
||||||
|
|
||||||
|
function toISO(date: Date | string): string {
|
||||||
|
return new Date(date).toISOString();
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Strip PII, credentials, infrastructure info from a User record. */
|
||||||
|
function safeUser(user: IUser): ISafeUser {
|
||||||
|
return {
|
||||||
|
_id: String(user._id),
|
||||||
|
displayName: user.displayName,
|
||||||
|
persona: user.persona,
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Strip secrets and infrastructure info from an AiProvider record. */
|
||||||
|
function safeProvider(provider: IAiProvider): ISafeAiProvider {
|
||||||
|
return {
|
||||||
|
_id: String(provider._id),
|
||||||
|
name: provider.name,
|
||||||
|
apiType: provider.apiType,
|
||||||
|
enabled: provider.enabled,
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Strip sensitive info from a project record. */
|
||||||
|
function safeProject(project: { name: string; slug: string; description?: string; gitUrl?: string; system?: string }): ISafeProject {
|
||||||
|
return {
|
||||||
|
_id: String((project as any)._id),
|
||||||
|
name: project.name,
|
||||||
|
slug: project.slug,
|
||||||
|
description: project.description,
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Strip `prompts.system` from a turn and normalize dates. */
|
||||||
|
function scrubTurn(turn: IChatTurn): IExportTurn {
|
||||||
|
return {
|
||||||
|
_id: String(turn._id),
|
||||||
|
createdAt: toISO(turn.createdAt),
|
||||||
|
mode: turn.mode,
|
||||||
|
status: turn.status,
|
||||||
|
prompts: {
|
||||||
|
user: turn.prompts.user,
|
||||||
|
},
|
||||||
|
blocks: turn.blocks || [],
|
||||||
|
toolCalls: (turn.toolCalls || []).map(scrubToolCall),
|
||||||
|
subagents: (turn.subagents || []).map(scrubSubagent),
|
||||||
|
stats: turn.stats,
|
||||||
|
errorMessage: turn.errorMessage,
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Scrub a tool call for export — no changes needed to the core structure,
|
||||||
|
* but we strip any subagent-embedded provider/user references. */
|
||||||
|
function scrubToolCall(tc: IChatToolCall): IChatToolCall {
|
||||||
|
const result: IChatToolCall = {
|
||||||
|
callId: tc.callId,
|
||||||
|
name: tc.name,
|
||||||
|
parameters: tc.parameters,
|
||||||
|
response: tc.response,
|
||||||
|
};
|
||||||
|
if (tc.subagent) {
|
||||||
|
result.subagent = scrubSubagent(tc.subagent);
|
||||||
|
}
|
||||||
|
return result;
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Scrub a subagent process for export. */
|
||||||
|
function scrubSubagent(sa: IChatSubagentProcess): IChatSubagentProcess {
|
||||||
|
return {
|
||||||
|
prompt: sa.prompt,
|
||||||
|
thinking: sa.thinking,
|
||||||
|
response: sa.response,
|
||||||
|
toolCalls: (sa.toolCalls || []).map(scrubToolCall),
|
||||||
|
stats: sa.stats,
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
// ── Markdown formatter ─────────────────────────────────────────────────
|
||||||
|
|
||||||
|
function formatAsMarkdown(data: IChatExportData): string {
|
||||||
|
const { session, turns, user, provider, project, modelConfig, gadgetCodeVersion } = data;
|
||||||
|
const safeUser_ = safeUser(user);
|
||||||
|
const safeProvider_ = safeProvider(provider);
|
||||||
|
|
||||||
|
const finishedTurns = turns.filter(
|
||||||
|
(t) => t.status !== ChatTurnStatus.Processing,
|
||||||
|
);
|
||||||
|
const totalInputTokens = finishedTurns.reduce((sum, t) => sum + (t.stats?.inputTokens || 0), 0);
|
||||||
|
const totalOutputTokens = finishedTurns.reduce((sum, t) => sum + (t.stats?.responseTokens || 0), 0);
|
||||||
|
const totalThinkingTokens = finishedTurns.reduce((sum, t) => sum + (t.stats?.thinkingTokenCount || 0), 0);
|
||||||
|
|
||||||
|
const lines: string[] = [];
|
||||||
|
|
||||||
|
// ── Header ──
|
||||||
|
lines.push(`# ${session.name}`);
|
||||||
|
lines.push("");
|
||||||
|
lines.push(`**Session ID:** ${session._id}`);
|
||||||
|
lines.push(`**Created:** ${toISO(session.createdAt)}`);
|
||||||
|
if (session.lastMessageAt) {
|
||||||
|
lines.push(`**Last Message:** ${toISO(session.lastMessageAt)}`);
|
||||||
|
}
|
||||||
|
lines.push(`**Mode:** ${session.mode}`);
|
||||||
|
lines.push(`**Project:** ${project.name} (${project.slug})`);
|
||||||
|
lines.push(`**AI Provider:** ${safeProvider_.name} (${safeProvider_.apiType})`);
|
||||||
|
lines.push(`**Model:** ${session.selectedModel}`);
|
||||||
|
lines.push(`**Reasoning Effort:** ${session.reasoningEffort || "off"}`);
|
||||||
|
lines.push(`**Context Window:** ${session.numCtx || modelConfig.params.numCtx}`);
|
||||||
|
lines.push(`**Exported by:** Gadget Code v${gadgetCodeVersion}`);
|
||||||
|
lines.push(`**Exported at:** ${data.exportedAt.toISOString()}`);
|
||||||
|
lines.push("");
|
||||||
|
lines.push(`**Turns:** ${finishedTurns.length}`);
|
||||||
|
lines.push(`**Tool Calls:** ${session.stats?.toolCallCount || 0}`);
|
||||||
|
lines.push(`**Total Input Tokens:** ${totalInputTokens.toLocaleString()}`);
|
||||||
|
lines.push(`**Total Output Tokens:** ${totalOutputTokens.toLocaleString()}`);
|
||||||
|
if (totalThinkingTokens > 0) {
|
||||||
|
lines.push(`**Total Thinking Tokens:** ${totalThinkingTokens.toLocaleString()}`);
|
||||||
|
}
|
||||||
|
lines.push("");
|
||||||
|
|
||||||
|
// ── Pins ──
|
||||||
|
if (session.pins && session.pins.length > 0) {
|
||||||
|
lines.push("## Pinned Context");
|
||||||
|
lines.push("");
|
||||||
|
for (const pin of session.pins) {
|
||||||
|
lines.push(`- ${pin.content}`);
|
||||||
|
}
|
||||||
|
lines.push("");
|
||||||
|
}
|
||||||
|
|
||||||
|
lines.push("---");
|
||||||
|
lines.push("");
|
||||||
|
|
||||||
|
// ── Turns ──
|
||||||
|
for (const [i, turn] of finishedTurns.entries()) {
|
||||||
|
const scrubbed = scrubTurn(turn);
|
||||||
|
lines.push(`## Turn ${i + 1}`);
|
||||||
|
lines.push("");
|
||||||
|
lines.push(`**Timestamp:** ${scrubbed.createdAt}`);
|
||||||
|
lines.push(`**Mode:** ${scrubbed.mode}`);
|
||||||
|
lines.push(`**Status:** ${scrubbed.status}`);
|
||||||
|
if (scrubbed.stats?.durationLabel) {
|
||||||
|
lines.push(`**Duration:** ${scrubbed.stats.durationLabel}`);
|
||||||
|
}
|
||||||
|
lines.push("");
|
||||||
|
|
||||||
|
// User prompt
|
||||||
|
lines.push("### User");
|
||||||
|
lines.push("");
|
||||||
|
lines.push(scrubbed.prompts.user);
|
||||||
|
lines.push("");
|
||||||
|
|
||||||
|
// Thinking blocks
|
||||||
|
const thinkingBlocks = scrubbed.blocks.filter(
|
||||||
|
(b) => b.mode === "thinking",
|
||||||
|
) as IChatTurnBlockThinking[];
|
||||||
|
if (thinkingBlocks.length > 0) {
|
||||||
|
lines.push("### Thinking");
|
||||||
|
lines.push("");
|
||||||
|
for (const block of thinkingBlocks) {
|
||||||
|
lines.push(block.content);
|
||||||
|
lines.push("");
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Response blocks
|
||||||
|
const responseBlocks = scrubbed.blocks.filter(
|
||||||
|
(b) => b.mode === "responding",
|
||||||
|
) as IChatTurnBlockResponding[];
|
||||||
|
if (responseBlocks.length > 0) {
|
||||||
|
lines.push("### Assistant");
|
||||||
|
lines.push("");
|
||||||
|
for (const block of responseBlocks) {
|
||||||
|
lines.push(block.content);
|
||||||
|
lines.push("");
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Tool call blocks
|
||||||
|
const toolBlocks = scrubbed.blocks.filter(
|
||||||
|
(b) => b.mode === "tool",
|
||||||
|
) as IChatTurnBlockTool[];
|
||||||
|
if (toolBlocks.length > 0 || scrubbed.toolCalls.length > 0) {
|
||||||
|
lines.push("### Tool Calls");
|
||||||
|
lines.push("");
|
||||||
|
|
||||||
|
// From blocks
|
||||||
|
for (const block of toolBlocks) {
|
||||||
|
const tc = block.content;
|
||||||
|
lines.push(`#### ${tc.name}`);
|
||||||
|
lines.push("");
|
||||||
|
lines.push("**Call ID:** " + tc.callId);
|
||||||
|
lines.push("");
|
||||||
|
lines.push("**Parameters:**");
|
||||||
|
lines.push("```json");
|
||||||
|
try {
|
||||||
|
lines.push(JSON.stringify(JSON.parse(tc.parameters), null, 2));
|
||||||
|
} catch {
|
||||||
|
lines.push(tc.parameters);
|
||||||
|
}
|
||||||
|
lines.push("```");
|
||||||
|
lines.push("");
|
||||||
|
if (tc.response) {
|
||||||
|
lines.push("**Response:**");
|
||||||
|
lines.push("```");
|
||||||
|
// Truncate very long responses for readability
|
||||||
|
const resp = tc.response.length > 5000
|
||||||
|
? tc.response.slice(0, 5000) + "\n... (truncated for readability)"
|
||||||
|
: tc.response;
|
||||||
|
lines.push(resp);
|
||||||
|
lines.push("```");
|
||||||
|
lines.push("");
|
||||||
|
}
|
||||||
|
if (tc.subagent) {
|
||||||
|
lines.push(formatSubagentMarkdown(tc.subagent));
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// From toolCalls array (if not already covered by blocks)
|
||||||
|
const blockCallIds = new Set(toolBlocks.map((b) => b.content.callId));
|
||||||
|
for (const tc of scrubbed.toolCalls) {
|
||||||
|
if (blockCallIds.has(tc.callId)) continue;
|
||||||
|
lines.push(`#### ${tc.name}`);
|
||||||
|
lines.push("");
|
||||||
|
lines.push("**Call ID:** " + tc.callId);
|
||||||
|
lines.push("");
|
||||||
|
lines.push("**Parameters:**");
|
||||||
|
lines.push("```json");
|
||||||
|
try {
|
||||||
|
lines.push(JSON.stringify(JSON.parse(tc.parameters), null, 2));
|
||||||
|
} catch {
|
||||||
|
lines.push(tc.parameters);
|
||||||
|
}
|
||||||
|
lines.push("```");
|
||||||
|
lines.push("");
|
||||||
|
if (tc.response) {
|
||||||
|
lines.push("**Response:**");
|
||||||
|
lines.push("```");
|
||||||
|
const resp = tc.response.length > 5000
|
||||||
|
? tc.response.slice(0, 5000) + "\n... (truncated for readability)"
|
||||||
|
: tc.response;
|
||||||
|
lines.push(resp);
|
||||||
|
lines.push("```");
|
||||||
|
lines.push("");
|
||||||
|
}
|
||||||
|
if (tc.subagent) {
|
||||||
|
lines.push(formatSubagentMarkdown(tc.subagent));
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Subagents
|
||||||
|
if (scrubbed.subagents.length > 0) {
|
||||||
|
lines.push("### Subagents");
|
||||||
|
lines.push("");
|
||||||
|
for (const sa of scrubbed.subagents) {
|
||||||
|
lines.push(formatSubagentMarkdown(sa));
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Error
|
||||||
|
if (scrubbed.errorMessage) {
|
||||||
|
lines.push("### Error");
|
||||||
|
lines.push("");
|
||||||
|
lines.push(`> ${scrubbed.errorMessage}`);
|
||||||
|
lines.push("");
|
||||||
|
}
|
||||||
|
|
||||||
|
// Turn stats
|
||||||
|
lines.push(`*Token usage: ${scrubbed.stats?.inputTokens || 0} input / ${scrubbed.stats?.responseTokens || 0} output${scrubbed.stats?.thinkingTokenCount ? ` / ${scrubbed.stats.thinkingTokenCount} thinking` : ""}*`);
|
||||||
|
lines.push("");
|
||||||
|
lines.push("---");
|
||||||
|
lines.push("");
|
||||||
|
}
|
||||||
|
|
||||||
|
return lines.join("\n");
|
||||||
|
}
|
||||||
|
|
||||||
|
function formatSubagentMarkdown(sa: IChatSubagentProcess): string {
|
||||||
|
const lines: string[] = [];
|
||||||
|
lines.push(`##### Subagent`);
|
||||||
|
lines.push("");
|
||||||
|
lines.push("**Prompt:** " + sa.prompt.slice(0, 200) + (sa.prompt.length > 200 ? "..." : ""));
|
||||||
|
lines.push("");
|
||||||
|
if (sa.thinking) {
|
||||||
|
lines.push("**Thinking:**");
|
||||||
|
lines.push("> " + sa.thinking.slice(0, 1000).replace(/\n/g, "\n> "));
|
||||||
|
lines.push("");
|
||||||
|
}
|
||||||
|
if (sa.response) {
|
||||||
|
lines.push("**Response:**");
|
||||||
|
lines.push(sa.response.slice(0, 2000) + (sa.response.length > 2000 ? "\n... (truncated)" : ""));
|
||||||
|
lines.push("");
|
||||||
|
}
|
||||||
|
if (sa.stats) {
|
||||||
|
lines.push(`*Subagent tokens: ${sa.stats.inputTokens || 0} input / ${sa.stats.responseTokens || 0} output / ${sa.stats.thinkingTokenCount || 0} thinking — ${sa.stats.durationLabel || "unknown"}*`);
|
||||||
|
lines.push("");
|
||||||
|
}
|
||||||
|
return lines.join("\n");
|
||||||
|
}
|
||||||
|
|
||||||
|
// ── JSON formatter ─────────────────────────────────────────────────────
|
||||||
|
|
||||||
|
function formatAsJson(data: IChatExportData): string {
|
||||||
|
const { session, turns, user, provider, project, modelConfig, gadgetCodeVersion } = data;
|
||||||
|
|
||||||
|
const finishedTurns = turns.filter(
|
||||||
|
(t) => t.status !== ChatTurnStatus.Processing,
|
||||||
|
);
|
||||||
|
const totalInputTokens = finishedTurns.reduce((sum, t) => sum + (t.stats?.inputTokens || 0), 0);
|
||||||
|
const totalOutputTokens = finishedTurns.reduce((sum, t) => sum + (t.stats?.responseTokens || 0), 0);
|
||||||
|
|
||||||
|
const exportObj = {
|
||||||
|
exportMetadata: {
|
||||||
|
format: "gadget-code-chat-export-v1",
|
||||||
|
version: "1.0.0",
|
||||||
|
exportedAt: data.exportedAt.toISOString(),
|
||||||
|
gadgetCodeVersion,
|
||||||
|
gadgetCodeUrl: "https://g4dge7.com/",
|
||||||
|
},
|
||||||
|
session: {
|
||||||
|
_id: String(session._id),
|
||||||
|
name: session.name,
|
||||||
|
mode: session.mode,
|
||||||
|
createdAt: toISO(session.createdAt),
|
||||||
|
lastMessageAt: session.lastMessageAt ? toISO(session.lastMessageAt) : undefined,
|
||||||
|
selectedModel: session.selectedModel,
|
||||||
|
reasoningEffort: session.reasoningEffort || "off",
|
||||||
|
numCtx: session.numCtx || modelConfig.params.numCtx,
|
||||||
|
stats: session.stats,
|
||||||
|
pins: session.pins || [],
|
||||||
|
},
|
||||||
|
user: safeUser(user),
|
||||||
|
provider: safeProvider(provider),
|
||||||
|
project: safeProject(project),
|
||||||
|
modelConfig: {
|
||||||
|
modelId: modelConfig.modelId,
|
||||||
|
params: modelConfig.params,
|
||||||
|
},
|
||||||
|
turns: finishedTurns.map(scrubTurn),
|
||||||
|
aggregateStats: {
|
||||||
|
turnCount: finishedTurns.length,
|
||||||
|
totalInputTokens,
|
||||||
|
totalOutputTokens,
|
||||||
|
},
|
||||||
|
};
|
||||||
|
|
||||||
|
return JSON.stringify(exportObj, null, 2);
|
||||||
|
}
|
||||||
|
|
||||||
|
// ── Companion Markdown (for JSON exports) ───────────────────────────────
|
||||||
|
|
||||||
|
function formatCompanionMarkdown(data: IChatExportData): string {
|
||||||
|
const { session, user, provider, modelConfig, gadgetCodeVersion } = data;
|
||||||
|
const safeUser_ = safeUser(user);
|
||||||
|
const safeProvider_ = safeProvider(provider);
|
||||||
|
|
||||||
|
const finishedTurns = (data.turns || []).filter(
|
||||||
|
(t) => t.status !== ChatTurnStatus.Processing,
|
||||||
|
);
|
||||||
|
const totalInputTokens = finishedTurns.reduce((sum, t) => sum + (t.stats?.inputTokens || 0), 0);
|
||||||
|
const totalOutputTokens = finishedTurns.reduce((sum, t) => sum + (t.stats?.responseTokens || 0), 0);
|
||||||
|
|
||||||
|
const lines: string[] = [];
|
||||||
|
|
||||||
|
lines.push("# Chat Export — Companion Document");
|
||||||
|
lines.push("");
|
||||||
|
lines.push("## Export Details");
|
||||||
|
lines.push("");
|
||||||
|
lines.push(`- **Format:** Gadget Code Chat Export v1 (JSON)`);
|
||||||
|
lines.push(`- **Exported:** ${data.exportedAt.toISOString()}`);
|
||||||
|
lines.push(`- **Gadget Code Version:** ${gadgetCodeVersion}`);
|
||||||
|
lines.push(`- **Get Gadget Code:** https://g4dge7.com/`);
|
||||||
|
lines.push("");
|
||||||
|
|
||||||
|
lines.push("## Authors");
|
||||||
|
lines.push("");
|
||||||
|
lines.push("### Human Author");
|
||||||
|
lines.push(`- **Name:** ${safeUser_.displayName}`);
|
||||||
|
lines.push("");
|
||||||
|
|
||||||
|
lines.push("### AI Agent");
|
||||||
|
lines.push(`- **Agent:** Gadget Code version ${gadgetCodeVersion}`);
|
||||||
|
lines.push(`- **Provider:** ${safeProvider_.name} (${safeProvider_.apiType})`);
|
||||||
|
lines.push(`- **Model:** ${modelConfig.modelId}`);
|
||||||
|
lines.push(`- **Parameters:**`);
|
||||||
|
lines.push(` - Temperature: ${modelConfig.params.temperature}`);
|
||||||
|
lines.push(` - Top P: ${modelConfig.params.topP}`);
|
||||||
|
lines.push(` - Top K: ${modelConfig.params.topK}`);
|
||||||
|
lines.push(` - Context Window: ${modelConfig.params.numCtx}`);
|
||||||
|
lines.push(` - Max Completion Tokens: ${modelConfig.params.maxCompletionTokens}`);
|
||||||
|
lines.push(` - Num Predict: ${modelConfig.params.numPredict}`);
|
||||||
|
lines.push(` - Reasoning: ${modelConfig.params.reasoning}`);
|
||||||
|
lines.push("");
|
||||||
|
|
||||||
|
lines.push("## Session Summary");
|
||||||
|
lines.push("");
|
||||||
|
lines.push(`- **Session:** ${session.name}`);
|
||||||
|
lines.push(`- **Session ID:** ${session._id}`);
|
||||||
|
lines.push(`- **Mode:** ${session.mode}`);
|
||||||
|
lines.push(`- **Turns:** ${finishedTurns.length}`);
|
||||||
|
lines.push(`- **Tool Calls:** ${session.stats?.toolCallCount || 0}`);
|
||||||
|
lines.push(`- **Total Input Tokens:** ${totalInputTokens.toLocaleString()}`);
|
||||||
|
lines.push(`- **Total Output Tokens:** ${totalOutputTokens.toLocaleString()}`);
|
||||||
|
lines.push("");
|
||||||
|
|
||||||
|
lines.push("## About This Export");
|
||||||
|
lines.push("");
|
||||||
|
lines.push("This Markdown document accompanies a JSON export of a Gadget Code chat session.");
|
||||||
|
lines.push("The JSON file contains the complete machine-readable conversation data, suitable");
|
||||||
|
lines.push("for import into other systems, analysis, or archival purposes.");
|
||||||
|
lines.push("");
|
||||||
|
lines.push("The JSON export format is designed to be self-contained and interoperable. It");
|
||||||
|
lines.push("includes all conversation turns, tool calls, subagent interactions, and session");
|
||||||
|
lines.push("metadata needed to reconstruct the full conversation in another system.");
|
||||||
|
lines.push("");
|
||||||
|
lines.push("For more information about Gadget Code, visit https://g4dge7.com/");
|
||||||
|
|
||||||
|
return lines.join("\n");
|
||||||
|
}
|
||||||
|
|
||||||
|
// ── Tool implementation ───────────────────────────────────────────────
|
||||||
|
|
||||||
|
export class ExportTool extends GadgetTool {
|
||||||
|
private _getSessionData: (() => Promise<IChatExportData>) | null = null;
|
||||||
|
|
||||||
|
setSessionDataProvider(fn: () => Promise<IChatExportData>): void {
|
||||||
|
this._getSessionData = fn;
|
||||||
|
}
|
||||||
|
|
||||||
|
get name(): string {
|
||||||
|
return "chat_export";
|
||||||
|
}
|
||||||
|
|
||||||
|
get category(): string {
|
||||||
|
return "chat";
|
||||||
|
}
|
||||||
|
|
||||||
|
get definition(): IToolDefinition {
|
||||||
|
return {
|
||||||
|
type: "function",
|
||||||
|
function: {
|
||||||
|
name: this.name,
|
||||||
|
description:
|
||||||
|
"Export the current chat session and all its conversation turns to disk as a document. " +
|
||||||
|
"The default format is Markdown, which produces a well-formatted conversation log including " +
|
||||||
|
"tool calls, responses, thinking blocks, and related metadata. " +
|
||||||
|
"The JSON format produces a complete machine-readable export suitable for import into " +
|
||||||
|
"other systems, analysis, or archival — it is always accompanied by a companion Markdown " +
|
||||||
|
"document that describes and credits the export. " +
|
||||||
|
"All exports exclude sensitive information (emails, API keys, hostnames, system prompts).",
|
||||||
|
parameters: {
|
||||||
|
type: "object",
|
||||||
|
properties: {
|
||||||
|
format: {
|
||||||
|
type: "string",
|
||||||
|
enum: [...VALID_FORMATS],
|
||||||
|
description:
|
||||||
|
"Export format: 'markdown' for a human-readable conversation log, " +
|
||||||
|
"'json' for a complete machine-readable export with a companion Markdown document. " +
|
||||||
|
"Defaults to 'markdown'.",
|
||||||
|
},
|
||||||
|
},
|
||||||
|
required: ["format"],
|
||||||
|
},
|
||||||
|
},
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
async execute(args: IToolArguments, logger: IAiLogger): Promise<string> {
|
||||||
|
const { format } = args;
|
||||||
|
|
||||||
|
// ── Validate format parameter ──
|
||||||
|
if (!format) {
|
||||||
|
return formatError({
|
||||||
|
code: "MISSING_PARAMETER",
|
||||||
|
message: "The 'format' parameter is required.",
|
||||||
|
parameter: "format",
|
||||||
|
expected: "Either 'markdown' or 'json'",
|
||||||
|
recoveryHint: "Specify 'markdown' for a readable log or 'json' for a machine-readable export.",
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
|
if (!VALID_FORMATS.includes(format as ExportFormat)) {
|
||||||
|
return formatError({
|
||||||
|
code: "INVALID_PARAMETER",
|
||||||
|
message: `Invalid format: '${format}'. Must be one of: ${VALID_FORMATS.join(", ")}`,
|
||||||
|
parameter: "format",
|
||||||
|
expected: `One of: ${VALID_FORMATS.join(", ")}`,
|
||||||
|
recoveryHint: "Use 'markdown' for a human-readable conversation log or 'json' for a complete data export.",
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
|
// ── Check session data provider ──
|
||||||
|
if (!this._getSessionData) {
|
||||||
|
return formatError({
|
||||||
|
code: "OPERATION_FAILED",
|
||||||
|
message: "ExportTool has not been initialized with a session data provider.",
|
||||||
|
recoveryHint: "This is an internal initialization error. The agent service must wire the data provider during startup.",
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
|
// ── Get export data ──
|
||||||
|
let data: IChatExportData;
|
||||||
|
try {
|
||||||
|
data = await this._getSessionData();
|
||||||
|
} catch (error) {
|
||||||
|
const message = error instanceof Error ? error.message : String(error);
|
||||||
|
logger.error("failed to get export data", { error: message });
|
||||||
|
return formatError({
|
||||||
|
code: "OPERATION_FAILED",
|
||||||
|
message: `Failed to retrieve session data for export: ${message}`,
|
||||||
|
recoveryHint: "Ensure the chat session is active and try again.",
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
|
// ── Determine file paths ──
|
||||||
|
const projectDir = this.toolbox.env.workspace?.projectDir;
|
||||||
|
if (!projectDir) {
|
||||||
|
return formatError({
|
||||||
|
code: "OPERATION_FAILED",
|
||||||
|
message: "Workspace project directory is not available. The workspace must be initialized before exporting.",
|
||||||
|
recoveryHint: "Ensure the agent is running within an active workspace.",
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
|
const exportDir = path.join(projectDir, ".gadget", "exports");
|
||||||
|
const baseName = sanitizeSessionName(data.session.name);
|
||||||
|
const timestamp = formatTimestamp(data.exportedAt);
|
||||||
|
const exportFormat = format as ExportFormat;
|
||||||
|
|
||||||
|
logger.debug("exporting chat session", {
|
||||||
|
sessionId: String(data.session._id),
|
||||||
|
format: exportFormat,
|
||||||
|
turnCount: data.turns.length,
|
||||||
|
});
|
||||||
|
|
||||||
|
try {
|
||||||
|
// Ensure export directory exists
|
||||||
|
await fs.mkdir(exportDir, { recursive: true });
|
||||||
|
|
||||||
|
const results: string[] = [];
|
||||||
|
|
||||||
|
if (exportFormat === "markdown") {
|
||||||
|
// ── Markdown export ──
|
||||||
|
const content = formatAsMarkdown(data);
|
||||||
|
const fileName = `${baseName}-${timestamp}.md`;
|
||||||
|
const filePath = path.join(exportDir, fileName);
|
||||||
|
await fs.writeFile(filePath, content, "utf-8");
|
||||||
|
const byteCount = Buffer.byteLength(content, "utf-8");
|
||||||
|
results.push(`.gadget/exports/${fileName}`);
|
||||||
|
|
||||||
|
// Build success response
|
||||||
|
const finishedTurns = data.turns.filter(
|
||||||
|
(t) => t.status !== ChatTurnStatus.Processing,
|
||||||
|
);
|
||||||
|
const totalInput = finishedTurns.reduce((s, t) => s + (t.stats?.inputTokens || 0), 0);
|
||||||
|
const totalOutput = finishedTurns.reduce((s, t) => s + (t.stats?.responseTokens || 0), 0);
|
||||||
|
|
||||||
|
return [
|
||||||
|
"CHAT EXPORT SUCCESS",
|
||||||
|
`Format: markdown`,
|
||||||
|
`Session: ${data.session.name}`,
|
||||||
|
`Turns: ${finishedTurns.length}`,
|
||||||
|
`Total Input Tokens: ${totalInput.toLocaleString()}`,
|
||||||
|
`Total Output Tokens: ${totalOutput.toLocaleString()}`,
|
||||||
|
`Bytes: ${byteCount.toLocaleString()}`,
|
||||||
|
`Files:`,
|
||||||
|
...results.map((r) => ` - ${r}`),
|
||||||
|
].join("\n");
|
||||||
|
}
|
||||||
|
|
||||||
|
// ── JSON export (always with companion markdown) ──
|
||||||
|
const jsonContent = formatAsJson(data);
|
||||||
|
const jsonFileName = `${baseName}-${timestamp}.json`;
|
||||||
|
const jsonFilePath = path.join(exportDir, jsonFileName);
|
||||||
|
await fs.writeFile(jsonFilePath, jsonContent, "utf-8");
|
||||||
|
const jsonBytes = Buffer.byteLength(jsonContent, "utf-8");
|
||||||
|
results.push(`.gadget/exports/${jsonFileName}`);
|
||||||
|
|
||||||
|
const companionContent = formatCompanionMarkdown(data);
|
||||||
|
const companionFileName = `${baseName}-${timestamp}-readme.md`;
|
||||||
|
const companionFilePath = path.join(exportDir, companionFileName);
|
||||||
|
await fs.writeFile(companionFilePath, companionContent, "utf-8");
|
||||||
|
const companionBytes = Buffer.byteLength(companionContent, "utf-8");
|
||||||
|
results.push(`.gadget/exports/${companionFileName}`);
|
||||||
|
|
||||||
|
// Build success response
|
||||||
|
const finishedTurns = data.turns.filter(
|
||||||
|
(t) => t.status !== ChatTurnStatus.Processing,
|
||||||
|
);
|
||||||
|
const totalInput = finishedTurns.reduce((s, t) => s + (t.stats?.inputTokens || 0), 0);
|
||||||
|
const totalOutput = finishedTurns.reduce((s, t) => s + (t.stats?.responseTokens || 0), 0);
|
||||||
|
|
||||||
|
return [
|
||||||
|
"CHAT EXPORT SUCCESS",
|
||||||
|
`Format: json + companion markdown`,
|
||||||
|
`Session: ${data.session.name}`,
|
||||||
|
`Turns: ${finishedTurns.length}`,
|
||||||
|
`Total Input Tokens: ${totalInput.toLocaleString()}`,
|
||||||
|
`Total Output Tokens: ${totalOutput.toLocaleString()}`,
|
||||||
|
`JSON Bytes: ${jsonBytes.toLocaleString()}`,
|
||||||
|
`Readme Bytes: ${companionBytes.toLocaleString()}`,
|
||||||
|
`Files:`,
|
||||||
|
...results.map((r) => ` - ${r}`),
|
||||||
|
].join("\n");
|
||||||
|
} catch (error) {
|
||||||
|
const message = error instanceof Error ? error.message : String(error);
|
||||||
|
logger.error("failed to write export file", { error: message });
|
||||||
|
return formatError({
|
||||||
|
code: "OPERATION_FAILED",
|
||||||
|
message: `Failed to write export file: ${message}`,
|
||||||
|
recoveryHint: "Check filesystem permissions and available disk space, then try again.",
|
||||||
|
});
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
@ -3,3 +3,4 @@
|
|||||||
// Licensed under the Apache License, Version 2.0
|
// Licensed under the Apache License, Version 2.0
|
||||||
|
|
||||||
export { SubagentTool } from "./subagent.ts";
|
export { SubagentTool } from "./subagent.ts";
|
||||||
|
export { ExportTool, type IChatExportData, type ISafeUser, type ISafeAiProvider, type ISafeProject, type IExportTurn } from "./export.ts";
|
||||||
|
|||||||
@ -170,6 +170,11 @@ export interface IAiModelProbeResult {
|
|||||||
};
|
};
|
||||||
}
|
}
|
||||||
|
|
||||||
|
export interface IAiEmbeddingResponse {
|
||||||
|
embedding: number[];
|
||||||
|
model: string;
|
||||||
|
}
|
||||||
|
|
||||||
export abstract class AiApi {
|
export abstract class AiApi {
|
||||||
protected env: IAiEnvironment;
|
protected env: IAiEnvironment;
|
||||||
protected provider: IAiProvider;
|
protected provider: IAiProvider;
|
||||||
@ -196,6 +201,11 @@ export abstract class AiApi {
|
|||||||
streamCallback?: IAiResponseStreamFn,
|
streamCallback?: IAiResponseStreamFn,
|
||||||
): Promise<IAiChatResponse>;
|
): Promise<IAiChatResponse>;
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Generate an embedding vector for the given text.
|
||||||
|
*/
|
||||||
|
abstract embeddings(modelId: string, text: string, dimensions?: number): Promise<IAiEmbeddingResponse>;
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* Forcefully abort any in-progress API request.
|
* Forcefully abort any in-progress API request.
|
||||||
* Provider-specific implementations (e.g., Ollama) should terminate
|
* Provider-specific implementations (e.g., Ollama) should terminate
|
||||||
|
|||||||
@ -21,6 +21,7 @@ export {
|
|||||||
AiApi,
|
AiApi,
|
||||||
type IAiModelListResult,
|
type IAiModelListResult,
|
||||||
type IAiModelProbeResult,
|
type IAiModelProbeResult,
|
||||||
|
type IAiEmbeddingResponse,
|
||||||
} from "./api.js";
|
} from "./api.js";
|
||||||
|
|
||||||
export type {
|
export type {
|
||||||
|
|||||||
@ -19,6 +19,7 @@ import {
|
|||||||
IAiModelProbeResult,
|
IAiModelProbeResult,
|
||||||
IAiProvider,
|
IAiProvider,
|
||||||
IAiResponseStreamFn,
|
IAiResponseStreamFn,
|
||||||
|
IAiEmbeddingResponse,
|
||||||
} from "./api.js";
|
} from "./api.js";
|
||||||
import { IAiEnvironment } from "./config/env.ts";
|
import { IAiEnvironment } from "./config/env.ts";
|
||||||
import type { Message as OllamaMessage } from "ollama";
|
import type { Message as OllamaMessage } from "ollama";
|
||||||
@ -367,4 +368,9 @@ export class OllamaAiApi extends AiApi {
|
|||||||
this.assertNonEmptyChatResponse(chatResponse);
|
this.assertNonEmptyChatResponse(chatResponse);
|
||||||
return chatResponse;
|
return chatResponse;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
async embeddings(modelId: string, text: string, dimensions?: number): Promise<IAiEmbeddingResponse> {
|
||||||
|
const response = await this.client.embed({ model: modelId, input: text, dimensions });
|
||||||
|
return { embedding: response.embeddings[0]!, model: modelId };
|
||||||
|
}
|
||||||
}
|
}
|
||||||
|
|||||||
@ -17,6 +17,7 @@ import {
|
|||||||
IAiModelProbeResult,
|
IAiModelProbeResult,
|
||||||
IAiProvider,
|
IAiProvider,
|
||||||
IAiResponseStreamFn,
|
IAiResponseStreamFn,
|
||||||
|
IAiEmbeddingResponse,
|
||||||
} from "./api.js";
|
} from "./api.js";
|
||||||
import {
|
import {
|
||||||
ChatCompletionFunctionTool,
|
ChatCompletionFunctionTool,
|
||||||
@ -64,6 +65,12 @@ interface StreamingToolCallAccumulator {
|
|||||||
};
|
};
|
||||||
}
|
}
|
||||||
|
|
||||||
|
interface OpenAiUsage {
|
||||||
|
promptTokens: number;
|
||||||
|
completionTokens: number;
|
||||||
|
reasoningTokens: number;
|
||||||
|
}
|
||||||
|
|
||||||
interface OpenAiChatIterationResult {
|
interface OpenAiChatIterationResult {
|
||||||
response: string;
|
response: string;
|
||||||
thinking?: string;
|
thinking?: string;
|
||||||
@ -80,6 +87,7 @@ interface OpenAiChatIterationResult {
|
|||||||
chunkCount: number;
|
chunkCount: number;
|
||||||
contentDeltaCount: number;
|
contentDeltaCount: number;
|
||||||
toolDeltaCount: number;
|
toolDeltaCount: number;
|
||||||
|
usage?: OpenAiUsage;
|
||||||
}
|
}
|
||||||
|
|
||||||
export class OpenAiApi extends AiApi {
|
export class OpenAiApi extends AiApi {
|
||||||
@ -187,17 +195,13 @@ export class OpenAiApi extends AiApi {
|
|||||||
options: IAiGenerateOptions,
|
options: IAiGenerateOptions,
|
||||||
streamCallback?: IAiResponseStreamFn,
|
streamCallback?: IAiResponseStreamFn,
|
||||||
): Promise<IAiGenerateResponse> {
|
): Promise<IAiGenerateResponse> {
|
||||||
await this.log.debug("OpenAiApi.generate called", {
|
|
||||||
provider: model.provider.name,
|
|
||||||
modelId: model.modelId,
|
|
||||||
});
|
|
||||||
|
|
||||||
if (options.signal?.aborted) {
|
if (options.signal?.aborted) {
|
||||||
throw new DOMException("The operation was aborted", "AbortError");
|
throw new DOMException("The operation was aborted", "AbortError");
|
||||||
}
|
}
|
||||||
|
|
||||||
const startTime = Date.now();
|
const startTime = Date.now();
|
||||||
const response = await this.client.chat.completions.create({
|
const response = await this.client.chat.completions.create(
|
||||||
|
{
|
||||||
model: model.modelId,
|
model: model.modelId,
|
||||||
messages: [
|
messages: [
|
||||||
...(options.systemPrompt
|
...(options.systemPrompt
|
||||||
@ -206,6 +210,7 @@ export class OpenAiApi extends AiApi {
|
|||||||
{ role: "user" as const, content: options.prompt },
|
{ role: "user" as const, content: options.prompt },
|
||||||
],
|
],
|
||||||
stream: true,
|
stream: true,
|
||||||
|
stream_options: { include_usage: true },
|
||||||
...(model.params.maxCompletionTokens
|
...(model.params.maxCompletionTokens
|
||||||
? { max_completion_tokens: model.params.maxCompletionTokens }
|
? { max_completion_tokens: model.params.maxCompletionTokens }
|
||||||
: {}),
|
: {}),
|
||||||
@ -219,16 +224,24 @@ export class OpenAiApi extends AiApi {
|
|||||||
| "high",
|
| "high",
|
||||||
}
|
}
|
||||||
: {}),
|
: {}),
|
||||||
}, options.signal ? { signal: options.signal } : undefined);
|
},
|
||||||
|
options.signal ? { signal: options.signal } : undefined,
|
||||||
|
);
|
||||||
|
|
||||||
let accumulatedResponse = "";
|
let accumulatedResponse = "";
|
||||||
let accumulatedThinking = "";
|
let accumulatedThinking = "";
|
||||||
|
let usage: OpenAiUsage | undefined;
|
||||||
|
|
||||||
for await (const chunk of response) {
|
for await (const chunk of response) {
|
||||||
if (options.signal?.aborted) {
|
if (options.signal?.aborted) {
|
||||||
throw new DOMException("The operation was aborted", "AbortError");
|
throw new DOMException("The operation was aborted", "AbortError");
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// Capture usage from the final streaming chunk
|
||||||
|
if (chunk.usage) {
|
||||||
|
usage = this.extractUsage(chunk.usage);
|
||||||
|
}
|
||||||
|
|
||||||
const delta = chunk.choices[0]?.delta;
|
const delta = chunk.choices[0]?.delta;
|
||||||
if (delta) {
|
if (delta) {
|
||||||
if (delta.content) {
|
if (delta.content) {
|
||||||
@ -265,9 +278,9 @@ export class OpenAiApi extends AiApi {
|
|||||||
text: numeral(durationMs / 1000).format("hh:mm:ss"),
|
text: numeral(durationMs / 1000).format("hh:mm:ss"),
|
||||||
},
|
},
|
||||||
tokenCounts: {
|
tokenCounts: {
|
||||||
input: 0,
|
input: usage?.promptTokens ?? 0,
|
||||||
response: 0,
|
response: usage?.completionTokens ?? 0,
|
||||||
thinking: 0,
|
thinking: usage?.reasoningTokens ?? 0,
|
||||||
},
|
},
|
||||||
},
|
},
|
||||||
};
|
};
|
||||||
@ -278,11 +291,6 @@ export class OpenAiApi extends AiApi {
|
|||||||
options: IAiChatOptions,
|
options: IAiChatOptions,
|
||||||
streamCallback?: IAiResponseStreamFn,
|
streamCallback?: IAiResponseStreamFn,
|
||||||
): Promise<IAiChatResponse> {
|
): Promise<IAiChatResponse> {
|
||||||
await this.log.debug("OpenAiApi.chat called", {
|
|
||||||
provider: model.provider.name,
|
|
||||||
modelId: model.modelId,
|
|
||||||
});
|
|
||||||
|
|
||||||
const startTime = Date.now();
|
const startTime = Date.now();
|
||||||
const messages = this.buildMessages(options);
|
const messages = this.buildMessages(options);
|
||||||
const tools = this.buildTools(options);
|
const tools = this.buildTools(options);
|
||||||
@ -295,27 +303,16 @@ export class OpenAiApi extends AiApi {
|
|||||||
options.signal,
|
options.signal,
|
||||||
);
|
);
|
||||||
|
|
||||||
await this.log.debug("OpenAI chat stream iteration finished", {
|
|
||||||
chunkCount: iteration.chunkCount,
|
|
||||||
contentDeltaCount: iteration.contentDeltaCount,
|
|
||||||
toolDeltaCount: iteration.toolDeltaCount,
|
|
||||||
responseLength: iteration.response.length,
|
|
||||||
thinkingLength: iteration.thinking?.length || 0,
|
|
||||||
toolCallCount: iteration.toolCalls.length,
|
|
||||||
finishReason: iteration.finishReason,
|
|
||||||
});
|
|
||||||
|
|
||||||
if (this.isEmptyIteration(iteration)) {
|
if (this.isEmptyIteration(iteration)) {
|
||||||
iteration = await this.readNonStreamingChatCompletion(model, messages, tools, options.signal);
|
iteration = await this.readNonStreamingChatCompletion(
|
||||||
|
model,
|
||||||
|
messages,
|
||||||
|
tools,
|
||||||
|
options.signal,
|
||||||
|
);
|
||||||
if (streamCallback && iteration.response) {
|
if (streamCallback && iteration.response) {
|
||||||
await streamCallback({ type: "response", data: iteration.response });
|
await streamCallback({ type: "response", data: iteration.response });
|
||||||
}
|
}
|
||||||
await this.log.warn("OpenAI stream was empty; used non-streaming fallback", {
|
|
||||||
responseLength: iteration.response.length,
|
|
||||||
thinkingLength: iteration.thinking?.length || 0,
|
|
||||||
toolCallCount: iteration.toolCalls.length,
|
|
||||||
finishReason: iteration.finishReason,
|
|
||||||
});
|
|
||||||
}
|
}
|
||||||
|
|
||||||
if (this.isEmptyIteration(iteration)) {
|
if (this.isEmptyIteration(iteration)) {
|
||||||
@ -325,7 +322,7 @@ export class OpenAiApi extends AiApi {
|
|||||||
thinking: undefined,
|
thinking: undefined,
|
||||||
toolCalls: undefined,
|
toolCalls: undefined,
|
||||||
toolCallResults: undefined,
|
toolCallResults: undefined,
|
||||||
stats: this.buildStats(startTime),
|
stats: this.buildStats(startTime, iteration.usage),
|
||||||
});
|
});
|
||||||
}
|
}
|
||||||
|
|
||||||
@ -333,20 +330,32 @@ export class OpenAiApi extends AiApi {
|
|||||||
done: true,
|
done: true,
|
||||||
response: iteration.response,
|
response: iteration.response,
|
||||||
thinking: iteration.thinking,
|
thinking: iteration.thinking,
|
||||||
toolCalls: iteration.toolCalls.length > 0 ? iteration.toolCalls : undefined,
|
toolCalls:
|
||||||
stats: this.buildStats(startTime),
|
iteration.toolCalls.length > 0 ? iteration.toolCalls : undefined,
|
||||||
|
stats: this.buildStats(startTime, iteration.usage),
|
||||||
};
|
};
|
||||||
this.assertNonEmptyChatResponse(finalResponse);
|
this.assertNonEmptyChatResponse(finalResponse);
|
||||||
return finalResponse;
|
return finalResponse;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
private extractUsage(usage: {
|
||||||
|
prompt_tokens: number;
|
||||||
|
completion_tokens: number;
|
||||||
|
completion_tokens_details?: { reasoning_tokens?: number };
|
||||||
|
}): OpenAiUsage {
|
||||||
|
return {
|
||||||
|
promptTokens: usage.prompt_tokens,
|
||||||
|
completionTokens: usage.completion_tokens,
|
||||||
|
reasoningTokens: usage.completion_tokens_details?.reasoning_tokens ?? 0,
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
private buildMessages(options: IAiChatOptions): ChatCompletionMessageParam[] {
|
private buildMessages(options: IAiChatOptions): ChatCompletionMessageParam[] {
|
||||||
const messages: ChatCompletionMessageParam[] = [];
|
const messages: ChatCompletionMessageParam[] = [];
|
||||||
if (options.systemPrompt) {
|
if (options.systemPrompt) {
|
||||||
messages.push({ role: "system", content: options.systemPrompt });
|
messages.push({ role: "system", content: options.systemPrompt });
|
||||||
}
|
}
|
||||||
for (const msg of options.context || []) {
|
for (const msg of options.context || []) {
|
||||||
if (!msg.content?.trim()) continue;
|
|
||||||
if (msg.role === "tool") {
|
if (msg.role === "tool") {
|
||||||
messages.push({
|
messages.push({
|
||||||
role: "tool",
|
role: "tool",
|
||||||
@ -355,7 +364,11 @@ export class OpenAiApi extends AiApi {
|
|||||||
});
|
});
|
||||||
continue;
|
continue;
|
||||||
}
|
}
|
||||||
if (msg.role === "assistant" && msg.toolCalls && msg.toolCalls.length > 0) {
|
if (
|
||||||
|
msg.role === "assistant" &&
|
||||||
|
msg.toolCalls &&
|
||||||
|
msg.toolCalls.length > 0
|
||||||
|
) {
|
||||||
messages.push({
|
messages.push({
|
||||||
role: "assistant",
|
role: "assistant",
|
||||||
content: msg.content,
|
content: msg.content,
|
||||||
@ -375,14 +388,16 @@ export class OpenAiApi extends AiApi {
|
|||||||
}
|
}
|
||||||
|
|
||||||
private buildTools(options: IAiChatOptions): ChatCompletionTool[] {
|
private buildTools(options: IAiChatOptions): ChatCompletionTool[] {
|
||||||
return (options.tools || []).map((tool): ChatCompletionFunctionTool => ({
|
return (options.tools || []).map(
|
||||||
|
(tool): ChatCompletionFunctionTool => ({
|
||||||
type: tool.definition.type,
|
type: tool.definition.type,
|
||||||
function: {
|
function: {
|
||||||
name: tool.definition.function.name,
|
name: tool.definition.function.name,
|
||||||
description: tool.definition.function.description,
|
description: tool.definition.function.description,
|
||||||
parameters: tool.definition.function.parameters,
|
parameters: tool.definition.function.parameters,
|
||||||
},
|
},
|
||||||
}));
|
}),
|
||||||
|
);
|
||||||
}
|
}
|
||||||
|
|
||||||
private async readStreamingChatCompletion(
|
private async readStreamingChatCompletion(
|
||||||
@ -396,11 +411,13 @@ export class OpenAiApi extends AiApi {
|
|||||||
throw new DOMException("The operation was aborted", "AbortError");
|
throw new DOMException("The operation was aborted", "AbortError");
|
||||||
}
|
}
|
||||||
|
|
||||||
const response = await this.client.chat.completions.create({
|
const response = await this.client.chat.completions.create(
|
||||||
|
{
|
||||||
model: model.modelId,
|
model: model.modelId,
|
||||||
messages,
|
messages,
|
||||||
tools,
|
tools,
|
||||||
stream: true,
|
stream: true,
|
||||||
|
stream_options: { include_usage: true },
|
||||||
...(model.params.maxCompletionTokens
|
...(model.params.maxCompletionTokens
|
||||||
? { max_completion_tokens: model.params.maxCompletionTokens }
|
? { max_completion_tokens: model.params.maxCompletionTokens }
|
||||||
: {}),
|
: {}),
|
||||||
@ -414,7 +431,9 @@ export class OpenAiApi extends AiApi {
|
|||||||
| "high",
|
| "high",
|
||||||
}
|
}
|
||||||
: {}),
|
: {}),
|
||||||
}, signal ? { signal } : undefined);
|
},
|
||||||
|
signal ? { signal } : undefined,
|
||||||
|
);
|
||||||
|
|
||||||
let content = "";
|
let content = "";
|
||||||
let thinking = "";
|
let thinking = "";
|
||||||
@ -423,6 +442,7 @@ export class OpenAiApi extends AiApi {
|
|||||||
let toolDeltaCount = 0;
|
let toolDeltaCount = 0;
|
||||||
let finishReason: string | null | undefined;
|
let finishReason: string | null | undefined;
|
||||||
const toolCallMap = new Map<number, StreamingToolCallAccumulator>();
|
const toolCallMap = new Map<number, StreamingToolCallAccumulator>();
|
||||||
|
let usage: OpenAiUsage | undefined;
|
||||||
|
|
||||||
for await (const chunk of response) {
|
for await (const chunk of response) {
|
||||||
if (signal?.aborted) {
|
if (signal?.aborted) {
|
||||||
@ -431,6 +451,12 @@ export class OpenAiApi extends AiApi {
|
|||||||
|
|
||||||
chunkCount++;
|
chunkCount++;
|
||||||
finishReason = chunk.choices[0]?.finish_reason ?? finishReason;
|
finishReason = chunk.choices[0]?.finish_reason ?? finishReason;
|
||||||
|
|
||||||
|
// Capture usage from the final streaming chunk
|
||||||
|
if (chunk.usage) {
|
||||||
|
usage = this.extractUsage(chunk.usage);
|
||||||
|
}
|
||||||
|
|
||||||
const delta = chunk.choices[0]?.delta;
|
const delta = chunk.choices[0]?.delta;
|
||||||
if (!delta) continue;
|
if (!delta) continue;
|
||||||
|
|
||||||
@ -444,7 +470,10 @@ export class OpenAiApi extends AiApi {
|
|||||||
if ("reasoning" in delta && delta.reasoning) {
|
if ("reasoning" in delta && delta.reasoning) {
|
||||||
thinking += delta.reasoning as string;
|
thinking += delta.reasoning as string;
|
||||||
if (streamCallback) {
|
if (streamCallback) {
|
||||||
await streamCallback({ type: "thinking", data: delta.reasoning as string });
|
await streamCallback({
|
||||||
|
type: "thinking",
|
||||||
|
data: delta.reasoning as string,
|
||||||
|
});
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
if (delta.tool_calls) {
|
if (delta.tool_calls) {
|
||||||
@ -463,6 +492,7 @@ export class OpenAiApi extends AiApi {
|
|||||||
chunkCount,
|
chunkCount,
|
||||||
contentDeltaCount,
|
contentDeltaCount,
|
||||||
toolDeltaCount,
|
toolDeltaCount,
|
||||||
|
usage,
|
||||||
};
|
};
|
||||||
}
|
}
|
||||||
|
|
||||||
@ -476,7 +506,8 @@ export class OpenAiApi extends AiApi {
|
|||||||
throw new DOMException("The operation was aborted", "AbortError");
|
throw new DOMException("The operation was aborted", "AbortError");
|
||||||
}
|
}
|
||||||
|
|
||||||
const response = await this.client.chat.completions.create({
|
const response = await this.client.chat.completions.create(
|
||||||
|
{
|
||||||
model: model.modelId,
|
model: model.modelId,
|
||||||
messages,
|
messages,
|
||||||
tools,
|
tools,
|
||||||
@ -494,10 +525,16 @@ export class OpenAiApi extends AiApi {
|
|||||||
| "high",
|
| "high",
|
||||||
}
|
}
|
||||||
: {}),
|
: {}),
|
||||||
}, signal ? { signal } : undefined);
|
},
|
||||||
|
signal ? { signal } : undefined,
|
||||||
|
);
|
||||||
|
|
||||||
const choice = response.choices[0];
|
const choice = response.choices[0];
|
||||||
const message = choice?.message;
|
const message = choice?.message;
|
||||||
const content = typeof message?.content === "string" ? message.content : "";
|
const content = typeof message?.content === "string" ? message.content : "";
|
||||||
|
const usage = response.usage
|
||||||
|
? this.extractUsage(response.usage)
|
||||||
|
: undefined;
|
||||||
const assistantToolCalls = (message?.tool_calls || [])
|
const assistantToolCalls = (message?.tool_calls || [])
|
||||||
.filter((tc) => tc.type === "function")
|
.filter((tc) => tc.type === "function")
|
||||||
.map((tc) => ({
|
.map((tc) => ({
|
||||||
@ -524,6 +561,7 @@ export class OpenAiApi extends AiApi {
|
|||||||
chunkCount: 0,
|
chunkCount: 0,
|
||||||
contentDeltaCount: content ? 1 : 0,
|
contentDeltaCount: content ? 1 : 0,
|
||||||
toolDeltaCount: assistantToolCalls.length,
|
toolDeltaCount: assistantToolCalls.length,
|
||||||
|
usage,
|
||||||
};
|
};
|
||||||
}
|
}
|
||||||
|
|
||||||
@ -592,7 +630,10 @@ export class OpenAiApi extends AiApi {
|
|||||||
);
|
);
|
||||||
}
|
}
|
||||||
|
|
||||||
private buildStats(startTime: number): IAiChatResponse["stats"] {
|
private buildStats(
|
||||||
|
startTime: number,
|
||||||
|
usage?: OpenAiUsage,
|
||||||
|
): IAiChatResponse["stats"] {
|
||||||
const seconds = (Date.now() - startTime) / 1000;
|
const seconds = (Date.now() - startTime) / 1000;
|
||||||
return {
|
return {
|
||||||
duration: {
|
duration: {
|
||||||
@ -600,10 +641,19 @@ export class OpenAiApi extends AiApi {
|
|||||||
text: numeral(seconds).format("hh:mm:ss"),
|
text: numeral(seconds).format("hh:mm:ss"),
|
||||||
},
|
},
|
||||||
tokenCounts: {
|
tokenCounts: {
|
||||||
input: 0,
|
input: usage?.promptTokens ?? 0,
|
||||||
response: 0,
|
response: usage?.completionTokens ?? 0,
|
||||||
thinking: 0,
|
thinking: usage?.reasoningTokens ?? 0,
|
||||||
},
|
},
|
||||||
};
|
};
|
||||||
}
|
}
|
||||||
|
|
||||||
|
async embeddings(modelId: string, text: string, dimensions?: number): Promise<IAiEmbeddingResponse> {
|
||||||
|
const response = await this.client.embeddings.create({
|
||||||
|
model: modelId,
|
||||||
|
input: text,
|
||||||
|
dimensions,
|
||||||
|
});
|
||||||
|
return { embedding: response.data[0]!.embedding, model: modelId };
|
||||||
|
}
|
||||||
}
|
}
|
||||||
|
|||||||
@ -36,6 +36,7 @@ export interface IChatSession {
|
|||||||
selectedModel: string;
|
selectedModel: string;
|
||||||
reasoningEffort?: ReasoningEffort;
|
reasoningEffort?: ReasoningEffort;
|
||||||
numCtx?: number;
|
numCtx?: number;
|
||||||
|
contextWindowUsage: number;
|
||||||
stats: {
|
stats: {
|
||||||
turnCount: number;
|
turnCount: number;
|
||||||
toolCallCount: number;
|
toolCallCount: number;
|
||||||
|
|||||||
@ -41,10 +41,23 @@ export type ToolCallMessage = (
|
|||||||
response: string,
|
response: string,
|
||||||
) => void;
|
) => void;
|
||||||
|
|
||||||
|
export interface IWorkOrderCompleteStats {
|
||||||
|
masterInputTokens: number;
|
||||||
|
masterOutputTokens: number;
|
||||||
|
masterThinkingTokens: number;
|
||||||
|
toolCallCount: number;
|
||||||
|
durationMs: number;
|
||||||
|
/** Aggregate (master + subagent) totals for session-level accounting */
|
||||||
|
totalInputTokens?: number;
|
||||||
|
totalOutputTokens?: number;
|
||||||
|
totalToolCallCount?: number;
|
||||||
|
}
|
||||||
|
|
||||||
export type WorkOrderCompleteMessage = (
|
export type WorkOrderCompleteMessage = (
|
||||||
workOrderId: string,
|
workOrderId: string,
|
||||||
success: boolean,
|
success: boolean,
|
||||||
message?: string,
|
message?: string,
|
||||||
|
stats?: IWorkOrderCompleteStats,
|
||||||
) => void;
|
) => void;
|
||||||
|
|
||||||
export type RequestCrashRecoveryMessage = (data: {
|
export type RequestCrashRecoveryMessage = (data: {
|
||||||
|
|||||||
@ -40,6 +40,15 @@ export interface GadgetCodeConfig {
|
|||||||
host: string;
|
host: string;
|
||||||
database: string;
|
database: string;
|
||||||
};
|
};
|
||||||
|
qdrant: {
|
||||||
|
host: string;
|
||||||
|
port?: number; // default: 6333
|
||||||
|
apiKey?: string; // optional API key for secured Qdrant
|
||||||
|
collection: string;
|
||||||
|
providerId: string; // AiProvider._id (GadgetId type not available here)
|
||||||
|
embeddingModel: string;
|
||||||
|
vectorSize: number;
|
||||||
|
};
|
||||||
redis: {
|
redis: {
|
||||||
host?: string;
|
host?: string;
|
||||||
port?: number;
|
port?: number;
|
||||||
|
|||||||
141
pnpm-lock.yaml
141
pnpm-lock.yaml
@ -25,6 +25,12 @@ importers:
|
|||||||
'@gadget/config':
|
'@gadget/config':
|
||||||
specifier: workspace:*
|
specifier: workspace:*
|
||||||
version: link:../packages/config
|
version: link:../packages/config
|
||||||
|
'@langchain/textsplitters':
|
||||||
|
specifier: ^1.0.1
|
||||||
|
version: 1.0.1(@langchain/core@1.1.47(openai@6.34.0(ws@8.18.3)(zod@4.4.3))(ws@8.18.3))
|
||||||
|
'@qdrant/js-client-rest':
|
||||||
|
specifier: ^1.12.0
|
||||||
|
version: 1.18.0(typescript@5.9.3)
|
||||||
'@react-three/drei':
|
'@react-three/drei':
|
||||||
specifier: ^10.7.7
|
specifier: ^10.7.7
|
||||||
version: 10.7.7(@react-three/fiber@9.6.1(@types/react@19.2.14)(react-dom@19.2.5(react@19.2.5))(react@19.2.5)(three@0.184.0))(@types/react@19.2.14)(@types/three@0.184.0)(react-dom@19.2.5(react@19.2.5))(react@19.2.5)(three@0.184.0)
|
version: 10.7.7(@react-three/fiber@9.6.1(@types/react@19.2.14)(react-dom@19.2.5(react@19.2.5))(react@19.2.5)(three@0.184.0))(@types/react@19.2.14)(@types/three@0.184.0)(react-dom@19.2.5(react@19.2.5))(react@19.2.5)(three@0.184.0)
|
||||||
@ -407,7 +413,7 @@ importers:
|
|||||||
version: 0.6.3
|
version: 0.6.3
|
||||||
openai:
|
openai:
|
||||||
specifier: ^6.34.0
|
specifier: ^6.34.0
|
||||||
version: 6.34.0(ws@8.18.3)
|
version: 6.34.0(ws@8.18.3)(zod@4.4.3)
|
||||||
playwright:
|
playwright:
|
||||||
specifier: 1.59.1
|
specifier: 1.59.1
|
||||||
version: 1.59.1
|
version: 1.59.1
|
||||||
@ -496,7 +502,7 @@ importers:
|
|||||||
version: 0.6.3
|
version: 0.6.3
|
||||||
openai:
|
openai:
|
||||||
specifier: ^6.34.0
|
specifier: ^6.34.0
|
||||||
version: 6.34.0(ws@8.18.3)
|
version: 6.34.0(ws@8.18.3)(zod@4.4.3)
|
||||||
devDependencies:
|
devDependencies:
|
||||||
'@types/node':
|
'@types/node':
|
||||||
specifier: ^25.6.0
|
specifier: ^25.6.0
|
||||||
@ -642,6 +648,9 @@ packages:
|
|||||||
resolution: {integrity: sha512-ctxtJ/eA+t+6q2++vj5j7FYX3nRu311q1wfYH3xjlLOsczhlhxAg2FWNUXhpGvAw3BWo1xBcvOV6/YLc2r5FJw==}
|
resolution: {integrity: sha512-ctxtJ/eA+t+6q2++vj5j7FYX3nRu311q1wfYH3xjlLOsczhlhxAg2FWNUXhpGvAw3BWo1xBcvOV6/YLc2r5FJw==}
|
||||||
hasBin: true
|
hasBin: true
|
||||||
|
|
||||||
|
'@cfworker/json-schema@4.1.1':
|
||||||
|
resolution: {integrity: sha512-gAmrUZSGtKc3AiBL71iNWxDsyUC5uMaKKGdvzYsBoTW/xi42JQHl7eKV2OYzCUqvc+D2RCcf7EXY2iCyFIk6og==}
|
||||||
|
|
||||||
'@codemirror/autocomplete@6.20.2':
|
'@codemirror/autocomplete@6.20.2':
|
||||||
resolution: {integrity: sha512-G5FPkgIiLjOgZMjqVjvuKQ1rGPtHogLldJr33eFJdVLtmwY+giGrlv/ewljLz6b9BSQLkjxuwBc6g6omDM+YxQ==}
|
resolution: {integrity: sha512-G5FPkgIiLjOgZMjqVjvuKQ1rGPtHogLldJr33eFJdVLtmwY+giGrlv/ewljLz6b9BSQLkjxuwBc6g6omDM+YxQ==}
|
||||||
|
|
||||||
@ -1246,6 +1255,16 @@ packages:
|
|||||||
'@kwsites/promise-deferred@1.1.1':
|
'@kwsites/promise-deferred@1.1.1':
|
||||||
resolution: {integrity: sha512-GaHYm+c0O9MjZRu0ongGBRbinu8gVAMd2UZjji6jVmqKtZluZnptXGWhz1E8j8D2HJ3f/yMxKAUC0b+57wncIw==}
|
resolution: {integrity: sha512-GaHYm+c0O9MjZRu0ongGBRbinu8gVAMd2UZjji6jVmqKtZluZnptXGWhz1E8j8D2HJ3f/yMxKAUC0b+57wncIw==}
|
||||||
|
|
||||||
|
'@langchain/core@1.1.47':
|
||||||
|
resolution: {integrity: sha512-+fiPu6ZFnJMrZyKeM77OIVPoMPAY6OKWacnPlojHtXTbMMzb2cEOKAJV0U07cDl86NHSCIYYa0i4CyKZzXbHQQ==}
|
||||||
|
engines: {node: '>=20'}
|
||||||
|
|
||||||
|
'@langchain/textsplitters@1.0.1':
|
||||||
|
resolution: {integrity: sha512-rheJlB01iVtrOUzttscutRgLybPH9qR79EyzBEbf1u97ljWyuxQfCwIWK+SjoQTM9O8M7GGLLRBSYE26Jmcoww==}
|
||||||
|
engines: {node: '>=20'}
|
||||||
|
peerDependencies:
|
||||||
|
'@langchain/core': ^1.0.0
|
||||||
|
|
||||||
'@lezer/common@1.5.2':
|
'@lezer/common@1.5.2':
|
||||||
resolution: {integrity: sha512-sxQE460fPZyU3sdc8lafxiPwJHBzZRy/udNFynGQky1SePYBdhkBl1kOagA9uT3pxR8K09bOrmTUqA9wb/PjSQ==}
|
resolution: {integrity: sha512-sxQE460fPZyU3sdc8lafxiPwJHBzZRy/udNFynGQky1SePYBdhkBl1kOagA9uT3pxR8K09bOrmTUqA9wb/PjSQ==}
|
||||||
|
|
||||||
@ -1374,6 +1393,16 @@ packages:
|
|||||||
engines: {node: '>=18'}
|
engines: {node: '>=18'}
|
||||||
hasBin: true
|
hasBin: true
|
||||||
|
|
||||||
|
'@qdrant/js-client-rest@1.18.0':
|
||||||
|
resolution: {integrity: sha512-/0dqX5uV9chC1DnYSnU4gNMrDqse/pt6hHg3Rqqpl5isH7xl1xSNvffjzBoxycDD79luWn7Ho6Rh/61sOs5DNw==}
|
||||||
|
engines: {node: '>=18.17.0', pnpm: '>=8'}
|
||||||
|
peerDependencies:
|
||||||
|
typescript: '>=4.7'
|
||||||
|
|
||||||
|
'@qdrant/openapi-typescript-fetch@1.2.6':
|
||||||
|
resolution: {integrity: sha512-oQG/FejNpItrxRHoyctYvT3rwGZOnK4jr3JdppO/c78ktDvkWiPXPHNsrDf33K9sZdRb6PR7gi4noIapu5q4HA==}
|
||||||
|
engines: {node: '>=18.0.0', pnpm: '>=8'}
|
||||||
|
|
||||||
'@react-three/drei@10.7.7':
|
'@react-three/drei@10.7.7':
|
||||||
resolution: {integrity: sha512-ff+J5iloR0k4tC++QtD/j9u3w5fzfgFAWDtAGQah9pF2B1YgOq/5JxqY0/aVoQG5r3xSZz0cv5tk2YuBob4xEQ==}
|
resolution: {integrity: sha512-ff+J5iloR0k4tC++QtD/j9u3w5fzfgFAWDtAGQah9pF2B1YgOq/5JxqY0/aVoQG5r3xSZz0cv5tk2YuBob4xEQ==}
|
||||||
peerDependencies:
|
peerDependencies:
|
||||||
@ -2859,6 +2888,9 @@ packages:
|
|||||||
js-stringify@1.0.2:
|
js-stringify@1.0.2:
|
||||||
resolution: {integrity: sha512-rtS5ATOo2Q5k1G+DADISilDA6lv79zIiwFd6CcjuIxGKLFm5C+RLImRscVap9k55i+MOZwgliw+NejvkLuGD5g==}
|
resolution: {integrity: sha512-rtS5ATOo2Q5k1G+DADISilDA6lv79zIiwFd6CcjuIxGKLFm5C+RLImRscVap9k55i+MOZwgliw+NejvkLuGD5g==}
|
||||||
|
|
||||||
|
js-tiktoken@1.0.21:
|
||||||
|
resolution: {integrity: sha512-biOj/6M5qdgx5TKjDnFT1ymSpM5tbd3ylwDtrQvFQSu0Z7bBYko2dF+W/aUkXUPuk6IVpRxk/3Q2sHOzGlS36g==}
|
||||||
|
|
||||||
js-tokens@4.0.0:
|
js-tokens@4.0.0:
|
||||||
resolution: {integrity: sha512-RdJUflcE3cUzKiMqQgsCu06FPu9UdIJO0beYbPhHN4k6apgJtifcoCtT9bcxOpYBtpD2kCM6Sbzg4CausW/PKQ==}
|
resolution: {integrity: sha512-RdJUflcE3cUzKiMqQgsCu06FPu9UdIJO0beYbPhHN4k6apgJtifcoCtT9bcxOpYBtpD2kCM6Sbzg4CausW/PKQ==}
|
||||||
|
|
||||||
@ -2910,6 +2942,26 @@ packages:
|
|||||||
resolution: {integrity: sha512-C3iHfuGUXK2u8/ipq9LfjFfXFxAZMQJJq7vLS45r3D9Y2xQ/m4S8zaR4zMLFWh9AsNPXmcFfUDhTEO8UIC/V6Q==}
|
resolution: {integrity: sha512-C3iHfuGUXK2u8/ipq9LfjFfXFxAZMQJJq7vLS45r3D9Y2xQ/m4S8zaR4zMLFWh9AsNPXmcFfUDhTEO8UIC/V6Q==}
|
||||||
engines: {node: '>=12.0.0'}
|
engines: {node: '>=12.0.0'}
|
||||||
|
|
||||||
|
langsmith@0.7.1:
|
||||||
|
resolution: {integrity: sha512-Wjk90UjNoY5cBHMlNAC/eZx5clI8jnjBOBW8uJu8+MWBtx0QesNjsUiLtjI+I3UnrpxFFpDqGXcnhBjH654Mqg==}
|
||||||
|
peerDependencies:
|
||||||
|
'@opentelemetry/api': '*'
|
||||||
|
'@opentelemetry/exporter-trace-otlp-proto': '*'
|
||||||
|
'@opentelemetry/sdk-trace-base': '*'
|
||||||
|
openai: '*'
|
||||||
|
ws: '>=7'
|
||||||
|
peerDependenciesMeta:
|
||||||
|
'@opentelemetry/api':
|
||||||
|
optional: true
|
||||||
|
'@opentelemetry/exporter-trace-otlp-proto':
|
||||||
|
optional: true
|
||||||
|
'@opentelemetry/sdk-trace-base':
|
||||||
|
optional: true
|
||||||
|
openai:
|
||||||
|
optional: true
|
||||||
|
ws:
|
||||||
|
optional: true
|
||||||
|
|
||||||
lazy@1.0.11:
|
lazy@1.0.11:
|
||||||
resolution: {integrity: sha512-Y+CjUfLmIpoUCCRl0ub4smrYtGGr5AOa2AKOaWelGHOGz33X/Y/KizefGqbkwfz44+cnq/+9habclf8vOmu2LA==}
|
resolution: {integrity: sha512-Y+CjUfLmIpoUCCRl0ub4smrYtGGr5AOa2AKOaWelGHOGz33X/Y/KizefGqbkwfz44+cnq/+9habclf8vOmu2LA==}
|
||||||
engines: {node: '>=0.2.0'}
|
engines: {node: '>=0.2.0'}
|
||||||
@ -3215,6 +3267,10 @@ packages:
|
|||||||
resolution: {integrity: sha512-mo+QTzKlx8R7E5ylSXxWzGoXoZbOsRMpyitcht8By2KHvMbf3tjwosZ/Mu/XYU6UuJ3VZnODIrak5ZrPiPyB6A==}
|
resolution: {integrity: sha512-mo+QTzKlx8R7E5ylSXxWzGoXoZbOsRMpyitcht8By2KHvMbf3tjwosZ/Mu/XYU6UuJ3VZnODIrak5ZrPiPyB6A==}
|
||||||
engines: {node: '>= 10.16.0'}
|
engines: {node: '>= 10.16.0'}
|
||||||
|
|
||||||
|
mustache@4.2.0:
|
||||||
|
resolution: {integrity: sha512-71ippSywq5Yb7/tVYyGbkBggbU8H3u5Rz56fH60jGFgr8uHwxs+aSKeqmluIVzM0m0kB7xQjKS6qPfd0b2ZoqQ==}
|
||||||
|
hasBin: true
|
||||||
|
|
||||||
mute-stream@3.0.0:
|
mute-stream@3.0.0:
|
||||||
resolution: {integrity: sha512-dkEJPVvun4FryqBmZ5KhDo0K9iDXAwn08tMLDinNdRBNPcYEDiWYysLcc6k3mjTMlbP9KyylvRpd4wFtwrT9rw==}
|
resolution: {integrity: sha512-dkEJPVvun4FryqBmZ5KhDo0K9iDXAwn08tMLDinNdRBNPcYEDiWYysLcc6k3mjTMlbP9KyylvRpd4wFtwrT9rw==}
|
||||||
engines: {node: ^20.17.0 || >=22.9.0}
|
engines: {node: ^20.17.0 || >=22.9.0}
|
||||||
@ -3334,6 +3390,18 @@ packages:
|
|||||||
resolution: {integrity: sha512-PRT7ZORmwu2MEFt4/fv3Q+mEfN4zetKxufQrkShY2oGvUms9r8otu5HfdyIFHkYXjO7laNsoVGmM2MANfuTA8g==}
|
resolution: {integrity: sha512-PRT7ZORmwu2MEFt4/fv3Q+mEfN4zetKxufQrkShY2oGvUms9r8otu5HfdyIFHkYXjO7laNsoVGmM2MANfuTA8g==}
|
||||||
engines: {node: '>=0.10.0'}
|
engines: {node: '>=0.10.0'}
|
||||||
|
|
||||||
|
p-finally@1.0.0:
|
||||||
|
resolution: {integrity: sha512-LICb2p9CB7FS+0eR1oqWnHhp0FljGLZCWBE9aix0Uye9W8LTQPwMTYVGWQWIw9RdQiDg4+epXQODwIYJtSJaow==}
|
||||||
|
engines: {node: '>=4'}
|
||||||
|
|
||||||
|
p-queue@6.6.2:
|
||||||
|
resolution: {integrity: sha512-RwFpb72c/BhQLEXIZ5K2e+AhgNVmIejGlTgiB9MzZ0e93GRvqZ7uSi0dvRF7/XIXDeNkra2fNHBxTyPDGySpjQ==}
|
||||||
|
engines: {node: '>=8'}
|
||||||
|
|
||||||
|
p-timeout@3.2.0:
|
||||||
|
resolution: {integrity: sha512-rhIwUycgwwKcP9yTOOFK/AKsAopjjCakVqLHePO3CC6Mir1Z99xT+R63jZxAT5lFZLa2inS5h+ZS2GvR99/FBg==}
|
||||||
|
engines: {node: '>=8'}
|
||||||
|
|
||||||
parse-node-version@1.0.1:
|
parse-node-version@1.0.1:
|
||||||
resolution: {integrity: sha512-3YHlOa/JgH6Mnpr05jP9eDG254US9ek25LyIxZlDItp2iJtwyaXQb57lBYLdT3MowkUFYEV2XXNAYIPlESvJlA==}
|
resolution: {integrity: sha512-3YHlOa/JgH6Mnpr05jP9eDG254US9ek25LyIxZlDItp2iJtwyaXQb57lBYLdT3MowkUFYEV2XXNAYIPlESvJlA==}
|
||||||
engines: {node: '>= 0.10'}
|
engines: {node: '>= 0.10'}
|
||||||
@ -4018,6 +4086,10 @@ packages:
|
|||||||
undici-types@7.19.2:
|
undici-types@7.19.2:
|
||||||
resolution: {integrity: sha512-qYVnV5OEm2AW8cJMCpdV20CDyaN3g0AjDlOGf1OW4iaDEx8MwdtChUp4zu4H0VP3nDRF/8RKWH+IPp9uW0YGZg==}
|
resolution: {integrity: sha512-qYVnV5OEm2AW8cJMCpdV20CDyaN3g0AjDlOGf1OW4iaDEx8MwdtChUp4zu4H0VP3nDRF/8RKWH+IPp9uW0YGZg==}
|
||||||
|
|
||||||
|
undici@6.25.0:
|
||||||
|
resolution: {integrity: sha512-ZgpWDC5gmNiuY9CnLVXEH8rl50xhRCuLNA97fAUnKi8RRuV4E6KG31pDTsLVUKnohJE0I3XDrTeEydAXRw47xg==}
|
||||||
|
engines: {node: '>=18.17'}
|
||||||
|
|
||||||
undici@7.25.0:
|
undici@7.25.0:
|
||||||
resolution: {integrity: sha512-xXnp4kTyor2Zq+J1FfPI6Eq3ew5h6Vl0F/8d9XU5zZQf1tX9s2Su1/3PiMmUANFULpmksxkClamIZcaUqryHsQ==}
|
resolution: {integrity: sha512-xXnp4kTyor2Zq+J1FfPI6Eq3ew5h6Vl0F/8d9XU5zZQf1tX9s2Su1/3PiMmUANFULpmksxkClamIZcaUqryHsQ==}
|
||||||
engines: {node: '>=20.18.1'}
|
engines: {node: '>=20.18.1'}
|
||||||
@ -4281,6 +4353,9 @@ packages:
|
|||||||
yauzl@2.10.0:
|
yauzl@2.10.0:
|
||||||
resolution: {integrity: sha512-p4a9I6X6nu6IhoGmBqAcbJy1mlC4j27vEPZX9F4L4/vZT3Lyq1VkFHw/V/PUcB9Buo+DG3iHkT0x3Qya58zc3g==}
|
resolution: {integrity: sha512-p4a9I6X6nu6IhoGmBqAcbJy1mlC4j27vEPZX9F4L4/vZT3Lyq1VkFHw/V/PUcB9Buo+DG3iHkT0x3Qya58zc3g==}
|
||||||
|
|
||||||
|
zod@4.4.3:
|
||||||
|
resolution: {integrity: sha512-ytENFjIJFl2UwYglde2jchW2Hwm4GJFLDiSXWdTrJQBIN9Fcyp7n4DhxJEiWNAJMV1/BqWfW/kkg71UDcHJyTQ==}
|
||||||
|
|
||||||
zustand@4.5.7:
|
zustand@4.5.7:
|
||||||
resolution: {integrity: sha512-CHOUy7mu3lbD6o6LJLfllpjkzhHXSBlX8B9+qPddUsIfeF5S/UZ5q0kmCsnRqT1UHFQZchNFDDzMbQsuesHWlw==}
|
resolution: {integrity: sha512-CHOUy7mu3lbD6o6LJLfllpjkzhHXSBlX8B9+qPddUsIfeF5S/UZ5q0kmCsnRqT1UHFQZchNFDDzMbQsuesHWlw==}
|
||||||
engines: {node: '>=12.7.0'}
|
engines: {node: '>=12.7.0'}
|
||||||
@ -4365,6 +4440,8 @@ snapshots:
|
|||||||
dependencies:
|
dependencies:
|
||||||
css-tree: 3.2.1
|
css-tree: 3.2.1
|
||||||
|
|
||||||
|
'@cfworker/json-schema@4.1.1': {}
|
||||||
|
|
||||||
'@codemirror/autocomplete@6.20.2':
|
'@codemirror/autocomplete@6.20.2':
|
||||||
dependencies:
|
dependencies:
|
||||||
'@codemirror/language': 6.12.3
|
'@codemirror/language': 6.12.3
|
||||||
@ -4890,6 +4967,27 @@ snapshots:
|
|||||||
|
|
||||||
'@kwsites/promise-deferred@1.1.1': {}
|
'@kwsites/promise-deferred@1.1.1': {}
|
||||||
|
|
||||||
|
'@langchain/core@1.1.47(openai@6.34.0(ws@8.18.3)(zod@4.4.3))(ws@8.18.3)':
|
||||||
|
dependencies:
|
||||||
|
'@cfworker/json-schema': 4.1.1
|
||||||
|
'@standard-schema/spec': 1.1.0
|
||||||
|
js-tiktoken: 1.0.21
|
||||||
|
langsmith: 0.7.1(openai@6.34.0(ws@8.18.3)(zod@4.4.3))(ws@8.18.3)
|
||||||
|
mustache: 4.2.0
|
||||||
|
p-queue: 6.6.2
|
||||||
|
zod: 4.4.3
|
||||||
|
transitivePeerDependencies:
|
||||||
|
- '@opentelemetry/api'
|
||||||
|
- '@opentelemetry/exporter-trace-otlp-proto'
|
||||||
|
- '@opentelemetry/sdk-trace-base'
|
||||||
|
- openai
|
||||||
|
- ws
|
||||||
|
|
||||||
|
'@langchain/textsplitters@1.0.1(@langchain/core@1.1.47(openai@6.34.0(ws@8.18.3)(zod@4.4.3))(ws@8.18.3))':
|
||||||
|
dependencies:
|
||||||
|
'@langchain/core': 1.1.47(openai@6.34.0(ws@8.18.3)(zod@4.4.3))(ws@8.18.3)
|
||||||
|
js-tiktoken: 1.0.21
|
||||||
|
|
||||||
'@lezer/common@1.5.2': {}
|
'@lezer/common@1.5.2': {}
|
||||||
|
|
||||||
'@lezer/cpp@1.1.5':
|
'@lezer/cpp@1.1.5':
|
||||||
@ -5039,6 +5137,14 @@ snapshots:
|
|||||||
dependencies:
|
dependencies:
|
||||||
playwright: 1.59.1
|
playwright: 1.59.1
|
||||||
|
|
||||||
|
'@qdrant/js-client-rest@1.18.0(typescript@5.9.3)':
|
||||||
|
dependencies:
|
||||||
|
'@qdrant/openapi-typescript-fetch': 1.2.6
|
||||||
|
typescript: 5.9.3
|
||||||
|
undici: 6.25.0
|
||||||
|
|
||||||
|
'@qdrant/openapi-typescript-fetch@1.2.6': {}
|
||||||
|
|
||||||
'@react-three/drei@10.7.7(@react-three/fiber@9.6.1(@types/react@19.2.14)(react-dom@19.2.5(react@19.2.5))(react@19.2.5)(three@0.184.0))(@types/react@19.2.14)(@types/three@0.184.0)(react-dom@19.2.5(react@19.2.5))(react@19.2.5)(three@0.184.0)':
|
'@react-three/drei@10.7.7(@react-three/fiber@9.6.1(@types/react@19.2.14)(react-dom@19.2.5(react@19.2.5))(react@19.2.5)(three@0.184.0))(@types/react@19.2.14)(@types/three@0.184.0)(react-dom@19.2.5(react@19.2.5))(react@19.2.5)(three@0.184.0)':
|
||||||
dependencies:
|
dependencies:
|
||||||
'@babel/runtime': 7.29.2
|
'@babel/runtime': 7.29.2
|
||||||
@ -6639,6 +6745,10 @@ snapshots:
|
|||||||
|
|
||||||
js-stringify@1.0.2: {}
|
js-stringify@1.0.2: {}
|
||||||
|
|
||||||
|
js-tiktoken@1.0.21:
|
||||||
|
dependencies:
|
||||||
|
base64-js: 1.5.1
|
||||||
|
|
||||||
js-tokens@4.0.0: {}
|
js-tokens@4.0.0: {}
|
||||||
|
|
||||||
js-yaml@4.1.1:
|
js-yaml@4.1.1:
|
||||||
@ -6738,6 +6848,13 @@ snapshots:
|
|||||||
|
|
||||||
kareem@2.6.3: {}
|
kareem@2.6.3: {}
|
||||||
|
|
||||||
|
langsmith@0.7.1(openai@6.34.0(ws@8.18.3)(zod@4.4.3))(ws@8.18.3):
|
||||||
|
dependencies:
|
||||||
|
p-queue: 6.6.2
|
||||||
|
optionalDependencies:
|
||||||
|
openai: 6.34.0(ws@8.18.3)(zod@4.4.3)
|
||||||
|
ws: 8.18.3
|
||||||
|
|
||||||
lazy@1.0.11: {}
|
lazy@1.0.11: {}
|
||||||
|
|
||||||
lcid@1.0.0:
|
lcid@1.0.0:
|
||||||
@ -7010,6 +7127,8 @@ snapshots:
|
|||||||
concat-stream: 2.0.0
|
concat-stream: 2.0.0
|
||||||
type-is: 1.6.18
|
type-is: 1.6.18
|
||||||
|
|
||||||
|
mustache@4.2.0: {}
|
||||||
|
|
||||||
mute-stream@3.0.0: {}
|
mute-stream@3.0.0: {}
|
||||||
|
|
||||||
mylas@2.1.14: {}
|
mylas@2.1.14: {}
|
||||||
@ -7084,9 +7203,10 @@ snapshots:
|
|||||||
dependencies:
|
dependencies:
|
||||||
wrappy: 1.0.2
|
wrappy: 1.0.2
|
||||||
|
|
||||||
openai@6.34.0(ws@8.18.3):
|
openai@6.34.0(ws@8.18.3)(zod@4.4.3):
|
||||||
optionalDependencies:
|
optionalDependencies:
|
||||||
ws: 8.18.3
|
ws: 8.18.3
|
||||||
|
zod: 4.4.3
|
||||||
|
|
||||||
opn@5.3.0:
|
opn@5.3.0:
|
||||||
dependencies:
|
dependencies:
|
||||||
@ -7096,6 +7216,17 @@ snapshots:
|
|||||||
dependencies:
|
dependencies:
|
||||||
lcid: 1.0.0
|
lcid: 1.0.0
|
||||||
|
|
||||||
|
p-finally@1.0.0: {}
|
||||||
|
|
||||||
|
p-queue@6.6.2:
|
||||||
|
dependencies:
|
||||||
|
eventemitter3: 4.0.7
|
||||||
|
p-timeout: 3.2.0
|
||||||
|
|
||||||
|
p-timeout@3.2.0:
|
||||||
|
dependencies:
|
||||||
|
p-finally: 1.0.0
|
||||||
|
|
||||||
parse-node-version@1.0.1: {}
|
parse-node-version@1.0.1: {}
|
||||||
|
|
||||||
parse5@7.3.0:
|
parse5@7.3.0:
|
||||||
@ -7845,6 +7976,8 @@ snapshots:
|
|||||||
|
|
||||||
undici-types@7.19.2: {}
|
undici-types@7.19.2: {}
|
||||||
|
|
||||||
|
undici@6.25.0: {}
|
||||||
|
|
||||||
undici@7.25.0: {}
|
undici@7.25.0: {}
|
||||||
|
|
||||||
undici@8.1.0: {}
|
undici@8.1.0: {}
|
||||||
@ -8103,6 +8236,8 @@ snapshots:
|
|||||||
buffer-crc32: 0.2.13
|
buffer-crc32: 0.2.13
|
||||||
fd-slicer: 1.1.0
|
fd-slicer: 1.1.0
|
||||||
|
|
||||||
|
zod@4.4.3: {}
|
||||||
|
|
||||||
zustand@4.5.7(@types/react@19.2.14)(react@19.2.5):
|
zustand@4.5.7(@types/react@19.2.14)(react@19.2.5):
|
||||||
dependencies:
|
dependencies:
|
||||||
use-sync-external-store: 1.6.0(react@19.2.5)
|
use-sync-external-store: 1.6.0(react@19.2.5)
|
||||||
|
|||||||
22
repairs.md
Normal file
22
repairs.md
Normal file
@ -0,0 +1,22 @@
|
|||||||
|
Summary of Changes (6 files, +36/-19 lines)
|
||||||
|
Root Cause
|
||||||
|
The token-economics commit (00969cf) correctly added subagent token aggregation to the DB write path (drone-session.ts), but the in-memory and frontend paths only used master-only values. This meant:
|
||||||
|
-
|
||||||
|
During a session: UI showed lower token counts than reality
|
||||||
|
-
|
||||||
|
After page refresh: numbers jumped up (DB values loaded)
|
||||||
|
-
|
||||||
|
Session toolCallCount missed subagent tool calls entirely
|
||||||
|
Fixes
|
||||||
|
1. packages/api/src/messages/drone.ts — Extended IWorkOrderCompleteStats with optional aggregate fields (totalInputTokens?, totalOutputTokens?, totalToolCallCount?)
|
||||||
|
2. gadget-code/src/lib/drone-session.ts — Now computes totalToolCallCount (master + subagent tool calls) alongside existing token aggregates, passes enrichedStats (with aggregate fields populated) to codeSession.onWorkOrderComplete()
|
||||||
|
3. gadget-code/src/lib/code-session.ts — In-memory session stats use stats.totalInputTokens ?? stats.masterInputTokens (aggregate when available, master fallback)
|
||||||
|
4. gadget-code/frontend/src/lib/api.ts — Frontend type matches the shared interface
|
||||||
|
5. gadget-code/frontend/src/pages/ChatSessionView.tsx — Frontend session stats update uses aggregate values; also now correctly increments toolCallCount (was previously missing)
|
||||||
|
6. gadget-drone/src/services/agent.ts — Three fixes:
|
||||||
|
-
|
||||||
|
Removed local IWorkOrderCompleteStats redefinition, now imports from @gadget/api
|
||||||
|
-
|
||||||
|
Master loop token accumulation uses optional chaining (response.stats?.tokenCounts?.input ?? 0) matching the subagent loop
|
||||||
|
-
|
||||||
|
agent:complete emit now includes stats field from subagent data
|
||||||
Loading…
Reference in New Issue
Block a user