docs: add gadget-tasks documentation, update README and agent-toolbox
- README.md: Added gadget-tasks and @gadget/ai-toolbox to projects table, added Scheduled Tasks architecture section with diagram, updated monorepo structure, added gadget-tasks to dev server instructions, added doc link - docs/agent-toolbox.md: Updated to reflect @gadget/ai-toolbox extraction from @gadget/ai. Changed IAiEnvironment → GadgetToolboxEnvironment, AiTool → GadgetTool, updated import paths, config examples, and added Tool Categories section. Clarified gadget-tasks is NOT a consumer. - docs/gadget-tasks.md: New documentation covering architecture, per-task execution flow, configuration, startup/shutdown sequences, concurrency, work order tracking, heartbeat, error recovery, and getting started.
This commit is contained in:
parent
f742a1d765
commit
4d8f403d78
51
README.md
51
README.md
@ -22,11 +22,14 @@ pnpm dev
|
|||||||
## Projects
|
## Projects
|
||||||
|
|
||||||
| Package | Role |
|
| Package | Role |
|
||||||
| -------------- | ------------------------------------------------------------------------ |
|
| ------------------ | ------------------------------------------------------------------------ |
|
||||||
| `gadget-code` | Web service — agentic IDE, browser UI, API server |
|
| `gadget-code` | Web service — agentic IDE, browser UI, API server |
|
||||||
| `gadget-drone` | Worker process — runs the agentic workflow loop in workspace directories |
|
| `gadget-drone` | Worker process — runs the agentic workflow loop in workspace directories |
|
||||||
|
| `gadget-tasks` | Scheduled task worker — headless IDE client for cron-driven tasks |
|
||||||
| `@gadget/ai` | Shared AI API abstraction — Ollama and OpenAI |
|
| `@gadget/ai` | Shared AI API abstraction — Ollama and OpenAI |
|
||||||
|
| `@gadget/ai-toolbox` | Shared AI tool implementations — search, file, plan, subagent tools |
|
||||||
| `@gadget/api` | Shared TypeScript interfaces — common types across all packages |
|
| `@gadget/api` | Shared TypeScript interfaces — common types across all packages |
|
||||||
|
| `@gadget/config` | Shared YAML config loader — per-package configuration |
|
||||||
|
|
||||||
## Architecture
|
## Architecture
|
||||||
|
|
||||||
@ -50,6 +53,30 @@ pnpm dev
|
|||||||
6. **Streaming response** flows back: thinking → response → tool calls
|
6. **Streaming response** flows back: thinking → response → tool calls
|
||||||
7. **Results persist** in MongoDB as ChatTurn records
|
7. **Results persist** in MongoDB as ChatTurn records
|
||||||
|
|
||||||
|
### Scheduled Tasks (gadget-tasks)
|
||||||
|
|
||||||
|
gadget-tasks is a **headless IDE client** that automates the browser IDE flow on a cron schedule. It does not duplicate AI, database, or workspace code — it drives the existing gadget-code platform via REST API and Socket.IO, the same protocol the browser IDE uses.
|
||||||
|
|
||||||
|
```
|
||||||
|
┌──────────────┐ ┌──────────────┐ ┌──────────────┐
|
||||||
|
│ gadget-tasks │────▶│ gadget-code │────▶│ gadget-drone │
|
||||||
|
│ (Scheduler) │◀────│ (Express 5) │◀────│ (Worker) │
|
||||||
|
└──────────────┘ └──────────────┘ └──────────────┘
|
||||||
|
│ │ │
|
||||||
|
│ REST + Socket.IO │ MongoDB │ Files
|
||||||
|
│ JWT Auth │ Redis │ Git
|
||||||
|
```
|
||||||
|
|
||||||
|
**Per-task flow:**
|
||||||
|
1. CronJob fires → create ChatSession via REST API
|
||||||
|
2. Lock drone via Socket.IO `requestSessionLock`
|
||||||
|
3. Set workspace mode to Agent via `requestWorkspaceMode`
|
||||||
|
4. Submit task prompt via `submitPrompt` → drone processes work order
|
||||||
|
5. Receive `workOrderComplete` → release session lock
|
||||||
|
6. Update `task.lastRun` via `PATCH /projects/:id/tasks/:taskId/lastRun`
|
||||||
|
|
||||||
|
See [gadget-tasks documentation](./docs/gadget-tasks.md) for setup and usage.
|
||||||
|
|
||||||
## AI Provider Setup
|
## AI Provider Setup
|
||||||
|
|
||||||
Before using AI features, add a provider via CLI:
|
Before using AI features, add a provider via CLI:
|
||||||
@ -120,6 +147,21 @@ pnpm dev
|
|||||||
# Drone worker (in a workspace directory)
|
# Drone worker (in a workspace directory)
|
||||||
cd ~/my-gadget-workspace
|
cd ~/my-gadget-workspace
|
||||||
pnpm --filter gadget-drone dev
|
pnpm --filter gadget-drone dev
|
||||||
|
|
||||||
|
# Task scheduler (requires running gadget-code + drone)
|
||||||
|
cd gadget-tasks
|
||||||
|
pnpm dev -- --user=admin@example.com --password=secret
|
||||||
|
```
|
||||||
|
|
||||||
|
### Global Install (CLI)
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Link gadget-drone and gadget-tasks globally for CLI use
|
||||||
|
pnpm link:global
|
||||||
|
|
||||||
|
# Now available as system commands
|
||||||
|
gadget-drone
|
||||||
|
gadget-tasks --user=admin@example.com --password=secret
|
||||||
```
|
```
|
||||||
|
|
||||||
### Testing
|
### Testing
|
||||||
@ -183,13 +225,16 @@ See [`packages/ai/README.md`](./packages/ai/README.md) for the full API referenc
|
|||||||
- [Workspace Management](./docs/gadget-workspace.md)
|
- [Workspace Management](./docs/gadget-workspace.md)
|
||||||
- [Drone Documentation](./gadget-drone/docs/gadget-drone.md)
|
- [Drone Documentation](./gadget-drone/docs/gadget-drone.md)
|
||||||
- [UI Design Guide](./gadget-code/docs/ui-design-guide.md)
|
- [UI Design Guide](./gadget-code/docs/ui-design-guide.md)
|
||||||
|
- [gadget-tasks Documentation](./docs/gadget-tasks.md)
|
||||||
|
|
||||||
## Monorepo Structure
|
## Monorepo Structure
|
||||||
|
|
||||||
```
|
```
|
||||||
gadget/
|
gadget/
|
||||||
├── packages/ai/ # @gadget/ai — AI API abstraction
|
├── packages/ai/ # @gadget/ai — AI API abstraction
|
||||||
|
├── packages/ai-toolbox/ # @gadget/ai-toolbox — Shared tool implementations
|
||||||
├── packages/api/ # @gadget/api — Shared interfaces
|
├── packages/api/ # @gadget/api — Shared interfaces
|
||||||
|
├── packages/config/ # @gadget/config — YAML config loader
|
||||||
├── gadget-code/ # Web service + browser IDE
|
├── gadget-code/ # Web service + browser IDE
|
||||||
│ ├── src/ # Backend (Express, Socket.IO, Mongoose)
|
│ ├── src/ # Backend (Express, Socket.IO, Mongoose)
|
||||||
│ ├── frontend/ # Frontend (React, Vite, Tailwind)
|
│ ├── frontend/ # Frontend (React, Vite, Tailwind)
|
||||||
@ -197,6 +242,8 @@ gadget/
|
|||||||
├── gadget-drone/ # Worker process
|
├── gadget-drone/ # Worker process
|
||||||
│ ├── src/ # Drone implementation
|
│ ├── src/ # Drone implementation
|
||||||
│ └── docs/ # Drone documentation
|
│ └── docs/ # Drone documentation
|
||||||
|
├── gadget-tasks/ # Scheduled task worker
|
||||||
|
│ └── src/ # Headless IDE client + cron scheduler
|
||||||
└── docs/ # Architecture & protocol docs
|
└── docs/ # Architecture & protocol docs
|
||||||
```
|
```
|
||||||
|
|
||||||
@ -207,4 +254,4 @@ Apache 2.0 — See [LICENSE](./LICENSE) for details.
|
|||||||
---
|
---
|
||||||
|
|
||||||
**Status:** Production-ready foundation with complete Chat Session UI
|
**Status:** Production-ready foundation with complete Chat Session UI
|
||||||
**Last Updated:** April 29, 2026
|
**Last Updated:** May 17, 2026
|
||||||
|
|||||||
@ -2,9 +2,9 @@
|
|||||||
|
|
||||||
## Overview
|
## Overview
|
||||||
|
|
||||||
The Agent Toolbox is a foundational component of the Gadget platform that enables AI agents to perform real-world actions through a standardized, extensible tool system. Tools are functions that agents can invoke to interact with external services, manipulate files, search the web, and execute various operations.
|
The Agent Toolbox (`@gadget/ai-toolbox`) is a shared package providing AI agent tool implementations for the Gadget platform. It was extracted from `@gadget/ai` into its own package so that agent tools can be maintained independently of the AI provider abstraction layer.
|
||||||
|
|
||||||
## Architecture
|
**Consumers:** `gadget-drone` (the only current consumer). gadget-tasks does **not** use `@gadget/ai-toolbox` — it drives the gadget-code platform via REST API and Socket.IO as a headless IDE client, delegating all AI and tool execution to drones.
|
||||||
|
|
||||||
### Design Philosophy
|
### Design Philosophy
|
||||||
|
|
||||||
@ -15,13 +15,14 @@ The toolbox system is built on these core principles:
|
|||||||
3. **Extensibility**: Easy to add new tools without modifying core infrastructure
|
3. **Extensibility**: Easy to add new tools without modifying core infrastructure
|
||||||
4. **Security**: Tools require explicit credentials configured in the environment
|
4. **Security**: Tools require explicit credentials configured in the environment
|
||||||
5. **Error Handling**: Comprehensive error reporting with recovery hints
|
5. **Error Handling**: Comprehensive error reporting with recovery hints
|
||||||
|
6. **Shared Implementation**: One tool codebase used by all agent processes
|
||||||
|
|
||||||
### Component Overview
|
### Component Overview
|
||||||
|
|
||||||
```
|
```
|
||||||
┌─────────────────┐
|
┌─────────────────┐
|
||||||
│ AiToolbox │ ← Manages tool registration and lookup
|
│ AiToolbox │ ← Manages tool registration and lookup
|
||||||
│ - env │ ← Holds IAiEnvironment with credentials
|
│ - env │ ← Holds GadgetToolboxEnvironment with credentials
|
||||||
│ - tools │ ← Map of all registered tools
|
│ - tools │ ← Map of all registered tools
|
||||||
│ - modeSets │ ← Tools organized by ChatSessionMode
|
│ - modeSets │ ← Tools organized by ChatSessionMode
|
||||||
└────────┬────────┘
|
└────────┬────────┘
|
||||||
@ -29,7 +30,7 @@ The toolbox system is built on these core principles:
|
|||||||
│ register()
|
│ register()
|
||||||
▼
|
▼
|
||||||
┌─────────────────┐
|
┌─────────────────┐
|
||||||
│ AiTool │ ← Abstract base class for all tools
|
│ GadgetTool │ ← Abstract base class for all tools
|
||||||
│ - name │ ← Unique tool identifier
|
│ - name │ ← Unique tool identifier
|
||||||
│ - category │ ← Tool category (search, file, etc.)
|
│ - category │ ← Tool category (search, file, etc.)
|
||||||
│ - definition │ ← JSON Schema for AI provider
|
│ - definition │ ← JSON Schema for AI provider
|
||||||
@ -41,19 +42,21 @@ The toolbox system is built on these core principles:
|
|||||||
▼
|
▼
|
||||||
┌─────────────────┐
|
┌─────────────────┐
|
||||||
│ GoogleSearch │ ← Example tool implementation
|
│ GoogleSearch │ ← Example tool implementation
|
||||||
│ GrepTool │ ← Future tools...
|
│ FileReadTool │ ← Read files from the workspace
|
||||||
|
│ ShellTool │ ← Execute shell commands
|
||||||
|
│ SubagentTool │ ← Spawn subagents for tasks
|
||||||
│ ... │
|
│ ... │
|
||||||
└─────────────────┘
|
└─────────────────┘
|
||||||
```
|
```
|
||||||
|
|
||||||
## Core Interfaces
|
## Core Interfaces
|
||||||
|
|
||||||
### IAiEnvironment
|
### GadgetToolboxEnvironment
|
||||||
|
|
||||||
The `IAiEnvironment` interface carries configuration and credentials from the application layer (which has access to YAML configs) down to the `@gadget/ai` package (which cannot read configs directly).
|
The `GadgetToolboxEnvironment` interface carries configuration and credentials from the application layer (which has access to YAML configs) down to the `@gadget/ai-toolbox` package (which cannot read configs directly). It also carries workspace and project context that tools need.
|
||||||
|
|
||||||
```typescript
|
```typescript
|
||||||
export interface IAiEnvironment {
|
export interface GadgetToolboxEnvironment {
|
||||||
NODE_ENV: string;
|
NODE_ENV: string;
|
||||||
services?: {
|
services?: {
|
||||||
google?: {
|
google?: {
|
||||||
@ -62,24 +65,24 @@ export interface IAiEnvironment {
|
|||||||
engineId?: string;
|
engineId?: string;
|
||||||
};
|
};
|
||||||
};
|
};
|
||||||
github?: {
|
|
||||||
token?: string;
|
|
||||||
};
|
};
|
||||||
slack?: {
|
workspace?: {
|
||||||
token?: string;
|
workspaceDir?: string;
|
||||||
signingSecret?: string;
|
projectDir?: string;
|
||||||
};
|
cacheDir?: string;
|
||||||
[key: string]: unknown;
|
|
||||||
};
|
};
|
||||||
|
project?: IProject;
|
||||||
|
chatSessionMode?: ChatSessionMode;
|
||||||
}
|
}
|
||||||
```
|
```
|
||||||
|
|
||||||
**Key Design Decisions:**
|
**Key Design Decisions:**
|
||||||
|
|
||||||
- **Services Object**: Credentials are organized by service (google, github, slack, etc.)
|
- **Services Object**: Credentials are organized by service (google, etc.)
|
||||||
- **Optional Everything**: All fields are optional to support partial configurations
|
- **Optional Everything**: All fields are optional to support partial configurations
|
||||||
- **Extensible**: The `[key: string]: unknown` index signature allows future services without breaking changes
|
- **Workspace Context**: Tools that work with files need `workspaceDir`, `projectDir`, and `cacheDir`
|
||||||
- **Type Safety**: Known services have typed interfaces
|
- **Project Context**: Tools can access the current `IProject` and `ChatSessionMode`
|
||||||
|
- **Extensible**: Future services can be added without breaking changes
|
||||||
|
|
||||||
### AiToolbox
|
### AiToolbox
|
||||||
|
|
||||||
@ -87,19 +90,28 @@ The toolbox manages tool registration, organization, and retrieval:
|
|||||||
|
|
||||||
```typescript
|
```typescript
|
||||||
export class AiToolbox {
|
export class AiToolbox {
|
||||||
constructor(env: IAiEnvironment);
|
constructor(env: GadgetToolboxEnvironment);
|
||||||
|
|
||||||
// Register a tool for use by agents
|
// Register a tool for use by agents
|
||||||
register(tool: AiTool, modes?: string[]): void;
|
register(tool: IAiTool, modes?: string[]): void;
|
||||||
|
|
||||||
// Get a tool by name (for system tools)
|
// Get a tool by name (for system tools)
|
||||||
getTool(name: string): AiTool | undefined;
|
getTool(name: string): IAiTool | undefined;
|
||||||
|
|
||||||
// Get all tools for a specific mode
|
// Get all tools for a specific mode
|
||||||
getModeSet(mode: string): ToolSet | undefined;
|
getModeSet(mode: string): ToolSet | undefined;
|
||||||
|
|
||||||
|
// Get tool names for a specific mode
|
||||||
|
getToolNamesForMode(mode: string): string[];
|
||||||
|
|
||||||
|
// Update workspace context (called when project changes)
|
||||||
|
updateWorkspace(workspace: { workspaceDir?: string; projectDir?: string; cacheDir?: string }): void;
|
||||||
|
|
||||||
|
// Update project context (called when session mode changes)
|
||||||
|
updateProjectContext(project: IProject, mode: ChatSessionMode): void;
|
||||||
|
|
||||||
// Access environment credentials
|
// Access environment credentials
|
||||||
get env(): IAiEnvironment;
|
get env(): GadgetToolboxEnvironment;
|
||||||
}
|
}
|
||||||
```
|
```
|
||||||
|
|
||||||
@ -108,14 +120,18 @@ export class AiToolbox {
|
|||||||
- **System Tools** (no modes): Called by the platform itself (e.g., auto-naming chat sessions)
|
- **System Tools** (no modes): Called by the platform itself (e.g., auto-naming chat sessions)
|
||||||
- **Agent Tools** (with modes): Available to AI agents in specific ChatSessionModes (e.g., "code", "research", "debug")
|
- **Agent Tools** (with modes): Available to AI agents in specific ChatSessionModes (e.g., "code", "research", "debug")
|
||||||
|
|
||||||
### AiTool
|
### GadgetTool
|
||||||
|
|
||||||
All tools extend this abstract base class:
|
All tools extend this abstract base class:
|
||||||
|
|
||||||
```typescript
|
```typescript
|
||||||
export abstract class AiTool {
|
export abstract class GadgetTool implements IAiTool {
|
||||||
protected _toolbox: AiToolbox;
|
protected _toolbox: AiToolbox;
|
||||||
|
|
||||||
|
constructor(toolbox: AiToolbox);
|
||||||
|
|
||||||
|
get toolbox(): AiToolbox;
|
||||||
|
|
||||||
// Unique identifier for the tool
|
// Unique identifier for the tool
|
||||||
abstract get name(): string;
|
abstract get name(): string;
|
||||||
|
|
||||||
@ -155,9 +171,9 @@ The tool execution flow demonstrates the abstraction layer between the common to
|
|||||||
|
|
||||||
```
|
```
|
||||||
┌─────────────────────────────────────────────────────────────┐
|
┌─────────────────────────────────────────────────────────────┐
|
||||||
│ Application Layer (gadget-drone / gadget-code) │
|
│ Application Layer (gadget-drone) │
|
||||||
│ - Reads YAML config │
|
│ - Reads YAML config │
|
||||||
│ - Constructs IAiEnvironment │
|
│ - Constructs GadgetToolboxEnvironment │
|
||||||
│ - Creates AiToolbox │
|
│ - Creates AiToolbox │
|
||||||
│ - Registers tools │
|
│ - Registers tools │
|
||||||
└────────────────────┬────────────────────────────────────────┘
|
└────────────────────┬────────────────────────────────────────┘
|
||||||
@ -293,16 +309,6 @@ This allows complex multi-step operations where the AI can:
|
|||||||
|
|
||||||
Tools requiring credentials need them configured in your YAML config files.
|
Tools requiring credentials need them configured in your YAML config files.
|
||||||
|
|
||||||
#### gadget-code.yaml
|
|
||||||
|
|
||||||
```yaml
|
|
||||||
# Add to gadget-code.yaml
|
|
||||||
google:
|
|
||||||
cse:
|
|
||||||
apiKey: "${GOOGLE_CSE_API_KEY}"
|
|
||||||
engineId: "${GOOGLE_CSE_ENGINE_ID}"
|
|
||||||
```
|
|
||||||
|
|
||||||
#### gadget-drone.yaml
|
#### gadget-drone.yaml
|
||||||
|
|
||||||
```yaml
|
```yaml
|
||||||
@ -313,6 +319,8 @@ google:
|
|||||||
engineId: "${GOOGLE_CSE_ENGINE_ID}"
|
engineId: "${GOOGLE_CSE_ENGINE_ID}"
|
||||||
```
|
```
|
||||||
|
|
||||||
|
> **Note:** `gadget-code` and `gadget-tasks` do not need toolbox credentials. gadget-code builds prompts but doesn't execute tools directly. gadget-tasks is a headless IDE client that delegates all tool execution to drones via the platform.
|
||||||
|
|
||||||
### Environment Variables
|
### Environment Variables
|
||||||
|
|
||||||
Set the actual values in your shell or deployment environment:
|
Set the actual values in your shell or deployment environment:
|
||||||
@ -324,14 +332,14 @@ export GOOGLE_CSE_ENGINE_ID="your-engine-id-here"
|
|||||||
|
|
||||||
### TypeScript Usage
|
### TypeScript Usage
|
||||||
|
|
||||||
In consumer applications, construct the environment and pass to the AI API:
|
In consumer applications, construct the environment and pass to the toolbox:
|
||||||
|
|
||||||
```typescript
|
```typescript
|
||||||
// gadget-drone/src/services/ai.ts
|
// gadget-drone/src/services/ai.ts
|
||||||
import { createAiApi, IAiEnvironment } from "@gadget/ai";
|
import { AiToolbox, GadgetToolboxEnvironment } from "@gadget/ai-toolbox";
|
||||||
import env from "../config/env.js";
|
import env from "../config/env.js";
|
||||||
|
|
||||||
const aiEnv: IAiEnvironment = {
|
const toolboxEnv: GadgetToolboxEnvironment = {
|
||||||
NODE_ENV: env.NODE_ENV,
|
NODE_ENV: env.NODE_ENV,
|
||||||
services: {
|
services: {
|
||||||
google: {
|
google: {
|
||||||
@ -343,7 +351,12 @@ const aiEnv: IAiEnvironment = {
|
|||||||
},
|
},
|
||||||
};
|
};
|
||||||
|
|
||||||
const api = createAiApi(aiEnv, providerConfig, logger);
|
const toolbox = new AiToolbox(toolboxEnv);
|
||||||
|
|
||||||
|
// Register tools for specific chat session modes
|
||||||
|
toolbox.register(new GoogleSearchTool(toolbox), ["code", "research"]);
|
||||||
|
toolbox.register(new FileReadTool(toolbox), ["code", "plan", "build"]);
|
||||||
|
toolbox.register(new SubagentTool(toolbox), ["code"]);
|
||||||
```
|
```
|
||||||
|
|
||||||
## Example Tool: Google Search
|
## Example Tool: Google Search
|
||||||
@ -450,7 +463,7 @@ Tool Call: search_google({
|
|||||||
|
|
||||||
### Implementation Details
|
### Implementation Details
|
||||||
|
|
||||||
**File**: `packages/ai/src/tools/search/google.ts`
|
**File**: `packages/ai-toolbox/src/network/search-google.ts`
|
||||||
|
|
||||||
**Key Methods:**
|
**Key Methods:**
|
||||||
|
|
||||||
@ -560,14 +573,19 @@ RECOVERY HINT: Provide a 'query' parameter with your search terms and try again.
|
|||||||
|
|
||||||
### Step 1: Create Tool Class
|
### Step 1: Create Tool Class
|
||||||
|
|
||||||
Extend `AiTool` and implement the required methods:
|
Extend `GadgetTool` and implement the required methods:
|
||||||
|
|
||||||
```typescript
|
```typescript
|
||||||
import { AiTool, IToolArguments, IToolDefinition } from "../tool.js";
|
import { GadgetTool } from "@gadget/ai-toolbox";
|
||||||
import { IAiLogger } from "../../api.js";
|
import type { AiToolbox } from "@gadget/ai-toolbox";
|
||||||
import { formatError } from "../tool-error.js";
|
import type { IToolArguments, IToolDefinition, IAiLogger } from "@gadget/ai";
|
||||||
|
import { formatError } from "@gadget/ai";
|
||||||
|
|
||||||
|
export class MyNewTool extends GadgetTool {
|
||||||
|
constructor(toolbox: AiToolbox) {
|
||||||
|
super(toolbox);
|
||||||
|
}
|
||||||
|
|
||||||
export class MyNewTool extends AiTool {
|
|
||||||
get name(): string {
|
get name(): string {
|
||||||
return "my_tool_name";
|
return "my_tool_name";
|
||||||
}
|
}
|
||||||
@ -616,11 +634,14 @@ export class MyNewTool extends AiTool {
|
|||||||
throw new Error("API key not configured in environment");
|
throw new Error("API key not configured in environment");
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// Access workspace context
|
||||||
|
const projectDir = this.toolbox.env.workspace?.projectDir;
|
||||||
|
|
||||||
// Perform the operation
|
// Perform the operation
|
||||||
logger.debug("executing my tool", { args });
|
logger.debug("executing my tool", { args });
|
||||||
|
|
||||||
try {
|
try {
|
||||||
const result = await this.doSomething(args.param1);
|
const result = await this.doSomething(args.param1 as string);
|
||||||
return `Operation successful: ${result}`;
|
return `Operation successful: ${result}`;
|
||||||
} catch (error) {
|
} catch (error) {
|
||||||
return formatError({
|
return formatError({
|
||||||
@ -639,10 +660,10 @@ export class MyNewTool extends AiTool {
|
|||||||
|
|
||||||
### Step 2: Add Credentials (if needed)
|
### Step 2: Add Credentials (if needed)
|
||||||
|
|
||||||
Update `IAiEnvironment` in `packages/ai/src/config/env.ts`:
|
Update `GadgetToolboxEnvironment` in `packages/ai-toolbox/src/toolbox.ts`:
|
||||||
|
|
||||||
```typescript
|
```typescript
|
||||||
export interface IAiEnvironment {
|
export interface GadgetToolboxEnvironment {
|
||||||
NODE_ENV: string;
|
NODE_ENV: string;
|
||||||
services?: {
|
services?: {
|
||||||
google?: {
|
google?: {
|
||||||
@ -655,40 +676,34 @@ export interface IAiEnvironment {
|
|||||||
apiKey?: string;
|
apiKey?: string;
|
||||||
endpoint?: string;
|
endpoint?: string;
|
||||||
};
|
};
|
||||||
[key: string]: unknown;
|
|
||||||
};
|
};
|
||||||
|
workspace?: {
|
||||||
|
workspaceDir?: string;
|
||||||
|
projectDir?: string;
|
||||||
|
cacheDir?: string;
|
||||||
|
};
|
||||||
|
project?: IProject;
|
||||||
|
chatSessionMode?: ChatSessionMode;
|
||||||
}
|
}
|
||||||
```
|
```
|
||||||
|
|
||||||
Update config types in `packages/config/src/types.ts`:
|
Update config types in `packages/config/src/types.ts` and consumer config readers to populate the environment.
|
||||||
|
|
||||||
```typescript
|
|
||||||
export interface GadgetDroneConfig {
|
|
||||||
// ... existing fields
|
|
||||||
myService?: {
|
|
||||||
apiKey: string;
|
|
||||||
endpoint: string;
|
|
||||||
};
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
Update consumer config readers to populate the environment.
|
|
||||||
|
|
||||||
### Step 3: Register the Tool
|
### Step 3: Register the Tool
|
||||||
|
|
||||||
In gadget-drone startup code (to be implemented):
|
In gadget-drone startup code:
|
||||||
|
|
||||||
```typescript
|
```typescript
|
||||||
import { AiToolbox } from "@gadget/ai";
|
import { AiToolbox } from "@gadget/ai-toolbox";
|
||||||
import { MyNewTool } from "@gadget/ai/tools/my-tool.js";
|
import { MyNewTool } from "@gadget/ai-toolbox";
|
||||||
|
|
||||||
const toolbox = new AiToolbox(aiEnv);
|
const toolbox = new AiToolbox(toolboxEnv);
|
||||||
|
|
||||||
// Register as system tool (no modes)
|
// Register as system tool (no modes — called by the platform itself)
|
||||||
toolbox.register(new MyNewTool());
|
toolbox.register(new MyNewTool(toolbox));
|
||||||
|
|
||||||
// Or register for specific modes
|
// Or register for specific chat session modes
|
||||||
toolbox.register(new MyNewTool(), ["code", "debug"]);
|
toolbox.register(new MyNewTool(toolbox), ["code", "debug"]);
|
||||||
```
|
```
|
||||||
|
|
||||||
## Testing Tools
|
## Testing Tools
|
||||||
@ -743,7 +758,7 @@ import { AiToolbox } from "../src/toolbox.js";
|
|||||||
|
|
||||||
describe("GoogleSearchTool integration", () => {
|
describe("GoogleSearchTool integration", () => {
|
||||||
it("should call Google CSE API with correct parameters", async () => {
|
it("should call Google CSE API with correct parameters", async () => {
|
||||||
const env = {
|
const env: GadgetToolboxEnvironment = {
|
||||||
NODE_ENV: "test",
|
NODE_ENV: "test",
|
||||||
services: {
|
services: {
|
||||||
google: {
|
google: {
|
||||||
@ -853,8 +868,21 @@ Planned improvements to the toolbox system:
|
|||||||
4. **Tool Metadata**: Version, author, usage statistics
|
4. **Tool Metadata**: Version, author, usage statistics
|
||||||
5. **Result Streaming**: Stream large results back to AI
|
5. **Result Streaming**: Stream large results back to AI
|
||||||
|
|
||||||
|
## Tool Categories
|
||||||
|
|
||||||
|
The `@gadget/ai-toolbox` package organizes tools into the following categories:
|
||||||
|
|
||||||
|
| Category | Directory | Tools | Description |
|
||||||
|
|----------|-----------|-------|-------------|
|
||||||
|
| Network | `src/network/` | `GoogleSearchTool`, `FetchUrlTool`, `WebFetcherTool` | Web search and URL fetching |
|
||||||
|
| System | `src/system/` | `FileReadTool`, `FileWriteTool`, `FileEditTool`, `FileListTool`, `GlobTool`, `GrepTool`, `ShellTool` | File system and shell operations |
|
||||||
|
| Plan | `src/plan/` | `PlanFileReadTool`, `PlanFileWriteTool`, `PlanFileEditTool`, `PlanFileListTool` | Plan-mode file operations |
|
||||||
|
| Chat | `src/chat/` | `SubagentTool` | Subagent spawning for multi-agent workflows |
|
||||||
|
| Project | `src/project/` | `ListSkillsTool`, `ReadSkillTool` | Project skill access |
|
||||||
|
|
||||||
## Related Documentation
|
## Related Documentation
|
||||||
|
|
||||||
- [Configuration Guide](./configuration.md) - Setting up tool credentials
|
- [Configuration Guide](./configuration.md) - Setting up tool credentials
|
||||||
- [Architecture Overview](./architecture.md) - System architecture
|
- [Architecture Overview](./architecture.md) - System architecture
|
||||||
- [AI API Reference](../packages/ai/README.md) - `@gadget/ai` package documentation
|
- [AI API Reference](../packages/ai/README.md) - `@gadget/ai` package documentation
|
||||||
|
- [gadget-tasks Documentation](./gadget-tasks.md) - Scheduled task worker (does NOT use @gadget/ai-toolbox)
|
||||||
|
|||||||
220
docs/gadget-tasks.md
Normal file
220
docs/gadget-tasks.md
Normal file
@ -0,0 +1,220 @@
|
|||||||
|
# gadget-tasks Documentation
|
||||||
|
|
||||||
|
## Overview
|
||||||
|
|
||||||
|
gadget-tasks is the Gadget Code **scheduled task worker** — a headless IDE client that automates the browser IDE flow on a cron schedule. It drives the gadget-code platform via REST API and Socket.IO, using the exact same protocol the browser IDE uses, but without any UI.
|
||||||
|
|
||||||
|
**Key design principle:** gadget-tasks contains **zero duplicated code**. No Mongoose models, no AI API calls, no prompt templates, no workspace management. All of that flows through the gadget-code platform, which delegates to drones for execution.
|
||||||
|
|
||||||
|
## Architecture
|
||||||
|
|
||||||
|
```
|
||||||
|
┌──────────────────┐ ┌──────────────────┐ ┌──────────────────┐
|
||||||
|
│ gadget-tasks │────▶│ gadget-code │────▶│ gadget-drone │
|
||||||
|
│ (Headless IDE) │◀────│ (Platform) │◀────│ (AWL Worker) │
|
||||||
|
└──────────────────┘ └──────────────────┘ └──────────────────┘
|
||||||
|
│ │ │
|
||||||
|
│ REST API │ MongoDB │ Files
|
||||||
|
│ Socket.IO │ Redis │ Git
|
||||||
|
│ JWT Auth │ Socket.IO relay │ AI API
|
||||||
|
```
|
||||||
|
|
||||||
|
gadget-tasks acts as a **programmatic IDE user**:
|
||||||
|
- Authenticates with the platform (same as logging into the browser IDE)
|
||||||
|
- Selects a drone (same as clicking a drone in the UI)
|
||||||
|
- Creates chat sessions, locks drones, submits prompts (same as typing in the chat)
|
||||||
|
- Waits for work order completion (same as watching the streaming response)
|
||||||
|
- Updates task records when done
|
||||||
|
|
||||||
|
## Per-Task Execution Flow
|
||||||
|
|
||||||
|
When a CronJob fires for a scheduled task:
|
||||||
|
|
||||||
|
1. **Create ChatSession** — `POST /api/v1/chat-sessions`
|
||||||
|
2. **Lock drone** — Socket.IO `requestSessionLock(drone, project, session)`
|
||||||
|
3. **Set workspace mode** — Socket.IO `requestWorkspaceMode(drone, project, session, "agent")`
|
||||||
|
4. **Submit prompt** — Socket.IO `submitPrompt(task.content)` → creates ChatTurn with canonical system prompt → routes work order to drone
|
||||||
|
5. **Wait for completion** — Socket.IO receives `workOrderComplete(turnId, success, message)`
|
||||||
|
6. **Release drone lock** — Socket.IO `releaseSessionLock(drone, project, session)`
|
||||||
|
7. **Update task.lastRun** — `PATCH /api/v1/projects/:id/tasks/:taskId/lastRun`
|
||||||
|
|
||||||
|
Steps 2–6 use the **same Socket.IO protocol** the browser IDE uses. gadget-code builds the system prompt from its canonical templates, routes the work order to the drone, and persists all results as ChatTurn records — identical to interactive use.
|
||||||
|
|
||||||
|
## Source Structure
|
||||||
|
|
||||||
|
```
|
||||||
|
gadget-tasks/
|
||||||
|
├── gadget-tasks.yaml # YAML configuration
|
||||||
|
├── package.json
|
||||||
|
├── tsconfig.json
|
||||||
|
└── src/
|
||||||
|
├── gadget-tasks.ts # Main entry — startup, shutdown, drone selection
|
||||||
|
├── config/
|
||||||
|
│ └── env.ts # Config loader — platform.baseUrl, redis, concurrency
|
||||||
|
├── lib/
|
||||||
|
│ ├── process.ts # GadgetProcess base class
|
||||||
|
│ └── service.ts # GadgetService base class
|
||||||
|
└── services/
|
||||||
|
├── platform.ts # REST API + Socket.IO headless IDE client
|
||||||
|
├── scheduler.ts # CronJob management, concurrency control
|
||||||
|
└── lock.ts # Redis singleton lock (prevents duplicate instances)
|
||||||
|
```
|
||||||
|
|
||||||
|
### Key Services
|
||||||
|
|
||||||
|
| Service | File | Purpose |
|
||||||
|
|---------|------|---------|
|
||||||
|
| **PlatformService** | `src/services/platform.ts` | REST API client (auth, projects, sessions, drones) + Socket.IO client (session lock, workspace mode, prompt submission, work order tracking) |
|
||||||
|
| **SchedulerService** | `src/services/scheduler.ts` | Creates CronJobs from task crontab expressions, enforces concurrency limit, delegates to PlatformService |
|
||||||
|
| **TaskLockService** | `src/services/lock.ts` | Redis-based singleton lock — prevents multiple gadget-tasks instances from running simultaneously |
|
||||||
|
|
||||||
|
## Configuration
|
||||||
|
|
||||||
|
### gadget-tasks.yaml
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
timezone: America/New_York
|
||||||
|
platform:
|
||||||
|
baseUrl: https://code-dev.g4dge7.com:5174
|
||||||
|
redis:
|
||||||
|
host: localhost
|
||||||
|
port: 6379
|
||||||
|
# password: optional
|
||||||
|
# keyPrefix: defaults to "gadget:"
|
||||||
|
concurrency: 1
|
||||||
|
logging:
|
||||||
|
console:
|
||||||
|
enabled: true
|
||||||
|
file:
|
||||||
|
enabled: true
|
||||||
|
path: ~/logs/gadget-tasks
|
||||||
|
# name: defaults to "gadget-tasks"
|
||||||
|
# maxWritesPerFile: defaults to 10000
|
||||||
|
# maxFiles: defaults to 10
|
||||||
|
```
|
||||||
|
|
||||||
|
| Field | Required | Default | Description |
|
||||||
|
|-------|----------|---------|-------------|
|
||||||
|
| `timezone` | No | `America/New_York` | Timezone for CronJob scheduling |
|
||||||
|
| `platform.baseUrl` | **Yes** | — | URL of the gadget-code platform |
|
||||||
|
| `redis.host` | No | `localhost` | Redis host for singleton lock |
|
||||||
|
| `redis.port` | No | `6379` | Redis port |
|
||||||
|
| `redis.password` | No | — | Redis password |
|
||||||
|
| `redis.keyPrefix` | No | `gadget:` | Redis key prefix |
|
||||||
|
| `concurrency` | No | `1` | Max concurrent tasks (sequential by default) |
|
||||||
|
| `logging.console.enabled` | No | `false` | Enable console logging |
|
||||||
|
| `logging.file.enabled` | No | `false` | Enable file logging |
|
||||||
|
| `logging.file.path` | No | `./logs` | Log file directory |
|
||||||
|
|
||||||
|
### Credentials
|
||||||
|
|
||||||
|
Email and password are **not stored in the config file**. They are provided at startup:
|
||||||
|
|
||||||
|
- **CLI args:** `--user=admin@example.com --password=secret`
|
||||||
|
- **Interactive prompt:** If no CLI args, you'll be prompted (same pattern as gadget-drone)
|
||||||
|
|
||||||
|
This avoids storing credentials in config files that may be checked into version control.
|
||||||
|
|
||||||
|
## Startup Sequence
|
||||||
|
|
||||||
|
```
|
||||||
|
1. hookProcessSignals() — SIGINT handler
|
||||||
|
2. Acquire Redis singleton lock — exits if another instance is running
|
||||||
|
3. Get user credentials — CLI args or interactive prompt
|
||||||
|
4. Authenticate with platform — POST /api/v1/auth/sign-in → JWT
|
||||||
|
5. Select a drone — auto-select if only one, otherwise interactive list
|
||||||
|
6. Connect Socket.IO — JWT auth, websocket transport
|
||||||
|
7. Start scheduler — empty initially
|
||||||
|
8. Fetch projects — GET /api/v1/projects
|
||||||
|
9. Schedule enabled tasks — create CronJob for each task's crontab
|
||||||
|
10. Start heartbeat — 19s interval, prevents drone timeout
|
||||||
|
```
|
||||||
|
|
||||||
|
## Shutdown Sequence
|
||||||
|
|
||||||
|
```
|
||||||
|
1. Stop all CronJobs — SchedulerService.stop()
|
||||||
|
2. Disconnect Socket.IO — reject pending work orders
|
||||||
|
3. Release Redis lock — allow another instance to start
|
||||||
|
```
|
||||||
|
|
||||||
|
## Getting Started
|
||||||
|
|
||||||
|
### Prerequisites
|
||||||
|
|
||||||
|
- gadget-code platform running (backend + frontend)
|
||||||
|
- At least one gadget-drone registered and online
|
||||||
|
- Redis running on localhost:6379 (or as configured)
|
||||||
|
- A user account on the platform with projects that have tasks defined
|
||||||
|
|
||||||
|
### Running
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# From the monorepo root
|
||||||
|
cd gadget-tasks
|
||||||
|
|
||||||
|
# Development mode (TypeScript, auto-restart)
|
||||||
|
pnpm dev -- --user=admin@example.com --password=secret
|
||||||
|
|
||||||
|
# Or with interactive credential prompt
|
||||||
|
pnpm dev
|
||||||
|
|
||||||
|
# Production (build first)
|
||||||
|
pnpm build
|
||||||
|
pnpm start -- --user=admin@example.com --password=secret
|
||||||
|
```
|
||||||
|
|
||||||
|
### Global CLI
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Link globally from the monorepo root
|
||||||
|
pnpm link:global
|
||||||
|
|
||||||
|
# Now available as a system command
|
||||||
|
gadget-tasks --user=admin@example.com --password=secret
|
||||||
|
```
|
||||||
|
|
||||||
|
## Task Execution Details
|
||||||
|
|
||||||
|
### Concurrency
|
||||||
|
|
||||||
|
By default, `concurrency: 1` means tasks execute **sequentially**. If a task fires while the drone is busy with another task, the new task waits for the drone to become available. This is the correct model — drones process one work order at a time.
|
||||||
|
|
||||||
|
If you increase `concurrency`, you need multiple drones available. Each task requires a drone lock, and a drone can only be locked to one session at a time.
|
||||||
|
|
||||||
|
### Work Order Tracking
|
||||||
|
|
||||||
|
When `submitPrompt` is called, the callback provides a `turnId`. gadget-tasks stores a Promise resolver in a `pendingWorkOrders` map keyed by `turnId`. When the server emits `workOrderComplete(turnId, success, message)`, the corresponding Promise is resolved, unblocking the task execution.
|
||||||
|
|
||||||
|
### Session Heartbeat
|
||||||
|
|
||||||
|
gadget-tasks sends a `sessionHeartbeat` every 19 seconds (same interval as the browser IDE). This prevents the drone's 120-second heartbeat timeout from firing while a task is active.
|
||||||
|
|
||||||
|
### Error Recovery
|
||||||
|
|
||||||
|
If gadget-tasks crashes while a task is being processed:
|
||||||
|
- The drone continues processing the work order independently
|
||||||
|
- The session and ChatTurn records are preserved in the database
|
||||||
|
- On restart, gadget-tasks creates **new** sessions for tasks that fire
|
||||||
|
- Old sessions are visible in the browser IDE via their ChatTurn records
|
||||||
|
|
||||||
|
No special crash recovery is needed in gadget-tasks (unlike gadget-drone, which has crash recovery for incomplete work orders).
|
||||||
|
|
||||||
|
## What gadget-tasks Does NOT Do
|
||||||
|
|
||||||
|
| ❌ Does NOT | ✅ Instead |
|
||||||
|
|------------|----------|
|
||||||
|
| Connect to MongoDB | Uses REST API to read/write data |
|
||||||
|
| Define Mongoose models | Uses `@gadget/api` TypeScript interfaces |
|
||||||
|
| Call AI APIs | Submits prompts through the platform → drone |
|
||||||
|
| Build system prompts | gadget-code builds prompts from canonical templates |
|
||||||
|
| Execute tools | The drone executes tools via `@gadget/ai-toolbox` |
|
||||||
|
| Manage workspaces | The drone manages workspace directories |
|
||||||
|
| Store credentials in config | Provides them at startup via CLI or prompt |
|
||||||
|
|
||||||
|
## Related Documentation
|
||||||
|
|
||||||
|
- [README](../README.md) — Project overview and quick start
|
||||||
|
- [Agent Toolbox](./agent-toolbox.md) — `@gadget/ai-toolbox` tool implementations (used by drones, not gadget-tasks)
|
||||||
|
- [Socket Protocol](./socket-protocol.md) — Socket.IO protocol between IDE, platform, and drone
|
||||||
|
- [Drone Documentation](../gadget-drone/docs/gadget-drone.md) — How drones execute work orders
|
||||||
@ -9,11 +9,7 @@ overrides:
|
|||||||
|
|
||||||
importers:
|
importers:
|
||||||
|
|
||||||
.:
|
.: {}
|
||||||
dependencies:
|
|
||||||
list:
|
|
||||||
specifier: link:../../snap/code/198/.local/share/pnpm/global/5/node_modules/list
|
|
||||||
version: link:../../snap/code/198/.local/share/pnpm/global/5/node_modules/list
|
|
||||||
|
|
||||||
gadget-code:
|
gadget-code:
|
||||||
dependencies:
|
dependencies:
|
||||||
|
|||||||
Loading…
Reference in New Issue
Block a user