docs: add gadget-tasks documentation, update README and agent-toolbox
- README.md: Added gadget-tasks and @gadget/ai-toolbox to projects table, added Scheduled Tasks architecture section with diagram, updated monorepo structure, added gadget-tasks to dev server instructions, added doc link - docs/agent-toolbox.md: Updated to reflect @gadget/ai-toolbox extraction from @gadget/ai. Changed IAiEnvironment → GadgetToolboxEnvironment, AiTool → GadgetTool, updated import paths, config examples, and added Tool Categories section. Clarified gadget-tasks is NOT a consumer. - docs/gadget-tasks.md: New documentation covering architecture, per-task execution flow, configuration, startup/shutdown sequences, concurrency, work order tracking, heartbeat, error recovery, and getting started.
This commit is contained in:
parent
f742a1d765
commit
4d8f403d78
51
README.md
51
README.md
@ -22,11 +22,14 @@ pnpm dev
|
||||
## Projects
|
||||
|
||||
| Package | Role |
|
||||
| -------------- | ------------------------------------------------------------------------ |
|
||||
| ------------------ | ------------------------------------------------------------------------ |
|
||||
| `gadget-code` | Web service — agentic IDE, browser UI, API server |
|
||||
| `gadget-drone` | Worker process — runs the agentic workflow loop in workspace directories |
|
||||
| `gadget-tasks` | Scheduled task worker — headless IDE client for cron-driven tasks |
|
||||
| `@gadget/ai` | Shared AI API abstraction — Ollama and OpenAI |
|
||||
| `@gadget/ai-toolbox` | Shared AI tool implementations — search, file, plan, subagent tools |
|
||||
| `@gadget/api` | Shared TypeScript interfaces — common types across all packages |
|
||||
| `@gadget/config` | Shared YAML config loader — per-package configuration |
|
||||
|
||||
## Architecture
|
||||
|
||||
@ -50,6 +53,30 @@ pnpm dev
|
||||
6. **Streaming response** flows back: thinking → response → tool calls
|
||||
7. **Results persist** in MongoDB as ChatTurn records
|
||||
|
||||
### Scheduled Tasks (gadget-tasks)
|
||||
|
||||
gadget-tasks is a **headless IDE client** that automates the browser IDE flow on a cron schedule. It does not duplicate AI, database, or workspace code — it drives the existing gadget-code platform via REST API and Socket.IO, the same protocol the browser IDE uses.
|
||||
|
||||
```
|
||||
┌──────────────┐ ┌──────────────┐ ┌──────────────┐
|
||||
│ gadget-tasks │────▶│ gadget-code │────▶│ gadget-drone │
|
||||
│ (Scheduler) │◀────│ (Express 5) │◀────│ (Worker) │
|
||||
└──────────────┘ └──────────────┘ └──────────────┘
|
||||
│ │ │
|
||||
│ REST + Socket.IO │ MongoDB │ Files
|
||||
│ JWT Auth │ Redis │ Git
|
||||
```
|
||||
|
||||
**Per-task flow:**
|
||||
1. CronJob fires → create ChatSession via REST API
|
||||
2. Lock drone via Socket.IO `requestSessionLock`
|
||||
3. Set workspace mode to Agent via `requestWorkspaceMode`
|
||||
4. Submit task prompt via `submitPrompt` → drone processes work order
|
||||
5. Receive `workOrderComplete` → release session lock
|
||||
6. Update `task.lastRun` via `PATCH /projects/:id/tasks/:taskId/lastRun`
|
||||
|
||||
See [gadget-tasks documentation](./docs/gadget-tasks.md) for setup and usage.
|
||||
|
||||
## AI Provider Setup
|
||||
|
||||
Before using AI features, add a provider via CLI:
|
||||
@ -120,6 +147,21 @@ pnpm dev
|
||||
# Drone worker (in a workspace directory)
|
||||
cd ~/my-gadget-workspace
|
||||
pnpm --filter gadget-drone dev
|
||||
|
||||
# Task scheduler (requires running gadget-code + drone)
|
||||
cd gadget-tasks
|
||||
pnpm dev -- --user=admin@example.com --password=secret
|
||||
```
|
||||
|
||||
### Global Install (CLI)
|
||||
|
||||
```bash
|
||||
# Link gadget-drone and gadget-tasks globally for CLI use
|
||||
pnpm link:global
|
||||
|
||||
# Now available as system commands
|
||||
gadget-drone
|
||||
gadget-tasks --user=admin@example.com --password=secret
|
||||
```
|
||||
|
||||
### Testing
|
||||
@ -183,13 +225,16 @@ See [`packages/ai/README.md`](./packages/ai/README.md) for the full API referenc
|
||||
- [Workspace Management](./docs/gadget-workspace.md)
|
||||
- [Drone Documentation](./gadget-drone/docs/gadget-drone.md)
|
||||
- [UI Design Guide](./gadget-code/docs/ui-design-guide.md)
|
||||
- [gadget-tasks Documentation](./docs/gadget-tasks.md)
|
||||
|
||||
## Monorepo Structure
|
||||
|
||||
```
|
||||
gadget/
|
||||
├── packages/ai/ # @gadget/ai — AI API abstraction
|
||||
├── packages/ai-toolbox/ # @gadget/ai-toolbox — Shared tool implementations
|
||||
├── packages/api/ # @gadget/api — Shared interfaces
|
||||
├── packages/config/ # @gadget/config — YAML config loader
|
||||
├── gadget-code/ # Web service + browser IDE
|
||||
│ ├── src/ # Backend (Express, Socket.IO, Mongoose)
|
||||
│ ├── frontend/ # Frontend (React, Vite, Tailwind)
|
||||
@ -197,6 +242,8 @@ gadget/
|
||||
├── gadget-drone/ # Worker process
|
||||
│ ├── src/ # Drone implementation
|
||||
│ └── docs/ # Drone documentation
|
||||
├── gadget-tasks/ # Scheduled task worker
|
||||
│ └── src/ # Headless IDE client + cron scheduler
|
||||
└── docs/ # Architecture & protocol docs
|
||||
```
|
||||
|
||||
@ -207,4 +254,4 @@ Apache 2.0 — See [LICENSE](./LICENSE) for details.
|
||||
---
|
||||
|
||||
**Status:** Production-ready foundation with complete Chat Session UI
|
||||
**Last Updated:** April 29, 2026
|
||||
**Last Updated:** May 17, 2026
|
||||
|
||||
@ -2,9 +2,9 @@
|
||||
|
||||
## Overview
|
||||
|
||||
The Agent Toolbox is a foundational component of the Gadget platform that enables AI agents to perform real-world actions through a standardized, extensible tool system. Tools are functions that agents can invoke to interact with external services, manipulate files, search the web, and execute various operations.
|
||||
The Agent Toolbox (`@gadget/ai-toolbox`) is a shared package providing AI agent tool implementations for the Gadget platform. It was extracted from `@gadget/ai` into its own package so that agent tools can be maintained independently of the AI provider abstraction layer.
|
||||
|
||||
## Architecture
|
||||
**Consumers:** `gadget-drone` (the only current consumer). gadget-tasks does **not** use `@gadget/ai-toolbox` — it drives the gadget-code platform via REST API and Socket.IO as a headless IDE client, delegating all AI and tool execution to drones.
|
||||
|
||||
### Design Philosophy
|
||||
|
||||
@ -15,13 +15,14 @@ The toolbox system is built on these core principles:
|
||||
3. **Extensibility**: Easy to add new tools without modifying core infrastructure
|
||||
4. **Security**: Tools require explicit credentials configured in the environment
|
||||
5. **Error Handling**: Comprehensive error reporting with recovery hints
|
||||
6. **Shared Implementation**: One tool codebase used by all agent processes
|
||||
|
||||
### Component Overview
|
||||
|
||||
```
|
||||
┌─────────────────┐
|
||||
│ AiToolbox │ ← Manages tool registration and lookup
|
||||
│ - env │ ← Holds IAiEnvironment with credentials
|
||||
│ - env │ ← Holds GadgetToolboxEnvironment with credentials
|
||||
│ - tools │ ← Map of all registered tools
|
||||
│ - modeSets │ ← Tools organized by ChatSessionMode
|
||||
└────────┬────────┘
|
||||
@ -29,7 +30,7 @@ The toolbox system is built on these core principles:
|
||||
│ register()
|
||||
▼
|
||||
┌─────────────────┐
|
||||
│ AiTool │ ← Abstract base class for all tools
|
||||
│ GadgetTool │ ← Abstract base class for all tools
|
||||
│ - name │ ← Unique tool identifier
|
||||
│ - category │ ← Tool category (search, file, etc.)
|
||||
│ - definition │ ← JSON Schema for AI provider
|
||||
@ -41,19 +42,21 @@ The toolbox system is built on these core principles:
|
||||
▼
|
||||
┌─────────────────┐
|
||||
│ GoogleSearch │ ← Example tool implementation
|
||||
│ GrepTool │ ← Future tools...
|
||||
│ FileReadTool │ ← Read files from the workspace
|
||||
│ ShellTool │ ← Execute shell commands
|
||||
│ SubagentTool │ ← Spawn subagents for tasks
|
||||
│ ... │
|
||||
└─────────────────┘
|
||||
```
|
||||
|
||||
## Core Interfaces
|
||||
|
||||
### IAiEnvironment
|
||||
### GadgetToolboxEnvironment
|
||||
|
||||
The `IAiEnvironment` interface carries configuration and credentials from the application layer (which has access to YAML configs) down to the `@gadget/ai` package (which cannot read configs directly).
|
||||
The `GadgetToolboxEnvironment` interface carries configuration and credentials from the application layer (which has access to YAML configs) down to the `@gadget/ai-toolbox` package (which cannot read configs directly). It also carries workspace and project context that tools need.
|
||||
|
||||
```typescript
|
||||
export interface IAiEnvironment {
|
||||
export interface GadgetToolboxEnvironment {
|
||||
NODE_ENV: string;
|
||||
services?: {
|
||||
google?: {
|
||||
@ -62,24 +65,24 @@ export interface IAiEnvironment {
|
||||
engineId?: string;
|
||||
};
|
||||
};
|
||||
github?: {
|
||||
token?: string;
|
||||
};
|
||||
slack?: {
|
||||
token?: string;
|
||||
signingSecret?: string;
|
||||
};
|
||||
[key: string]: unknown;
|
||||
workspace?: {
|
||||
workspaceDir?: string;
|
||||
projectDir?: string;
|
||||
cacheDir?: string;
|
||||
};
|
||||
project?: IProject;
|
||||
chatSessionMode?: ChatSessionMode;
|
||||
}
|
||||
```
|
||||
|
||||
**Key Design Decisions:**
|
||||
|
||||
- **Services Object**: Credentials are organized by service (google, github, slack, etc.)
|
||||
- **Services Object**: Credentials are organized by service (google, etc.)
|
||||
- **Optional Everything**: All fields are optional to support partial configurations
|
||||
- **Extensible**: The `[key: string]: unknown` index signature allows future services without breaking changes
|
||||
- **Type Safety**: Known services have typed interfaces
|
||||
- **Workspace Context**: Tools that work with files need `workspaceDir`, `projectDir`, and `cacheDir`
|
||||
- **Project Context**: Tools can access the current `IProject` and `ChatSessionMode`
|
||||
- **Extensible**: Future services can be added without breaking changes
|
||||
|
||||
### AiToolbox
|
||||
|
||||
@ -87,19 +90,28 @@ The toolbox manages tool registration, organization, and retrieval:
|
||||
|
||||
```typescript
|
||||
export class AiToolbox {
|
||||
constructor(env: IAiEnvironment);
|
||||
constructor(env: GadgetToolboxEnvironment);
|
||||
|
||||
// Register a tool for use by agents
|
||||
register(tool: AiTool, modes?: string[]): void;
|
||||
register(tool: IAiTool, modes?: string[]): void;
|
||||
|
||||
// Get a tool by name (for system tools)
|
||||
getTool(name: string): AiTool | undefined;
|
||||
getTool(name: string): IAiTool | undefined;
|
||||
|
||||
// Get all tools for a specific mode
|
||||
getModeSet(mode: string): ToolSet | undefined;
|
||||
|
||||
// Get tool names for a specific mode
|
||||
getToolNamesForMode(mode: string): string[];
|
||||
|
||||
// Update workspace context (called when project changes)
|
||||
updateWorkspace(workspace: { workspaceDir?: string; projectDir?: string; cacheDir?: string }): void;
|
||||
|
||||
// Update project context (called when session mode changes)
|
||||
updateProjectContext(project: IProject, mode: ChatSessionMode): void;
|
||||
|
||||
// Access environment credentials
|
||||
get env(): IAiEnvironment;
|
||||
get env(): GadgetToolboxEnvironment;
|
||||
}
|
||||
```
|
||||
|
||||
@ -108,14 +120,18 @@ export class AiToolbox {
|
||||
- **System Tools** (no modes): Called by the platform itself (e.g., auto-naming chat sessions)
|
||||
- **Agent Tools** (with modes): Available to AI agents in specific ChatSessionModes (e.g., "code", "research", "debug")
|
||||
|
||||
### AiTool
|
||||
### GadgetTool
|
||||
|
||||
All tools extend this abstract base class:
|
||||
|
||||
```typescript
|
||||
export abstract class AiTool {
|
||||
export abstract class GadgetTool implements IAiTool {
|
||||
protected _toolbox: AiToolbox;
|
||||
|
||||
constructor(toolbox: AiToolbox);
|
||||
|
||||
get toolbox(): AiToolbox;
|
||||
|
||||
// Unique identifier for the tool
|
||||
abstract get name(): string;
|
||||
|
||||
@ -155,9 +171,9 @@ The tool execution flow demonstrates the abstraction layer between the common to
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────────────────────┐
|
||||
│ Application Layer (gadget-drone / gadget-code) │
|
||||
│ Application Layer (gadget-drone) │
|
||||
│ - Reads YAML config │
|
||||
│ - Constructs IAiEnvironment │
|
||||
│ - Constructs GadgetToolboxEnvironment │
|
||||
│ - Creates AiToolbox │
|
||||
│ - Registers tools │
|
||||
└────────────────────┬────────────────────────────────────────┘
|
||||
@ -293,16 +309,6 @@ This allows complex multi-step operations where the AI can:
|
||||
|
||||
Tools requiring credentials need them configured in your YAML config files.
|
||||
|
||||
#### gadget-code.yaml
|
||||
|
||||
```yaml
|
||||
# Add to gadget-code.yaml
|
||||
google:
|
||||
cse:
|
||||
apiKey: "${GOOGLE_CSE_API_KEY}"
|
||||
engineId: "${GOOGLE_CSE_ENGINE_ID}"
|
||||
```
|
||||
|
||||
#### gadget-drone.yaml
|
||||
|
||||
```yaml
|
||||
@ -313,6 +319,8 @@ google:
|
||||
engineId: "${GOOGLE_CSE_ENGINE_ID}"
|
||||
```
|
||||
|
||||
> **Note:** `gadget-code` and `gadget-tasks` do not need toolbox credentials. gadget-code builds prompts but doesn't execute tools directly. gadget-tasks is a headless IDE client that delegates all tool execution to drones via the platform.
|
||||
|
||||
### Environment Variables
|
||||
|
||||
Set the actual values in your shell or deployment environment:
|
||||
@ -324,14 +332,14 @@ export GOOGLE_CSE_ENGINE_ID="your-engine-id-here"
|
||||
|
||||
### TypeScript Usage
|
||||
|
||||
In consumer applications, construct the environment and pass to the AI API:
|
||||
In consumer applications, construct the environment and pass to the toolbox:
|
||||
|
||||
```typescript
|
||||
// gadget-drone/src/services/ai.ts
|
||||
import { createAiApi, IAiEnvironment } from "@gadget/ai";
|
||||
import { AiToolbox, GadgetToolboxEnvironment } from "@gadget/ai-toolbox";
|
||||
import env from "../config/env.js";
|
||||
|
||||
const aiEnv: IAiEnvironment = {
|
||||
const toolboxEnv: GadgetToolboxEnvironment = {
|
||||
NODE_ENV: env.NODE_ENV,
|
||||
services: {
|
||||
google: {
|
||||
@ -343,7 +351,12 @@ const aiEnv: IAiEnvironment = {
|
||||
},
|
||||
};
|
||||
|
||||
const api = createAiApi(aiEnv, providerConfig, logger);
|
||||
const toolbox = new AiToolbox(toolboxEnv);
|
||||
|
||||
// Register tools for specific chat session modes
|
||||
toolbox.register(new GoogleSearchTool(toolbox), ["code", "research"]);
|
||||
toolbox.register(new FileReadTool(toolbox), ["code", "plan", "build"]);
|
||||
toolbox.register(new SubagentTool(toolbox), ["code"]);
|
||||
```
|
||||
|
||||
## Example Tool: Google Search
|
||||
@ -450,7 +463,7 @@ Tool Call: search_google({
|
||||
|
||||
### Implementation Details
|
||||
|
||||
**File**: `packages/ai/src/tools/search/google.ts`
|
||||
**File**: `packages/ai-toolbox/src/network/search-google.ts`
|
||||
|
||||
**Key Methods:**
|
||||
|
||||
@ -560,14 +573,19 @@ RECOVERY HINT: Provide a 'query' parameter with your search terms and try again.
|
||||
|
||||
### Step 1: Create Tool Class
|
||||
|
||||
Extend `AiTool` and implement the required methods:
|
||||
Extend `GadgetTool` and implement the required methods:
|
||||
|
||||
```typescript
|
||||
import { AiTool, IToolArguments, IToolDefinition } from "../tool.js";
|
||||
import { IAiLogger } from "../../api.js";
|
||||
import { formatError } from "../tool-error.js";
|
||||
import { GadgetTool } from "@gadget/ai-toolbox";
|
||||
import type { AiToolbox } from "@gadget/ai-toolbox";
|
||||
import type { IToolArguments, IToolDefinition, IAiLogger } from "@gadget/ai";
|
||||
import { formatError } from "@gadget/ai";
|
||||
|
||||
export class MyNewTool extends GadgetTool {
|
||||
constructor(toolbox: AiToolbox) {
|
||||
super(toolbox);
|
||||
}
|
||||
|
||||
export class MyNewTool extends AiTool {
|
||||
get name(): string {
|
||||
return "my_tool_name";
|
||||
}
|
||||
@ -616,11 +634,14 @@ export class MyNewTool extends AiTool {
|
||||
throw new Error("API key not configured in environment");
|
||||
}
|
||||
|
||||
// Access workspace context
|
||||
const projectDir = this.toolbox.env.workspace?.projectDir;
|
||||
|
||||
// Perform the operation
|
||||
logger.debug("executing my tool", { args });
|
||||
|
||||
try {
|
||||
const result = await this.doSomething(args.param1);
|
||||
const result = await this.doSomething(args.param1 as string);
|
||||
return `Operation successful: ${result}`;
|
||||
} catch (error) {
|
||||
return formatError({
|
||||
@ -639,10 +660,10 @@ export class MyNewTool extends AiTool {
|
||||
|
||||
### Step 2: Add Credentials (if needed)
|
||||
|
||||
Update `IAiEnvironment` in `packages/ai/src/config/env.ts`:
|
||||
Update `GadgetToolboxEnvironment` in `packages/ai-toolbox/src/toolbox.ts`:
|
||||
|
||||
```typescript
|
||||
export interface IAiEnvironment {
|
||||
export interface GadgetToolboxEnvironment {
|
||||
NODE_ENV: string;
|
||||
services?: {
|
||||
google?: {
|
||||
@ -655,40 +676,34 @@ export interface IAiEnvironment {
|
||||
apiKey?: string;
|
||||
endpoint?: string;
|
||||
};
|
||||
[key: string]: unknown;
|
||||
};
|
||||
workspace?: {
|
||||
workspaceDir?: string;
|
||||
projectDir?: string;
|
||||
cacheDir?: string;
|
||||
};
|
||||
project?: IProject;
|
||||
chatSessionMode?: ChatSessionMode;
|
||||
}
|
||||
```
|
||||
|
||||
Update config types in `packages/config/src/types.ts`:
|
||||
|
||||
```typescript
|
||||
export interface GadgetDroneConfig {
|
||||
// ... existing fields
|
||||
myService?: {
|
||||
apiKey: string;
|
||||
endpoint: string;
|
||||
};
|
||||
}
|
||||
```
|
||||
|
||||
Update consumer config readers to populate the environment.
|
||||
Update config types in `packages/config/src/types.ts` and consumer config readers to populate the environment.
|
||||
|
||||
### Step 3: Register the Tool
|
||||
|
||||
In gadget-drone startup code (to be implemented):
|
||||
In gadget-drone startup code:
|
||||
|
||||
```typescript
|
||||
import { AiToolbox } from "@gadget/ai";
|
||||
import { MyNewTool } from "@gadget/ai/tools/my-tool.js";
|
||||
import { AiToolbox } from "@gadget/ai-toolbox";
|
||||
import { MyNewTool } from "@gadget/ai-toolbox";
|
||||
|
||||
const toolbox = new AiToolbox(aiEnv);
|
||||
const toolbox = new AiToolbox(toolboxEnv);
|
||||
|
||||
// Register as system tool (no modes)
|
||||
toolbox.register(new MyNewTool());
|
||||
// Register as system tool (no modes — called by the platform itself)
|
||||
toolbox.register(new MyNewTool(toolbox));
|
||||
|
||||
// Or register for specific modes
|
||||
toolbox.register(new MyNewTool(), ["code", "debug"]);
|
||||
// Or register for specific chat session modes
|
||||
toolbox.register(new MyNewTool(toolbox), ["code", "debug"]);
|
||||
```
|
||||
|
||||
## Testing Tools
|
||||
@ -743,7 +758,7 @@ import { AiToolbox } from "../src/toolbox.js";
|
||||
|
||||
describe("GoogleSearchTool integration", () => {
|
||||
it("should call Google CSE API with correct parameters", async () => {
|
||||
const env = {
|
||||
const env: GadgetToolboxEnvironment = {
|
||||
NODE_ENV: "test",
|
||||
services: {
|
||||
google: {
|
||||
@ -853,8 +868,21 @@ Planned improvements to the toolbox system:
|
||||
4. **Tool Metadata**: Version, author, usage statistics
|
||||
5. **Result Streaming**: Stream large results back to AI
|
||||
|
||||
## Tool Categories
|
||||
|
||||
The `@gadget/ai-toolbox` package organizes tools into the following categories:
|
||||
|
||||
| Category | Directory | Tools | Description |
|
||||
|----------|-----------|-------|-------------|
|
||||
| Network | `src/network/` | `GoogleSearchTool`, `FetchUrlTool`, `WebFetcherTool` | Web search and URL fetching |
|
||||
| System | `src/system/` | `FileReadTool`, `FileWriteTool`, `FileEditTool`, `FileListTool`, `GlobTool`, `GrepTool`, `ShellTool` | File system and shell operations |
|
||||
| Plan | `src/plan/` | `PlanFileReadTool`, `PlanFileWriteTool`, `PlanFileEditTool`, `PlanFileListTool` | Plan-mode file operations |
|
||||
| Chat | `src/chat/` | `SubagentTool` | Subagent spawning for multi-agent workflows |
|
||||
| Project | `src/project/` | `ListSkillsTool`, `ReadSkillTool` | Project skill access |
|
||||
|
||||
## Related Documentation
|
||||
|
||||
- [Configuration Guide](./configuration.md) - Setting up tool credentials
|
||||
- [Architecture Overview](./architecture.md) - System architecture
|
||||
- [AI API Reference](../packages/ai/README.md) - `@gadget/ai` package documentation
|
||||
- [gadget-tasks Documentation](./gadget-tasks.md) - Scheduled task worker (does NOT use @gadget/ai-toolbox)
|
||||
|
||||
220
docs/gadget-tasks.md
Normal file
220
docs/gadget-tasks.md
Normal file
@ -0,0 +1,220 @@
|
||||
# gadget-tasks Documentation
|
||||
|
||||
## Overview
|
||||
|
||||
gadget-tasks is the Gadget Code **scheduled task worker** — a headless IDE client that automates the browser IDE flow on a cron schedule. It drives the gadget-code platform via REST API and Socket.IO, using the exact same protocol the browser IDE uses, but without any UI.
|
||||
|
||||
**Key design principle:** gadget-tasks contains **zero duplicated code**. No Mongoose models, no AI API calls, no prompt templates, no workspace management. All of that flows through the gadget-code platform, which delegates to drones for execution.
|
||||
|
||||
## Architecture
|
||||
|
||||
```
|
||||
┌──────────────────┐ ┌──────────────────┐ ┌──────────────────┐
|
||||
│ gadget-tasks │────▶│ gadget-code │────▶│ gadget-drone │
|
||||
│ (Headless IDE) │◀────│ (Platform) │◀────│ (AWL Worker) │
|
||||
└──────────────────┘ └──────────────────┘ └──────────────────┘
|
||||
│ │ │
|
||||
│ REST API │ MongoDB │ Files
|
||||
│ Socket.IO │ Redis │ Git
|
||||
│ JWT Auth │ Socket.IO relay │ AI API
|
||||
```
|
||||
|
||||
gadget-tasks acts as a **programmatic IDE user**:
|
||||
- Authenticates with the platform (same as logging into the browser IDE)
|
||||
- Selects a drone (same as clicking a drone in the UI)
|
||||
- Creates chat sessions, locks drones, submits prompts (same as typing in the chat)
|
||||
- Waits for work order completion (same as watching the streaming response)
|
||||
- Updates task records when done
|
||||
|
||||
## Per-Task Execution Flow
|
||||
|
||||
When a CronJob fires for a scheduled task:
|
||||
|
||||
1. **Create ChatSession** — `POST /api/v1/chat-sessions`
|
||||
2. **Lock drone** — Socket.IO `requestSessionLock(drone, project, session)`
|
||||
3. **Set workspace mode** — Socket.IO `requestWorkspaceMode(drone, project, session, "agent")`
|
||||
4. **Submit prompt** — Socket.IO `submitPrompt(task.content)` → creates ChatTurn with canonical system prompt → routes work order to drone
|
||||
5. **Wait for completion** — Socket.IO receives `workOrderComplete(turnId, success, message)`
|
||||
6. **Release drone lock** — Socket.IO `releaseSessionLock(drone, project, session)`
|
||||
7. **Update task.lastRun** — `PATCH /api/v1/projects/:id/tasks/:taskId/lastRun`
|
||||
|
||||
Steps 2–6 use the **same Socket.IO protocol** the browser IDE uses. gadget-code builds the system prompt from its canonical templates, routes the work order to the drone, and persists all results as ChatTurn records — identical to interactive use.
|
||||
|
||||
## Source Structure
|
||||
|
||||
```
|
||||
gadget-tasks/
|
||||
├── gadget-tasks.yaml # YAML configuration
|
||||
├── package.json
|
||||
├── tsconfig.json
|
||||
└── src/
|
||||
├── gadget-tasks.ts # Main entry — startup, shutdown, drone selection
|
||||
├── config/
|
||||
│ └── env.ts # Config loader — platform.baseUrl, redis, concurrency
|
||||
├── lib/
|
||||
│ ├── process.ts # GadgetProcess base class
|
||||
│ └── service.ts # GadgetService base class
|
||||
└── services/
|
||||
├── platform.ts # REST API + Socket.IO headless IDE client
|
||||
├── scheduler.ts # CronJob management, concurrency control
|
||||
└── lock.ts # Redis singleton lock (prevents duplicate instances)
|
||||
```
|
||||
|
||||
### Key Services
|
||||
|
||||
| Service | File | Purpose |
|
||||
|---------|------|---------|
|
||||
| **PlatformService** | `src/services/platform.ts` | REST API client (auth, projects, sessions, drones) + Socket.IO client (session lock, workspace mode, prompt submission, work order tracking) |
|
||||
| **SchedulerService** | `src/services/scheduler.ts` | Creates CronJobs from task crontab expressions, enforces concurrency limit, delegates to PlatformService |
|
||||
| **TaskLockService** | `src/services/lock.ts` | Redis-based singleton lock — prevents multiple gadget-tasks instances from running simultaneously |
|
||||
|
||||
## Configuration
|
||||
|
||||
### gadget-tasks.yaml
|
||||
|
||||
```yaml
|
||||
timezone: America/New_York
|
||||
platform:
|
||||
baseUrl: https://code-dev.g4dge7.com:5174
|
||||
redis:
|
||||
host: localhost
|
||||
port: 6379
|
||||
# password: optional
|
||||
# keyPrefix: defaults to "gadget:"
|
||||
concurrency: 1
|
||||
logging:
|
||||
console:
|
||||
enabled: true
|
||||
file:
|
||||
enabled: true
|
||||
path: ~/logs/gadget-tasks
|
||||
# name: defaults to "gadget-tasks"
|
||||
# maxWritesPerFile: defaults to 10000
|
||||
# maxFiles: defaults to 10
|
||||
```
|
||||
|
||||
| Field | Required | Default | Description |
|
||||
|-------|----------|---------|-------------|
|
||||
| `timezone` | No | `America/New_York` | Timezone for CronJob scheduling |
|
||||
| `platform.baseUrl` | **Yes** | — | URL of the gadget-code platform |
|
||||
| `redis.host` | No | `localhost` | Redis host for singleton lock |
|
||||
| `redis.port` | No | `6379` | Redis port |
|
||||
| `redis.password` | No | — | Redis password |
|
||||
| `redis.keyPrefix` | No | `gadget:` | Redis key prefix |
|
||||
| `concurrency` | No | `1` | Max concurrent tasks (sequential by default) |
|
||||
| `logging.console.enabled` | No | `false` | Enable console logging |
|
||||
| `logging.file.enabled` | No | `false` | Enable file logging |
|
||||
| `logging.file.path` | No | `./logs` | Log file directory |
|
||||
|
||||
### Credentials
|
||||
|
||||
Email and password are **not stored in the config file**. They are provided at startup:
|
||||
|
||||
- **CLI args:** `--user=admin@example.com --password=secret`
|
||||
- **Interactive prompt:** If no CLI args, you'll be prompted (same pattern as gadget-drone)
|
||||
|
||||
This avoids storing credentials in config files that may be checked into version control.
|
||||
|
||||
## Startup Sequence
|
||||
|
||||
```
|
||||
1. hookProcessSignals() — SIGINT handler
|
||||
2. Acquire Redis singleton lock — exits if another instance is running
|
||||
3. Get user credentials — CLI args or interactive prompt
|
||||
4. Authenticate with platform — POST /api/v1/auth/sign-in → JWT
|
||||
5. Select a drone — auto-select if only one, otherwise interactive list
|
||||
6. Connect Socket.IO — JWT auth, websocket transport
|
||||
7. Start scheduler — empty initially
|
||||
8. Fetch projects — GET /api/v1/projects
|
||||
9. Schedule enabled tasks — create CronJob for each task's crontab
|
||||
10. Start heartbeat — 19s interval, prevents drone timeout
|
||||
```
|
||||
|
||||
## Shutdown Sequence
|
||||
|
||||
```
|
||||
1. Stop all CronJobs — SchedulerService.stop()
|
||||
2. Disconnect Socket.IO — reject pending work orders
|
||||
3. Release Redis lock — allow another instance to start
|
||||
```
|
||||
|
||||
## Getting Started
|
||||
|
||||
### Prerequisites
|
||||
|
||||
- gadget-code platform running (backend + frontend)
|
||||
- At least one gadget-drone registered and online
|
||||
- Redis running on localhost:6379 (or as configured)
|
||||
- A user account on the platform with projects that have tasks defined
|
||||
|
||||
### Running
|
||||
|
||||
```bash
|
||||
# From the monorepo root
|
||||
cd gadget-tasks
|
||||
|
||||
# Development mode (TypeScript, auto-restart)
|
||||
pnpm dev -- --user=admin@example.com --password=secret
|
||||
|
||||
# Or with interactive credential prompt
|
||||
pnpm dev
|
||||
|
||||
# Production (build first)
|
||||
pnpm build
|
||||
pnpm start -- --user=admin@example.com --password=secret
|
||||
```
|
||||
|
||||
### Global CLI
|
||||
|
||||
```bash
|
||||
# Link globally from the monorepo root
|
||||
pnpm link:global
|
||||
|
||||
# Now available as a system command
|
||||
gadget-tasks --user=admin@example.com --password=secret
|
||||
```
|
||||
|
||||
## Task Execution Details
|
||||
|
||||
### Concurrency
|
||||
|
||||
By default, `concurrency: 1` means tasks execute **sequentially**. If a task fires while the drone is busy with another task, the new task waits for the drone to become available. This is the correct model — drones process one work order at a time.
|
||||
|
||||
If you increase `concurrency`, you need multiple drones available. Each task requires a drone lock, and a drone can only be locked to one session at a time.
|
||||
|
||||
### Work Order Tracking
|
||||
|
||||
When `submitPrompt` is called, the callback provides a `turnId`. gadget-tasks stores a Promise resolver in a `pendingWorkOrders` map keyed by `turnId`. When the server emits `workOrderComplete(turnId, success, message)`, the corresponding Promise is resolved, unblocking the task execution.
|
||||
|
||||
### Session Heartbeat
|
||||
|
||||
gadget-tasks sends a `sessionHeartbeat` every 19 seconds (same interval as the browser IDE). This prevents the drone's 120-second heartbeat timeout from firing while a task is active.
|
||||
|
||||
### Error Recovery
|
||||
|
||||
If gadget-tasks crashes while a task is being processed:
|
||||
- The drone continues processing the work order independently
|
||||
- The session and ChatTurn records are preserved in the database
|
||||
- On restart, gadget-tasks creates **new** sessions for tasks that fire
|
||||
- Old sessions are visible in the browser IDE via their ChatTurn records
|
||||
|
||||
No special crash recovery is needed in gadget-tasks (unlike gadget-drone, which has crash recovery for incomplete work orders).
|
||||
|
||||
## What gadget-tasks Does NOT Do
|
||||
|
||||
| ❌ Does NOT | ✅ Instead |
|
||||
|------------|----------|
|
||||
| Connect to MongoDB | Uses REST API to read/write data |
|
||||
| Define Mongoose models | Uses `@gadget/api` TypeScript interfaces |
|
||||
| Call AI APIs | Submits prompts through the platform → drone |
|
||||
| Build system prompts | gadget-code builds prompts from canonical templates |
|
||||
| Execute tools | The drone executes tools via `@gadget/ai-toolbox` |
|
||||
| Manage workspaces | The drone manages workspace directories |
|
||||
| Store credentials in config | Provides them at startup via CLI or prompt |
|
||||
|
||||
## Related Documentation
|
||||
|
||||
- [README](../README.md) — Project overview and quick start
|
||||
- [Agent Toolbox](./agent-toolbox.md) — `@gadget/ai-toolbox` tool implementations (used by drones, not gadget-tasks)
|
||||
- [Socket Protocol](./socket-protocol.md) — Socket.IO protocol between IDE, platform, and drone
|
||||
- [Drone Documentation](../gadget-drone/docs/gadget-drone.md) — How drones execute work orders
|
||||
@ -9,11 +9,7 @@ overrides:
|
||||
|
||||
importers:
|
||||
|
||||
.:
|
||||
dependencies:
|
||||
list:
|
||||
specifier: link:../../snap/code/198/.local/share/pnpm/global/5/node_modules/list
|
||||
version: link:../../snap/code/198/.local/share/pnpm/global/5/node_modules/list
|
||||
.: {}
|
||||
|
||||
gadget-code:
|
||||
dependencies:
|
||||
|
||||
Loading…
Reference in New Issue
Block a user