233 lines
7.8 KiB
Markdown
233 lines
7.8 KiB
Markdown
# Global Installation Implementation Summary
|
|
|
|
## Overview
|
|
|
|
This document summarizes the changes made to enable global installation of `gadget-code` and `gadget-drone` with YAML-based configuration.
|
|
|
|
## Key Changes
|
|
|
|
### 1. MongoDB/Mongoose Cleanup ✅
|
|
|
|
**Problem**: `gadget-drone` had unnecessary `mongoose` dependency for type utilities only.
|
|
|
|
**Solution**:
|
|
- Created `ObjectId` utility in `@gadget/api` (`packages/api/src/lib/objectid.ts`)
|
|
- Exported `Types` from `@gadget/api` for drone to use
|
|
- Removed `mongoose` from `gadget-drone/package.json` dependencies
|
|
- Removed Redis configuration from `gadget-drone` (was never used)
|
|
|
|
**Files Changed**:
|
|
- `packages/api/src/lib/objectid.ts` (new)
|
|
- `packages/api/src/index.ts` (export Types)
|
|
- `packages/api/package.json` (kept mongoose for database models)
|
|
- `gadget-drone/package.json` (removed mongoose)
|
|
- `gadget-drone/src/config/env.ts` (removed redis config)
|
|
- `gadget-drone/src/services/agent.ts` (import Types from @gadget/api)
|
|
- `gadget-drone/src/services/ai.ts` (import Types from @gadget/api)
|
|
|
|
### 2. Configuration System ✅
|
|
|
|
**Problem**: `.env` files tied to project directories, not suitable for global installation.
|
|
|
|
**Solution**: Created `@gadget/config` package for YAML configuration loading.
|
|
|
|
**New Package**: `packages/config/`
|
|
- `src/types.ts` - TypeScript interfaces for config schemas
|
|
- `src/loader.ts` - YAML loading with env var substitution
|
|
- `src/index.ts` - Public API
|
|
|
|
**Features**:
|
|
- Searches `~/.config/gadget/` first, then `/etc/gadget/`
|
|
- Environment variable substitution with `${VAR_NAME}` syntax
|
|
- Path resolution (supports `~` for home directory)
|
|
- Clear error messages with documentation links
|
|
|
|
### 3. gadget-code Updates ✅
|
|
|
|
**Changes**:
|
|
- Removed `dotenv` dependency
|
|
- Added `@gadget/config` dependency
|
|
- Updated `src/config/env.ts` to load YAML config
|
|
- Changed `env.root` to `env.installDir` (calculated from `__dirname`)
|
|
- Updated all path references to use `installDir`:
|
|
- `src/web-app.ts` (asset paths)
|
|
- `src/controllers/api.ts` (controller loading)
|
|
- `src/controllers/api/v1.ts` (child controller loading)
|
|
- Added `bin` entry to `package.json` for global commands
|
|
|
|
**Configuration File**: `~/.config/gadget/gadget-code.yaml`
|
|
- All settings from old `.env` file migrated
|
|
- Organized hierarchically (site, auth, session, mongodb, etc.)
|
|
- Sensitive values use environment variable substitution
|
|
|
|
### 4. gadget-drone Updates ✅
|
|
|
|
**Changes**:
|
|
- Removed `dotenv` dependency
|
|
- Added `@gadget/config` dependency
|
|
- Removed `mongoose` dependency (now uses Types from @gadget/api)
|
|
- Added `numeral` dependency (was missing but used in logging)
|
|
- Updated `src/config/env.ts` to load YAML config
|
|
- Changed `env.root` to `env.installDir`
|
|
- Added `bin` entry to `package.json` for global command
|
|
- Workspace initialization already correctly uses `process.cwd()`
|
|
|
|
**Configuration File**: `~/.config/gadget/gadget-drone.yaml`
|
|
- Platform connection settings (baseUrl, apiKey)
|
|
- Logging configuration with XDG-compliant default paths
|
|
- Environment variable substitution for secrets
|
|
|
|
### 5. Documentation ✅
|
|
|
|
**New File**: `docs/configuration.md`
|
|
- Complete configuration reference for both applications
|
|
- Environment variable substitution guide
|
|
- Migration path from `.env` files
|
|
- Troubleshooting section
|
|
- Security best practices
|
|
- XDG Base Directory specification compliance
|
|
|
|
### 6. Example Configuration Files ✅
|
|
|
|
**Created**:
|
|
- `~/.config/gadget/gadget-code.yaml` - Full example with all settings
|
|
- `~/.config/gadget/gadget-drone.yaml` - Drone configuration example
|
|
|
|
Both files use environment variable substitution for sensitive values.
|
|
|
|
## Architecture Changes
|
|
|
|
### Path Handling
|
|
|
|
**Before**:
|
|
```typescript
|
|
const ROOT_DIR = path.resolve(__dirname, "..", "..");
|
|
// Used for: assets, controllers, logs, everything
|
|
```
|
|
|
|
**After**:
|
|
```typescript
|
|
const INSTALL_DIR = path.resolve(__dirname, "..", "..");
|
|
// Used for: assets, controllers (where code is installed)
|
|
|
|
// Workspace operations use process.cwd()
|
|
// (where drone is run, separate from installation)
|
|
```
|
|
|
|
### Configuration Loading
|
|
|
|
**Before**:
|
|
```typescript
|
|
import "dotenv/config";
|
|
const value = process.env.DTP_SOME_VALUE;
|
|
```
|
|
|
|
**After**:
|
|
```typescript
|
|
import { loadGadgetCodeConfig } from "@gadget/config";
|
|
const config = loadGadgetCodeConfig();
|
|
const value = config.some.value; // with ${ENV_VAR} substitution
|
|
```
|
|
|
|
### Dependency Injection
|
|
|
|
**Before**: All values from environment variables
|
|
**After**: Hierarchical YAML with env var overrides for secrets only
|
|
|
|
## Testing Results
|
|
|
|
### ✅ gadget-drone
|
|
- Loads YAML configuration successfully
|
|
- Initializes workspace in `process.cwd()`
|
|
- Starts services correctly
|
|
- Prompts for credentials (as expected)
|
|
|
|
### ✅ gadget-code
|
|
- Loads YAML configuration successfully
|
|
- Resolves paths from `installDir`
|
|
- Loads controllers correctly
|
|
- Starts Express app with proper asset paths
|
|
- MongoDB/Redis connection errors expected (services not running)
|
|
|
|
## Migration Guide
|
|
|
|
### For Developers
|
|
|
|
1. **Install dependencies**: `pnpm install`
|
|
2. **Build packages**: `pnpm -r build`
|
|
3. **Set environment variables**: Export required secrets
|
|
4. **Run with pnpm**: Use `pnpm dev` commands
|
|
|
|
### For Production Deployment
|
|
|
|
1. **Publish packages** to npm (or private registry)
|
|
2. **Install globally**: `npm install -g gadget-code @gadget/drone`
|
|
3. **Create config files** in `~/.config/gadget/` or `/etc/gadget/`
|
|
4. **Set environment variables** for secrets
|
|
5. **Run commands**: `gadget-code-web`, `gadget-drone`
|
|
|
|
## Breaking Changes
|
|
|
|
1. **`.env` files no longer supported** - Must use YAML configuration
|
|
2. **Configuration location changed** - Now in `~/.config/gadget/` or `/etc/gadget/`
|
|
3. **Environment variables** - Only for secrets, not general configuration
|
|
4. **Path resolution** - `env.root` replaced with `env.installDir`
|
|
|
|
## Benefits
|
|
|
|
1. ✅ **True global installation** - Run from any directory
|
|
2. ✅ **Centralized configuration** - All config in standard locations
|
|
3. ✅ **Better secrets management** - Env vars for sensitive data only
|
|
4. ✅ **Multiple drone instances** - Each workspace independent
|
|
5. ✅ **Cleaner architecture** - Separation of install location vs workspace
|
|
6. ✅ **XDG compliance** - Logs in `~/.local/state/`
|
|
7. ✅ **No MongoDB/Redis in drone** - Pure HTTP + Socket.IO client
|
|
|
|
## Next Steps
|
|
|
|
1. **Update AGENTS.md files** - Document new development workflow
|
|
2. **Add migration script** (optional) - Convert `.env` to YAML
|
|
3. **Test on clean system** - Verify installation from scratch
|
|
4. **Update CI/CD** - Adjust for YAML configuration
|
|
5. **Consider publishing** - Release to npm for true global installation
|
|
|
|
## Files Summary
|
|
|
|
### New Files
|
|
- `packages/config/package.json`
|
|
- `packages/config/tsconfig.json`
|
|
- `packages/config/src/index.ts`
|
|
- `packages/config/src/types.ts`
|
|
- `packages/config/src/loader.ts`
|
|
- `packages/api/src/lib/objectid.ts`
|
|
- `docs/configuration.md`
|
|
- `~/.config/gadget/gadget-code.yaml` (example)
|
|
- `~/.config/gadget/gadget-drone.yaml` (example)
|
|
|
|
### Modified Files
|
|
- `packages/api/src/index.ts`
|
|
- `packages/api/package.json`
|
|
- `gadget-code/package.json`
|
|
- `gadget-code/src/config/env.ts`
|
|
- `gadget-code/src/web-app.ts`
|
|
- `gadget-code/src/controllers/api.ts`
|
|
- `gadget-code/src/controllers/api/v1.ts`
|
|
- `gadget-drone/package.json`
|
|
- `gadget-drone/src/config/env.ts`
|
|
- `gadget-drone/src/services/agent.ts`
|
|
- `gadget-drone/src/services/ai.ts`
|
|
- `gadget-drone/src/gadget-drone.ts`
|
|
|
|
### Removed Dependencies
|
|
- `gadget-code`: `dotenv`
|
|
- `gadget-drone`: `dotenv`, `mongoose`
|
|
|
|
### Added Dependencies
|
|
- `gadget-code`: `@gadget/config` (workspace)
|
|
- `gadget-drone`: `@gadget/config` (workspace), `numeral`
|
|
- `packages/config`: `js-yaml`, `@types/js-yaml`
|
|
|
|
## Conclusion
|
|
|
|
The global installation initiative is complete. Both `gadget-code` and `gadget-drone` can now be installed globally and run from any directory, with configuration managed through YAML files in standard locations. The drone process is now properly isolated with no database dependencies, communicating only via HTTP and Socket.IO.
|