project-standalo-note-to-app/skills/guardrail-orchestrator/scripts/MIGRATION_GUIDE.md

# Task Session Migration Guide

## Overview

This guide explains how to migrate task sessions from the old flat file structure to the new directory-based structure.

## Background

### Old Structure (Flat Files)
```
.workflow/versions/v001/task_sessions/
├── task_design.yml
├── task_implementation.yml
└── task_review.yml
```

### New Structure (Directories)
```
.workflow/versions/v001/task_sessions/
├── task_design/
│   ├── session.yml       # Session data (execution info)
│   ├── task.yml          # Task snapshot (definition at execution time)
│   └── operations.log    # Human-readable operation log
├── task_implementation/
│   ├── session.yml
│   ├── task.yml
│   └── operations.log
└── task_review/
    ├── session.yml
    ├── task.yml
    └── operations.log
```

## Benefits of New Structure

1. **Better Organization**: Each task session has its own directory
2. **Snapshot Preservation**: Task definitions are captured at execution time
3. **Human-Readable Logs**: Operations log provides easy-to-read history
4. **Extensibility**: Easy to add attachments, artifacts, or outputs per task
5. **Backwards Compatible**: Old code can still read from either structure

## Migration Script

### Location
```
skills/guardrail-orchestrator/scripts/migrate_task_sessions.py
```

### Usage

#### Dry Run (Recommended First Step)
```bash
python3 skills/guardrail-orchestrator/scripts/migrate_task_sessions.py --dry-run
```

This will:
- Find all flat task session files
- Report what would be migrated
- Show actions that would be taken
- **NOT make any changes**

#### Live Migration
```bash
python3 skills/guardrail-orchestrator/scripts/migrate_task_sessions.py
```

This will:
- Create directory for each task session
- Move session data to `session.yml`
- Create `task.yml` snapshot
- Generate `operations.log`
- Delete original flat files

## Migration Process

### What the Script Does

For each flat task session file (e.g., `task_design.yml`):

1. **Create Directory**: `task_sessions/task_design/`

2. **Move Session Data**:
   - Read original `task_design.yml`
   - Save to `task_design/session.yml`
   - Delete original file

3. **Create Task Snapshot**:
   - Look for `tasks/task_design.yml`
   - If found: Copy and add snapshot metadata
   - If not found: Create minimal task.yml from session data
   - Save to `task_design/task.yml`

4. **Create Operations Log**:
   - Initialize `task_design/operations.log`
   - Add migration note
   - If session has operations array, convert to log format
   - Human-readable format with timestamps

### Task Snapshot Metadata

When a task definition is found, these fields are added:
```yaml
snapshotted_at: '2025-12-16T12:00:00'
source_path: 'tasks/task_design.yml'
status_at_snapshot: 'completed'
migration_note: 'Created during migration from flat file structure'
```

### Operations Log Format

```
# Operations Log for task_design
# Migrated: 2025-12-16T12:00:00
# Format: [timestamp] OPERATION target_type: target_id (path)
======================================================================

[2025-12-16T12:00:00] MIGRATION: Converted from flat file structure

# Historical operations from session data:
[2025-12-16T11:00:00] CREATE file: auth.ts (app/lib/auth.ts)
    Summary: Created authentication module

[2025-12-16T11:15:00] UPDATE entity: User (app/lib/types.ts)
    Summary: Added email field to User type
```

## Migration Results

### Success Output
```
======================================================================
                          Migration Summary
======================================================================
Total files processed: 3
Successful migrations: 3
Failed migrations: 0

Migration completed successfully!

Next steps:
  1. Verify migrated files in .workflow/versions/*/task_sessions/
  2. Check that each task has session.yml, task.yml, and operations.log
  3. Test the system to ensure compatibility
```

### Dry Run Output
```
Processing: v001/task_design.yml
----------------------------------------------------------------------
  Would create directory: .workflow/versions/v001/task_sessions/task_design
  Would move task_design.yml to .workflow/versions/v001/task_sessions/task_design/session.yml
  Would create .workflow/versions/v001/task_sessions/task_design/task.yml (if source exists)
  Would create .workflow/versions/v001/task_sessions/task_design/operations.log

This was a DRY RUN. No files were modified.
Run without --dry-run to perform the migration.
```

## Verification Steps

After migration, verify the structure:

```bash
# Check directory structure
ls -la .workflow/versions/v001/task_sessions/task_design/

# Should show:
# session.yml
# task.yml
# operations.log

# Verify session data
cat .workflow/versions/v001/task_sessions/task_design/session.yml

# Verify task snapshot
cat .workflow/versions/v001/task_sessions/task_design/task.yml

# Check operations log
cat .workflow/versions/v001/task_sessions/task_design/operations.log
```

## Backwards Compatibility

The `version_manager.py` module includes backwards-compatible loading:

```python
def load_task_session(version: str, task_id: str) -> Optional[dict]:
    """Load a task session from directory or flat file (backwards compatible)."""
    # Try new directory structure first
    session_dir = get_version_dir(version) / 'task_sessions' / task_id
    session_path = session_dir / 'session.yml'

    if session_path.exists():
        return load_yaml(str(session_path))

    # Fallback to old flat file structure
    old_path = get_version_dir(version) / 'task_sessions' / f'{task_id}.yml'
    if old_path.exists():
        return load_yaml(str(old_path))

    return None
```

This means:
- New code works with both structures
- No breaking changes for existing workflows
- Migration can be done gradually
- Rollback is possible if needed

## Troubleshooting

### No Files Found
If the script reports "No flat task session files found":
- Check that `.workflow/versions/` exists
- Verify that task sessions are in expected location
- Confirm files have `.yml` or `.yaml` extension
- May indicate all sessions are already migrated

### Task File Not Found
If `tasks/task_id.yml` doesn't exist:
- Script creates minimal task.yml from session data
- Warning is logged but migration continues
- Check `task.yml` has `migration_note` field

### Migration Errors
If migration fails:
- Review error message in output
- Check file permissions
- Verify disk space
- Try dry-run mode to diagnose

### Rollback (If Needed)
To rollback a migration:
1. Stop any running workflows
2. For each migrated directory:
   ```bash
   # Copy session.yml back to flat file
   cp .workflow/versions/v001/task_sessions/task_design/session.yml \
      .workflow/versions/v001/task_sessions/task_design.yml

   # Remove directory
   rm -rf .workflow/versions/v001/task_sessions/task_design/
   ```

## Best Practices

1. **Always dry-run first**: Use `--dry-run` to preview changes
2. **Backup before migration**: Copy `.workflow/` directory
3. **Migrate per version**: Test one version before migrating all
4. **Verify after migration**: Check files and run system tests
5. **Keep old backups**: Don't delete backups immediately

## Integration with Workflow System

After migration, all workflow operations work seamlessly:

```python
# Start task session (creates directory structure)
session = create_workflow_session("new feature", None)
task_session = create_task_session(session, "task_api", "create", "backend")

# Load task session (works with both structures)
task = load_task_session("v001", "task_design")

# Log operations (appends to operations.log)
log_operation(task, "CREATE", "file", "api.ts", target_path="app/api/api.ts")
```

## Additional Resources

- `version_manager.py`: Core versioning system
- `workflow_manager.py`: Workflow orchestration
- `.workflow/operations.log`: Global operations log
- `.workflow/index.yml`: Version index