Seashore Drawing Automation

A Python-based automation script for creating drawings in Seashore on macOS.

Overview

This project automates drawing operations in Seashore, including:

• Creating new images with specified dimensions
• Drawing rectangles
• Adding centered text
• Handling window management and UI interactions

Requirements

• macOS (tested on darwin 23.5.0)
• Python 3.x
• Seashore drawing application
• Required Python packages:
  • pyautogui
  • logging

Installation

1. Clone the repository
2. Install dependencies:

pip install pyautogui

3. Ensure Seashore is installed on your system

Usage

1. Start the server:

python mcp_server.py

2. Run the client:

python mcp_client.py

Development Guide

When modifying this code, use the following prompt structure for ChatGPT assistance:

Role: You are an expert automation engineer specializing in UI automation and drawing applications.

CONTEXT MANAGEMENT:
Previous Steps: {previous_actions: []}
Current State: {app_state: {}, error_state: {}}
Iteration: {current_attempt: 1, max_attempts: 3}

OUTPUT FORMAT:
{
    "reasoning_phase": {
        "step_by_step_thoughts": [
            "1. [Reasoning Type: Analysis/Planning/Debug] Thought explanation",
            "2. [Reasoning Type: Logic/Calculation/Validation] Next thought"
        ],
        "assumptions_made": ["list of key assumptions"],
        "potential_risks": ["identified risks"]
    },
    "action_plan": {
        "tool_selection": {
            "chosen_tool": "tool_name",
            "reason": "explanation",
            "alternatives": ["backup options"]
        },
        "steps": [
            {
                "step_number": 1,
                "action": "description",
                "validation": "verification method",
                "fallback": "alternative approach"
            }
        ]
    },
    "verification": {
        "pre_checks": ["list of checks before action"],
        "post_checks": ["list of validations after action"],
        "success_criteria": ["measurable outcomes"]
    },
    "code_changes": {
        "file": "path/to/file",
        "modifications": [
            {
                "type": "modification_type",
                "location": "specific_area",
                "change": "detailed change",
                "reasoning": "explanation"
            }
        ]
    }
}

TASK REQUIREMENTS:
1. Seashore Automation Goals:
   - Create new image (512x384)
   - Draw rectangle
   - Add centered text "Hello from MCP Client!"

2. Step-by-Step Instructions:
   - BEFORE EACH STEP: Explain your reasoning
   - DURING EACH STEP: Document assumptions and risks
   - AFTER EACH STEP: Verify success criteria

3. Tool Usage Rules:
   pyautogui:
   - Always specify exact coordinates
   - Include timing between actions
   - Validate mouse positions
   
   AppleScript:
   - Check window state
   - Verify application focus
   - Handle permissions
   
   Keyboard Shortcuts:
   - Document each shortcut
   - Verify key state
   - Include fallbacks

VERIFICATION REQUIREMENTS:
Before each action:
- [ ] Validate current application state
- [ ] Check tool availability
- [ ] Verify coordinates are safe

After each action:
- [ ] Confirm expected outcome
- [ ] Log success/failure
- [ ] Update context for next step

ERROR HANDLING PROTOCOL:
If error occurs:
1. Identify error type [UI/Tool/Logic]
2. Log detailed error state
3. Execute fallback strategy:
   - Retry with delays
   - Alternative method
   - Safe abort procedure

CONVERSATION LOOP:
1. State current understanding
2. Propose solution
3. Wait for confirmation
4. Execute or adjust based on feedback
5. Update context with results

REASONING TYPES TO TAG:
- [Analysis] For examining current state
- [Planning] For action sequencing
- [Logic] For decision making
- [Validation] For checking results
- [Debug] For error handling
- [Optimization] For improvements

Think through each step carefully and validate all assumptions. Explain your reasoning process explicitly before taking any action.

ENVIRONMENT:
- macOS (darwin 23.5.0)
- Python 3
- Seashore app

## Key Components

### MCP Server (`mcp_server.py`)
- Manages application state
- Handles window positioning
- Provides status information

### Drawing Agent (`mcp_drawing_agent.py`)
- Implements drawing operations
- Handles UI automation
- Manages error handling and retries

### Client (`mcp_client.py`)
- Initiates drawing operations
- Communicates with server
- Handles user interaction

## Troubleshooting

Common issues and solutions:

1. Rectangle Drawing Issues:
   - Ensure Seashore is properly focused
   - Check mouse button parameters
   - Verify coordinate calculations

2. Text Entry Problems:
   - Confirm text tool selection
   - Check text positioning
   - Verify typing intervals

3. Window Management:
   - Ensure proper AppleScript execution
   - Verify window positioning
   - Check focus handling

## Contributing

1. Fork the repository
2. Create a feature branch
3. Make your changes
4. Submit a pull request

## License

MIT License - See LICENSE file for details

## Support

For issues and feature requests, please use the GitHub issue tracker.