GCP-AI IDE Integration

This project provides a Cloud-Based NLP Control Interface for GCP, allowing natural language commands to be executed against Google Cloud resources via an IAP-secured Cloud Run service.

Architecture Overview

1. Bootstrap Script (bootstrap.sh): Executed once (e.g., in Cloud Shell) to set up all initial GCP resources and build necessary container images using Cloud Build.
2. Artifact Registry: Stores custom-built container images (Terraform runner, API service).
3. GCS Bucket: Stores Terraform state.
4. Terraform (terraform/): Defines GCP infrastructure (Service Account, Cloud Run service, Secret Manager, IAP).
5. Cloud Build: Used by the bootstrap script to build images and run Terraform.
6. Cloud Run Service (mcp-api): Hosts the Flask application, secured by IAP.
7. Flask Application (containers/mcp-api/): Receives authenticated requests, interprets simple commands (e.g., list_projects), and interacts with GCP APIs using the attached service account.

Deployment

This project uses a single script for end-to-end deployment.

Prerequisites

• gcloud CLI installed and authenticated.
• Docker installed and running (for initial image builds if not using Cloud Build for everything).
• Permissions to enable APIs, create repositories, buckets, service accounts, Cloud Run services, Cloud Build jobs, and manage IAM in the target GCP project.
• A cloned copy of this repository.

Steps

1. Navigate to the repository root directory.

2. Run the bootstrap script, providing your Project ID and desired Region:

   chmod +x bootstrap.sh
   ./bootstrap.sh <YOUR_GCP_PROJECT_ID> <YOUR_REGION>
   # Example: ./bootstrap.sh <YOUR_GCP_PROJECT_ID> us-central1 
   

This script performs the following actions:

• Sets the gcloud project.
• Enables required GCP APIs.
• Creates an Artifact Registry repository (mcp-internal).
• Creates a GCS bucket for Terraform state (<project-id>-tfstate).
• Builds a custom Terraform image and pushes it to Artifact Registry using the default Cloud Build pool.
• Creates a private Cloud Build worker pool (mcp-pool) peered with the default VPC network.
• Runs terraform init and terraform apply using the private pool and the custom Terraform image.
• Builds the Flask API container image and pushes it to Artifact Registry using the default Cloud Build pool.
• Deploys the Flask API image to the Cloud Run service (mcp-api) using the default Cloud Build pool.
• Outputs the final API URL.

Detailed logs for each phase are saved to .log files in the execution directory.

Signed-URL Bootstrap Workflow (No Git, No Key Files)

What Has Been Automated

• Personal Gmail Auth: You log in to Google Cloud with your personal Gmail (e.g., cklose@gmail.com) using gcloud auth login. No service account key files are needed.
• Service Account Creation: The script automatically creates a gcp-storage-uploader service account with the necessary Storage permissions.
• Impersonation Setup: Your Gmail is granted the Service Account Token Creator role on this service account, allowing you to impersonate it for secure operations.
• Tarball Upload: The script packages your project as project.tgz and uploads it to a GCS bootstrap bucket using your user credentials.
• Signed URL Generation: The script generates a signed download URL for the tarball using service account impersonation (no manual key download required).
• Cloud Shell Bootstrap: The script prints a one-liner for Cloud Shell to pull and extract your project, then run bootstrap.sh.

How to Use

1. Authenticate:

   gcloud auth login
   # Log in with your personal Gmail account
   

2. Run the Upload Script:

   $env:GCP_PROJECT_ID='<YOUR_GCP_PROJECT_ID>'
   $env:GCP_BUCKET_NAME='<YOUR_GCS_BUCKET>'
   ./push_to_cloudshell.ps1
   

3. Copy the Cloud Shell One-Liner:
   • Paste it into Cloud Shell to extract your project and run the bootstrap script.

Security & Simplicity

• No Git credential helpers, no manual key files, no Cloud Source Repositories.
• All permissions are managed via IAM and impersonation.
• You only need to log in as yourself and have the right IAM roles.

Troubleshooting

• If impersonation fails, ensure your Gmail has the Service Account Token Creator role on the gcp-storage-uploader service account.
• If the signed URL expires, just re-run the script to generate a fresh one.

Next Steps

1. Test the End-to-End Flow:
   • Paste the generated Cloud Shell one-liner into Cloud Shell and confirm your project boots and runs bootstrap.sh.
2. Automate Further:
   • Optionally, add this workflow to CI/CD (e.g., GitHub Actions) to push new versions automatically.
   • Add a Cloud Shell alias for repeated pulls, as described above.
3. Project Hardening:
   • Review and restrict IAM permissions as needed for least privilege.
   • Harden your bootstrap and deployment scripts for error handling and idempotency.
4. Expand Documentation:
   • Document any customizations, advanced usage, or troubleshooting tips as your workflow evolves.

Fast Cloud Shell Bootstrap (No Git Required)

If you want to move your project into Cloud Shell or Cloud Build without Git, credential helpers, or ZIP uploads, use the signed-URL workflow below. This is 100% scripted and works from PowerShell, bash, or CI:

1. Upload with PowerShell (Windows, Mac, Linux)

Run the provided script to package and upload your project:

$env:GCP_PROJECT_ID='<YOUR_GCP_PROJECT_ID>'
$env:GCP_BUCKET_NAME='<YOUR_GCS_BUCKET>'
./push_to_cloudshell.ps1

This will:

• Package your project as project.tgz
• Ensure a GCS bootstrap bucket exists
• Generate a signed URL for upload (no credentials needed)
• Upload the tarball
• Print a one-liner for Cloud Shell

2. Pull & Unpack in Cloud Shell

Copy the green one-liner from the script output and paste it into Cloud Shell:

curl -s <SIGNED_URL> | tar xz && cd gcp_admin2 && ./bootstrap.sh

This will extract your project and run the bootstrap script.

Optional: Add Cloud Shell Alias

Add this to your ~/.bashrc in Cloud Shell for fast pulls:

alias pullproj='curl -s $(gsutil signurl -d 15m ~/.sign.json gs://<YOUR_GCS_BUCKET>/project.tgz | tail -1) | tar xz'

Now, after each push, just type pullproj in Cloud Shell.

Why This Beats CSR + Git Helpers

• No credential helper needed; upload uses signed URL (no auth required)
• No API enable or triggers
• Works behind locked-down proxies (HTTPS PUT)
• Git is optional (commit locally, tarball when ready)
• Easy to script into CI pipelines

Testing the API

Once the deployment is complete, you can test the API using curl and gcloud to get an identity token:

1. Get the API URL (output from the script, or run gcloud run services describe mcp-api --region <YOUR_REGION> --format='value(status.url)').

2. Send a test request:

   API_URL="<YOUR_API_URL>"
   curl -H "Authorization: Bearer $(gcloud auth print-identity-token --audiences=$API_URL)" -H "Content-Type: application/json" -d '{"message": "list_projects"}' $API_URL/chat
   

   Replace <YOUR_API_URL> with the actual URL.

Troubleshooting

• Consult the detailed logs generated by bootstrap.sh (bootstrap-*.log, bootstrap-*.log.tf.log, bootstrap-*.log.api.log).
• Check the Cloud Build history in the GCP Console for detailed build step logs.
• Verify resources (AR repo, GCS bucket, Cloud Run service, SA, Secret) exist in the GCP Console.
• Use gcloud commands to check status (e.g., gcloud run services describe mcp-api ..., gcloud artifacts docker images list ...).
• Refer to troubleshooting.md for common issues (Note: Some specific solutions may be outdated due to the script change).

Development

• Infrastructure: Modify files in the terraform/ directory.
• API Logic: Modify containers/mcp-api/main.py.
• Dependencies: Update containers/mcp-api/requirements.txt.

After making changes, re-run the ./bootstrap.sh script to apply infrastructure updates and redeploy the API.

Available Commands

Currently, the API supports these commands:

• list_projects - Lists all accessible GCP projects
• list_buckets - Lists all accessible GCS buckets

IDE Integration

Follow these steps to integrate with your IDE:

1. Obtain a Google ID token:

gcloud auth print-identity-token --audiences=<YOUR_API_URL>

2. Send a request to the API with the token in the Authorization header:

Authorization: Bearer <YOUR_TOKEN>

Roadmap

• Enhanced natural language processing with Vertex AI
• Additional GCP commands (VM operations, BigQuery queries, etc.)
• Improved error handling and user feedback
• Security hardening with VPC Service Controls

Contributing

Contributions are welcome! Please open an issue or PR for any changes.

---

🚀 GCP Signed-URL Bootstrap & Natural Language IDE Integration

What This Accomplishes

• Zero-touch project upload & bootstrap: Upload your entire project to Google Cloud Shell using a signed URL—no git, no zip, no service account keys.
• Automated IAM & API setup: Scripts handle all permissions and API enablement needed for GCP automation.
• Org policy/VPC-SC block bypass: By using a personal sandbox project, you can prototype and automate without waiting for admin intervention.
• Full logging & diagnostics: Every step and error is logged for audit and troubleshooting.
• Natural language IDE integration: Use Windsurf (this IDE) to communicate with your GCP projects via natural language commands.

How We Got Here

1. Problem: Org policies and VPC Service Controls blocked API enablement and service account creation, even for Project Owners.
2. Solution:
   • Built a PowerShell script to tar, upload, and generate a signed URL for the project.
   • Created a bootstrap script for Cloud Shell that auto-detects project/region, sets up IAM, enables APIs, and logs everything.
   • Automated IAM role assignment and fast-fail diagnostics for permissions.
   • Used a personal sandbox GCP project to bypass org-level blocks.
3. Result:
   • Fully automated, repeatable, and portable GCP bootstrap workflow.
   • No manual IAM, API, or upload steps required.

How to Repeat This Workflow

A. Create a new sandbox project:

gcloud projects create <YOUR_GCP_PROJECT_ID> --set-as-default

B. On your laptop, run:

$env:GCP_PROJECT_ID='<YOUR_GCP_PROJECT_ID>'
$env:GCP_BUCKET_NAME='<YOUR_GCS_BUCKET>'
./push_to_cloudshell.ps1

C. In Cloud Shell (in your new project):

• Copy-paste the command output by the script:

curl -s '<signed-url>' | tar xz && cd gcp_admin2 && chmod +x bootstrap.sh && ./bootstrap.sh

D. Bootstrap is now fully automated!

Example: Using Windsurf (This IDE) for Natural Language GCP Automation

Once your project is bootstrapped, you can use Windsurf to:

• Query GCP resources:

  "List all Cloud Run services in my project."

• Deploy infrastructure:

  "Deploy the latest version of my Flask API to Cloud Run."

• Automate IAM and API setup:

  "Grant my colleague editor access and enable the Vision API."

• Troubleshoot errors:

  "Why did my terraform apply fail? Show me the latest error logs."

Windsurf interprets your natural language requests and executes the necessary GCP commands/scripts, giving you a conversational interface to your cloud projects.

---

Bottom Line:

• You now have a fully automated, script-driven, and natural-language-enabled workflow for GCP project bootstrapping and management.
• Use your sandbox for rapid prototyping, and repeat this process for any new project or environment.

For more details, see log5.md for a full audit trail and troubleshooting history.