Skip to content

Add GCP infrastructure scripts and configurations#5

Merged
greg-ogs merged 25 commits into
mainfrom
testing
May 9, 2025
Merged

Add GCP infrastructure scripts and configurations#5
greg-ogs merged 25 commits into
mainfrom
testing

Conversation

@greg-ogs

@greg-ogs greg-ogs commented May 9, 2025

Copy link
Copy Markdown
Owner
### Summary
This pull request introduces comprehensive GCP infrastructure setup and configurations using Terraform and custom scripts. It includes resource definitions, managed instance groups, firewall rules, service accounts, VPC setups, and autoscaling policies.

### Key Changes
- Addition of GCP managed instance group configurations for autoscaling and health checks.
- Implementation of Terraform scripts for creating VPCs, subnets, and other network resources.
- Scripts for MySQL setup, Docker registry creation, and service account provisioning.
- Introduction of Docker files and authentication mechanisms for easier deployment.
- Setup of HTTP Load Balancer with associated path-based routing and static IP reservation.
- Updates to `.gitignore`, variable refactoring, and enhanced security handling via secrets.

greg-ogs added 24 commits April 25, 2025 15:06
Create a managed instance group using an nginx template and configure health checks, autoscaling, and named ports. Include firewall updates for HTTP and health check traffic, and log instance group details for verification.
…proxy setup Kubernetes: Sql pod using embedded proxy (1 container).

Implement a script to create a GCP service account with roles for Artifact Registry, GCS, and Cloud SQL. Add a script for creating a Docker registry in Artifact Registry. Include setup files (Dockerfile, docker-compose, entrypoint) for an embedded SQL proxy Flask application.
Introduced a `Dockerfile` and `entrypoint.sh` for the sidecar-proxy service to handle Flask and Cloud SQL Proxy setup. Updated `docker_registry.sh` script to improve registry handling and add logic for tagging and pushing images. This enhances support for custom workloads and streamlines image deployment.
Added detailed configurations for MySQL instances, Private Services Access (PSA), and secret management into task scripts. Refactored network, subnet, and firewall setups while enabling relevant APIs (SQL Admin, Secret Manager) to streamline the deployment workflow. Included new regional Managed Instance Group for alchemy with autoscaling and additional metadata handling.
Added an environment variable to specify the path for Google Application Credentials. This ensures proper authentication for accessing Google Cloud resources during the script execution.
Uncommented and finalized scripts to set up a MySQL database, create instance templates, and configure a regional Managed Instance Group (MIG) for task automation. Added secret retrieval for database credentials, enabled Private Services Access, and incorporated autoscaling and health checks for reliability. Introduced a new `alchemy-mig.sh` script to manage and configure an additional regional MIG specifically for the "alchemy" service.
Cleaned up the script by removing MySQL-related variables and redundant secret retrievals. Focused the script to streamline Managed Instance Group (MIG) setup for the "alchemy" service. Ensured optimal regional configuration with essential parameters only.
Introduce a new script `task_8.sh` and refine `task_3.sh` to replace standard images with machine images for instance creation and configuration. Unused commented code is removed to improve clarity and maintainability.
Introduce scripts for managing VMs, custom VPCs, private service access, and peering configurations in GCP. Include SQL setup for Alchemy application and related changes in instance groups.
Updated container image versions and ports in `docker-compose.yml` and `Dockerfile`. Enhanced authentication by integrating Google Cloud secrets and adding Cloud SDK installation in the Docker image. Re-enabled and refined the Docker registry creation script for better image handling.
Introduced a new script to automate the creation of a private Google Kubernetes Engine (GKE) cluster with detailed specifications, including network setup, Cloud NAT configuration, and node pool for SQL. Additionally, removed an unnecessary space in an existing script to maintain code consistency.
Modified the Managed Instance Group (MIG) setup to operate within a single zone instead of multiple zones for better alignment with the specified configuration. Updated all relevant commands to replace regional arguments with zonal ones and adjusted associated variables accordingly.
This script sets up an HTTP Load Balancer with path-based routing and a reserved static IP. It routes `/nginx` requests to the nginx Managed Instance Group (MIG) and other requests to the alchemy MIG. The configuration includes backend services, a URL map, an HTTP proxy, and a forwarding rule.
Created a Dockerfile to use the latest Terraform image, setting up a working directory and entry point for Terraform commands. Updated `.gitignore` to exclude specific Terraform-related directories.
Introduce a Terraform setup to manage GCP resources, specifically creating a VPC and subnetworks. Updated the Dockerfile to include a service account configuration comment. Adjusted `.gitignore` to exclude Terraform-related files and credentials for better security.
Introduced a Google Compute Project Metadata resource to manage SSH keys. Added a Google Cloud Storage bucket with a unique name. Refactored variables and outputs into separate files for better structure and maintainability.
Renamed student-related variables across the Terraform configuration. Introduced a new service account resource with IAM binding for Storage Object Creator role. Updated outputs to include the service account email for reference.
Introduce SSH and HTTP firewall rules for secure VPC access. Add new variables for safe IP ranges and load balancer configurations. Implement a GitHub CodeQL workflow for advanced code scanning and security analysis.
These changes add outputs for VPC ID, subnetwork IDs, bucket ID, and project metadata ID. This enhances the configuration's flexibility by exposing these values for use in other modules or external integrations.
This commit introduces a new application.tf file in the base Terraform directory. The file likely contains foundational configurations necessary to set up the application infrastructure. This is a key step in structuring the Terraform codebase.
Moved compute-related configurations to a dedicated directory. Added remote state data source to reference base outputs and defined GCP provider, variables, and necessary configurations for compute resources.
Replaced the `project_id` variable with remote state output for better consistency and removed unused variable definition. Simplified `.gitignore` by consolidating patterns related to `.gcp`.
Introduced instance templates and regional instance group managers for `us-central1` and `us-east1`, along with HTTP health checks and auto-healing policies. Included a startup script to configure nginx, fetch instance metadata, and update the default page. Updated project ID references and refined outputs for compatibility.
@greg-ogs greg-ogs self-assigned this May 9, 2025
@greg-ogs greg-ogs added the enhancement New feature or request label May 9, 2025
# Conflicts:
#	terraform/base/outputs.tf
@greg-ogs greg-ogs merged commit 6c76cac into main May 9, 2025
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant