E.Huang
Back to projects

RapidAPI Platform

A collection of eight Go APIs published on the RapidAPI marketplace covering AI rankings, federal contracts, and data aggregation — all provisioned through a single reusable Terraform module for consistent cloud deployment.

GoTerraformGoogle Cloud FunctionsCloud RunFirestorePlaywrightOpenAPI

Each API in the collection covers a different data domain — AI leaderboard rankings, fashion resale inventory, federal contract opportunities — but they all ship through the same Go service contract and Terraform module. A single parameterized module wires each Cloud Function to its GCS source archive, IAM bindings, and Secret Manager entries, so adding a new API means adding one entry to terraform.tfvars and writing the Go handler.

The multi-source aggregators handle partial failure differently depending on the data. The fashion-resale API returns a 200 with per-platform error details when individual sources are down. The AI rankings aggregator takes a different approach: it serves stale data for any source that becomes unavailable and refreshes it in the background, preventing concurrent requests from hammering the upstream during recovery. A model name normalizer reconciles naming differences across leaderboards — one source timestamps its model names, another doesn’t — so the merged dataset is consistent.

Federal contracts required a different architecture. SAM.gov enforces daily per-subsystem rate quotas, so the data can’t be scraped on request. Instead, a Cloud Run Job runs on two schedules — an incremental refresh every 6 hours and a full refresh weekly — writing pre-aggregated documents to Firestore that the Cloud Function reads. Per-subsystem call budgets prevent any one category from exhausting the daily quota; if a budget runs out, the job marks the run as data-truncated rather than failing, so every document in Firestore is complete to some level even if not fully current.