{"componentChunkName":"component---src-templates-mdproject-jsx","path":"/projects/bkv_gas_targets","result":{"data":{"allGithubFile":{"nodes":[{"base":"bkv_gas_targets.md","relativeDirectory":"projects","childMarkdownRemark":{"id":"84bd7e6e-57fb-5bec-97eb-60f52ef4cef0","html":"<h1>Gas Targets App (React/FastAPI) - BKV Corporation</h1>\n<p>Software Engineer/Developer: Manuel Solalinde <a href=\"mailto:manolosolalinde@gmail.com\">(manolosolalinde@gmail.com)</a></p>\n<p>Every month, reservoir and production engineers must <strong>set the monthly gas-production target for every well</strong>. Doing this well-by-well was a slow, manual grind; this full-stack web application lets an engineer <strong>process the entire well set in a single analysis</strong>, turning that process into about an hour of compute. It combines an interpolation/trend tool purpose-built for this task with guided manual review of the edge cases, then exports the finalized targets as the operator CSV consumed downstream. *(This page documents the <strong>v2.1.0</strong> architecture — a React frontend with a FastAPI backend on AWS App Runner — since superseded by a unified Streamlit redesign on ECS Fargate.)* The heavy lifting — reading raw production history from the corporate data warehouse and fitting a trend per well — runs as a <strong>parallel, queue-driven Lambda fan-out</strong>, so the full well set is processed in one pass instead of one well at a time.</p>\n<h2>Overview</h2>\n<p>Each month an engineer creates an <strong>analysis</strong> (defined by its start/end date) and walks it through a four-stage workflow in the app. The application loads the analysis data from the corporate <strong>Amazon Redshift</strong> warehouse into a private working database, generates a production trend for every well in parallel, then surfaces two case-study reviews so the engineer can spot wells that need manual overrides before the final targets are exported.</p>\n<h2>Workflow</h2>\n<p>The user flow is organized as a set of sequential tabs:</p>\n<ul>\n<li><strong>Database control</strong> — start/stop the RDS working-database instance on demand, so the database only runs (and only bills) while an analysis is in progress.</li>\n<li><strong>Analysis selection</strong> — create a new analysis (the start/end dates form its ID), load an existing one, or delete one to reset its trends and overrides.</li>\n<li><strong>Update DB &#x26; Generate Trends</strong> — two one-time-per-analysis operations: <em>Update DB</em> pulls header, raw production, and restim data from Redshift into the working DB; <em>Generate Trends</em> fans out one job per well to compute its trend. Both run asynchronously and report live status, well counts, and any wells that errored.</li>\n<li><strong>Overrides &#x26; Well Review</strong> — two case studies flag wells that need attention: a <em>Last-7-days production review</em> (wells producing in the recent window) and a <em>Trend review</em> (wells whose computed trend disagrees with their producing status). The engineer reviews each flagged well's timeseries — production vs. forecast, target, and budget — and uploads an Excel overrides sheet (well ID, code, override volume, comments) that is immediately reflected back in the review tables.</li>\n<li><strong>Download final output</strong> — export the finalized monthly gas-targets operator CSV, with all manual overrides applied.</li>\n</ul>\n<h2>Architecture</h2>\n<p>The system is a containerized, cloud-native stack provisioned entirely as code with <strong>Pulumi</strong>:</p>\n<ul>\n<li><strong>Frontend</strong> — a <strong>React</strong> (Redux) single-page app served by an <strong>AWS App Runner</strong> service, authenticated through <strong>Azure AD</strong>, talking to the backend over the internet.</li>\n<li><strong>Backend</strong> — a <strong>Python / FastAPI</strong> API on a second <strong>App Runner</strong> service. It reaches the working database through a <strong>VPC connector</strong> and pushes work onto <strong>SQS</strong> using IAM authentication.</li>\n<li><strong>Worker</strong> — a <strong>Lambda</strong> function triggered by SQS messages. It reads production data from a <strong>cross-account corporate Amazon Redshift</strong> warehouse over scoped access, and reads/writes the working database.</li>\n<li><strong>Data</strong> — a <strong>PostgreSQL RDS</strong> working database holds the per-analysis header, raw, restim, trends, and overrides tables.</li>\n<li><strong>Queue</strong> — an <strong>SQS</strong> queue decouples the API from the workers and is the trigger that drives the parallel fan-out.</li>\n</ul>\n<p>The asynchronous operations follow a consistent pattern: the frontend calls a backend endpoint to kick off the work, then polls a state endpoint until the work reports <code class=\"language-text\">finished</code>. The backend enqueues one SQS message per unit of work (per table for the DB update, per well for trend generation); each message triggers an independent Lambda invocation, and a per-well job-status table tracks each well so the state endpoint knows when the whole batch is complete.</p>\n<h4>Design decisions &#x26; trade-offs</h4>\n<ul>\n<li><strong>On-demand working database</strong> — rather than a shared, always-on warehouse, the RDS working DB is started and stopped per analysis, so it only runs (and only bills) while an analysis is in progress. This keeps cost bounded and leaves the engineer in control of when compute is live.</li>\n<li><strong>Queue-driven fan-out over direct invocation</strong> — instead of the backend invoking workers directly, an <strong>SQS</strong> queue drives the Lambda fan-out. This decouples the API from worker scaling and isolates per-well failures, so one bad well can't take down the batch.</li>\n<li><strong>Production-faithful local development</strong> — locally the environment can't reach the queue or the warehouse, so Redshift reads are <strong>mocked</strong> and the backend <strong>invokes the Lambda directly</strong>. This reproduces the production data flow without any cloud dependencies.</li>\n</ul>\n<h2>CI/CD &#x26; Local Development</h2>\n<ul>\n<li><strong>Deployment</strong> — automated with <strong>GitHub Actions</strong>, which runs the backend test suite, builds Docker images for the frontend, backend, and Lambda, pushes them to <strong>Amazon ECR</strong>, then updates the App Runner / Lambda services.</li>\n<li><strong>Local environment</strong> — a <code class=\"language-text\">docker compose</code> stack runs the frontend, backend, database, and Lambda together on a developer machine. Because the local environment can't reach Redshift or SQS, the warehouse reads are <strong>mocked</strong> and the backend <strong>invokes the Lambda directly</strong> instead of going through the queue — reproducing the production data flow without the cloud dependencies.</li>\n</ul>\n<blockquote>\n<p><em>Note: Screenshots and architecture diagrams are omitted, as they display proprietary client data and systems. Available on request, subject to client approval.</em></p>\n</blockquote>","frontmatter":{"title":"Gas Targets App (React/FastAPI)","image_url":"./images/covers/gas_targets.png","description":"Web application that generates and sets monthly gas-production targets for every well.","technologies":"Python, JavaScript, React, Redux, FastAPI, SQLAlchemy, PostgreSQL, Amazon Redshift, AWS (App Runner, Lambda, SQS, RDS, VPC, ECR, IAM, Elastic IP), Pulumi, Docker, Docker Compose, GitHub Actions, Azure AD, Pandas, Plotly.js, MkDocs.","date":"2026-02-01T00:00:00.000Z"}},"user":"manolosolalinde","repository":"cv"}]},"site":{"siteMetadata":{"title":"Manuel Solalinde's Portfolio","description":"Manuel Solalinde's Portfolio and Featured Projects","author":"Manuel Solalinde"}}},"pageContext":{"uid":"bkv_gas_targets.md","rootQuery":"\n    query mdProjectQuery($uid: String) {\n        allGithubFile(filter: {base: {eq: $uid}, relativeDirectory: {eq: \"projects\"}}) {\n            nodes {\n              base\n              relativeDirectory\n              childMarkdownRemark {\n                id\n                html\n                frontmatter {\n                  title\n                  image_url\n                  description\n                  technologies\n                  date\n                }\n              }\n              user\n              repository\n            }\n        }\n        site {\n            siteMetadata {\n                title\n                description\n                author\n            }\n        }\n    }\n"}},"staticQueryHashes":["3649515864"]}