Lead Software Engineer

Description The Lead Software Engineer serves as the primary technical lead responsible for the full-stack software operations, maintenance, troubleshooting, and continuous improvement of NOAA’s Office of Water Prediction (OWP) enterprise web and data dissemination platform. This includes sustaining the flagship National Water Prediction Service (NWPS), the Precipitation Frequency Data Server (PFDS), the National Snow Analysis (NSA), River Forecast Center (RFC) water supply web portals, and the weather.gov Content Management System (CMS). Operating within the NWS Enterprise Cloud (NEC) Amazon Web Services (AWS) public cloud environment and legacy on-premises systems, the Lead Software Engineer oversees a cloud-native, containerized architecture (AWS Fargate, Aurora PostgreSQL, S3, EC2) and automated CI/CD pipelines. The position is responsible for delivering Tier 3 operational support, resolving critical application defects, optimizing cloud compute costs, leading small O&M functional enhancements, and enforcing strict NOAA security, coding, and quality control standards Requirements System Operations & High Availability: Maintain, sustain, and improve NWPS and related OWP web applications to reliably deliver mission-critical hydrologic observations, river forecasts, and flood inundation maps to emergency managers and the public with 24x7 high availability during high-impact weather events. Tier 3 Operational Support & Incident Response: Provide Tier 3 technical support for operational systems, responding to urgent operational issues within 1 hour (or 30 minutes after regular business hours). Conduct thorough troubleshooting, deliver Root Cause Analysis (RCA) documentation, and track ticket resolutions in VLab Redmine/GitLab. Full-Stack Maintenance & Small Enhancements: Implement minor functional updates and O&M enhancements, including Quantitative Precipitation Estimate (QPE) processing fixes, bulk HydroNotes, dynamic hydrograph configuration, CMS gauge updates, and KML/KMZ data creation. DevSecOps & Automated CI/CD Pipelines: Build, maintain, and optimize automated CI/CD pipelines in VLab GitLab and GitHub. Integrate automated static code analysis, unit testing, dependency scanning, and vulnerability patches prior to code promotion. Code Quality & Testing Governance: Enforce the mandatory =80% statement test coverage threshold across all frontend (JavaScript) and backend (Python, Go, C/C++) code logic, while maintaining an Acceptable Quality Level (AQL) of 0 medium or high-level static linting errors. Infrastructure as Code (IaC) & Cloud Optimization: Refactor and maintain cloud-agnostic Terraform modules to eliminate configuration drift, optimize AWS auto-scaling groups, manage IAM roles/secrets, and reduce compute/storage costs. Release & Change Control Management: Prepare technical documentation, conduct peer code reviews, and present planned software changes to the NOAA Change Control Board (CCB) for formal release approval. Stakeholder & Partner Collaboration: Participate in weekly status meetings with regional hydrologists, Tier 1/Tier 2 support teams, and OWP leadership. Maintain comprehensive developer documentation, API schemas, and architecture diagrams in version-controlled repositories. Experience Required Education: Bachelor’s Degree in Computer Science, Information Systems, Software Engineering, Mathematics, or a related computational field. Total Professional Experience: Minimum of 5 years (with 10+ years highly preferred) of progressive experience in software engineering, web application development, and cloud-optimized data dissemination solutions. Domain & Application Experience: Demonstrated experience building, scaling, and maintaining mission-critical, public-facing, cloud-native web applications serving high-concurrency user volumes and high-volume scientific/geospatial datasets. DevOps & Agile Cadence: Proven track record working within Agile/Scrum development methodologies, managing software repositories in GitLab/GitHub, and enforcing CI/CD test automation harnesses. Production Support: Direct experience providing Tier 3 operational support, after-hours callout support, and SLA-driven incident management for high-availability systems. Technical Skills Required Programming Languages: Proficiency in Python (following PEP 8 guidelines), C/C++ (following SEI CERT and Google C++ Style Guides), Go, JavaScript/TypeScript, and Bash scripting. AWS Cloud Services: Direct hands-on experience with AWS cloud-native architecture, including AWS Fargate / ECS container orchestration, Amazon Aurora PostgreSQL, Amazon S3, Amazon EC2, AWS Lambda, CloudWatch, and IAM/Secrets Manager. Infrastructure as Code (IaC) & Containers: Proficiency with Terraform for cloud infrastructure automation, Docker containerization, and container registry image security scanning. Software Testing & QA Frameworks: Expertise designing test suites using Pytest, Gradle, CMake, and automated quality/security tools (e.g., Dependabot, AddressSanitizer, Ruff, Flake8, Black). Geospatial & Data Technologies: Experience integrating geospatial tools and frameworks, including Esri ArcGIS Enterprise on Linux, FOSS4G open-source tools, Open Geospatial Consortium (OGC) standards (WMS, WFS), Cloud-Optimized GeoTIFFs (COGs), Zarr, GeoJSON, and NetCDF formats. Version Control & CI/CD Tooling: Expert knowledge of VLab GitLab, GitHub Actions, VLab Redmine ticketing, Git branching strategies (Conventional Branching), and automated pipeline security scanning. Security & Accessibility Compliance: Deep understanding of federal IT security standards (FISMA, NIST controls) and accessibility testing for WCAG 2.1 AA and Section 508 compliance.