Senior Backend Engineer - Monitoring and Anomaly Detection (Monetization) - Gitlab

The Observability, Monitoring, and Integrations team sits within GitLab's Monetization section and owns end-to-end observability, monitoring, and detection across the systems that power how customers buy and use GitLab. As a Backend Engineer on the Fulfillment Workflow Monitoring (Catch All) team, you'll build and operate telemetry, detection, and reconciliation tooling that identifies billing, data, and event anomalies across CustomersDot, Salesforce, and Zuora before they affect revenue or the customer experience. This is a greenfield team, and you'll help build it from the ground up. You'll shape its operating rhythm, incident handling model, and quality bar while using telemetry, artificial intelligence, and machine learning to proactively detect and resolve issues. Because Monetization changes can directly affect revenue and often involve confidential or financial information, you'll use strong judgment, careful rollout practices, and a high bar for reliability. We're opening this role at the Senior Software Engineer and Staff Software Engineer levels. Automated billing anomaly detection for event drops, abuse spikes, and transactional discrepancies End-to-end telemetry, tracing, and reconciliation across CustomersDot, Salesforce, Zuora, and usage and billing pipelines What you’ll do - Design, build, and operate metrics, logs, and traces across the Monetization stack using tools such as Prometheus and Grafana. - Implement automated detection for billing, data, and event anomalies, and route alerts to the designated feature teams. - Develop reconciliation and data integrity checks across usage and billing pipelines. - Define and track service reliability targets (SLOs) and service level indicators (SLIs) to measure reliability, write runbooks, and take part in incident handling. - Explore artificial intelligence and machine learning techniques to predict system anomalies and accelerate resolution. - Review merge requests and offer feedback to other Monetization engineers. - Collaborate with Product, Finance, Support, and other partners to turn operational needs into reliable tooling. What you’ll bring - Professional experience with Ruby on Rails. - A background in site reliability or observability engineering, including monitoring, alerting, SLOs, SLIs, runbooks, incident handling, and tools such as Prometheus, Grafana, and OpenTelemetry. - Experience building anomaly detection, monitoring, or risk management tooling. - Exposure to data stores for reporting and insights, especially ClickHouse, and the change data capture and event streaming pipelines that feed them, such as Siphon and NATS JetStream. - Experience with billing, financial, or other business-critical systems. - Experience owning a project from concept to production, including proposal, discussion, execution, and monitoring. - Clear, concise communication about complex technical, architectural, and organizational problems in English, with the ability to propose thorough, iterative solutions in a remote and largely asynchronous work environment. - Additional relevant experience includes working knowledge of Python for anomaly detection and data work, and experience with Zuora or Salesforce.