Senior Data Engineer - Advanced Data Lake (GCP Hybrid Cloud)

Senior Data Engineer - Advanced Data Lake (GCP Hybrid Cloud)

Mayo Clinic is top-ranked in more specialties than any other care provider according to U.S. News & World Report. We prioritize patient needs while investing in our employees through competitive compensation, comprehensive benefit plans, and continuous education and advancement opportunities, ensuring a long and successful career.

Benefits Highlights

• Medical: Multiple plan options.
• Dental: Delta Dental or reimbursement account for flexible coverage.
• Vision: Affordable plan with national network.
• Pre-Tax Savings: HSA and FSAs for eligible expenses.
• Retirement: Competitive retirement package to secure your future.

Responsibilities

We are seeking a talented Senior Data Engineer to join our Advanced Data Lake (ADL) team. This is an infrastructure-heavy, hybrid cloud role with Google Cloud Platform (GCP) as a core requirement. You will build and operate enterprise Data Lakehouse platforms supporting large-scale analytics and digital transformation.

Key responsibilities include architecting and maintaining automated data pipelines for ingesting, transforming, and integrating complex datasets. You will utilize DataStream for real-time data movement and Dataflow for processing at scale. Composer/Airflow will be leveraged for seamless scheduling, monitoring, and automation of pipeline operations. Infrastructure provisioning and workflow management will be handled with Terraform and Dataform to ensure reproducibility and adherence to best practices. All code and pipeline assets will be managed through git repositories, with CI/CD automation and streamlined releases enabled by Azure DevOps (ADO). Changes will be governed by ServiceNow processes to ensure traceability, auditability, and operational compliance.

Core duties involve working with cross-functional teams to translate business needs into pipeline specifications, building and optimizing data models for advanced analytics, and maintaining data quality and security throughout all processes. You will automate workflow monitoring and proactively resolve data issues, applying strong technical and problem-solving skills.

The ideal candidate must be proficient in Python and SQL, with significant experience in Google Cloud Platform (especially Dataflow and DataStream), Terraform, Dataform, and orchestration with Composer/Airflow. Experience managing code in git repositories, working with Azure DevOps workflows, and following ServiceNow change management processes is required. Strong communication skills and the ability to manage multiple priorities in a remote, team-oriented environment are also necessary.

This role involves developing and deploying data pipelines, integrations, and transformations to support analytics and machine learning applications as part of an assigned product team, utilizing various open-source programming languages and vended software. The position requires independent judgment and maintaining an understanding of the organization's current solutions, coding languages, and tools. Consultative services may be provided to departments/divisions and leadership committees. Demonstrated experience in designing, building, and installing data systems applied to the Department of Data & Analytics technology framework is required.

You will partner with product owners and Analytics and Machine Learning delivery teams to:

• Identify and retrieve data.
• Conduct exploratory analysis.
• Pipeline and transform data.
• Identify and visualize trends.
• Build and validate analytical models.
• Translate qualitative and quantitative assessments into actionable insights.

This is a full time remote position within the United States. Mayo Clinic will not sponsor or transfer visas for this position including F1 OPT STEM.

Qualifications

A Bachelor's degree in engineering, mathematics, computer science, information technology, health science, or another analytical/quantitative field and a minimum of five years of professional or research experience in data visualization, data engineering, or analytical modeling techniques; OR an Associate’s degree in a relevant field and a minimum of seven years of professional or research experience in these areas. In-depth business or practice knowledge will also be considered.

The incumbent must manage a varied workload with multiple priorities, stay current on healthcare trends, and adapt to enterprise changes. Required skills include:

• Interpersonal skills and time management skills.
• Demonstrated experience working on cross functional teams.
• Strong analytical skills and the ability to identify and recommend solutions.
• Commitment to customer service.
• Excellent verbal and written communication skills, attention to detail, and a high capacity for learning and problem resolution.

Required Technical Expertise

• Advanced experience in SQL.
• Strong experience in scripting languages such as Python, JavaScript, PHP, C++ or Java & API integration.
• Experience in hybrid data processing methods (batch and streaming) such as Apache Spark, Hive, Pig, Kafka.
• Experience with big data, statistics, and machine learning.
• Ability to navigate linux and windows operating systems.

Preferred Technical Expertise

• Knowledge of workflow scheduling (Apache Airflow Google Composer).
• Infrastructure as code (Kubernetes, Docker).
• CI/CD (Jenkins, Github Actions).
• Experience in DataOps/DevOps and agile methodologies.
• Experience with hybrid data virtualization such as Denodo.
• Working knowledge of Tableau, Power BI, SAS, ThoughtSpot, DASH, d3, React, Snowflake, SSIS, and Google Big Query.
• Google Cloud Platform (GCP) certification is preferred.

Required and Preferred Qualifications

• GCP Professional Data Engineer certification is required.
• Preferred qualifications include hybrid or multi-cloud experience.
• Familiarity with enterprise data governance, metadata, and lineage tools is preferred.
• Experience working in large, regulated environments is preferred.

Compensation and Details

• Exemption Status: Exempt
• Compensation Detail: $138,257.60 - $200,512.00 / year
• Benefits Eligible: Yes
• Schedule: Full Time (80 Hours/Pay Period)
• Schedule Details: Monday - Friday, 8:00 a.m. - 5:00 p.m.
• Weekend Schedule: May be required to provide 24/7 on-call support.
• International Assignment: No

Site Description

Mayo Clinic locations include three major campuses in Phoenix/Scottsdale, Arizona, Jacksonville, Florida, Rochester, Minnesota, Mayo Clinic Health System campuses throughout Midwestern communities, and international locations. Each location offers a unique environment for employees to thrive.

Equal Opportunity

All qualified applicants will receive consideration for employment without regard to race, color, religion, sex, gender identity, sexual orientation, national origin, protected veteran status or disability status. Learn more about the "EOE is the Law". Mayo Clinic participates in E-Verify and may provide the Social Security Administration and, if necessary, the Department of Homeland Security with information from each new employee's Form I-9 to confirm work authorization.

Recruiter: Ted Keefe