Senior Data Engineer (Databricks, AWS, Python, MLOps)
Job Description
We are seeking a Senior Data Engineer with deep Databricks, Python and MLOps experience for a 16-month, 100% remote contract supporting a Life Sciences organization. You will provide technical leadership on lakehouse architecture and ETL/ELT pipelines, operationalize machine learning workflows, and partner directly with clinical and medical affairs teams to deliver trusted data. The ideal candidate will have 10+ years of data engineering within Life Sciences, Pharma or Medical Device organizations, with 5+ years each of Databricks and Python/PySpark. AWS is strongly preferred, though Azure or GCP experience will also be considered. Exposure to AI/ML, MLflow and Delta Lake in a large enterprise Life Sciences environment is a plus. Requirements • 10+ years of data engineering in Life Sciences, Pharma or Medical Device organizations • 5+ years of Databricks experience • 5+ years of Python / PySpark experience • Cloud experience with AWS, Azure or GCP (strong preference for AWS) • Experience building ETL/ELT pipelines • Hands-on MLOps experience • Experience with lakehouse architecture • Proven technical leadership • Experience working with clinical and medical affairs teams • Nice to have: AI/ML, MLflow, Delta Lake and large enterprise Life Sciences environments • Independent work authorization only (no sponsorship)
Responsibilities
- Design and build ETL/ELT pipelines on Databricks
- Architect and evolve the lakehouse platform on AWS
- Develop data processing in Python and PySpark
- Implement MLOps practices to deploy and monitor ML workflows
- Manage experiments and model lifecycle with MLflow where applicable
- Build reliable Delta Lake tables for analytics and ML
- Partner with clinical and medical affairs teams on data requirements
- Provide technical leadership and guide engineering standards