Data Engineer (Azure Databricks)
Job Description
We are seeking a Data Engineer with strong Azure Databricks experience for a remote contract-to-hire opportunity (12+ months, W2 only). This role is intended to convert to a permanent position. You will design, build and optimize large-scale ETL/ELT pipelines across data lakes and data warehouses, working with both structured and unstructured data while upholding data quality, governance, security and compliance standards. The ideal candidate will have 5+ years of data engineering, strong SQL and Python/Spark skills, and exposure to AI, machine learning or advanced analytics workloads. Experience across Azure, AWS or GCP cloud data platforms is valuable, with Azure Databricks as the core of the role. Requirements • 5+ years of data engineering experience • Strong SQL and relational database skills • Hands-on ETL/ELT pipeline development • Strong Azure Databricks experience • Experience with Python / Spark or similar • Experience with cloud data platforms (Azure, AWS or GCP) • Knowledge of data warehouses, data lakes and dimensional modeling • Experience with data quality, governance, security and compliance • AI, machine learning or advanced analytics experience • W2 only; open to USC, Green Card, GC EAD, TN and H4 EAD candidates
Responsibilities
- Design and develop large-scale data pipelines on Azure Databricks
- Build and optimize ETL/ELT processes for performance and cost
- Develop data models for data warehouses and data lakes
- Process structured and unstructured data from multiple sources
- Write and tune complex SQL against relational databases
- Implement data quality, governance and security controls
- Support AI, machine learning and advanced analytics use cases with reliable data
- Collaborate with analytics and business teams on data requirements