Skip to main content

About the role

Overview: As a Data Engineer, you will design and optimize scalable data pipelines using Python and Azure Databricks to support data-driven solutions. You will build ETL/ELT workflows, work with large datasets, and apply Spark-based processing within Databricks. You will ensure data quality, security, and governance while collaborating with cross-functional teams to deliver data-driven insights. This role offers a chance to contribute to cloud-based data platforms and CI/CD-enabled deployments in a growing digital transformation environment.
Responsibilities: Design, develop, and maintain scalable data pipelines using Python and Azure Databricks Build and optimize ETL/ELT workflows for ingesting, transforming, and processing large datasets Develop data solutions using Apache Spark within the Databricks environment Integrate data from multiple structured and unstructured sources Ensure data quality, reliability, security, and governance across data platforms Collaborate with Data Scientists, Analysts, Architects, and Business stakeholders to deliver data solutions Implement performance tuning and optimization for large-scale data processing workloads Develop and maintain data models, metadata management, and documentation Monitor and troubleshoot production data pipelines and resolve data-related issues Support CI/CD implementation and deployment automation for data engineering solutions
Key requirements: Strong expertise in Python and Azure Databricks Hands-on experience with cloud-based data engineering and big data processing Data transformation using Azure services and Databricks Experience with Apache Spark in Databricks Ability to build scalable data pipelines and ETL/ELT workflows Knowledge of data quality, security, and governance Collaboration with cross-functional teams Strong communication Problem-solving mindset Python Azure Databricks Apache Spark

Matching similar jobs

JOB OVERVIEW

Experience level

Senior

Location

Washington, DC

Occupation

Data Warehousing Specialists

Industry

Computer Systems Design Services

Posted

yesterday

Tired of running searches?

Rank the roles you'd take once, and matches like these arrive on their own.

CREATE PROFILE (opens in a new tab)