Data Engineer
wiproWashington, DC
About the role
Overview:
As a Data Engineer, you will design and optimize scalable data pipelines using Python and Azure Databricks to support data-driven solutions. You will build ETL/ELT workflows, work with large datasets, and apply Spark-based processing within Databricks. You will ensure data quality, security, and governance while collaborating with cross-functional teams to deliver data-driven insights. This role offers a chance to contribute to cloud-based data platforms and CI/CD-enabled deployments in a growing digital transformation environment.
Responsibilities:
Design, develop, and maintain scalable data pipelines using Python and Azure Databricks Build and optimize ETL/ELT workflows for ingesting, transforming, and processing large datasets Develop data solutions using Apache Spark within the Databricks environment Integrate data from multiple structured and unstructured sources Ensure data quality, reliability, security, and governance across data platforms Collaborate with Data Scientists, Analysts, Architects, and Business stakeholders to deliver data solutions Implement performance tuning and optimization for large-scale data processing workloads Develop and maintain data models, metadata management, and documentation Monitor and troubleshoot production data pipelines and resolve data-related issues Support CI/CD implementation and deployment automation for data engineering solutions
Key requirements:
Strong expertise in Python and Azure Databricks Hands-on experience with cloud-based data engineering and big data processing Data transformation using Azure services and Databricks Experience with Apache Spark in Databricks Ability to build scalable data pipelines and ETL/ELT workflows Knowledge of data quality, security, and governance Collaboration with cross-functional teams Strong communication Problem-solving mindset Python Azure Databricks Apache Spark
Matching similar jobs
JOB OVERVIEW
Experience level
Senior
Location
Washington, DC
Occupation
Data Warehousing Specialists
Industry
Computer Systems Design Services
Posted
yesterday
Tired of running searches?
Rank the roles you'd take once, and matches like these arrive on their own.
CREATE PROFILE (opens in a new tab)