Senior Data Engineer

Location: San Jose, CA – Local Candidates Only

Job Type: Contract

Work Arrangement: Onsite – No Remote

Responsibilities: Lead end-to-end migration, transformation, and loading of high-volume, complex data into PostgreSQL.Design, build, and maintain scalable ETL/ELT pipelines processing millions of records. Map and reconcile complex legacy data structures across multiple business entities. Develop data validation and reconciliation frameworks to ensure data integrity and completeness. Optimize PostgreSQL performance through query tuning, indexing, partitioning, and batch/incremental loading. Troubleshoot data quality issues, schema mismatches, and pipeline failures. Document data mappings, transformation logic, data lineage, and migration runbooks. Collaborate with business and technical stakeholders on migration requirements and timelines. Establish monitoring, logging, and alerting for pipeline health and data quality. Mentor junior data engineers and contribute to engineering best practices.

Required Qualifications: 7 years of experience in Data Engineering. Strong to expert-level Python skills. Deep hands-on experience with PostgreSQL and/or Oracle. Proven experience designing and managing ETL/ELT pipelines at scale. Experience with large-scale data migration projects involving millions of records. Strong experience with data mapping, transformation, validation, and reconciliation. Knowledge of data modeling, normalization, indexing, and partitioning. Experience with Git and CI/CD practices. Strong troubleshooting and problem-solving skills.

Preferred Qualifications: Experience with Airflow, Dagster, or Prefect. Experience with AWS, GCP, or Azure. Exposure to NoSQL and cross-database migrations. Experience working in regulated or high-stakes data environments. Experience with Docker and Infrastructure-as-Code. Familiarity with Great Expectations, dbt tests, or similar data quality frameworks.

What Success Looks Like: Fully migrated and validated datasets in PostgreSQL with zero critical data loss or corruption. Documented, repeatable, and optimized ETL/ELT pipelines. Clear data lineage and audit trails for migration and validation activities.