Lead Data Engineer (Python, Pyspark, AWS)

Descrição da Vaga

Requisitos

• Candidates must demonstrate strong, hands-on experience in:
• Python and/or PySpark
• Apache Spark or similar distributed data processing frameworks
• Cloud data warehouses such as Snowflake and/or Redshift
• AWS ecosystem, including S3, EC2, EMR, Glue, Athena, Lambda, and CloudWatch
• Strong SQL and advanced data modeling
• Modern data tooling:
• Apache Airflow
• DBT
• GitLab & CI/CD pipelines
• Docker & Kubernetes
• Designing scalable data pipelines and enterprise-grade architecture patterns
• Implementing governance, data quality, metadata management, and compliance standards
• Leadership & Strategic Expectations (Critical for Success)
• This is not an individual contributor-only role
• Proven experience in a Lead or Architect capacity
• Ability to define technical strategy and influence engineering standards across teams
• Strong mentorship capability with structured coaching examples
• Clear stakeholder management experience across product, analytics, and business functions
• Ability to articulate measurable business impact and success metrics
• Strong documentation practices aligned with governance and compliance requirements
• Ownership of design reviews, code reviews, and process maturity
• Important Candidate Fit Criteria (Based on Prior Interview Feedback)
• To succeed in this role, candidates must clearly demonstrate:
• Impact Articulation – Ability to quantify business value, performance improvements, cost savings, or scalability gains from past initiatives
• Depth of Mentorship – Concrete examples of structured mentorship, coaching frameworks, and measurable team growth outcomes
• Documentation & Governance Mindset – Strong commitment to documentation, data lineage, governance frameworks, and compliance standards
• Strategic Leadership – Evidence of leading platform evolution, defining architecture standards, and influencing cross-team processes—not just delivering technical tasks
• Stakeholder Management – Experience partnering with business and product teams to translate requirements into scalable technical solutions
• Candidates who primarily demonstrate strong technical skills but lack leadership depth, governance orientation, stakeholder engagement, or strategic platform ownership may not be the right fit for this Lead-level opportunity

Responsabilidades

• Client is seeking a Lead Data Engineer to architect, scale, and evolve a modern enterprise data platform supporting retail and consumer brands
• This is a high-impact leadership role combining deep hands-on engineering expertise with architectural ownership and cross-functional influence
• The Lead Data Engineer will own the end-to-end design and delivery of scalable, secure, and cost-efficient data solutions built on AWS and cloud data warehouse technologies
• The role requires strong technical depth in distributed data processing and cloud-native tooling, along with the ability to define engineering standards, mentor teams, and translate business requirements into measurable outcomes
• This individual will drive platform evolution, enforce governance and quality frameworks, and lead cross-team initiatives to enable advanced analytics and AI-driven solutions

Benefícios