Senior Data Engineer

Type: 6 Month Contract

Location: Brazil, Fully Remote

Rate: $20/hr USD - $30/hr USD

Job Description

Insight Global is seeking a Data Validation Engineer for an end client to join its QA Automation organization and serve as the team's subject matter expert for data quality and validation. This individual will be responsible for validating large-scale datasets, analytics platforms, and ETL pipelines that support our clients streaming and digital products. The ideal candidate will have strong Python and SQL skills, experience working with BigQuery and cloud-based data environments, and a proven background validating data accuracy, completeness, and consistency. This role will also leverage AI tools and large language models (LLMs) to enhance data validation processes, automate analysis, improve anomaly detection, and accelerate root cause investigation. The Data Validation Engineer will partner closely with QA Automation Engineers, Data Engineers, and business stakeholders to ensure trusted data is available for reporting, analytics, and downstream testing efforts.

Responsibilities

• Own end-to-end data validation efforts across large-scale data pipelines and analytics platforms

• Validate data movement through ETL/ELT processes and identify discrepancies between source and target systems

• Develop and maintain Python-based data validation, reconciliation, and quality-checking scripts

• Write SQL queries to analyze data, investigate issues, and verify business logic

• Perform source-to-target validation across BigQuery and cloud-based data platforms

• Troubleshoot data quality issues and perform root cause analysis

• Leverage AI/LLM technologies (Claude, ChatGPT, Copilot, etc.) to generate and optimize validation rules, SQL queries, reconciliation logic, and analysis summaries

• Utilize AI-assisted techniques for anomaly detection, trend analysis, and identification of data inconsistencies

• Partner with QA Automation Engineers to ensure reliable, trusted data is available for downstream testing efforts

• Support validation of analytics generated across web, mobile, and connected TV (CTV) platforms

• Collaborate within Agile teams using Jira and Confluence

Required Skills & Experience

• 5+ years of experience in Data Validation, Data Quality, Big Data QA, ETL Testing, or a related field

• Strong Python development and writing code from scratch

• Advanced SQL skills with hands-on experience writing complex queries

• Experience validating large-scale datasets

• Experience testing ETL/ELT data pipelines

• Hands-on experience with BigQuery

• Experience performing source-to-target data reconciliation

• Experience troubleshooting data discrepancies and conducting root cause analysis

• Strong understanding of data quality principles, including:

o Data accuracy

o Data completeness

o Data consistency

o Data integrity

o Duplicate detection

• Experience working within GCP or AWS cloud data environments

• Experience with reporting tools

• Experience leveraging AI/LLM tools to improve data validation, analysis, or engineering workflows

• Experience creating or enhancing validation logic, reconciliation processes, or data quality checks using AI-assisted approaches