Data Engineer

Andela exists to connect brilliance and opportunity. Since 2014, we have been dedicated to breaking down global barriers and accelerating the future of work for both technologists and organizations around the world. For technologists, Andela offers competitive long-term career opportunities with leading organizations, access to a global community of professionals, and educational opportunities with leading technology providers.

At Andela, we’re deeply passionate about creating long-lasting and transformative growth opportunities for all - and doing it in an E.P.I.C. way! We’re excited to continue building our remote-first team with incredible people like you. After applying for this role, you will join our Andela Community of brilliant technologists by passing a technical screening and live interview. As a community member, you’ll have access to a multitude of exclusive technologist roles.

Join Andela today to access this opportunity and more in our global marketplace!

Our roles are typically filled at lightning speed, so if you’re considering applying, get your application in quickly!

About the role

We are looking for an experienced Data Engineer to support end-to-end data initiatives throughout the consulting engagement lifecycle. In this role, you will work directly with clients, analyze and transform complex datasets, design scalable data solutions, and automate data workflows. You will collaborate with cross-functional teams including data science and delivery teams while ensuring data quality, reliability, and accessibility. This position requires strong technical expertise in Azure-based data engineering technologies combined with excellent communication and problem-solving skills.

Responsibilities

• Participate in all data-related aspects of the consulting engagement lifecycle, including discovery, implementation, measurement, and continuous improvement.
• Load, analyze, and profile client data to identify transformation requirements and areas requiring client involvement.
• Design data mart solutions to support discovery, implementation, measurement, and long-term continuous improvement initiatives.
• Load data into proprietary measurement tools.
• Make data available to delivery teams, including Data Science and Profit Realization teams.
• Design inbound data merge processes for ongoing data refreshes.
• Automate end-to-end data processes with validation, monitoring, and notification capabilities.
• Design and automate data pipelines for cloud-hosted client-facing dashboards.
• Design and automate modeling extracts for continuous improvement models.
• Design and automate services and applications supporting online models and client-facing interfaces.
• Support ongoing data requests and automation initiatives from delivery teams.
• Troubleshoot production issues related to client data, software defects, or technical challenges.
• Build processes supporting data transformation, data structures, metadata, dependency management, and workload management.
• Perform root cause analysis on internal and external data and processes to answer business questions and identify improvement opportunities.
• Interface directly with clients and translate technical concepts into business context.

Must have skills

• Bachelor's degree in Computer Science or a related field, or equivalent experience as a Database Developer, ETL Developer, BI Developer, or similar role.
• 5+ years of professional experience.
• Strong experience with Azure Data Factory.
• Strong experience with Azure Databricks.
• Strong experience with Azure SQL.
• Strong experience with SQL Server.
• Experience with Python, including PySpark.
• Experience performing root cause analysis on internal and external data and processes.
• Experience building processes supporting data transformation, data structures, metadata, dependency management, and workload management.
• Proven experience manipulating, processing, and extracting value from large, disconnected datasets.
• Ability to communicate effectively with technical and non-technical audiences.
• Ability to work independently and collaboratively within a team environment.
• Strong analytical, problem-solving, decision-making, and attention-to-detail skills.
• Excellent verbal and written communication skills.
• Ability to simplify technical concepts and present information to senior-level executives.
• Results-driven mindset and ability to thrive in a fast-paced environment.
• Strong client-facing communication skills.
• Experience designing and automating data pipelines.
• Experience working with Databricks, Apache Spark, and Azure Data Factory.

Nice to have skills

• Experience with Agile methodologies such as Scrum and/or Kanban.
• Working knowledge of message queuing systems.
• Working knowledge of stream processing technologies.
• Experience with highly scalable Big Data stores.
• Experience with cloud-based technologies and cloud environments.
• Experience with Microsoft Azure cloud services beyond core data engineering tools.
• Experience building and optimizing big data pipelines, architectures, and datasets.
• Experience with big data technologies such as Hadoop, Spark, Kafka, and similar tools.
• Experience with Azure messaging services, including Event Grid, Event Hub, and Service Bus.
• Familiarity with scripting languages such as R, JavaScript, Scala, or similar.
• Understanding of JSON format and experience working with JSON documents.
• Additional strong communication skills.

Conditions

• 100% Remote role. Primary timezone: UTC-4. Requires at least 6 hours of overlap with EST. Location: open to candidates from countries across the Americas, Europe, Africa, and Asia-Pacific.
• 6-month, contractor role - renewable.
• Full-time dedication (40 hours/week).
• BYOD (Bring Your Own Device) required.

At Andela, we know our strengths lie in our diverse community whose talents, perspectives, backgrounds, and orientations we take pride in. Andela is committed to nurturing a work environment where all individuals are treated with respect and dignity. Everyone has the right to work in a professional atmosphere that promotes equal employment opportunities and prohibits discriminatory practices.

Andela provides equal employment opportunities to all employees and applicants without regard to factors including but not limited to race, color, religion, gender, sexual orientation, gender identity, national origin, age, disability, pregnancy (including breastfeeding), genetic information, HIV/AIDS or any other medical status, family or parental status, marital status, amnesty or status as a covered veteran in accordance with applicable federal, state and local laws.