Data Engineer (Azure & Databricks) – Hybrid @São Paulo

We’re looking for the special, unique and amazing YOU!

@ KCS IT, we look for the ones that stand out, for those who always want to be better and fight for it, and for those who have the same values that we do: dedication, energy, integrity, transparency, flexibility, trust, honesty, hard work, proactivity and teamwork.

At KCS, we stand for equality and value diversity. We create a safe, diverse environment where opportunities are equal for everyone! We do not discriminate based on age, ethnicity, sexual orientation, gender, disability or any factor other than merit. All applications with skills for the position are welcome!

We are looking for an Amazing: Data Engineer (Azure & Databricks) – HYBRID @São Paulo, Brazil

Seniority: Mid-Senior / Senior Level

Work Model: Hybrid - Minimum 2 days per week at the office in São Paulo.

The amazing you will have:

• Bachelor's degree in Computer Science, Software Engineering, Information Systems, or a related field;
• Proven experience in back-end Data Engineering, integrating heterogeneous data sources and developing enterprise Data Lake solutions;
• Strong proficiency in advanced SQL, including joins, CTEs, subqueries, window functions and query optimization;
• Hands-on experience with Python and PySpark for distributed data processing, automation, integrations and notebook development;
• Solid experience with Azure Data Factory (ADF) for data ingestion, pipeline development and orchestration;
• Experience with Azure Synapse Analytics for data processing, transformation and analytical workloads;
• Strong knowledge of Databricks for data engineering, notebook development and large-scale data transformations;
• Experience with Azure DevOps and Git, including version control, code reviews, automated testing and CI/CD pipelines;
• Experience integrating REST APIs and enterprise data sources, including authentication, pagination, error handling and incremental data loading;
• Strong understanding of ETL/ELT processes, relational and analytical data modeling, and Data Lake/Lakehouse architectures;
• Experience implementing Medallion Architecture (Bronze, Silver and Gold layers);
• Knowledge of data harmonization, source-to-target mapping, canonical data models and data dictionaries;
• Experience with data quality, governance, reconciliation, traceability and secure data access;
• Ability to develop, monitor, troubleshoot and optimize data pipelines, ensuring reliability, performance and cost efficiency;
• Ability to collaborate with international and cross-functional teams, translating business requirements into technical solutions;
• English communication skills for technical collaboration and documentation in an international environment;
• Availability to work on-site in São Paulo at least twice per week.

To turn yourself from amazing to unique, you’ll have (plus):

• Experience integrating enterprise platforms such as SAP ECC, SAP S/4HANA, Microsoft Dynamics 365, ServiceNow and/or Coupa;
• Knowledge of Procurement data and processes, including suppliers, purchase requisitions, purchase orders, contracts, sourcing, payment terms and savings;
• Experience with Delta Lake, Change Data Capture (CDC), incremental loading and schema evolution;
• Knowledge of Azure Data Lake Storage (ADLS) and Azure Key Vault;
• Experience with Apache Spark performance optimization;
• Familiarity with enterprise data cataloging and data lineage solutions;
• Experience working on global data integration projects involving multiple systems, countries and business regions;
• Knowledge of DataOps practices and Agile methodologies;
• Spanish communication skills will be considered an advantage;
• Related technical certifications in Azure Data Engineering, Databricks or similar technologies.

What will you be doing?

• Develop, maintain and enhance a Global Non-Trade Procurement Data Lake, supporting data integration and harmonization across different systems and regions;
• Design, build and optimize ETL/ELT pipelines using Azure Data Factory, Azure Synapse Analytics and Databricks;
• Develop SQL, Python and PySpark scripts for data ingestion, transformation and publication;
• Implement and maintain the Bronze, Silver and Gold layers of the Data Lake architecture;
• Integrate data from SAP, Dynamics 365, ServiceNow, Coupa and other enterprise systems;
• Develop and optimize analytical tables, queries and views;
• Implement data quality controls, reconciliation processes, schema management and monitoring solutions;
• Maintain technical documentation, data mappings, transformation rules and data lineage;
• Support automated testing, deployment and continuous improvement through Azure DevOps and CI/CD practices;
• Collaborate with global teams to ensure reliable, scalable and governed data solutions.

Why should you become part of our family?

• You can develop a career that fits you: Your career development is personalized, taking into consideration your needs and goals from short to long term.
• Interesting Challenges Ahead: You can work for several clients from different sectors of activity.
• Free training programs: Our training and certification programs in languages, tech, behavior and business will help you reach your full potential faster.
• International projects in Benelux: You can gain international experience in Benelux and balance a new way of living with work.
• Type of projects: Depending on the project, you might find one of these types:
• Hybrid Systems: It is important to balance work with socialization. That's why a hybrid system works for us and for you.
• Full Remote projects: If you want to work while enjoying the comfort of your home.
• Full Onsite projects: If you prefer the company of your colleagues!
• Take care of your well-being: Enjoy our free nutrition, psychologist, general medicine appointments and our yoga and personal training days… all remote!

Who are we?

Founded in 2008 and based in Lisbon, KCS IT is a consulting company in the field of Information Technology and Services, focused on creating value for our clients through three main areas: Consulting, Outsourcing, Innovation and Training.

Our commitment to talent development is unmistakable in the recent opening of the Porto, Leiria and Azores hubs, which aim to develop technology for the national and international market.

Since 2018, we have been elected one of the 10 major companies in the “Index of Excellence”, an initiative that aims to reward organizations that invest most in the development and satisfaction of their employees.