Lead Data Engineer

Role details

Our Purpose Mastercard powers economies and empowers people in 200+ countries and territories worldwide. Together with our customers, we’re helping build a sustainable economy where everyone can prosper. We support a wide range of digital payments choices, making transactions secure, simple, smart and accessible. Our technology and innovation, partnerships and networks combine to deliver a unique set of products and services that help people, businesses and governments realize their greatest potential. Title and Summary Lead Data Engineer Overview Mastercard powers the global economy by enabling secure, seamless, and intelligent payments across the world. Behind every transaction is a sophisticated technology ecosystem that processes billions of payment events with speed, resilience, and precision. As a Lead Data Engineer within the Data Collection & Engineering (DC&E) organisation, you will play a critical role in designing, building, and scaling Mastercard's next-generation data platforms and analytical ecosystems. You will develop high-performance, cloud-enabled data pipelines that power Mastercard's enterprise data warehouse and lakehouse environments, enabling advanced analytics, business intelligence, regulatory reporting, and data-driven decision making across the organisation. This role offers a unique opportunity to solve large-scale data engineering challenges, work with cutting-edge big data technologies, and contribute to a modern cloud transformation programme supporting global payment processing platforms. Key Responsibilities Data Engineering & Development • Design, develop, test, and deploy high-quality, secure, scalable, and resilient data pipelines using Apache Spark, Java/Scala across Hadoop and cloud-native object storage platforms. • Build and maintain batch and near real-time data processing frameworks capable of supporting petabyte-scale workloads. • Develop reusable engineering components and frameworks that accelerate data product delivery while maintaining enterprise standards. Architecture & Platform Engineering • Design and implement a "build once, run anywhere" architecture supporting seamless deployment across on-premises and public cloud environments without code changes. • Implement data lineage, metadata management, data cataloguing, data quality controls, and observability capabilities across the data ecosystem. • Collaborate with architects and platform teams to establish scalable design patterns and engineering best practices. Cloud Modernisation • Contribute to migration initiatives moving legacy ETL and data warehouse workloads from on-premises environments to cloud-native architectures. • Leverage cloud services such as Amazon S3, EMR, Glue, and related data services to improve scalability, reliability, and operational efficiency. • Drive adoption of modern lakehouse and distributed compute architectures. Delivery & Technical Leadership • Lead end-to-end development activities including requirement analysis, solution design, coding, testing, deployment, and production support. • Mentor and guide junior engineers through code reviews, technical coaching, and engineering best practices. • Partner with product owners, analysts, architects, and business stakeholders to deliver high-quality solutions within committed timelines. Operational Excellence • Troubleshoot complex production incidents and perform root cause analysis to identify and implement long-term remediation strategies. • Ensure compliance with Mastercard's engineering, security, quality assurance, and operational governance standards. • Continuously identify opportunities to improve performance, automation, monitoring, and process efficiency. Innovation • Evaluate emerging data technologies and conduct proof-of-concept (POC) initiatives to determine their applicability within Mastercard's data ecosystem. • Contribute to engineering innovation and continuous improvement initiatives across the organisation. Qualifications Required Experience • 10-12 years of experience delivering enterprise-scale Data Warehouse, Data Lake, or Data Lakehouse solutions. • Proven experience implementing multiple end-to-end data engineering projects within large-scale distributed computing environments. • Hands-on experience migrating ETL and analytics workloads from on-premises platforms to cloud-native environments. Technical Expertise • Strong development experience using:Apache Spark, Scala or Java, Hadoop ecosystem technologies, Object Storage platforms • Experience building orchestration and workflow solutions using: Apache Airflow/ Apache NiFi and Similar enterprise scheduling frameworks • Strong SQL expertise and experience with Relational and NoSQL database technologies: Oracle/ SQL Server, Cassandra, Dynamo DB etc. • Working knowledge of cloud platforms, preferably AWS: Amazon S3, EMR, AWS Glue, Cloud-native data services Professional Skills • Strong analytical and problem-solving capabilities. • Experience operating within Agile delivery environments. • Excellent written and verbal communication skills. • Proven ability to collaborate within geographically distributed and matrix-based teams. • Self-starter with strong ownership, accountability, and execution focus. • Ability to learn emerging technologies quickly and apply them effectively to business challenges. Corporate Security Responsibility All activities involving access to Mastercard assets, information, and networks comes with an inherent risk to the organization and, therefore, it is expected that every person working for, or on behalf of, Mastercard is responsible for information security and must: Abide by Mastercard’s security policies and practices; Ensure the confidentiality and integrity of the information being accessed; Report any suspected information security violation or breach, and Complete all periodic mandatory security trainings in accordance with Mastercard’s guidelines.