SUMMARY OF RESPONSIBILITIES
We are seeking a Senior Data Solution Engineer to lead the full lifecycle of data solutions projects with minimal guidance. The ideal candidate will have extensive experience with cloud infrastructure, big data technologies, and machine learning fundamentals, capable of architecting, implementing, and optimizing data pipelines that can process 10 to 100 gigabyte-scale datasets efficiently.
KEY AREAS OF RESPONSIBILITIES
- Lead end-to-end data solutions projects through all phases: discovery, planning, design, implementation, testing, operations, and optimization
- Architect and implement scalable data processing pipelines using modern technologies
- Design and enforce data governance frameworks and lineage tracking solutions
- Collaborate with stakeholders to translate business requirements into technical solutions
- Provide technical leadership and mentorship to junior team members
- Optimize existing data solutions for improved performance and cost-efficiency
- Implement and maintain data quality controls and monitoring systems
QUALIFICATIONS & EXPERIENCE
Minimum Qualifications
- Bachelor's degree in Computer Science, Engineering, or related field (or equivalent practical experience)
- Relevant certifications in AWS, data engineering, or cloud architecture
- 5+ years of experience in data engineering or solutions architecture
- Strong expertise with Kubernetes (k8s) for containerized applications
- Extensive experience with AWS data services (including but not limited to S3, Glue, Athena, EMR, Lakeformation)
- Proficiency with PySpark for large-scale data processing
- Experience with data lake technologies (Apache Iceberg, Apache Hive)
- Strong understanding of data lineage tracking and governance principles
- Advanced GitOps practices using GitLab and GitLab CI/CD pipelines
- Experience orchestrating complex data pipelines with workflow management tools
- Proven ability to process and analyze 10 to 100 gigabyte-scale datasets efficiently
- Working knowledge of ML concepts and frameworks
- Experience with CI/CD for data applications
- Terraform or other IaC tools for infrastructure provisioning
PERSONAL QUALITIES
- Self-motivated problem solver who can work with minimal guidance
- Excellent communication skills to translate technical concepts to non-technical stakeholders
- Strong project management capabilities
- Detail-oriented with a focus on data quality and reliability
- Collaborative mindset for cross-functional team environments
About the Company
Payments Network Malaysia Sdn Bhd
Embark on an exciting career journey with Payments Network Malaysia Sdn Bhd (PayNet), the heartbeat of Malaysia's financial markets!
As the national payments network and a pivotal infrastructure for Malaysia’s dynamic financial markets, PayNet is a linchpin in advancing the nation’s digital economy.
Our comprehensive suite of retail payment solutions - encompassing DuitNow (QR and P2P), JomPAY (Bill Payments), FPX (Online), MyDebit (Domestic Debit), MEPS (ATM), and IBG (Interbank GIRO) - not only offer wide accessibility but are seamlessly integrated into the fabric of daily life in Malaysia. These services have revolutionised the way Malaysians handle financial transactions, marking a significant leap in consumer convenience and efficiency.
At PayNet, our focus is on providing a safe, efficient, and innovative payments system. We are dedicated to improving and managing payment services that meet the evolving needs of consumers and businesses. Our work ensures the stability and reliability of Malaysia’s financial system, supporting the growth of the economy.
Learn more about our work and how we are contributing to Malaysia's financial future at www.paynet.my.
Join us in embracing digital payments and advancing Malaysia's financial landscape.