AWS Data Engineer
JOB DESCRIPTION
Key Responsibilities
- Design, develop, and maintain scalable data pipelines using PySpark on AWS.
- Build and orchestrate ETL/ELT workflows using AWS Glue and AWS Step Functions .
- Develop serverless applications and automation using AWS Lambda .
- Write clean, efficient, and maintainable Python/ PySpark code following engineering best practices.
- Provision and manage cloud infrastructure using Terraform (Infrastructure as Code).
- Implement and maintain CI/CD pipelines to automate code deployment, testing, and infrastructure changes.
- Monitor, troubleshoot, and optimize data pipelines for performance, reliability, and cost efficiency.
- Collaborate with business stakeholders to deliver data solutions. Follow DevOps, security, and coding standards throughout the engagement.
Required Skills
- Strong hands-on experience with PySpark and Python for data engineering.
- Experience developing ETL pipelines using AWS Glue .
- Proficiency with AWS Step Functions for workflow orchestration.
- Experience building serverless solutions using AWS Lambda .
- Hands-on experience with Terraform for Infrastructure as Code (IaC).
- Experience implementing CI/CD pipelines using tools such as GitLab, GitHub Actions, Jenkins, or similar.
- Good understanding of AWS services, data lakes, IAM, S3, CloudWatch, and monitoring.
Base Camp Digital, founded in Singapore in 2018 (formerly Ploy Asia), is a leading IT Solutions and Delivery company operating across Asia, with offices in Singapore, Hong Kong, Malaysia, and Thailand.
We provide targeted expertise for successful project outcomes tailored to modern technological demands.
Our core delivery expertise focuses on high-growth areas:
Enterprise Software Implementation: Delivery and support for major ERP/CRM systems, including deep platform expertise in:
SAP, Salesforce, Oracle, Workday, Pega, Microsoft, and ServiceNow.
Bespoke Software Development: Custom application development using key technologies like Java, .NET, and open-source stacks. Inluding Software Testing Manual and Automation.
Data, Analytics, and AI: Services spanning Data Engineering, Data Analysis, and advanced Artificial Intelligence (AI) solutions (Machine Learning, Computer Vision, LLMs).
Infrastructure Services: Full-range support, from basic desktop to complex Data Centre management.