Date: May 19, 2022 3:50:30 AM UTC
Company: Synagie
City: Selangor, Malaysia
Country: MY
Description:
Responsibilities – What You’ll Do
- Developing and maintaining the data pipelines and data platform to ensure consistency across data.
- Work closely with the Data Engineering, Data Products team on improving the product to better enhance the investment decision and processes.
- Design and develop APIs for 3rd party data into existing database.
- Look at daily operational data CI/CD to ensure the standards of stable data.
- Troubleshoot of the production system, assist, and implement the necessary mechanisms and tools to ensure the stability of the overall operation of the production system.
Qualifications – What You Should Have
- Familiar with Cloud solutions and services like AWS EC2, AWS S3, Databricks, Snowflake.
- Familiar with Python, R, NoSQL, SQL and other programming languages.
- Knowledge of Job Orchestration solutions like Apache Airflow, Control M, AWS Step Functions.
- Knowledge of database technologies like AWS Redshift, Snowflake, MongoDB and its basic administration.
- Experience with various data formats and ingestions procedures for Excel files, CSV, JSON, etc.
- Strong coding and trouble shooting ability.
- Willing to challenge questions that have no obvious answers and have a strong enthusiasm for learning new technologies.
- Min 3 years of experience in enterprise databases, data modeling, and data integration.
Going beyond – Good to Have
- Experience with administration of git repositories.
- Experience with Beautiful Soup, Scrapy, Selenium and other web automation tools.
- Data migration experience. (Data collection, transformation and destination audit)
- Experience with cloud providers other than AWS and Azure.
- Experience with building offline/private Docker and Kubernetes infrastructure.
Experience: 3 Year