31 Aug
|
Centraprise
|
Toronto
31 Aug
Centraprise
Toronto
Skills –
Well versed with Data , ETL Processes, SQL and high-level programming languages such Scala, Java, DevSecOps
Detailed :
What will you do?
- Work within an Agile cross functional team to design, develop and maintain data ingestion flows and evolve the platform to orchestrate them.
- Design and implement data pipelines, framework and ETL processes.
- Ensure data quality, security, and performance.
- Develop and optimize SQL queries, stored procedures, and views.
- Integrate Snowflake with other data sources and BI tools.
- Monitor and troubleshoot data jobs and platform issues.
- Develop and maintain comprehensive documentation for data pipelines, transformations, and data models.
What do you need to succeed?
- 8+ years of Experience with Big Data technologies used for ETL: Hadoop, Spark, Hive
- Well versed with Snowflake Platform.
- Robust SQL skills and knowledge of data modeling.
- Programming Experience with Scala or Java an API Development.
- Experience with ETL tools (e.g., Informatica, Talend, Apache Airflow).
- Familiarity with any of the cloud platforms (Preferably AWS).
- Knowledge on Gen AI to generate code and improve developer proficiency
- Knowledge of Python or other scripting languages is a plus.
- Excellent problem-solving and communication skills.
- Knowledge of SCM, Infrastructure-as-code, and CI/CD pipelines.
Nice to Have
- Experience with workflow management tools like: Apache Airflow
- Experience with Continuous Integration tools – Git Actions, GitHub, Automated Testing tools, Git, or similar tools.
- Experience with Docker/Kubernetes/Containers/OCP4
- Shell Scripting
- Bachelor’s or master’s degree in computer science, Data Engineering and, or a related field.
📌 Big Data Engineer (Toronto)
🏢 Centraprise
📍 Toronto