03 Oct
|
Galent
|
Toronto
We are looking for a Senior Backend Developer with 5+ years of experience in big data engineering, API integration, and AI-assisted development. The ideal candidate will design, build, and maintain scalable data pipelines and backend systems in a enterprise setting. Design and develop Spark-Scala applications for large-scale data processing on Hadoop/CDP clusters Build and optimize ETL/ELT pipelines using Spark DataFrames, Datasets and Spark SQL Tune Spark jobs for performance (partitioning, caching, broadcast joins, shuffle optimization) Write complex HiveQL / Spark SQL queries including window functions, CTEs, subqueries and aggregations Work with HDFS encryption zones and data governance requirements Unix / Shell Scripting Manage HDFS operations (hdfs dfs commands), file transfers, and data validation Build scripts and pipelines to extract data from REST APIs using curl and Python Leverage GitHub Copilot / AI coding assistants to accelerate development Use AI tools for code review, SQL generation,
script debugging and documentation Contribute to AI-assisted data quality and anomaly detection pipelines Explore and implement LLM-based automation for repetitive data engineering tasks Schedule and manage jobs using AAP (Ansible Automation Platform) / Control-M / cron Build and maintain Ansible playbooks for automated deployments Manage deployment pipelines including artifact versioning, Vault secret injection and environment-specific configuration Familiarity with CI/CD pipelines (Helios, GitHub Actions) Understanding of AML / Financial regulatory data domains.
All qualified applicants will receive consideration for employment without regard to race, color, religion, sex (including pregnancy, sexual orientation, or gender identity), national origin, citizenship status, age, disability, genetic information, protected veteran status, or any other characteristic protected by applicable law. #
📌 Full time developer (Toronto)
🏢 Galent
📍 Toronto