17 Nov
|
TinEye / Idee
|
Ontario
17 Nov
TinEye / Idee
Ontario
Build large-scale web crawling infrastructure in Toronto with Python, Go, Rust, or C++ and asynchronous network programming expertise. This is an in-office position.
TinEye seeks a Web Crawling and Scraping Engineer for its small Toronto development team. You will design crawling strategies and operate high-throughput ingestion systems. The work connects crawled data with indexing and matching for TinEye’s image search technologies.
Key Responsibilities: Architect crawling strategies for client-requested page prioritization
Maintain frontier scheduling, rate limiting, deduplication, parsing, and proxies
Improve monitoring and logging for crawler performance and health
Eliminate crawling and extraction bottlenecks for scalable resource usage
Document debugging procedures and support data flow into indexing
Requirements: 4+ years designing large-scale crawling and indexing infrastructure
Extensive distributed systems and real-time URL prioritization experience
Expertise with crawling frameworks, scraping libraries, and HTTP protocols
Experience with Puppeteer, Selenium, asynchronous I/O, queues, and storage
Knowledge of SQL and NoSQL databases with solid problem-solving skills
Apply web crawling, distributed systems, and asynchronous programming expertise to TinEye’s Toronto image search infrastructure.
#J-18808-Ljbffr
📌 Web Crawling Infrastructure Engineer (Ontario)
🏢 TinEye / Idee
📍 Ontario