Apply on Kit Job: kitjob.ca/job/2g8rrc

Senior QE Automation Developer – Async Chatbot (Advice Center) We are seeking a

Senior QE Automation Developer

to design and build a next-generation automation framework supporting an AI-driven

Async Advice Center Chatbot . This role sits at the intersection of

software engineering, quality engineering, and AI validation , with a strong emphasis on

hands‑on development

and

framework architecture .

You will lead the creation of

end‑to‑end automation solutions from the ground up , validating complex conversational flows across mobile, chatbot platforms, and AI‑driven routing systems.

Key Responsibilities

Design & Build Automation Frameworks

Architect and develop a

scalable, maintainable automation framework

for chatbot and conversational AI systems

Establish reusable components for UI, API, and conversational flow testing

AI & Chatbot Validation

Develop

LLM feeder and evaluator frameworks

to validate non‑deterministic AI responses

Intent recognition and response accuracy

Conversation flow and state transitions

Routing logic across bots and queues

Introduce

AI‑aware testing methodologies

(e.g., probabilistic validation, response scoring)

Quality Engineering & Test Strategy

Build

robust regression suites

to improve coverage and reduce production defects

Address current

quality gaps due to compressed sprint cycles

Integrate automation into CI/CD pipelines with strong

shift‑left testing practices

Platform & Integration Testing

Validate end‑to‑end flows across:

Mobile entry points

Chat platforms (e.g., LivePerson)

AI/LLM services

Ensure accurate

routing,



queue selection, and system orchestration

Evaluate and introduce

up-to-date automation tools and frameworks

where appropriate

Contribute to building an

in‑house solution to replace a commercial product , overcoming current limitations

Work within

developer repositories, PR workflows, and CI pipelines

Collaborate closely with developers, product, and platform teams

Promote

code quality, testability, and automation best practices

Required Qualifications

8+ years of experience in

Quality Engineering / Test Automation with strong development focus

Advanced programming expertise in

Java

Proven experience building

automation frameworks from scratch

Hands‑on experience with:

REST Assured

(API testing)

Strong understanding of:

End‑to‑end test architecture

CI/CD integration

Test data management and environment orchestration

Experience working in

Agile environments with rapid release cycles

Comfortable working in

code‑first QE models

(Git, PR reviews, dev collaboration)

Preferred / Nice‑to‑Have

Experience testing

chatbots or conversational AI systems

Exposure to

LLMs and AI-enabled systems , including:

Prompt engineering

Response evaluation frameworks

AI/agent‑based workflows

Familiarity with

Playwright

or modern browser automation tools

Experience with

LivePerson

or similar chat platforms

Prior exposure to

Botium

or chatbot testing tools

Experience designing

AI‑aware automation strategies

Disclaimer:

AI may be used in evaluating candidates. This posting is for an existing vacancy.

#J-18808-Ljbffr

Apply on Kit Job: kitjob.ca/job/2g8rrc

📌 Senior QE Automation Developer – Async Chatbot (Advice Center) (Toronto)
🏢 S.i. Systems
📍 Toronto