The jobs vertical is the perfect use case because it is highly fragmented, involves complex job application flows, has limited tech resource allocation within companies, requires highly accurate and resilient outcomes, and impacts the lives of millions of companies and individuals every day.

Our AI Agents in the Jobs Vertical

Our AI Agents enables the user to apply for a job seamlessly on any site, by:

1. Autonomously understanding the input fields required within the job application flow on the career site, regardless of page layout, input types, or applicant tracking system;
2. Asking the required questions seamlessly on-site, without redirecting the candidate;
3. Using these responses to complete the application on the career site on behalf of the candidate;
4. Tracking when a job application has been completed successfully.

This is done by leveraging proprietary Computer Vision and Natural Language Understanding models, providing a flexible, robust and scalable solution.

The Problem

Candidates applying for a job today still have to go through an experience that resembles the early stages of the Internet:

1. They have to perform a search on a job website;
2. When they find a job they like and click on it, they get redirected to the company career site;
3. They are asked to set up a profile by filling in their personal details;
4. They are asked screening questions specific to the job;
5. Only at this stage are they able to complete the job application.

Candidates hate this experience, and 95% of them drop out before completing the application.

Key Features

• Page Understanding and Grounding: web pages come with varied and complex sets of UI controls, which are interpreted, classified and grouped utilizing both visual appearance and page source code.
• Comprehensive Interaction Capabilities: the interaction with UI controls and widgets, often a tricky and non-trivial task, exploits a repertoire of diverse techniques, including classical and visual-based automation tasks.
• Tactical and Strategic Reasoning: at each step of a job application, the most proper action to be undertaken on a web page is evaluated dynamically, according to the local context as well as user profile and navigation history.
• Adaptation and Resilience: the highly varying and fast-evolving landscape of job application web flows is properly addressed thanks to an outstandingly flexible and self-adapting architecture.
• Vertical Design Generalizability: SonicJobs utilizes a set of proprietary models, specifically designed and trained in-house, exploiting a broad range of vision and language techniques to ensure precise and context-aware processing of web pages in the jobs vertical, while being flexible enough to expand into new verticals.

Page Understanding and Grounding

A single web page has hundreds of elements. Our AI Agents combine advanced Machine Learning techniques to hande the complex task of web page interpretation. Task predictions are performed through state-of-the-art Computer Vision and fused with the output of HTML analysis and Optical Character Recognition to yield a solid grounding of UI detections.

SonicJobs’ proprietary models are trained with a combination of Supervised, Unsupervised and Self-Supervised techniques, and leverage a continuously growing amount of proprietary real-world data coming from our production environment, along with synthetic data generated using in-house technology.

Comprehensive Interaction Capabilities

Input UI controls and widgets on web pages exhibit great variability and a broad range of implementation options, including custom-made date pickers, dynamically-populated lists, overlays and pop-ups. Our AI Agents understand these input controls and interact with them, which is a very challenging task.

The AI Agents are able to leverage the HTML structure of an input control, as well as its visual aspect and any surrounding readable information, to pick the most suitable technique to interact with it using a vast repertoire of options. The range of techniques guarantees ample coverage of the interaction patterns encountered in web pages.

Tactical and Strategic Reasoning

Building upon a deep domain-grounded understanding of web pages, the AI Agents identify the most proper set of actions to be undertaken in order to advance the job application flow, filtering elements on the page that are not required. This includes understanding screening questions, resume upload, and clicking on the ‘Next’ and “Apply” buttons, to progress along multiple pages within the job application flow.

If the outcome of the chosen action is not what is expected, a buddy Agent specialized on error detection advises on correction or recovery measures to allow the main Agent to proceed with the flow.

Moreover, the AI Agents implement a Strategic Reasoning level, which affects the navigation flow from a broader perspective, allowing the Agent to try alternative paths to maximize success. An example of Strategic Reasoning is the capability of the Agent to deal with conditional questions with multiple dependencies.

Adaptation and Resilience

The choice of relying on AI rather than classical automation to interact with web pages allows the AI Agents to adapt to small or big changes for known job application flows as well as completely new and unknown flows.

SonicJobs proprietary technology, based on tactical and strategic reasoning coupled with error recovery, provides unmatched resilience and sets the standard in the Web AI Agents space.

In addition, the unique architecture of the AI Agents, includes advances in Multi-Agents frameworks, Multimodal Models and Model Ensembles, allows for a rapid integration of new technological evolutions, such as those tied to LLMs and Foundational Models.

Vertical Design with Generalizability

For enterprise AI Agent solutions, the interconnectivity between the domain-specific application layer and the general-purpose reasoning layer needs to be very strong. This is because accuracy and reliability are very important for enterprises, particularly for mission-critical workflows. Consequently, the architecture of the Agents needs to be specific to the domain (e.g. jobs), as well as generalizable enough to be expanded to further domains.

Our AI Agent technology is built for scale, with the core models targeted at capturing features that are common across websites (e.g. UI elements detection).

We have built an ‘Adaptive Learning Flywheel,’ where each successfully completed task (e.g. job application on behalf of the user) within our system continuously improves the performance of our models. As more successful applications are completed by our AI Agents, the models are automatically refined and optimized, leading to higher accuracy and error resolution, which in turn leads to more successful applications and further enhances the system’s capabilities.

Future Direction

SonicJobs is building on the next multi-agent architecture for deeper long-form context tasks on the web by leveraging on the latest psychology-inspired AI concepts and leveraging on the power of LLMs, fine tuning and in-context learning, and will further improve the generalizability and scalability of our models.

The AI Agent revolution is only just beginning, and SonicJobs is at the forefront of it.