Patent Auto Classifier gets about two in three classifications right

The ANAO analyzed IP Australia's data and found that 64.7% of the Patent Auto Classifier's classifications were correct as of November 2025. Another 27.5% were missing and 7.8% incorrect.

  • 1 for an organization

    PAC routes incoming applications automatically, and 64.7% of its classifications were correct, though the time that it saves was not measured.

Depth runs from 1, a convenience or time saved, to 5, a life or a livelihood preserved.

Status
Partly reported benefit.
Evidence

foundstudiedreportedclaimed

An independent audit report supports the benefit.

The system’s part
Not isolated. The ANAO measured the correctness of PAC's own classifications, so the figures concern the system itself. It did not compare classification with and without PAC or measure the time that PAC saves. The ANAO describes PAC as an AI system that uses natural language processing and is retrained every year.
Not yet checked
LimitsCosts and harmsLearns from dataLater outcome

Download data Cite Suggest a correction

Benefit type
Repetitive Task Automation
System
Patent Auto Classifier (PAC)
Made by
IP Australia (built in-house and retrained every year)
Used by
IP Australia (patent examination)
Where
Australia (national)
Stage
In routine use
Date
, when the source was published
Kind
Aggregate result
Broad benefit
Processing Information, Identifying Objects, Actions, and Events
Collections
Scrutiny and findings
Last revised
1 November 2026

Sources

  1. Audit report, by authors with no tie to the system.

    Australian National Audit Office. Artificial Intelligence Use in IP Australia. Auditor-General Report No. 43 of 2025-26, published 29 June 2026. Paragraphs 3.25 and 3.26 and Figure 3.3 (archived copy of the report page).

    The ANAO analysed this data and as at November 2025, the model's classifications are on average 64.7 per cent correct, with 27.5 per cent of its classifications missing and 7.8 per cent of its classifications being incorrect

Details

AI system

Workflow

PAC read the text of each application and scored how relevant each International Patent Classification symbol was. It sent each application to the examination queue with the highest score, and examiners redirected the applications that it misrouted.

Follow-up

Conditions for replication

IP Australia retrains the model every year so that its performance does not drift. Examiners must catch and redirect the applications that PAC sends to the wrong queue.

Entry details

Sector

Intellectual property administration (patent examination).

Purpose

IP Australia built the Patent Auto Classifier to classify each incoming patent application by type of invention and send it to the right examination queue.

31 more questions not yet checked

model, release, provider, role, configuration, inputs and outputs, autonomy, human oversight, AI contribution, beneficiaries, reach, distribution and access, duration, every measurement question, every money question, ownership and control, dependencies, cross-border connections, geopolitical implications, replication attempts, availability, institutional decision, grounds, action and result, success criterion, every evidence question, observed or intended, alternative assessments, related entries, other identifiers, category mappings.

Cite this record

Peter H. Vartanian. (2026). Patent Auto Classifier gets about two in three classifications right (AIBD-1010). Artificial Intelligence Benefits Database, edition of 1 November 2026. https://petervartanian.xyz/aibd/records/AIBD-1010.html (archived at https://doi.org/10.7910/DVN/AADSII, version 7.0)

BibTeX Data Field definitions