In the past, it was challenging to rank and evaluate patent attorneys, law firms, or patent portfolios using quantitative measures. While previous reports have ranked firms or companies based on their filing activity, little effort has been made to evaluate the quality of an attorney’s work or a law firm’s performance, or to evaluate the quality of a patent portfolio. This was due to the unavailability of data from the patent office and the high cost and complexity of processing large data sets.
However, advancements in powerful computation and the availability of data from the US Patent Office and other sources, such as the Patent Trial and Appeal Board (PTAB) or PACER for patent litigation, have made this task both feasible and affordable.
Entity Name Resolution
In 2016, we began analyzing data to rank law firms and companies based on their activities for the first time. However, cleaning the data has been a major challenge for this type of analysis. Companies and law firms may file under different names and subsidiaries, and mergers can alter a company’s size and activity in non-organic ways. Additionally, errors in filings make it difficult to determine the correct entity name.
Over the past six years, the Patexia Data Science team has been perfecting its Entity Name Resolution algorithm to address this challenge. We can now automatically extract useful information from thousands of PDF documents, including the names of attorneys who worked on a particular patent application. While the algorithm is not perfect, its accuracy has improved over time. We use Natural Language Processing (NLP) techniques and Machine Learning to continually train our system and enhance our data quality.
Including Interviews as an Activity Indicator
After making it possible to rank attorneys based on filing activity, we dedicated ourselves to enhancing the ranking methodology by including interviews as an indicator of activity for patent attorneys. Interviews refer to a communication between a patent examiner and the applicant or the applicant’s representative, typically a patent attorney or agent. The purpose of the interview is to discuss and resolve any issues or concerns that the examiner may have with the patent application. Interviews can be conducted in person, by phone, or via video conference, and are usually scheduled by the applicant or their representative after a preliminary examination of the application by the patent examiner. The goal of the interview is to facilitate the patent prosecution process and ultimately obtain a patent with the broadest possible claims.
In the past, the number of issued patents or filed applications was been the primary metric for determining a patent attorney’s activity level. However, we recognize that interviews with patent examiners are also an important part of the patent prosecution process and can significantly impact the outcome of a patent application. As such, we have now added interviews as a factor in our activity rankings, providing a more comprehensive view of a patent attorney’s overall activity level. This improvement is part of our ongoing efforts to refine our ranking methodology and provide our clients with the most accurate and meaningful insights into the patent landscape.
Defining and Measuring Important Qualities
Aside from a clean data set, we had to define factors that were important from the perspective of different stakeholders (i.e., Corporations (Assignees), Law Firms, and Examiners). We came up with the following five high-level factors to measure the overall performance of any stakeholder:
- Activity – The number of issued patents or filed applications by a corporation and law firm, or the number of issued patents and interviews participated for attorneys
- Success – Obtaining a patent or patent claim from the perspective of the applicant, law firm, or attorney
- Quality – The claim quality, broadness, novelty, and market coverage that ultimately make a patent or a claim valuable.
- Efficiency – The speed and simplicity of the process of obtaining a patent
- Cost – The overall cost, including attorney’s fee and the USPTO fees, for obtaining and maintaining a patent
We then defined the performance of each of the stakeholders as a weighted average of those five factors. For example, we define the performance of law firms and companies as:
Performance (law firm / attorney) = a1. Success + a2 . Efficiency + a3 . Activity
Performance (company) = b1 . Success + b2 . Activity + b3 . Quality
In our ranking methodology, we distinguished the performance criteria between companies and law firms since the latter are typically not accountable for the quality of the invention itself, including the field, market, and complexity, among other factors. Their objective is to acquire a valid patent with the broadest claim feasible in a cost-effective and efficient manner. However, they are not the ones generating the invention.
On the other side, companies strive to secure the highest quality patents while also minimizing expenses. Although they have less control over the efficiency of the process, they can influence the quality of their inventions.
Important Signals
The Patexia Data Science team extracted more than 20 signals from a variety of data sources and constructed all of the above factors (e.g., success, quality, efficiency, activity) by combining different signals with different weights. For the purpose of this report, these are some of the key signals we used in our analysis:
- Filing Activity Count – A count of the number of issued patents per organization, attorney, examiner, etc.
- Interview Count – Number of interviews in which the patent attorney or agent participated during the patent prosecution process
- Allowance Rate (Application-Level) – The ratio of patents issued to applications filed for a certain period of time.
- Allowance Rate (Claim-Level) – The ratio of the claims allowed to claims applied for in the original application.
- Number of Office Actions – Average number of Office Actions during the examination process.
- Pendency – The number of days from filing date to issuance or abandonment of an application.
- Number of Time Extensions – The total number of times the attorney or law firm requested an extension of time to respond to an Office Action. Some of these extension requests may require additional fees of up to $3,160 for large entities, which often means a delay in the examination.
- Ratio of Average Number of Words in Independent Claims (patent to application) – The patent to application ratio of the average number of words in independent claims shows how much the claim changed from its original form. A number smaller than one shows that more limitations were added during the examination, which usually limits the coverage of the claim.
Some other factors that we considered for quality measurement were, for example, Forward and Backward Citation, Number of Words in Independent Claims. We are constantly adding new signals to improve our measurements and make them more relevant.
Normalization and Noise Removal
There are several players that could impact the outcome of patent examination:
- Company – Assignees or law firm clients (companies) may have different strategies when it comes to filing. These strategies could influence the work an attorney performs.
- Examiner – While the patent office provides a very detailed guideline for the examination process, each examiner may still act somewhat differently based on his/her judgment and understanding of the invention. The examiner’s assessment of an application could change the final outcome.
- Subject Matter – The field of invention, its complexity, novelty, etc., could change the outcome. For example, some fields are more mature (e.g., agriculture or civil), where inventions tend to be narrower. Some other fields are newer (e.g., IoT, drones). The amount of prior art obviously varies significantly based on the area of invention, which could change the examination time, claim broadness, and many other factors.
- Law Firms / Attorneys – As the person or organization in charge of patent prosecution, law firms and attorneys undoubtedly have a huge impact on the outcome. Depending on which firm takes over the patent prosecution – and even within the firm, depending on which attorney is in charge – we could see substantially different results.
In order to ensure fairness and eliminate the influence of various factors such as subject matter, examiner, or company, we made our measurements agnostic to these factors when calculating performance for a particular player, such as law firms. This was critical because even if the same attorney operates under similar conditions, they may achieve vastly different results depending on the company’s corporate culture. For example, the results may vary greatly when working with companies like Broadcom versus Qualcomm.
Different companies may have varying strategies when it comes to obtaining patents. While some may prioritize pursuing the broadest claim possible and use multiple Office Actions to achieve this, others may place greater emphasis on cost control and limit correspondence with the examiner to avoid incurring unnecessary expenses.
Due to the large amount of data we analyzed, which included over two million applications over a five-year period, we were able to use regression modeling to statistically eliminate the interference caused by the aforementioned factors when focusing on a particular factor. For instance, when calculating the performance of law firms, we removed any noise caused by individual companies, examiners, or a specific subject matter. It would not have been appropriate to compare without such noise removal.
Scope of Analysis and Timeline
When evaluating patent activity, a five-year window provides a more comprehensive view than a single year. This approach accounts for the average time of about three years required to obtain a patent, allowing for the identification of broader trends and patterns beyond mere activity rankings. Additionally, it minimizes the impact of incomplete data for a specific organization during part of a year. In this report, we analyzed approximately 1.8 million utility patents and 2.2 million utility applications issued or abandoned between January 1, 2018, and December 31, 2022. To address the lower volume of biotech companies’ patent activity compared to high-tech, we created separate metrics for these categories. Thus, the report is primarily structured around three categories.
- High-tech patents
- Biotech patents
- All patents (everything including high-tech and biotech but excluding design patents)
Since Design Patents, in general, have a different examination and prosecution process, and because a shorter amount of time is required to obtain a design patent, we decided to exclude them from most of this analysis. This helps us focus mainly on utility patents, which are the most popular type of patent.
To keep things simple, we divided the high-tech and biotech patents, using the Technology Center associated with each patent application by the USPTO.
Moreover, we limited our measurements to the top 2,000 most active law firms in different categories (i.e., high-tech, biotech, and overall). That means if, for example, a firm was ranked 2,200th in terms of Overall Activity, it was not included in the Overall Performance measurement.
Data Sources
For this report, we used several sources of data provided by the USPTO:
- USPTO Bulk Patent Data – A set of XML files provided by the patent office, which covers metadata of patents as well as claims, body, and description.
- USPTO Examiner Dataset – The patent office has started releasing examiner data on a yearly basis. This data covers all transactions related to each patent application, the status of applications, applicant and correspondence addresses, etc., often provided in a CSV format or JSON.
- USPTO Examiners Database – A list of all current examiners and their art units as well as technology centers.
Dive into Our Latest Patent Intelligence Report
Explore the latest insights and trends in patent prosecution with our newly released Patent Intelligence Report. Gain valuable insights into the ever-evolving landscape of intellectual property. Access the full report here!
