Since the launch of our inaugural IPR Intelligence Report in 2017, we have consistently enhanced and refined our ranking methodology with each passing year. Our commitment to improvement is underscored by our active engagement with the IP community, soliciting feedback through direct communication and surveys. This valuable input, coupled with the expertise we have amassed over time, has been instrumental in enhancing the relevance and utility of our rankings.
Here, we will provide a detailed exposition of the ranking methodology employed in our 2024 PTAB Intelligence Report. We eagerly anticipate your feedback, which will contribute to our preparations for next year’s report. For the current edition, our analysis spans a five-year period, encompassing data from July 1, 2019, through June 30, 2024.
General Considerations
We will proceed with the assumption that our readers possess a basic understanding of PTAB proceedings. In its simplest form, these proceedings constitute trial proceedings conducted at the Patent Trial and Appeal Board (PTAB). They serve the fundamental purpose of scrutinizing the patentability of one or more claims, with the specific ground for review varying depending on the type of proceeding.
The landscape of patent litigation and challenges underwent a transformative shift with the enactment of the Leahy-Smith America Invents Act (AIA) in September 2011. This landmark legislation introduced a set of new procedures that significantly impacted the way patents are scrutinized and disputed. IPRs and PGRs emerged as key components of this legislative overhaul. They were designed to offer a more efficient and cost-effective alternative to traditional litigation in federal courts. Here’s a brief overview of each:
- Inter Partes Review (IPR): IPR was introduced as an avenue for challenging the validity of existing patents. It allows third parties to petition the PTAB to review the claims of a granted patent. Unlike traditional litigation, IPR proceedings are conducted before the PTAB, which comprises administrative patent judges with specialized expertise. This streamlined process is designed to expedite the resolution of patent disputes.
- Post-Grant Review (PGR): PGR was established to provide a mechanism for challenging patents that were granted after the AIA’s enactment in March 2013. It allows a third party to request a comprehensive review of the patent’s validity based on a broader range of grounds, including issues such as patent eligibility and adequacy of written description. PGR is typically initiated within nine months of a patent grant.
Numbers for Performance and Success Rankings
In previous years, our approach necessitated the exclusion of firms or attorneys with limited activity when compiling performance rankings. However, we have since refined our methodology to encompass all individuals and entities with at least one concluded IPR or PGR proceeding.
For the computation of Success and Performance Scores, we exclusively considered cases that had reached a non-pending status, which includes the following four categories: Terminated-Denied, Terminated-Settled, Terminated-Adverse Judgment, and Final Written Decision Entered, as of September 2, 2024. Consequently, our Success and Performance rankings encompass all attorneys, law firms, and companies who have participated in at least one scorable case (refer to Table 1).
If an attorney, law firm, or company satisfies this minimum case requirement, they are eligible for inclusion in our Success or Performance Rankings, whether in the overall category or specifically as petitioners or patent owners.
In continuity with our approach from the previous report, we have chosen to incorporate the data of up to 1,000 of the most active attorneys, law firms, and companies. This comprehensive dataset includes their respective scores and rankings across all categories and is conveniently presented in the accompanying Excel spreadsheet.
Success Scores for Petitioners, Patent Owners, and Overall
To ascertain the performance ranking of each stakeholder, our initial step involves calculating their success in every IPR or PGR. This assessment is conducted promptly upon the termination of a case, whether due to settlement, denial, or the issuance of the Final Written Decision by the PTAB panel of judges. In our evaluation, we make diligent use of claim-level data whenever it is available, as it enhances our ability to comprehensively assess each party’s level of success.
The allocation of points to each party or representative is outlined in the following table (Table 6.1):
| Status | PO Success Points | Petitioner Success Points |
| Terminated-Denied | 1 Point | 0 Point |
| Terminated-Settled | 0.25 Point | 0.75 Point |
| FWD Entered | 1 – Petitioner Success Point | 0.5 + 0.5 x Claims InvalidatedClaims Listed |
| Terminated-Adverse Judgement | 1 Point | 0 Point |
Table 1 – Point Allocation for the Petitioner and the Patent Owner or their Representatives
In situations where not all claims are invalidated, we acknowledge the nuanced dynamics at play. If the surviving claims were originally part of a district court case, this development is generally favorable for the patent owner (or the plaintiff in the district court case). It allows them to continue pursuing potential damages for infringed claims in the district court, constituting a notable victory from their perspective.
However, in the context of an IPR or PGR, the attorneys representing the petitioner have also made a significant impact by invalidating some of the asserted claims. This achievement can serve as leverage during negotiations, potentially leading to more favorable settlement terms. Consequently, these attorneys deserve recognition for their efforts and are awarded a minimum of 0.5 points for the invalidation of one claim. If they successfully invalidate multiple claims, their credit for the outcome is calculated based on the formula outlined in Table 6.1. This approach acknowledges the intricate interplay between the parties involved and seeks to provide a comprehensive evaluation of their contributions and achievements.
For instance, if the petition has contested 10 claims, and only three of them were invalidated following the issuance of the Final Written Decision, the attorney or law firm representing the petitioner will be awarded 0.5 + 0.5 x 3/10, resulting in 0.65 points. Conversely, the attorney or law firm representing the patent owner will receive 1 – 0.65, equating to 0.35 points for that particular case.
In cases of settlement, although it may be perceived as a mutually beneficial outcome, it often indicates that the patent owner and their legal representatives deemed the case to be relatively weak in light of the existing prior art and the arguments presented by the petitioner. Consequently, they opted for settlement to minimize their potential losses rather than proceeding with the IPR. We acknowledge that settlements can arise from various other factors, such as broader strategies related to co-pending district court cases. Nevertheless, to ensure a consistent method of assessment across all cases, we have assigned 0.25 points to the patent owner or their representatives in settled cases and 0.75 points to the petitioner or their representatives. This approach acknowledges the differing implications of settlements for both parties and reflects the complexity of settlement dynamics in the context of IPR proceedings.
The “Terminated – Adverse Judgment” outcome represents another final scenario taken into account when calculating the success scores. In these instances, where the patent owner voluntarily concedes the invalidation of all claims, the points are attributed to the petitioner’s side, including their representatives, law firm, and attorney.
Once the success points for each party involved in a case are computed and assigned, we proceed to calculate the average points across all their cases as a measure of their overall success. To determine the patent owner or petitioner scores, we take the average of all the non-pending (concluded) patent owner or petitioner cases, specifically those falling under the categories of “Terminated-Denied,” “Terminated-Settled,” “Terminated-Adverse Judgment,” or “FWD Entered Statuses.”
For the Overall Success Score, we calculate the average score encompassing all cases associated with a particular attorney, law firm, or company. This calculation is performed after normalizing the petitioner or patent owner success scores, ensuring a comprehensive evaluation of their success across the entire spectrum of cases they are involved in.
Normalization Using Machine Learning
In the realm of Patent Trial and Appeal Board (PTAB) proceedings, the evaluation of individual success is a multifaceted endeavor. Numerous intricate variables come into play, such as the legal arguments presented by the involved parties, the decisions of the presiding PTAB judges, the specific patents and technologies in question, and the legal acumen of the attorneys representing each side. Moreover, expert witness testimonies and other relevant elements wield considerable influence over the final outcome of these proceedings.
Assessing the success of an individual, whether it be an attorney or a petitioner, within a single PTAB case can be exceedingly challenging. The interconnected nature of these factors makes it arduous to discern an individual’s true impact on the case’s resolution independently. However, by harnessing extensive datasets compiled from a multitude of PTAB cases, we employ the formidable capabilities of advanced computational techniques and machine learning algorithms to engage in a rigorous mathematical analysis of individual performance across diverse circumstances and cases.
Our approach to evaluating Success Scores for PTAB participants is underpinned by a commitment to fairness and precision. To mitigate the potential influence of external factors, including the success of other entities or the discretion of PTAB judges, we have developed a robust analytical model deeply rooted in cutting-edge machine learning methodologies. This model effectively segregates the performance of each entity—whether it be a company, attorney, or law firm—from external effects that could skew the analysis. As a result, we offer a more refined and impartial appraisal of individual success within the complex landscape of PTAB proceedings. Our scoring methodology is meticulously designed to furnish a transparent and dependable representation of each participant’s performance, untainted by undue influence from external variables.
In practice, we employ advanced analytical techniques, incorporating intricate adjustments to recalibrate the scores of all involved parties and their representatives, taking into account the various factors that can impact the outcome of PTAB cases. Once we have obtained these adjusted success scores for each participant, we calculate the average success score for each party or their representatives by considering their performance across all the PTAB cases in which they were engaged. This advanced approach culminates in a more precise and equitable evaluation of individual success within the intricate landscape of PTAB proceedings, facilitating a comprehensive understanding of their performance.
Activity Scores for Petitioners, Patent Owners, and Overall
We recognize that in-house counsel at companies tend to prioritize candidates with recent activity and experience over those whose activity occurred several years ago. Consequently, the Activity Score places greater emphasis on more recent cases as compared to older ones. To illustrate, in this model, an attorney who was involved in a total of 50 cases, all filed in 2019, would receive a lower ranking than another attorney with the same 50 cases but distributed over a span of last five years.
Furthermore, to calculate the Activity Score, we employ a logarithmic function. This approach helps prevent significant disparities between entities with varying case numbers, rendering comparisons more equitable and facilitating a more nuanced assessment of entities with different activity levels.
Performance Scores for Petitioners, Patent Owners, and Overall
To calculate the Performance Score, we’ve merged the Activity and Success Scores. This integration addresses the issue where extensive caseloads could potentially dilute performance measurements over time. Consequently, it becomes less than ideal to directly compare the performance scores of firms or attorneys with significantly different caseloads, even after excluding firms with low activity.
The “Performance Score,” which enables us to score and rank companies, attorneys, and firms based on both Activity and Success, is essentially a weighted average of the success and activity scores. This holistic approach serves to assist companies in identifying highly active and exceptionally successful law firms and attorneys for their cases—essentially, the most qualified professionals in the field.
Performance and Activity Rankings for Petitioners, Patent Owners and Overall
We ranked all companies, law firms, and attorneys based on their Activity Scores. However, when it came to determining Performance Rankings, we exclusively ranked those companies, law firms, or attorneys that fulfilled the minimum requirement of having at least one scorable case in the respective category. Consequently, there are companies, law firms, or attorneys who were active during the period of our study but did not receive performance scores due to their cases being either pending or non-scorable.
Performance of Judges
When it comes to PTAB Administrative Judges, we conducted an analysis focusing on the cases and statistics pertaining to the primary judge responsible for crafting the Final Written Decision. Our methodology mirrored that described earlier, exclusively considering cases with the statuses of Terminated-Denied or Final Written Decision. Cases that were settled were excluded from this subset, as judges play no role in the settlement process between the parties.
Each judge was assigned a score out of 100, which indicates the extent to which their judgments tended to favor the petitioner. Our comprehensive report encompasses the complete roster of PTAB Administrative Judges for the year 2023, offering insights into their activity (number of proceedings) and their performance, particularly in relation to petitioners or patent owners. A Performance Score approaching 100 suggests that the judge more frequently ruled in favor of the petitioner.
