Our calculations for identifying the most qualified attorneys, law firms or companies for each sub-category of intellectual property rely on a ranking methodology. This methodology serves as our recipe, determining how we measure the activity, success, and performance scores of all stakeholders. Once we have gathered all the necessary data, we use a proprietary algorithm to calculate each stakeholder’s final score which serves as the basis for our rankings. In this section, we provide an overview of how these scores are determined, which will ultimately be used in the calculation of the final rankings.
Entity Name Resolution
Cleaning up entity names is one of the most challenging parts of the ranking process, as it requires a team of highly skilled engineers and data scientists with access to powerful computational resources. The complexity of the task is due to the noise present in the data, which can make it difficult and time-consuming to sort through. It is crucial to resolve this problem through data cleaning, as companies and law firms may operate under various names and subsidiaries, leading to confusion in the data. Furthermore, mergers and acquisitions can significantly alter a company’s size and activity non-organically. Errors in filings can also cause difficulties in detecting the correct entity name, adding to the complexity of the process. For attorneys, the issue is even more challenging due to the potential for similar names and different filing formats, as well as frequent moves to different law firms. Addressing these challenges is a critical part of producing accurate and reliable rankings.
The Entity Name Resolution algorithm has been a focus of the Patexia Data Science team for several years. Although it is not yet perfect, we have made significant strides in improving its quality over time. Our approach involves leveraging advanced Natural Language Processing (NLP) techniques, alongside Machine Learning algorithms, to continuously enhance the accuracy and reliability of our data.
Defining and Measuring Important Qualities
In addition to a clean dataset, it is necessary to determine and extract crucial parameters from the vantage point of diverse stakeholders, including corporations, trademark owners, trademark firms, and attorneys. Our evaluation is based on four overarching factors as follows:
- Activity – The number of registered trademarks obtained by the trademark owners, law firms or attorneys during the period of our study.
- Success – The process of getting the trademark registered successfully and efficiently.
- Performance – The overall quality of work performed by the attorney, or law firm (or the trademark owner) which is a function of both activity and success.
- Cost – The overall cost, including attorney’s fees and USPTO fees, for obtaining and maintaining the trademark.
We defined the Performance and Success for each of the stakeholders as a weighted average of the other factors. For example, we define the Performance Score of law firms or attorneys as:
Performance (law firm / attorney) = a1. Success + a2. Activity
Important Signals
The Patexia Data Science team has extracted multiple signals from the raw trademark data to build the model and calculate various parameters for success and performance evaluations. These signals underwent assessment by the trademark community through a conducted survey before generating this report series. The following signals are among the essential ones that we have defined and utilized in our assessment:
- Activity Count – A count of the number of registered trademarks per trademark owner, attorney, or law firm for the period of the study.
- Applications Filed – The number of trademark applications filed during the period of the study.
- Applications Concluded – The sum of all trademark applications registered or abandoned during the period of the study.
- Registration Rate – The ratio of trademarks registered to all trademark applications concluded (either registered or abandoned) during the period of the study.
- Number of Office Actions – Average number of Office Actions per trademark application that was handled by the law firm or attorney during the period of study.
- Pendency – The number of days from the filing date to the registration or abandonment date of the trademark application.
- Number of Time Extensions – The total number of times the attorney or law firm requested an extension of time per application
- Opposition – The total number of oppositions that the trademark attorney, law firm, or the trademark owner has received for the entire portfolio during the period of the study.
We defined success as a weighted average of Registration rate, Office Actions and Oppositions. We are constantly adding new signals to improve our measurements and make them more relevant.
Normalization of Scores Using Regression Model
After the Success and Performance Scores are calculated, they are normalized based on their deviations from the mean value, and then adjusted out of 100 to present them in percentage form. However, to accurately measure the success of an attorney, law firm or company, it is essential to evaluate their success independent of all other factors. Therefore, we have developed a regression model that allows us to adjust for these factors and determine a true measure of success for each stakeholder.
We mathematically calculate the impact of different variables on the success or failure of an individual attorney, law firm, or company. This allows us to evaluate their success independent of other factors, and obtain a more accurate measure of their performance. For instance, when calculating the success of an attorney, we not only consider the success of the company they are representing for trademark, but also the role of the examiner. Similarly, when evaluating the success of a law firm, we take into account the success of the companies they represent and the examiners they work with. Lastly, when evaluating the success of a company in the trademark space, we consider the success of the law firms and attorneys they work with, as well as the examiners they interact with.
The implementation of this comprehensive approach has resulted in significant enhancements in the precision and impartiality of our rankings, ultimately offering our clients with more insightful and useful information. As a result, our rankings now provide a more accurate and fair representation of the performance and success of attorneys, law firms, and companies in the intellectual property industry. By continuously refining and improving our methodology, we aim to remain at the forefront of delivering reliable and insightful rankings to our clients.
Scope of Analysis and Timeline
As previously mentioned, our evaluation process considers a five-year timeframe instead of just a single year. The rationale behind this decision is multifaceted. Firstly, securing a trademark typically takes more than a year, thus examining a shorter period would not provide a comprehensive understanding of an attorney or law firm’s success. Additionally, attorney and law firm workloads often fluctuate from year to year, rendering a single year’s data insufficient for drawing meaningful conclusions. By analyzing data over a five-year window, we can identify general trends and patterns, rather than simply providing activity rankings. This enables us to evaluate different stakeholders using a larger dataset, while also reducing the impact of incomplete data for a particular organization during any given year. As a result, this approach enhances the accuracy and fairness of our rankings, providing clients with more valuable insights.
For the purpose of this report, we looked at a total of 2,510,218 trademark applications that were concluded (registered or refused) between January 1, 2018 through December 31, 2022. Due to the size of the data set, only the top 10,000 most active companies, law firms, and attorneys were included in our performance evaluations. Therefore, any firm that was ranked lower than 10,000th in terms of activity was not considered for performance measurement.
Data Sources
The data source used in this report is the USPTO Bulk Trademark Data, which is a comprehensive collection of XML files containing essential metadata of trademarks registered in the United States. The USPTO, being the primary agency responsible for granting patents and registering trademarks, maintains a rich collection of data that serves as a valuable resource for conducting empirical research and drawing meaningful insights. By utilizing this data, we were able to conduct a rigorous analysis of trademark activity, and provide valuable insights to our clients.
