A printed version of the IRS US Tax Regulations is over 75,000 pages long, and taxpayers and even tax professionals struggle to wrap their minds around such a volume of complex unstructured text. But can Machine Learning (ML) make much of this work unnecessary?
In this blog we discuss how the Artificial Intelligence science of Natural Language Processing (NLP) can be deployed. We also examine the challenges of using standard NLP tools to understand the meaning of tax rules – which are sometimes written in a legalistic, unnatural linguistic style, and what might be needed in what may be called “Unnatural Language Processing” (ULP).
Currently, a substantial amount of on-line consumer activity is automatically analyzed with NLP to assess sentiment and meaning of on-line text in areas such as restaurant reviews, customer complaints, talk-bots, message boards, social media and on-line communities. Until recently, much of the AI-NLP work has been focused on understanding this sort of commonplace language, the kind found in transcripts of spoken language, literature, news stories, and entertainment reviews.
However, regulations important to tax professionals are far more complex. For example, let us consider the ways NLP could improve tax understanding and the speed of tax advice. The first sentence in the US Tax Regulations is about income taxes (Title 26):
Right from the starting gate, we encounter nuanced sentence structure, indication of exceptions, and references to other sections – and the beginning of a list. This text is challenging for both NLP analysis and human comprehension.
How does a human go about the task of understanding this passage? To begin, the reader must understand the sentence – and this task is similar to that of understanding a restaurant review.
However, the structure of the sentence in the tax passage above is also different. It is actually so different from normal language that full comprehension can only be achieved by incorporating the meaning of other terms referenced in the text.
A diligent reader would need to read those referenced terms and merge the sentence with existing knowledge to fully comprehend what is means. Terms such as “gross income” can easily be identified. But not only those explicitly referenced is needed, but also those implied (synonyms, antonyms and related terms). Then exceptions in the tax regulation must be called out, and segregated, for clarity.
Thankfully, while it is hard to automate the comprehension of the passage above, it is straightforward for a linguistic system to organize the reference terms for effective human reading and comprehension.
As we have noted, the grammar and words in tax regulations are more sophisticated, and the sentence structure in tax regulations is different. For example, the average sentence length in English is 25-35 words, but in tax regulations, it is often far longer.
A randomly selected passage from the Federal unemployment tax act states
That sentence is almost 3 times the length of standard language (67 words), and is far more complex than those used in normal writing or conversation. If you doubt that, try reading the passage above to your non-tax spouse, or partner, and ask them what it means.
But there is even more complexity. In the tax writings, we often find lists that are preceded by words such as “unlike”, “such as”, “including” or “the following”. Each of these terms have different meanings. The term “unlike’ defines a term by showing what it is not. The terms “such as” and “includes”, provides a non-exhaustive example. While the term “the following’ provides a list of items that defines what it must be (rather confusing is it not?)
In short, the use of lists of terms compounds the difficulty of any attempt at applying NLP to tax regulations.
While overall important, the treatment of references and terms is particularly important in tax. From a linguistic standpoint, it is often results in an intricate mesh of references and individually defined terms found in the tax code, regulations, and court and letter rulings. So, it is important to understand how they got there.
US tax law begins with the Code, enacted by Congress. Then the IRS publishes Regulations, stating the implementation of Code. When a taxpayer (individual or corporation) requests clarification of regulations, the IRS sometimes responds with a Letter Ruling – which then also becomes part of regulatory refinement.
In addition, some tax law arises from court rulings. So all sources (Code, Regulations, Letter Rulings, and Case Law) must be processed and merged to construct a comprehensive text corpus. To add to this complexity, the cross-referencing between these categories is not standardized.
However the good news is that cross referencing can be done with modern NLP. This technology can surface all thematic references across the entire body of writing. Think of this as an “automated uber-search” approach, to get to the meaning and context to interpret the tax code.
In our experiment of applying standard NLP software packages to the 75,000 pages of codes and regulations, we found that approximately 86% of the approximately 900 tax regulation sections contained a list. Some of the lists were over 1,000 words long, with nested sub-headings, references, rules, and exceptions to rules.
So, while existing NLP software can identify a list by detecting “delimiters” such as “1)”, “à”, etc., they often performed poorly at capturing internal list structures, such as hierarchies, exceptions, and negations. The issue is that much of the information in tax regulations is found in lists, not sentences. So, while humans are great at finding meaning in lists, for machines this is rather complicated.
For example, meaning in a sentence is typically embodied in the verb (in NLP known as the ‘head’). The meaning of a list is a noun phrase, and the meaning is often stated in the introduction to the list. For example, in phrases such as “these are items specifically included in gross income:”, where “gross income items” is the meaning, and we expect to find a list populated with nouns designating concepts like “cash”, “payment in kind”, etc.
What we found in our experiment is that new specific list parsers for tax will be needed in order to perform NLP on list-oriented text – lack of a good NLP list processor is currently a fundamental impediment to applying it to professional tax writing.
What we also found is that quite a large part of the domain knowledge in tax is found inside tables. Unfortunately, in the regulation text it appears that these tables are constructed with even fewer standards than lists and the information is very different from a sentence (it is often numerical). So to make NLP work for tax, a new “numerical grammar scheme” needs to be created to understand tax codes.
The issue is that most of us learn the practice of writing sentences and hopefully conforming to generally agreed-upon grammar. In contrast, few such standards are taught, or even exist for lists or tables, though they increasingly comprise a large share of professional tax communication.
It seems that in writing lists in the US tax code, any ‘rule’ is applied. However, the US Treasury now has as part of the Plain Writing Act of 2010 created a writing style committee. This committee is now setting new standards so that the style may become more uniform in the future. This includes the construction of lists and tables.
The outcome from the work of this promising committee may significantly help us in automating the understanding of the tax code and regulation.
Even, with the current challenges, we believe that NLP can still provide some benefits. This includes:
So, will computers automate interpreting tax laws and regulations? For the near future, the answer is both ‘yes’ and ‘no’.
In the “yes” column, prospects are good for automating routine, repetitive tasks with classification algorithms aided by NLP. This includes tax category determinations, like answering questions regarding the status of a worker as either a contractor or employee, based on NLP analysis of their hiring agreements.
In the ‘no’ column, we note that more complex tax decision-making and tax advice functions may require creating tax specific NLPs with a consistent tax knowledge base.
There are many challenges in doing this, including linguistic “unnaturalness” of the documents, abundance of complex lists and tables, and unifying single concepts that are dispersed across several types of tax regulations.
In addition, the investment required to make a full “moon shot” for NLP expert systems may not be justified at the present. Therefore, it seems unlikely an all-knowing AI tax expert will be built in the near future – but very likely that the in the short run, human tax experts will be increasingly supported by AI and NLP, but not replaced by it.
That means that tax professionals must learn to embrace these digital assistants as another member of the team and provide organization, skill set and governance as they start embracing AI, ML, and NLP.
Co-written with Dr. Cas Milner, data scientist, former professor, and a consultant working with PwC on advanced analytic projects.
Principal, Advanced Tax Analytics & Innovation, PwC US