Machine learning (ML) in US tax regulations

Start adding items to your reading lists:
Save this item to:
This item has been saved to your reading list.

Bjarne Berg Principal, Advanced Tax Analytics & Innovation, PwC US August 29, 2017

A printed version of the IRS US Tax Regulations is over 75,000 pages long, and taxpayers and even tax professionals struggle to wrap their minds around such a volume of complex unstructured text.  But can Machine Learning (ML) make much of this work unnecessary?

In this blog we discuss how the Artificial Intelligence science of Natural Language Processing (NLP) can be deployed. We also examine the challenges of using standard NLP tools to understand the meaning of tax rules – which are sometimes written in a legalistic, unnatural linguistic style, and what might be needed in what may be called “Unnatural Language Processing” (ULP).

Tax and Natural Language Processing

Currently, a substantial amount of on-line consumer activity is automatically analyzed with NLP to assess sentiment and meaning of on-line text in areas such as restaurant reviews, customer complaints, talk-bots, message boards, social media and on-line communities. Until recently, much of the AI-NLP work has been focused on understanding this sort of commonplace language, the kind found in transcripts of spoken language, literature, news stories, and entertainment reviews. 

However, regulations important to tax professionals are far more complex. For example, let us consider the ways NLP could improve tax understanding and the speed of tax advice. The first sentence in the US Tax Regulations is about income taxes (Title 26):

  • ” § 1.61-1Gross income. (a) General definition. Gross income means all income from whatever source derived, unless excluded by law. Gross income includes income realized in any form, whether in money, property, or services. Income may be realized, therefore, in the form of services, meals, accommodations, stock, or other property, as well as in cash. Section 61 lists the more common items of gross income for purposes of illustration. For purposes of further illustration, § 1.61-14 mentions several miscellaneous items of gross income not listed specifically in section 61… “

Right from the starting gate, we encounter nuanced sentence structure, indication of exceptions, and references to other sections – and the beginning of a list. This text is challenging for both NLP analysis and human comprehension.

How does a human go about the task of understanding this passage? To begin, the reader must understand the sentence – and this task is similar to that of understanding a restaurant review.

However, the structure of the sentence in the tax passage above is also different. It is actually so different from normal language that full comprehension can only be achieved by incorporating the meaning of other terms referenced in the text.

A diligent reader would need to read those referenced terms and merge the sentence with existing knowledge to fully comprehend what is means. Terms such as “gross income” can easily be identified. But not only those explicitly referenced is needed, but also those implied (synonyms, antonyms and related terms). Then exceptions in the tax regulation must be called out, and segregated, for clarity.

Thankfully, while it is hard to automate the comprehension of the passage above, it is straightforward for a linguistic system to organize the reference terms for effective human reading and comprehension.

The Complexity of Tax Regulations

As we have noted, the grammar and words in tax regulations are more sophisticated, and the sentence structure in tax regulations is different. For example, the average sentence length in English is 25-35 words, but in tax regulations, it is often far longer.

A randomly selected passage from the Federal unemployment tax act states

  • “No reduced rate of contributions to a pooled fund or to a partially pooled account is permitted to a person (or group of persons) having individuals in his (or their) employ except on the basis of his (or their) experience with respect to unemployment or other factors bearing a direct relation to unemployment risk during not less than the 3 consecutive years immediately preceding the computation date."

That sentence is almost 3 times the length of standard language (67 words), and is far more complex than those used in normal writing or conversation.  If you doubt that, try reading the passage above to your non-tax spouse, or partner, and ask them what it means.

But there is even more complexity. In the tax writings, we often find lists that are preceded by words such as “unlike”, “such as”, “including” or “the following”. Each of these terms have different meanings. The term “unlike’ defines a term by showing what it is not.  The terms “such as” and “includes”, provides a non-exhaustive example. While the term “the following’ provides a list of items that defines what it must be (rather confusing is it not?)

In short, the use of lists of terms compounds the difficulty of any attempt at applying NLP to tax regulations. 

The Code, Regulations, Court and Letter Rulings in Tax

While overall important, the treatment of references and terms is particularly important in tax. From a linguistic standpoint, it is often results in an intricate mesh of references and individually defined terms found in the tax code, regulations, and court and letter rulings. So, it is important to understand how they got there.

US tax law begins with the Code, enacted by Congress. Then the IRS publishes Regulations, stating the implementation of Code. When a taxpayer (individual or corporation) requests clarification of regulations, the IRS sometimes responds with a Letter Ruling – which then also becomes part of regulatory refinement.

In addition, some tax law arises from court rulings. So all sources (Code, Regulations, Letter Rulings, and Case Law) must be processed and merged to construct a comprehensive text corpus. To add to this complexity, the cross-referencing between these categories is not standardized.

However the good news is that cross referencing can be done with modern NLP. This technology can surface all thematic references across the entire body of writing. Think of this as an “automated uber-search” approach, to get to the meaning and context to interpret the tax code.

An Experiment of Applying NLP to Tax regulations and Codes

In our experiment of applying standard NLP software packages to the 75,000 pages of codes and regulations, we found that approximately 86% of the approximately 900 tax regulation sections contained a list. Some of the lists were over 1,000 words long, with nested sub-headings, references, rules, and exceptions to rules.

So, while existing NLP software can identify a list by detecting “delimiters” such as “1)”, “à”, etc., they often performed poorly at capturing internal list structures, such as hierarchies, exceptions, and negations. The issue is that much of the information in tax regulations is found in lists, not sentences. So, while humans are great at finding meaning in lists, for machines this is rather complicated.

For example, meaning in a sentence is typically embodied in the verb (in NLP known as the ‘head’). The meaning of a list is a noun phrase, and the meaning is often stated in the introduction to the list. For example, in phrases such as “these are items specifically included in gross income:”, where “gross income items” is the meaning, and we expect to find a list populated with nouns designating concepts like “cash”, “payment in kind”, etc. 

What we found in our experiment is that new specific list parsers for tax will be needed in order to perform NLP on list-oriented text – lack of a good NLP list processor is currently a fundamental impediment to applying it to professional tax writing.

What we also found is that quite a large part of the domain knowledge in tax is found inside tables. Unfortunately, in the regulation text it appears that these tables are constructed with even fewer standards than lists and the information is very different from a sentence (it is often numerical). So to make NLP work for tax, a new “numerical grammar scheme” needs to be created to understand tax codes.

The issue is that most of us learn the practice of writing sentences and hopefully conforming to generally agreed-upon grammar. In contrast, few such standards are taught, or even exist for lists or tables, though they increasingly comprise a large share of professional tax communication.

It seems that in writing lists in the US tax code, any ‘rule’ is applied. However, the US Treasury now has as part of the Plain Writing Act of 2010 created a writing style committee. This committee is now setting new standards so that the style may become more uniform in the future. This includes the construction of lists and tables.

The outcome from the work of this promising committee may significantly help us in automating the understanding of the tax code and regulation.

Now what?

Even, with the current challenges, we believe that NLP can still provide some benefits. This includes:

  1. Providing a reading score for each of the sections of the tax regulations. This includes scores calculated to estimate the grade-level needed for a reader to comprehend a section of the tax code. While some sections are simple, we found that reading scored suggest that some tax sections may require two PhDs to understand (both of us have Ph.D.’s and one author had two Ph.Ds. and still did not understand that section).  If such readability scores existed, tax professionals could determine who needs to be involved in determining the right staff to be involved and when to bring in experts.
  2. NLP’s for tax in the future may provide unique insights from enhanced search capabilities (such as simple lookups of rates) to comprehensive views across document types, rulings, changes over time, commentary and legal interpretations. In doing so, NLPs may provide network diagrams of indexed associations, and support “thematic search” that visually displays the connections and references in a text passage. It may even facilitate navigation and exploration of complex topics such as “how have the rates of R&D credits for software development evolved over time last ten years”. We had significant success in doing this with digital assistants around financial rules in Europe and believe some of this might also be leveragable.
  3. Even more interesting, the NLPs of the future may provide distilled “meaning” of tax sections and automate the determination of the topic of a sentence, paragraph, and section. It may even uncover topics not listed in the section heading or title. This could be other parts of the regulations that has an impact on the tax rules, but is not explicitly mentioned. It may even rank, or score, areas impacted or related to individual rules.
  4. Finally, NLPs may even provide a linguistic style analysis and look for phrases indicating relative caution, warnings, and penalties, conditional versus imperative, to highlight areas of significant enforcement and reputational risks.

Future of NLP in Tax

So, will computers automate interpreting tax laws and regulations? For the near future, the answer is both ‘yes’ and ‘no’.

In the “yes” column, prospects are good for automating routine, repetitive tasks with classification algorithms aided by NLP. This includes tax category determinations, like answering questions regarding the status of a worker as either a contractor or employee, based on NLP analysis of their hiring agreements.

In the ‘no’ column, we note that more complex tax decision-making and tax advice functions may require creating tax specific NLPs with a consistent tax knowledge base.

There are many challenges in doing this, including linguistic “unnaturalness” of the documents, abundance of complex lists and tables, and unifying single concepts that are dispersed across several types of tax regulations.

In addition, the investment required to make a full “moon shot” for NLP expert systems may not be justified at the present. Therefore, it seems unlikely an all-knowing AI tax expert will be built in the near future – but very likely that the in the short run, human tax experts will be increasingly supported by AI and NLP, but not replaced by it.

That means that tax professionals must learn to embrace these digital assistants as another member of the team and provide organization, skill set and governance as they start embracing AI, ML, and NLP.

Co-written with Dr. Cas Milner, data scientist, former professor, and a consultant working with PwC on advanced analytic projects.

Contact us

Bjarne Berg

Principal, Advanced Tax Analytics & Innovation, PwC US

Follow us