Evaluating the Kansas Department of Labor’s Response to COVID-19 Unemployment Claims (Part 2)

Introduction

The Legislative Post Audit Committee requested this audit at its September 2, 2020 committee meeting.

Objectives, Scope, & Methodology

The audit included three questions. For reporting purposes, we divided those questions into two separate audit reports. This audit report is the second and final report and answers the following question:

1. What factors caused delays in the Kansas Department of Labor’s unemployment claims processing during the COVID-19 pandemic?

To answer this question, we spoke with officials from the Kansas Department of Labor (KDOL) and reviewed KDOL staffing, incident, and call center reports. We reviewed relevant reports from the U.S. Department of Labor’s Office of the Inspector General. We attempted to use U.S. Department of Labor unemployment claims data to compare processing times in Kansas to other states. However, the data contained significant errors so we couldn’t use it. However, we did contact officials from the Idaho, Nebraska, and Oklahoma labor departments for comparative information. Our work primarily focused on 2020 and early 2021, during the COVID-19 pandemic.

This audit also includes an updated unemployment insurance fraud estimate. In February 2021 we released the first part of this audit. In that audit we reported a preliminary estimate of how much fraud could have occurred in Kansas in 2020. In this report, we used KDOL claims data from January 2020 through February 2021 to provide a more precise estimate. We used KDOL claims data and a neural network computer model for our estimate. It is important to note that about 9,000 claims applications were missing from KDOL’s data used for our estimate. However, given that there were 1.08 million claims applications in total, the missing applications should have little effect on our overall conclusions. We also found inconsistencies and potential errors with the date payments were made in KDOL’s claims data. This did not affect our overall fraud estimate but could affect the distribution of payments by month. Finally, we were unable to report on KDOL’s detailed staffing numbers because we did not have a detailed report of staffing allocation.  

Important Disclosures

We conducted this performance audit in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. Overall, we believe the evidence obtained provides a reasonable basis for our findings and conclusions based on those audit objectives.

Audit standards require us to report our work on internal controls relevant to our audit objectives. In this audit we reviewed KDOL internal controls to ensure claims are processed timely and accurately. KDOL followed U.S. Department of Labor accuracy and timeliness quality control measures. The U.S. Department of Labor allowed states to suspend these measures early in the pandemic. As a result, parts of KDOL’s quality control process were temporarily suspended in 2020.

Finally, several members of this audit team were either victims of unemployment fraud, identity theft related to unemployment claims, or knew someone that was a victim. We concluded these events did not constitute an actual impairment to the team’s independence or ability to objectively complete this audit.

More specific details about the scope of our work and the methods we used are included throughout the report as appropriate.

Our audit reports and podcasts are available on our website (www.kslpa.org).

Rapid program changes, historically high unemployment claims, and an ill-equipped computer processing system created delays in claims processing during the pandemic in Kansas.

Unemployment Insurance Program Background

The Kansas Department of Labor (KDOL) administers the regular unemployment insurance program and gives financial aid to unemployed individuals.

• The regular unemployment insurance program is a joint program between federal and state governments. Although there are broad federal guidelines over the program, states establish their own criteria for who is eligible for regular unemployment insurance. States also decide the amount and duration of regular unemployment benefits.

• In Kansas, individuals must meet several criteria to qualify for regular unemployment benefits. For example, individuals must have worked for enough time and left work through no fault of their own (such as a medical emergency or layoffs, etc.). 

• Generally, Kansas employers are assessed a tax that funds the state’s unemployment insurance trust fund. That fund pays weekly benefits to unemployed individuals for 16 to 26 weeks depending on the state’s unemployment rates.

• Kansas employees do not contribute any money to the trust fund.

In 2020, the federal government created several temporary unemployment insurance programs to help individuals who lost their jobs due to COVID-19.

• The COVID-19 pandemic significantly increased the unemployment rate nationally and in Kansas. Before the pandemic, the national unemployment rate was about 4% in January 2020 (about 3% in Kansas). By April 2020, the national unemployment rate rose to about 15% (about 12% in Kansas).

• In March 2020, Congress passed the CARES Act to help individuals the pandemic negatively affected. The act included funding for new unemployment insurance programs.  

• The new programs differed from regular unemployment insurance in a few ways. For example, the new programs were entirely federally funded. Additionally, all the new programs were temporary and have either expired or are scheduled to expire in September 2021. 

• The new programs expanded unemployment benefits to include more people who lost their jobs due to COVID-19. For example,
  • The Pandemic Unemployment Assistance (PUA) program extended benefits to several new classes of workers. This included the self-employed (e.g., independent contractors) and gig workers like Uber drivers. Under this program, individuals not eligible for regular unemployment insurance could receive up to about $500 per week for 39 weeks (a maximum of about $19,500) under the terms of the original CARES Act in 2020.
  • The Federal Pandemic Unemployment Compensation (FPUC) program also gave an extra $600 per week to anyone already receiving unemployment benefits, for a period from late March through late July 2020. FPUC was renewed at a level of $300 per week in late December 2020 and expires in September 2021. FPUC was temporarily replaced by the Lost Wages Assistance (LWA) program in 2020. LWA is a disaster recovery fund administered by the Federal Emergency Management Agency, but it was temporarily used in place of FPUC during the pandemic.
  • Under the terms of the original CARES Act, the Pandemic Emergency Unemployment Compensation (PEUC) program provided up to 13 weeks of additional unemployment benefits to claimants once they exhausted all other unemployment benefits.

• Benefits paid out for the temporary federal unemployment programs do not come from a state’s unemployment trust fund. They are paid with federal funds.

Kansas and other states across the U.S. experienced claims processing delays during the pandemic.

• To receive benefits, individuals must first apply for unemployment insurance. Once submitted, a claim goes through several steps. Those include eligibility determination, calculating benefits, and submitting payment. It is important this processing happens timely. Delays in any part of this process could delay payment to individuals needing assistance.

• High unemployment rates contributed to delays in claims processing in Kansas and nationally. As mentioned above, by April 2020, the national unemployment rate was about 15% (about 12% in Kansas). That’s higher than any other time in recent history. For context, at their highest, unemployment rates were about 10% nationally and about 7% in Kansas during the Great Recession in 2009 and 2010.
  • In 2020, high unemployment rates meant many people were suddenly applying for benefits at once. Typically, unemployment rates rise over a few months, giving state labor departments time to respond to the increase in claims. However, the immediacy of the pandemic caused unemployment rates to surge very quickly.  This put a tremendous amount of strain on states’ unemployment processing systems, creating errors and processing delays.
  • In Kansas, there were several reports of individuals waiting several weeks or months to receive unemployment benefits. KDOL data showed individuals called its customer service call center about 12.5 million times in April 2020, sometimes calling multiple times a day. These calls significantly outnumbered the 33 full-trained customer service representatives KDOL reported at that time. This suggests many people needed assistance in applying for or receiving unemployment benefits. Federal and media reports during the pandemic showed these delays also occurred nationally during the pandemic.   

• We were unable to compare Kansas’ claims processing times to other states. We attempted to use federal unemployment claims data to compare Kansas to other states. However, the federal data contained significant errors that prevented us from using it. Additionally, that data only measured time to payment after an application was submitted and approved. It did not capture any delays in getting an application submitted.  

• This audit focused on identifying the main causes for Kansas’ delays in claims processing. We worked with KDOL officials to understand what caused these delays.  As discussed below, the main issues were the outdated computer system and its upkeep, as well as call center staffing issues.

Implementation and Computer System Issues

KDOL relied on an outdated, piecemeal, and poorly maintained unemployment computer system during the pandemic.   

• States use large, powerful computer systems to process unemployment claims. Among other things, these systems hold eligibility rules and historic claims data. Generally, when someone applies for benefits, their application runs through these systems to determine eligibility, benefit amount, and process payment.

• Kansas’ unemployment computer system was created in the early 1970’s and was centered around a mainframe computer. The mainframe operated on an older, lesser-used coding language. Over the years, the outdated coding language created challenges for KDOL. For example, there are few IT staff available that are still familiar with the mainframe’s coding language. This makes it difficult to maintain and update the system. Additionally, the outdated code requires KDOL staff to navigate between several screens to process a single claim. In some cases, staff are unable to use a mouse to enter or retrieve information. In these cases, all information must be hard coded into the system, taking time and special training.  

• Over the years, KDOL had to add modern programs around the outdated mainframe, creating a piecemeal system. Increasingly, the mainframe had to interact with programs that operated on modern computer code. For example, the state’s online application site operates on modern code. However, that site must communicate with the mainframe system. As a solution, KDOL installed an intermediate program called Rocket that allows the two systems to communicate. All programs, new and old, must work in unison to process claims. Periods of high claims volume stressed the connection between these programs, which led to system issues and delays. 

• Historically, KDOL did not properly document changes to the computer code, creating a risk for system error. Unemployment systems run on a significant amount of underlying computer code. Periodically, that code needs revision. When this happens, programmers should follow a uniform and well documented process to maintain system integrity. Poorly organized documentation and coding increase the risk for system error. That’s because programmers can’t be sure what changes were made, and how additional changes will impact the existing code. According to KDOL, staff over the years did not follow these best practices, resulting in a disorganized coding structure. Because of staff turnover and poor practices, current programmers use the system without full knowledge of how the code functions. According to KDOL, staff begun documenting changes to the coding structure during the pandemic.

Frequent changes to the state’s unemployment computer system during the pandemic created system errors and processing delays.

• The pandemic resulted in an extremely unique challenge for KDOL. Under normal conditions, KDOL does not have to make many major edits or changes to its unemployment computer system. When they did, the changes didn’t need to happen immediately, giving them more of an opportunity to test the changes before deployment. When the pandemic began, KDOL had to quickly build and deploy several changes to its unemployment computer system. The significant changes and hurried pace, combined with historically high unemployment claims and an ill-equipped computer processing system put KDOL at extremely high-risk for processing delays and errors.

• The pandemic spurred lots of sudden changes at the state and federal level. Government officials were trying to quickly implement new programs to help address high unemployment caused by the pandemic.
  • KDOL could not begin processing claims for the temporary federal unemployment programs until it received and implemented changes from the federal government. Federal documents showed it took between one and two weeks for the U.S. Department of Labor to issue detailed guidance to states on how to implement the federal pandemic programs. Once KDOL had the guidance, they began making necessary changes to their systems. For example, KDOL officials told us they had to build an entirely new program to administer the new PUA program, which took time. KDOL officials told us getting the new PUA system to effectively communicate with the outdated mainframe was extremely challenging and took a significant amount of resources.    
  • KDOL also had to respond to additional program revisions from the U.S. DOL throughout the pandemic. Figure 1 shows some of the key program revisions during the pandemic. As the figure shows, the U.S. DOL issued several program revisions during the pandemic. These revisions required KDOL to review new guidance, edit the underlying code, and test changes before deployment.

• Changes to the state’s coding structure to implement new federal programs and requirements created errors and delays. These changes caused problems because the state’s unemployment system operated on a disorganized coding structure. Despite an internal testing process, KDOL staff were unable to prevent all changes from creating system errors. This led to several claims processing issues. For example:
  • A coding issue made it appear that several claimants were no longer eligible for benefits when they still had multiple weeks of eligibility remaining. In this case, those claimants were denied payments because of the error.
  • Other coding issues denied claimants that were eligible for pandemic related programs. KDOL officials told us these coding errors mostly occurred as claimants were transitioning between unemployment programs. 

• Coding issues are not easy to identify or fix, which creates payment delays for claimants. In some cases, KDOL learned of these issues after the fact from customer service representatives taking calls from the public. Additionally, it takes specialized claims maintenance staff to review the claims and identify the issue. KDOL told us they do not have many of these specialists because it takes years of experience to gain the knowledge necessary for that position. Once identified, IT staff must also find where in the code the issue originated to fix it. These issues take time to fix, during which claimants may go without benefits.

During the pandemic, a surge in valid and fraudulent claims strained the state’s outdated and piecemeal unemployment system, leading to system failures and claim delays.

• Unemployment claims increased dramatically during the pandemic. Figure 2 shows claims filed from January 2020 through February 2021. As the figure shows, claims for the state’s regular unemployment program increased from 3,000 initial claims in February 2020 to about 66,000 claims at the end of March. That’s roughly a 22-fold increase in one month.

• High claims volumes strained the state’s outdated and piecemeal system, resulting in system failures and delays. As mentioned above, KDOL’s unemployment computer system consisted of modern and outdated programs. KDOL officials told us the outdated mainframe had issues communicating with modern systems during high-volume times. For example, the program responsible for connecting the mainframe to the modern online application site crashed periodically during the pandemic. Periods of high-claims volume contributed to these crashes. Claimants could not file online claims during this time.

• Fraudsters put additional strain on the state’s system. Fraudsters may be able to automate their attacks against states’ systems. In doing so, they can overwhelm state systems with a significant number of claims. This puts more stress on already strained systems. As part of this audit, we estimated about 630,000 of the 1.08 million unique claims applications (59%) from January 2020 to February 2021 could have been fraudulent attempts.  Our full fraud estimate is discussed in more detail below, but it’s important to note that not all 630,000 potentially fraudulent claims were paid.   

Staffing and Call Center Issues

Prior to the pandemic, KDOL had few staff to answer calls because of low unemployment and low federal funding levels. 

• KDOL customer service positions are federally funded. Federal funding is based on prior year unemployment program expenditures. Generally, funding increases and decreases with unemployment rates as the need for unemployment insurance changes. Kansas’s unemployment rate steadily declined from 2010 to 2019, reaching 3% in 2019; the lowest level since 1979. Over the same time, federal funding for Kansas’s unemployment program also declined.

• KDOL reported that because unemployment program funding was low, they only had 33 customer service representatives to answer phones in April 2020. 

• Total calls significantly outnumbered available staff at the beginning of the pandemic. Figure 3 shows the total number of incoming calls and calls answered during the pandemic. As the figure shows, KDOL reported a total of 12.5 million incoming calls in April 2020. That’s compared to just 33 fully trained customer service representatives. During this time individuals called multiple times a day because they couldn’t reach a customer service representative. This contributed to the 12.5 million calls in April 2020. 

• According to KDOL, up to 120 staff from other divisions and agencies helped answer calls during the spring of 2020. In total, KDOL reported answering only about 70,000 of the 12.5 million calls (about 1%) in April 2020.

Despite additional staff, the number of calls answered did not improve significantly during the pandemic, potentially leading to additional claims delays.  

• Not being able to talk to a customer service representative likely caused additional delays in claimants receiving unemployment benefits. Claimants call KDOL for several reasons. In some cases, they’re calling because they need help applying for benefits or to resolve problems with an existing claim. As discussed above, many claimants called KDOL during the pandemic, sometimes multiple times a day, without being able to reach a customer service representative. Not being able to reach a customer service representative likely resulted in additional delays for people needing assistance with their claim.   

• We were unable to compare detailed staffing trends to calls answered during the pandemic because of a lack of data. Beginning in July 2020, KDOL contracted with Accenture to provide additional surge staff during the pandemic. According to KDOL, Accenture staff helped answer phone calls, made out-bound calls, and helped with other administrative duties. KDOL gave us some staffing data, but we could not readily identify how many Accenture staff were answering phones on a given day or month. As a result, we were unable to compare detailed staffing levels to calls answered. Generally, KDOL reported adding up to about 500 temporary Accenture surge staff over the course of the pandemic. However, we could not verify those numbers.

• Despite added surge staff, the number of answered calls did not significantly increase during the pandemic. As Figure 3 shows, there was no clear increase in calls answered during the pandemic. That’s despite KDOL adding additional Accenture surge staff. KDOL suggested call complexity increased during the pandemic. It is possible more complex calls increased call times. Longer calls could have resulted in fewer calls answered per day. KDOL also reiterated that not all Accenture surge staff were answering phones, which could have limited the number of calls answered per day.  

Modernization Efforts

States with modern unemployment computer systems appeared better equipped to handle the challenges of the pandemic.

• According to a National Association of State Workforces report, 26 states (including Kansas) used outdated unemployment computer systems as of February 2021. Generally, an outdated system means it operates on an antiquated mainframe computer system. Like Kansas, states with outdated systems may depend on a combination of outdated and modern programs, resulting in a piecemeal system. The remaining 24 states upgraded to modern systems. Generally, modern systems do not rely on old mainframe computers and are built using modern coding languages. As a result, the necessary programs function more cohesively than the outdated, piecemeal systems.

• States with outdated unemployment systems appeared to encounter similar challenges as Kansas during the pandemic. An October 2020 report from the U.S. Department of Labor said outdated mainframe systems had compatibility issues with the new programs (like PUA). An April 2020 U.S. Department of Labor report found these compatibility issues likely resulted in delays processing claims during the pandemic.

• States with modern unemployment systems appeared better equipped to handle the challenges of the pandemic. A May 2021 U.S. Department of Labor report found that on average, states with modern systems implemented the PUA and PEUC programs a week to two weeks before other states. This is likely because it was easier to build the necessary pieces and make the needed changes to accommodate the new federal requirements. We also spoke to officials with the Nebraska and Idaho state labor departments. Officials confirmed their modern systems gave them more flexibility to quickly integrate new programs (like PUA) during the pandemic. Better integration also means modern systems are more stable and able to handle higher claims volume.

• We were unable to compare Kansas’ claims processing times to other states. We attempted to use federal unemployment claims data to determine how Kansas’ processing times compared to other states. However, the federal data contained significant errors that prevented us from using it. This included duplicated claims totals and inconsistencies between states. Additionally, that data only measured time to payment after an application was submitted and approved. It did not capture any delays in getting an application submitted.  

KDOL is in the process of modernizing its unemployment computer system.

• From 2005 to 2011 KDOL made efforts to modernize its unemployment system. During this time, KDOL made improvements to its system, but never fully modernized it. For example, KDOL added a new case management system and a new online application site. However, KDOL did not replace the outdated mainframe system before the modernization process was stopped in 2011. As a result, KDOL still uses a piecemeal system, centered around an outdated mainframe system. Current KDOL officials did not know why the project was stopped or why it wasn’t restarted. They suggested a lack of dedicated funding could have been a contributing factor.   

• As of July 2021, KDOL had restarted the process of modernizing its unemployment computer system. Officials told us before the pandemic they were gathering information from other states on modern unemployment systems. However, the pandemic paused their efforts. In March 2021, KDOL finalized a project modernization plan. Generally, the plan would replace KDOL’s current piecemeal and outdated system with a more self-contained system. The new system would also operate on modern computer code. In April 2021, KDOL posted a request for proposal for its modernization project.

• Passed in 2021, Kansas House Bill 2196 included several provisions for KDOL’s modernization project. The bill required that the system be in place by December 31, 2022. However, extensions to the deadline can be granted. The governor’s fiscal year 2022 budget report proposed investing about $37.5 million on the modernization project. The report proposed those funds come from federal and special revenue funds between fiscal years 2021 and 2022.  

• A new, modern system should be better suited to handle the unique challenges of a pandemic, recession, or other major unemployment events. Generally, a modern system can house all the necessary components to process claims within one integrated system. This reduces the risk for system failure during periods of high claims volume. The new system will also operate on modern code, making it easier to quickly add new programs or requirements should the need arise. Additionally, KDOL will have an opportunity to fix its disorganized coding structure as it transitions away from the mainframe computer code.  It’s likely a modern system will also eliminate the need for staff to navigate between multiple screens and hard code information when processing a claim. A modern system should also improve KDOL’s ability to run various metrics and reports on how its system is operating. 

Unemployment Fraud Update

In February 2021 we released a preliminary fraud estimate with the intent of releasing an updated estimate in this report.

• Fraud is a legal term used to describe specific criminal acts. Ultimately, only courts can decide whether fraud occurred. In this audit, we do not use fraud to refer to any legal determination. Rather, we use fraud to describe claims that displayed suspicious characteristics indicative of imposter fraud.

• Imposter fraud occurs when a fraudster uses stolen personal information to apply for unemployment benefits in other people’s names. This normally occurs in large quantities. We focused on imposter fraud because it appeared to be the most widespread fraud during the pandemic. Other types of improper payments or fraud, such as someone deliberately misrepresenting their employment information to try to increase their benefit amount, were not specifically accounted for in our fraud methodology. KDOL told us they also saw an increase in these other types of fraud during the pandemic.      

• In February 2021 we released a preliminary fraud estimate showing that about $600 million in unemployment fraud could have occurred in Kansas in calendar year 2020. This estimate was based on three key numbers: The number of claims KDOL reported stopping as potentially fraud from March 2020 through November 2020 (157,000), the total number of claims filed that during that time (650,000), and the total benefit amount paid ($2.6 billion) in 2020.

• As of February 2021, there was little information on how much fraud could have occurred in Kansas during the pandemic. We thought it prudent to release a preliminary estimate while we finished our detailed fraud analysis using KDOL’s claims data. Since then, we completed our detailed fraud estimate. That work is described below.

For this audit, we used an advanced computer model to create a more precise estimate of unemployment fraud in Kansas. 

• We used a neural network to estimate unemployment fraud in Kansas. A neural network is a form of machine learning used to replicate human decision making. This helps automate and expedite time-consuming tasks.

• Neural networks must first be trained to replicate human decisions. To accomplish this, we manually reviewed a random sample of 1,000 unique claims applications (out of about 737,000) for fraud. We looked for 26 things that can be indicators of fraud. For example, we looked for:     
  • Duplicated passwords or e-mail address. We counted the number of times the same password was used by different claimants. We reviewed password complexity and duplicate counts to determine the likelihood of fraud. We estimate it’s extremely unlikely (0.006%) that a six-character password, with random characters, would have at least one duplicate by chance out of 1 million claims. We also counted the number of times the same or similar e-mail address (within 2 characters) was used across claimants.
  • The accuracy of state employee application information. Some fraudsters targeted state employees because more of their information is publicly available. We cross-checked social security numbers in the claims data against Kansas state employee data to identify state employees. We then confirmed the accuracy of the names and dates of birth for those claims. Mismatches suggested imposter fraud.
  • We were unable to review checking account numbers, bank routing numbers, or IP addresses. Although this data exists in KDOL’s system, successfully querying that data would have taken additional time, delaying the release of the audit. 

• We used the results of our sample to train the neural network to identify fraud in all remaining claims. In total, we identified possible fraud in 575 of 1,000 (58%) claims sampled. We trained the model using 700 claims randomly selected from our sample of 1,000. We used the remaining 300 claims in the sample to test and validate the network’s accuracy. Once fully trained, we ran the neural network against all 1.08 million unemployment claims filed from January 2020 to February 2021. Appendix B has more information on our neural network methodology. 

We estimate about $700 million in potentially fraudulent payments could have been made in Kansas during the pandemic.  

• In total, Kansas paid about $2.8 billion in unemployment benefits from January 2020 through February 2021. Of that total, we estimate about $700 million (about 25%) could have been fraudulent.

• Our $700 million fraud estimate combined the results of our neural network with claims KDOL already flagged as potentially fraud. In its data, KDOL already flagged a significant number of fraudulent payments made during the pandemic. We used our neural network to identify additional fraud KDOL may have missed. Ultimately, we combined the results of KDOL’s work and our neural network to arrive at our $700 million estimate. Figure 4 summarizes the components of our estimate. As the figure shows:
  • LPA identified about $71 million in potentially fraudulent payments that KDOL did not (high confidence). This only included claims our model was at least 95% confident were fraud. For example, one claim we reviewed had the exact same password (9 characters, upper and lower cases, contained a special character and numbers), e-mail address, and residential address as 20 other claimants. In this case, it was very likely one person applied for benefits multiple times using other people’s information. Given the network’s confidence, we also had high confidence of fraud in these cases.
  • KDOL and LPA had consensus on $309 million in potentially fraudulent payments (high confidence). Both KDOL and our neural network flagged these payments as potential fraud, giving us high confidence of fraud in these cases.
  • KDOL identified about $306 million in potentially fraudulent payments that our model did not (low confidence). These are claims identified by KDOL as fraud, but not by our neural network. This could be for two reasons.  First, KDOL officials told us they incorrectly flagged many legitimate claims as fraud during the pandemic. Further, KDOL’s access to other fraud detection methods could also explain why our network didn’t flag some of these payments.  However, because our neural network did not flag these payments as fraud, KDOL officials were concerned much of this $306 million were legitimate claims they incorrectly flagged as fraud. Ultimately, there was no reliable way to determine how many of these payments were legitimate. As such, we included this with the other limitations to our estimate below.

• Our final fraud estimate is subject to a few key assumptions and limitations.
  • It is unlikely all the claims flagged in this estimate will end up being fraud. This would overstate our estimate. As noted above, KDOL flagged legitimate claims as fraud during the pandemic. These cases would overstate our estimate. 
  • It is possible fraud occurred that neither our model nor KDOL identified. This would understate our estimate. 
  • We were unable to use KDOL’s full 1099-G data as part of our estimate. A 1099-G form notifies individuals of taxes owed on state unemployment benefits. Some victims of fraud received these forms for benefits they never received. KDOL encouraged these individuals to contact them so they could amend their tax form. KDOL compiled a list of fraudulent claims based on the public’s feedback on 1099-G forms. This could have identified additional fraudulent claims not identified by KDOL or our model. We attempted to review that data to supplement our fraud estimate. However, time restraints and data issues prevented a full analysis. The data we used for our estimate contained some, but not all claims flagged as fraud from KDOL’s 1099-G review. As a result, it is possible additional fraud exists that was not captured in our estimate. This would understate our estimate.
  • KDOL’s claims data was missing 9,000 application records that received payment. We could not review those claims to determine fraud. This could understate our estimate. However, given there were 1.08 million claims, these missing applications likely have a minimal effect on our estimate.  

Of the estimated $700 million in fraudulent benefit payments, about half ($343 million) came from federal funds and half ($344 million) from state funds.  

• Of the roughly $2.8 billion in benefits paid from January 2020 through February 2021 (both fraudulent and valid), about $1.7 billion came from federal funds for temporary pandemic programs. The other $1.1 billion came from state funds for the state’s regular unemployment insurance program.    

• Of the estimated $700 million in fraudulent benefit payments, about half came from federal funds and half from state funds.
  • We estimated about $343 million in fraud from federal funds, mostly occurring in the spring and summer of 2020. Figure 5 shows the distribution of fraudulent payments during the pandemic, by state and federal funding source. As the figure shows, most of the fraudulent payments from federal programs occurred in the spring and summer of 2020. These fraudulent payments peaked in July 2020, before declining. This coincides with the first iteration of the FPUC program expiring in July 2020. As federal programs, these fraudulent payments did not affect the state’s unemployment trust fund.
  • We estimated about $344 million in fraud from state funds, mostly occurring at the end of 2020. As Figure 5 shows, most of the fraud to the state’s regular unemployment program occurred in late 2020, peaking at about $107 million in December 2020. These fraudulent payments did impact the state’s unemployment trust fund. The balance of the trust fund declined by about $711 million during this time. That means fraudulent payments could have accounted for about 48% of the decline. Legitimate payments likely accounted for the rest of the decline.

• Fraudulent payments declined significantly in February 2021, likely because of KDOL’s new identity verification process. In February 2021, KDOL implemented a new identity verification system to help combat cases of imposter fraud. Under the new system, all claimants must answer a series of questions that only they should know before they are allowed to apply for benefits. As Figure 5 shows, fraudulent payments from state funds decreased from about $100 million in January 2021 to $4 million in February 2021, after the identify verification system was implemented. That’s a decrease of about 96%.  

• Passed in 2021, Kansas House Bill 2196 included provisions to replenish the state’s unemployment trust fund with emergency federal pandemic funding. That included an initial payment of $250 million in 2021. Additional payments may be made pending the results of a future contracted audit required by the bill. Among other things, the audit will estimate how much fraud occurred in Kansas during the pandemic. 

We estimate about $2 billion in potentially fraudulent payments were prevented in Kansas during the pandemic.   

• There were a significant number of fraudulent attempts in Kansas that were never paid. Figure 6 compares total claims filed, fraudulent attempts filed, and fraudulent attempts paid during the pandemic. As the figure shows, a little more than half of claims filed during the pandemic were cases of attempted imposter fraud. However, as the figure also shows, not all these attempts were paid. We estimate about 30% of fraudulent attempts were paid, resulting in the estimated $700 million in fraud payments reported above. The other claims were likely stopped by KDOL fraud staff, reported by the public, or deemed ineligible for payment.

• We estimate about $2 billion in potentially fraudulent payments were prevented. It is difficult to know with certainty the value of the fraudulent attempts that were never paid. That’s because we can’t be sure which programs the fraudsters would have been eligible for, their weekly payment amounts, or how long they’d receive benefits. However, we applied the average unemployment benefit amount to the number of fraud attempts prevented to estimate this amount.

KDOL officials reported working with federal organizations and banks to identify and recover fraudulent payments, but no estimate on recovered funds was available.

• KDOL officials told us that the state’s two banks, Bank of America and U.S. Bank, are the last line of defense in identifying and preventing potentially fraudulent payments. The banks will suspend payment on accounts they consider suspicious. In these cases, payments are held by the bank until it can be returned to the state. KDOL reported about $7.4 million in potentially fraudulent payments that could be recouped from Bank of America. KDOL officials told us their initial review confirmed $3.9 million of that total as fraudulent, but still need to review the remaining $3.5 million. KDOL officials told us U.S. Bank did not have an estimate of fraudulent payment stopped at the time of this audit.

• KDOL officials told us they continue to work with federal investigators and law enforcement to investigate potentially fraudulent claims. A recent report from the U.S. Department of Labor estimated that nationally, about $87 billion in unemployment benefits could have been made improperly, with a significant portion attributed to fraud. To date, few cases, in Kansas or nationally, appear to have been prosecuted.

Conclusion

We did not make any additional conclusions for this audit.

Recommendations

We did not make any recommendations for this audit because the Kansas Department of Labor is already in the process of modernizing its unemployment insurance system.

Agency Response

On August 6, 2021 we provided the draft audit report to the Kansas Department of Labor (KDOL). We made some minor changes based on the department’s feedback.

KDOL’s response is included below. In its response KDOL suggested $306 million of claims KDOL flagged as fraud should not be included in our $700 million fraud estimate. The $306 million were claims KDOL flagged as fraud that our neural network did not detect. In its response, KDOL officials were concerned much of this $306 million may have been incorrectly flagged as fraud by KDOL. Ultimately, we did not remove the $306 million from our estimate. That’s for two main reasons.

• KDOL’s fraud detection process likely identified fraudulent claims that our neural network didn’t. Our neural network was only trained to identify wide-scale imposter fraud. KDOL’s process also focused on multiple types of fraud, like individual fraud and false employer schemes. Further, KDOL had access to additional data (bank account, routing numbers, and I.P. addresses), different fraud detection tools, and relied on the public to report fraud.

• There was no way to verify KDOL’s assertion that a significant amount of the $306 million it flagged as fraud were legitimate claims. This will require additional investigation by KDOL. We acknowledge it is possible our estimate included some legitimate claims that were incorrectly flagged as fraud by KDOL. This would overstate our estimate, which we noted in our report.

Finally, although some of the $306 million KDOL flagged as fraud may have been legitimate (which would overstate our estimate), it’s also likely that neither we nor KDOL have identified all fraudulent claims (which would understate our estimate). Consequently, we think our $700 million estimate is very reasonable.

Appendix A – Cited References

1. This appendix lists the major publications we relied on for this reportCOVID-19: States Cite Vulnerabilities in Detecting Fraud While Complying with the CARES Act UI Program Self-Certification Requirement (October, 2020). U.S. Department of Labor Office of the Inspector General.
2. COVID-19: States Struggled to Implement CARES Act Unemployment Insurance Programs (May, 2021). U.S. Department of Labor Office of the Inspector General.
3. Status of State UI IT Modernization Projects (February, 2020). National Association of State Workforces Agencies.

Appendix B – Fraud Analysis

This appendix contains further information about the methodology and results of the fraud analysis presented in this report.

We used a neural network model to help us find fraud using risk indicators.

• A neural network is a type of advanced machine learning model. Models like this are used to map the complex relationships that exist between a series of input variables and output variables. In this audit, the input variables were the fraud risk indicators (along with some basic application information), and the output variable was a binary determination of “fraud” or “not fraud”.

• Machine learning models like a neural network train themselves on a set of complete data before they can be used to predict or classify new data.
  • Complete data is data for which the outcome is already known. In this case, the outcome is whether a claim is fraudulent.
  • Training consists of running the model through repeated iterations of a learning algorithm, during which time the model teaches itself to minimize error in its predictions.
  • Once trained, the model can predict the outcomes for data that does not yet have an outcome.

We reviewed 1,000 sampled claims applications to generate a dataset to train the neural network.

• We chose a random sample of 1,000 applications from a universe of about 737,000 unique claims applications to review. Later, KDOL sent an additional 343,000 claims applications, increasing total claims to about 1.08 million. All claims data contained the same fields. Drawing a sample from the roughly 737,000 claims wasn’t problematic for two reasons. First, there was no evidence of significant differences in claims characteristics between the two data sets. Second, our final fraud estimate did not rely on the proportion of fraud found in the sample. That’s because we used a neural network, which searches for fraud on a claim-by-claim basis. 
  • Using 26 fraud indicators we reviewed each application in the sample to determine the possibility of fraud. Fraud indicators included duplicate passwords, security words, and e-mail addresses. We also reviewed residential, mailing, and employer addresses for suspicious information. We used those and other indicators to help us assess fraud.   
  • For each determination, we marked our conclusion (fraud or not fraud) with a confidence level of low, medium, or high.
  • We also used a formula backup to help designate fraud for ambiguous cases. The formula assigned a numerical weight to each fraud indicator, giving each claim a risk score. We relied on the score to help determine fraud when we could not reach consensus or had low confidence.

Various neural networks were trained and tested on the applications we reviewed.

• The 1,000 reviewed applications were randomly split into 3 distinct sets: the training set (700), the validation set (100), and the test set (200).
  • The neural network model reviews the training set while iterating through the training algorithm.
  • The validation set was used to compare models once they were trained. Ultimately, it’s how the preferred model was chosen. When validating, it was important to use data the models had not seen. This helped assess the models’ accuracy in predicting outcomes of new, unseen data. The final model was selected for its high accuracy and low degree of bias.
  • We then tested the validity of the final model. Once the preferred model was selected, we tested it one final time. To do this we used the final set of 200 applications the model had not seen. Thus, a model’s performance on the test set is a strong indicator of the model’s performance. The test showed the model predicted the correct outcome 183 of 200 times (91.5%).