Follow up Audit: Reviewing Agencies’ Implementation of Selected Performance Audit Recommendations

Introduction

The Legislative Post Audit Committee’s rules include a process to check on prior audit recommendations. That process has 2 primary components. First, the Post Auditor is required to follow up with each agency twice a year and have officials self-report on their progress in implementing recommendations. Second, the Post Auditor is required to prepare an audit proposal each year that lists “previous audit recommendations for which follow up audit work is necessary to independently ascertain whether such agency or other entity has implemented audit recommendations.”

On April 22, 2022, the Legislative Post Audit Committee approved the proposal staff prepared for that purpose. The proposal included 7 recommendations from 3 of our prior audits. The recommendations were for the Kansas Departments of Agriculture, Commerce, and Education (including a recommendation made to the Kansas Board of Education).  

Objectives, Scope, & Methodology

Our audit objective was to answer the following question:

1. To what extent have agencies implemented selected audit recommendations from performance audits issued in recent years?

To determine if the agencies and the Board implemented the recommendations, we interviewed officials, reviewed current audit work, and requested and reviewed documents and processes. Documentation included policies and procedures, spreadsheets, and annual reports.

We only did what was necessary to evaluate whether the agencies implemented the recommendations. We did not reevaluate the programs or problems found in the original audit. Therefore, a finding that a recommendation was implemented does not mean that the agencies completely fixed any underlying problems.

Important Disclosures

We conducted this performance audit in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. Overall, we believe the evidence obtained provides a reasonable basis for our findings and conclusions based on those audit objectives.

Audit standards require us to report our work on internal controls relevant to our audit objectives. For this limited scope audit, we reviewed whether recommended internal controls were implemented. We did not evaluate the effectiveness of those controls. Our audit reports and podcasts are available on our website (www.kslpa.org).

The 3 agencies and the Board fully implemented 1 of the 7 recommendations we reviewed for this audit.

The Kansas Department of Education fully implemented 1 of 3 recommendations from our 2019 audit.

In December 2019, we published an audit evaluating at-risk student counts, weights, and expenditures.

• The audit evaluated the Kansas Department of Education’s (KSDE) at-risk funding and expenditure process. Kansas school districts receive additional state funding to help students who are at risk of academic failure.
• Among other things, the audit found that in a sample of 20 districts, most at-risk spending was used for teachers and programs for all students, and did not appear to specifically address at-risk students, as required by law.
• The audit also found that the Kansas Department of Education (KSDE) had approved at-risk practices that did not target at risk students and were not clearly evidence-based.  Lastly, KSDE did not provide districts with guidance that reflected new at-risk spending requirements.
• To address these findings, we made 2 recommendations directed at KSDE, and 1 recommendation directed at the State Board of Education.

As of March 2023, KSDE fully implemented 1 and partially implemented the other recommendation, whereas the Board did not implement the third recommendation.

• LPA currently has an audit evaluating K-12 at-risk expenditures and statutory compliance. At the time of this follow up audit, the K-12 at-risk audit team had already gathered evidence from KSDE that helped answer our audit question.
• This work included comparing the guidance document KSDE provides to districts and relevant Kansas statutes, reviewing KSDE’s process identifying at-risk programs, and speaking to KSDE officials. We used this work to evaluate the status of KSDE and the Board’s progress on our recommendations.
• Figure 1 shows the results of our evaluation.

The Kansas Department of Agriculture partially implemented all 3 recommendations from our 2020 audit.

In November 2020, we published an audit evaluating the Department of Agriculture’s price verification inspection process.

• The audit evaluated the Marketplace Equity Protection Program within the Kansas Department of Agriculture’s (KDA) Weights and Measures division. The program is responsible for overseeing the accuracy of large and small scales, as well as point of sale systems across Kansas.
• The audit found that retail businesses failed most price verification inspections, and that KDA did not respond consistently to those failures.  Specifically, KDA didn’t conduct timely follow up inspections for about 75% of the failed inspections we reviewed. KDA also issued legal orders later than it could have, and its fines were small and often reduced further. The audit also found that staffing constraints contributed to the problems.
• To address these findings, we made 3 recommendations directed at KDA.

As of March 2023, KDA partially implemented all 3 recommendations.

• We requested and reviewed documents to verify KDA’s self-reported status for each of the 3 recommendations. Those documents included annual price verification program data reports, a price verification procedure and policy, a legal action escalation policy, and a failed-inspection spreadsheet. We also spoke with staff to learn more about what KDA has done to address each recommendation.
• Figure 2 shows the results of our evaluation.

We could not determine whether the Department of Commerce implemented a recommendation from our 2020 audit.

• In November 2020, we published an audit on the Angel Investor Tax Credit Program. This program gives investors a tax credit for investing in certain Kansas startups. This included a statutory requirement for participating businesses to stay in Kansas for at least 10 years. The Department of Commerce is responsible for determining whether a business qualifies, and for monitoring the program’s success.
• Our audit found that the law didn’t provide benchmarks for measuring program success. We recommended the Legislature consider amending statute to shorten the 10-year requirement and to clarify the program’s goals, including specific benchmarks for program success. The 2021 Legislature shortened the requirement to 5 years for non-bioscience businesses.
• The audit recommended Commerce proactively enforce statutorily required timeframes for qualifying businesses to remain in Kansas.
• As part of our biannual follow up process, Commerce staff stated that they seek to enforce contractual claw back provisions from companies that left the state during the term of the tax credit agreement. Staff told us they gather Department of Labor data and conduct site visits as part of their annual tracking activities.
• During our fieldwork, we reviewed a copy of a demand letter sent to one company that left the state prematurely. We also reviewed a spreadsheet with Angel Investor information. However, the data did not appear to be complete, and it was unclear when the spreadsheet was last updated. Commerce staff told us that the employee who may have additional evidence was on extended leave. No one else at Commerce had access to the necessary information during the course of this audit.
• As a result, we were unable to confirm whether the agency implemented our recommendation.

Recommendations

We did not make any recommendations for this audit.

Potential Issues for Further Consideration

Although we had unresolved questions about the following issue, more audit work would be needed to determine whether it represents an actual problem or not.

The Department of Agriculture told us their Weights and Measures division has ongoing staff vacancies.

• During this audit, staff told us 2 of 5 inspector positions in its Marketplace Equity Protection Program were vacant.
• The 2020 Price Verification audit noted similar vacancy problems in the Weights and Measures division.
• We did not explore what factors contribute to the chronic staffing constraints that prevent KDA to fully staff this division.

Agency Response

On April 5^th, 2023 we provided the draft audit report to the the Kansas Department of Education, the Kansas Department of Commerce, and the Kansas Department of Agriculture. Their responses are below. Agency officials generally agreed with our findings and conclusions.

KDA Response

The Kansas Department of Agriculture (KDA) Weights and Measures Program has worked to address the issues identified in the Legislative Post Audit Report, specifically those regarding point of sale (price verification) systems. In responding to these issues, the program has isolated some specific barriers that exist which complicate efforts to fully resolve the recommendations of the LPA report. This letter seeks to provide context surrounding the program’s challenges and outline some of these barriers in implementing all the LPA recommendations identified earlier.

The National Institute of Standards and Technology (NIST) provides specifications, tolerances and other technical requirements for weighing and measuring devices. Amendments to these standards are adopted by the National Conference on Weights and Measures, the professional association to which Kansas and other state programs belong. 

These NIST standards for point-of-sale systems, sometimes referred to as price verification systems, call for a passing score of 98%. This pass/fail rate is determined on inspection by randomly selecting items from a retail store shelf (50 or 100 items) and determining if the price displayed on the item, shelf or advertising correctly displays when the item is scanned at the check-out. A retail store may have one or several price scanners depending on the size of the store and the number of check-out lanes. During an inspection each price scanning device is tested.

Price Verification Statistics reports were provided to LPA as part of this audit. These reports from FY 2020 through March 2023 indicate an average pass rate of 47.7% and a fail rate of 52.3%. This is based on the number of price scanning devices tested and not the number of retail stores. Please note that a device may fail with overcharges or undercharges and that the ratio of overcharges to undercharges is one consideration when determining the enforcement action to be taken.

The criteria KDA’s Weights and Measures Program staff use to determine follow-up inspections is detailed in the Price Verification Escalation Process document provided to LPA.

Field inspection staff do not have sole discretion in determining whether to escalate a store to a legal summary order or civil penalty. The program supervisor, administrative staff and inspector utilize retail store inspection data tracking and the outlined process in the escalation process document to determine when follow-up inspections are scheduled. This time frame is determined by several factors: the store history of inspection percentage scores; whether that percentage is increasing or decreasing compared to past inspection(s); the ratio of overcharges to undercharges; and whether the store management has requested a compliance meeting or a training seminar. Any of those variables, as written into the escalation process, will affect the time frame of subsequent follow-up inspections. Every effort is made to schedule follow-up inspections in an acceptable time frame, balancing other small-scale and large-scale inspection assignments.

Funding a more robust price verification system inspection model with an increase in staff could be accomplished with an increase in state general fund dollars, a new statutory licensing and fee requirement, or a combination of the two. The decision not to ask for a state general fund budget enhancement this legislative session was made by me, taking into consideration the other agency funding needs regarding continuity of operations for our stakeholders. It is likely that retail stores would oppose a change in the statute requiring additional licensing and associated fees. Without considerable funding increases and in consideration of current pass/fail percentages, KDA may consider performing these inspections on a consumer complaint only basis and/or setting a more reasonable pass/fail score apart from the NIST standard. Finally, in the audit report section Potential Issues for Further Consideration, we would like to address the staff vacancies. There are currently fourteen (14) total positions in the Weights and Measures Program.

• One Weights and Measures Program manager
• One compliance manager
• One administrative assistant
• One petroleum/fuel inspection (PMEP) section supervisor
• Five PMEP field inspector positions
• One scale, package and point of sale devices (MEPP) section supervisor
• Four MEPP field inspector positions (who make price verification inspections and randomly inspect the accuracy of small and large scales)

Specifically, within the MEPP section, the following may provide some context to the position vacancies that existed in 2020 when the audit started and the current vacancies.

• The supervisor’s position has remained filled with the same employee since 2014 until very recently when the MEPP supervisor announced their resignation with their last day in a few weeks. (Incidentally, this supervisor has recently accepted a position at the National Institute of Standards.)
• Field position (MEPP) #1 – has remained filled with the same employee since 2013.
• Field position (MEPP) #2 -has remained filled with the same employee since 2015.
• Field position (MEPP) #3 – inspector resigned 10/2021; position refilled 02/2022; inspector resigned 07/2022; position currently filled as of 10/30/22.

Field position (MEPP) #4 – inspector resigned 12/2018; position refilled 02/2020; inspector resigned 05/2020; position refilled 07/2021; inspector resigned 11/2022 and the program is actively recruiting to fill this vacancy.

This serves to highlight many state agency recruiting and hiring difficulties. There are changing dynamics in the available work force, difficulty in recruitment of these types of inspection jobs that will not allow work from home, and challenges in retaining workers without the ability in the state system to meet private sector salaries. 

Thank you for the opportunity to outline some of the specific challenges that are being addressed by this program in its efforts to respond to the LPA report. We continue to explore solutions to these challenges that are common-sense and budget sensitive, while collaborating with the Kansas Legislature and our stakeholders. Please let us know if we can answer any questions or provide additional information.

KSDE Response

Thank you for the opportunity to respond to the Follow Up Audit to Reviewing At-Risk Student Counts, Weights, and Expenditures.  The Kansas State Department of Education has the following comments about the recommendations contained in the document. KSDE has two comments related to Recommendation 1.

• The guidance for clerical staff salaries paid with at-risk funds has been updated and is posted online. 
• KSDE’s guidance does suggest a limit on at-risk expenditures that may be spent for teacher salaries despite the lack of such a limit in statute. When school districts ask about this topic and the statute is silent, KSDE attempts to provide guidance that follows the intent of the statute.

Recommendation 3 refers to the State Board’s oversight of the process. After the original 2019 At-Risk Audit, State Board President Kathy Busch wrote in a follow-up letter that a subcommittee of State Board members met with KSDE staff and reviewed the process to verify that practices and programs are identified in accordance with statute.  That process has been implemented and followed by KSDE staff since that time.  This method of providing guidance to staff is consistent with the practice followed by the State Board.

Department of Commerce did not send a response

Appendix A – Cited References

This appendix lists the major publications we relied on for this report.

• K-12 Education: Evaluating At-Risk Student Counts, Weights, and Expenditures (December, 2019). Kansas Legislative Division of Post Audit.
• Evaluating the Department of Agriculture’s Price Verification Inspection Process (November, 2020). Kansas Legislative Division of Post Audit.
• Angel Investor Tax Credit Program (November, 2020). Kansas Legislative Division of Post Audit.