‘It’s Not Supposed to be Easy’: Measuring NMTC Impact Analysis

Published by Brad Stanhope on Wednesday, March 4, 2020
Journal cover thumb March 2020

Measuring the impact of new markets tax credit (NMTC) transactions is difficult, but important.

“It’s not supposed to be easy,” said Katie Frattaroli, vice president of business development at AMCREF Community Capital, a finance firm and community development entity. “If it was easy, everyone would be doing it. This is a challenging topic. Measuring impact is supposed to be hard and it’s supposed to be individualized.”

Frattaroli was part of a panel on economic and market research at the Novogradac 2020 New Markets Tax Credit Conference in San Diego in late January. Panel members discussed the growing importance of analysis of NMTC developments and the difficulties in accurately forecasting and measuring impact.

Blair Kincer, a partner in the Novogradac office in metropolitan Washington D.C., said analysis of NMTC developments typically falls into two areas–impact analysis that forecasts the benefit of the development and market analysis to help a developer determine demand.

Impact analysis was the focus of much of the San Diego discussion, highlighted by such topics as what to measure and the importance of third-party data.

What to Measure

One of the biggest issues in impact analysis is what to measure. The simplest metric is jobs created, but panelists said it goes beyond that.

“What we look at is four categories, four pillars,” said Frattaroli, “That allows us to rank employment, economic, environmental and community impact–because of the range of impact can be so different. That has helped us rate different projects across different regions with different impacts.”

Frattaroli said AMCREF instituted a scoring system, with each category having a maximum of 10 points. If a development scores below six in any category, it fails.

Panelists acknowledged that jobs created is still a leading measuring stick, but emphasized that it goes beyond a simple number.

“We find folks who work in our space will tend to have our measuring stick–maybe [measuring the impact in] jobs per million dollars of allocation,” said Peter Byford, NMTC senior program manager for consulting firm taxadvantagegroup (TAG). “Maybe they find 10 jobs per million [in NMTC allocation] as minimum, but what we try to do is dig in and do a little more nuanced version of that story.”

Byford stressed that certain jobs–particularly accessible or high-quality jobs–have a multiplicative impact. Byford cited an NMTC development in a rural area that created 103 jobs, a number that might not seem significant in a larger-population center, but had a huge impact where it was located.

Again, jobs aren’t always the key.

“The baseline for nonprofits is how many more low-income people you will serve with a project,” Byford said. “Then it becomes complicated. If you’re doing a museum, the number of low-income people moving through per million dollars of credits is significantly different from the number of students who would be affected through [a NMTC development focused on] building six more classrooms on a campus. One is in the thousands, one is in the tens, but the tens has a daily impact. How do you compare those two?”

Best Practices

When working with clients to measure impact, panelists stressed the importance of clarity.

“I totally agree with setting expectations and limitations early on,” said Byford. “It’s very collaborative. It’s important to identify how we can actually make our goals merge and make it work for both of us. We also need a framework around measurements. Not only goals, but what is the measuring stick?”

Byford warned against oversimplifying research.

“I really think the pitfall is that we simplify [impact] to a simple metric, or we pull a report in 2014 as measuring stick and never go back and see if it’s good, because it’s a report we can use,” he said. “We run the risk that, if we’re not careful, we can measure things that are no longer applicable to the way people live. That makes the program less valuable.”

Third-Party Data Importance

The CDFI Fund wants more than one source of impact data.

“The change we saw most recently that had the largest impact was language of the CDFI Fund in its FAQs,” said Abby Cohen, a partner in the Novogradac office in metro Washington, D.C. Cohen referred to an update that includes the phrase, “applicants will score more favorably if metrics are obtained from or informed by third-party sources rather than relying sole on the applicant’s own track record.”

Third-party data took on importance.

“We received many requests from clients to be a third party or find third-party sources,” Cohen said. “Right now, we’re working on developing third-party sources for some of the more common project types and impacts to better help clients earlier in the process when finalizing their project pipeline.”

The difficulty is often finding reliable third-party data. Frattaroli and Byford both cited specific organizations they’ve used.

“We’ve worked with this company in New Orleans called Life City, which helps businesses become more energy efficient [along with other standards],” said Frattaroli. “They developed their own database of environmental impact. We’ve worked together for a number of years to develop the plan for what we’re looking at and the different measures.”

Byford cited how TAG finds third-party data for educational developments.

“For schools, we try and start with state department of education and their best practices and we will often go one step further and go to educational groups,” Byford said. “We pull that data to justify the work we’re doing.”

Byford said that sometimes third-party data is available from other participants in the development.

“We sometimes work with third-party consultants who help design their plan,” he said. “You can pull third-party data from architects, design, programming consultants. They have data justifying why they’re adjusting what they’re doing.”

However, it’s not simple.

“Adding a layer of third-party data seems to make a lot of sense, but when you get into specialized products and specialized cases–and NMTCs tend to be specialized–third-party data gets hard,” Byford said. “It’s a little bit nuanced.”

Getting Info in Application

In practical terms, NMTC practitioners have to address the simple task of communicating analytic information–including the third-party metrics–into an NMTC application.

“How do you take all that squishy, qualitative value and wedge it down numerically?” asked Byford. “That’s the struggle we’re all slowly adjusting to and trying to get to, but I hope we continue to try and not give up. Not to just report the number of jobs and give third-party report that says it’s a good number.”

He said the issue is particularly difficult on smaller NMTC transactions, citing such circumstances as a small nonprofit wanting to do a $7 million NMTC development. Such organizations are not likely to be prepared to sift through vast amounts of information and analytics–they need help.

An Opportunity to be Proactive

The CDFI Fund’s requirement for impact analysis–without specifics–leaves room for interpretation.

“I think it creates an opportunity for us to look at everything,” said Frattaroli. “We can be more proactive as far as understanding what is impact and what is need, rather than the reactive approach that’s been in place.”

Byford said the lack of specificity is helpful.

“This is in keeping with original impact of statute: [The government says,] ‘We don’t know what your local community needs, so we’ll just tell you the things we don’t want you to do,’ ” said Byford. “You figure out how to work it out locally. We have this opportunity with more open-ended guidance to create our own measuring stick and not be boxed in. That’s the coolest thing about the NMTC program.”

In a data-driven world, measuring impact of NMTC investments will likely only gain importance.

“I think we’re evolving,” Kincer said. “This requirement for analysis of impact is new. The NMTC program isn’t that old. … I think [measuring impact] is going to be more and more important.”

Frattaroli pointed to specificity as a key.

“I hope we continue to pick apart [the difference between] what a job is and quality jobs, accessibility and how we measure value,” said Frattaroli. “I don’t think we need to get away from jobs, but we can further analyze the impact of one job versus another.”

She cited a specific example.

“When we’re talking about manufacturing generally, one of the things focused on is how well those workers are equipped,” said Frattaroli. “What is the company doing to promote education and advancement? While we don’t like to see failures, there will be. If there are layoffs, how well are those workers equipped to find other jobs? Are there benefits beyond the jobs? That’s economic impact to the community.”