 
|
|||||||||||||||||||||
              
|
Op-Ed:If it feels good, fudge it Economic studies that drive important regional policy decisions are, more often than not, flimsy and flawed. Paul R. Flora looks at their weaknesses and prescribes solutions Sunday, July 19, 1998 By Paul Flora
Faith-based economics is playing an ever greater role in public policy decisions being made around the Pittsburgh region. Not religious faith, but faith in blue-sky forecasts and feel-good analysis. Needless to say, relying on poor information, unfounded assumptions and uncritical analysis can lead to public policy mistakes. But accepting bad research also crowds out good analytical work and damages the credibility of all public decisions and the research on which they are based.
Research lacking credibility is endemic to all regions, as vested interests rally like minds to endorse and embrace carefully crafted promotional studies. Let the consumer -the media and the public - beware!
But a region's progress is hampered further when leaders fail to insist on objective, credible research on which to make decisions, or contribute themselves to corrupting the process by which sound information is produced. Unfortunately, public policy research for the Pittsburgh region is falling victim all too often to one or more of the following problems.
If you ask the wrong questions, you get the wrong answers. Correctly framing the problem to be addressed is critical to gathering good information and reaching sound conclusions.
The success of public economic development agencies is too often measured by assertions of firms attracted and jobs created. Rarely are such agencies' activities scrutinized and measured against specific, objective goals that can be directly attributed to the actions of the agency.
The current trials and tribulations of the Ben Franklin Technology Center of Western Pennsylvania result, in part, from a failure to frame issues correctly.
Setting aside the serious allegations of extravagant spending, nepotism and conflicts of interest, problems of which any board should be aware and responsible, many public agencies are also inadequately monitored for the efficacy of their work.
Responsible oversight of economic development agencies would include a close examination of what positive economic impacts are actually attributable to the agency's actions, rather than to other factors. With closer oversight of relevant factors, an agency's board should more readily become aware of ineffectiveness and the potential of mismanagement or other abuses.
This problem takes shape on a regional scale, when the Allegheny Conference on Community Development puts forward a goal of 100,000 new jobs over a six-year period.
The region was growing at a faster pace in the late 1980s. A region's economic growth is largely determined by national trends, U.S. industrial trends and the mix of industries within a region. The business acumen and economic health of specific large local firms is also critical.
What evidence will be offered that actions by the Allegheny Conference or any of the many economic development groups are responsible for some or all of the job growth in the region? Of what value is such a vague goal for assessing either the economy's progress or the value of specific economic development efforts?
If you torture it long enough, the data will confess. This favorite old saying among social scientists is intended to remind us that any desired result can be attained with enough effort by the analyst.
But even if mendacity isn't involved, many studies do suffer from poor methodologies which ignore basic economic considerations. Outright mistakes occur as well.
That such poor research is nevertheless accepted often occurs because there is little incentive for agencies to question results that appear favorable to their cause. And when agencies contract for an economic analysis of their agency or agency's project, seldom do hired consultants produce unwelcome news.
The most recent local example appears in a study prepared for the Pittsburgh Cultural Trust, titled "The Economic and Social Impact of the Not-for-Profit Arts Community on Allegheny County and the City of Pittsburgh."
In that report, researchers tallied five categories of expenditures, of which at least four are probably overstated.
The analysis included $38.5 million of local audience spending and $51.0 million of out-of-area spending - but did not account for what economists call "substitution effects."
For example, of the 63 arts organizations surveyed, if none existed, then local residents and out-of-area residents (which include persons from all the surrounding counties in our region) would still deploy their discretionary income in some fashion - shifting away from spending on the arts to more local spending on movies, sports, restaurants, shopping, education, housing or increased savings, as well as spending outside the region for cultural arts.
The absence of any assumptions about the magnitude of substitution effects is a major flaw that significantly overstates the direct spending impact. (Perhaps some assumption should have been posited for credibility and reasonableness.)
In addition, the analysis did not distinguish between one-time capital costs and ongoing operating costs, but lumped them together to generate a $54.3 million estimate of annual direct spending by the arts organizations. Capital costs are typically treated separately in such studies, particularly if construction costs are included. The study failed to indicate whether the $54.3 million includes major one-time capital expenditures.
Finally, the analysis included $3.4 million of direct spending by guest artists and contractors without accounting for the spending by local artists and contractors who leave Allegheny County on tour and for art shows. Without any evidence of magnitude of the outflow, the reasonable assumption would be to assume this is a wash.
Corroborating evidence that these three assumptions, and perhaps other data or assumptions, generated too-high estimates is apparent by the implicit multiplier of 4.7 that is derived by dividing the report's estimated 5,393 total jobs by the estimated 1,138 direct arts jobs. Economists tend to become skeptical when employment multipliers rise above 2. This multiplier of 4.7 screams for attention.
Just think happy thoughts. At times, despite the forces aligned against the possibility, a report's conclusions will be sound and (potentially) damaging to prevailing political objectives. But, of course, they need not be widely reported.
In 1995, Policy and Management Associates of Boston, Mass., delivered a report to the Urban Redevelopment Authority of Pittsburgh. The "Socio-economic and Financial Analysis of Redevelopment of the LTV-South Side Works Site in Pittsburgh" included an impact analysis of a casino on the site.
In the course of their work, they undertook case studies of riverboat casinos in the Midwest. Their findings cast serious doubt on the economic benefits to be derived from introducing casinos to a region's economy.
From their case studies, casinos appeared to drain as much, if not more, money from the regional economy than they attracted. The relatively negative findings were driven largely by an accurate representation of substitution effects and recognition that a large percentage of casino admissions would be local. While the casinos purchased many goods and services locally, sufficient amounts of money leaked out of the region to offset the small percentage of tourist dollars that were attracted into the region.
The extension of their case studies to the Pittsburgh market revealed an expectation that as many as 82 percent of all casino admissions would be by persons who live within 100 miles of Pittsburgh. The six-county metro area accounted for 70 percent of all casino admissions.
By the time the closely held study was final, the changing environment for casino gambling in Harrisburg had made the issue moot. The same month that the report was printed, the city of Pittsburgh extricated itself from its deal with Hospitality Franchise Systems Inc., which sought to develop a casino on the LTV property.
Regardless, findings from the URA's report would have been, and still are, valuable information for the public and decision-makers. No matter how embarrassing the report's findings may have been to the city administration, it clearly should have been disseminated - adding valuable information to the public debate on gambling.
If you build it, they will come, or the sun will come out tomorrow. A corollary to the feel-good Peter Pan mentality is the belief that no matter how things are today, they will be better tomorrow. And to justify their optimism, believers speculate a host of changes bearing unsubstantiated positive consequences.
As a prerequisite to obtain federal funds for regional highway and transit expenditures, the Southwestern Pennsylvania Regional Planning Commission must prepare a long-range plan, including forecasts of economic activity and demographic changes for a 20- to 25-year period. The forecasts help predict future levels of travel and influence assessments of future needs for more highway and transit infrastructure.
Higher forecasts attest to a greater need for federal funding and shape a better outlook if the regional analysis is examined by outside businesses or site-location specialists. Thus local political and business interests favor projections of rapid future growth - no matter how unlikely.
This bias has been pitched in a battle of wills for several years against academic researchers at the University of Pittsburgh, private sector economists and SPRPC's own staff to control the output of SPRPC's forecast advisory committee.
For more than a decade, SPRPC and Pitt's University Center for Social and Urban Research have operated an economic model, called REMI. Among other uses, this model provides long-range forecasts as input to SPRPC's long range plan. The model works reasonably well for economic forecasts, but its predictions of population levels have been consistently too high.
To counter this known problem, UCSUR has developed a way to adjust the model, in consultation with the economics faculty at Pitt and the model's developers at Regional Economic Modeling of Amherst, Mass. While not wholly satisfactory from a theoretical view, the adjustment addresses the clear dilemma of projecting population increases in the absence of any evidence that population growth has begun in the Pittsburgh region.
Despite historic experience and several independent forecasts to the contrary, SPRPC continues to be pressured by business interests not to adjust the model for the apparent overstated population forecasts. In fact, their representatives are pushing for SPRPC to alter the model so that significantly higher growth of new jobs and population is forecast.
Independence and objectivity. In general, university researchers enjoy the independence necessary to avoid conflicts of interest and to ensure an objective approach. But not always.
The integrity of our region's research institutions was a casualty of last fall's election campaign for the Regional Renaissance Initiative. Voters grow accustomed to hyperbole and misrepresentation from politicians as they pitch their pet projects. But for Carnegie Mellon University, Duquesne University and Pitt - as well as Price Waterhouse - to offer up a shaky economic study, claiming as many as 22,000 to 30,000 jobs from passage of the initiative, was a huge disappointment.
In contrast to their claims of independent, objective and thorough analysis with conservative assumptions, the research lacked independence, accepted unsubstantiated, sometimes heroic, assumptions and contained errors.
At the press conference for the study's release, three of the four institutional spokespersons raised their hand when asked if they personally belonged to the Regional Renaissance Partnership or its companion lobbying arm, the Community Alliance for Economic Development and Jobs. Only Vijai Singh, associate chancellor of Pitt, did not signify any personal association.
Further, Mark Kamlet, dean of the Heinz School of Public Policy and Management at Carnegie, provided one of four signatures (Patti Burns' among them) on a lobbying brochure mailed to voters urging a "yes" vote by the Community Alliance.
While it is within their individual democratic right to lend their name and voice their opinion, such a personal association does not create an environment for objective research.
But an even more egregious abuse of the claim of objectivity and independence in that study occurred. Harold Miller, the deputy executive director of the Allegheny Conference on Community Development and the western division of the Pennsylvania Economy League, was the key architect of the RRI legislation and served as managing director for the Regional Renaissance Partnership.
The economic impact report was conducted at the request of the Allegheny Conference on Community Development. Miller, in his dual roles on the Allegheny Conference and the Partnership, should have maintained an arm's length from the research. Instead, he chaired the working meetings of the research team.
The assumption that a 1.7-million-square foot mall (the size of Century III Mall) would be built between the two proposed North Side stadiums as a result of passage of RRI was the most apparent example of a potential lack of objectivity.
When questioned why the committee assumed a 1.7 million square foot North Side mall - providing 6,200 to 7,500 of the study's jobs - would result from the initative, Steven Manners, the assistant director of Pitt's University Center for Social and Urban Research, said, "We didn't create the package. [USX CEO] Tom Usher did. We only ran the numbers the Partnership gave us."
Including the mall was the first of several heroic assumptions and one, like some others, for which the authors provided no documented evidence or footnotes to attest to its reasonableness.
Most important, the analysis was based on a preordained scenario - assigned to the committee by the Regional Renaissance Partnership, rather than chosen as reasonable by the research committee.
Conclusion. The Pittsburgh region suffers from poor framing of its overall research agenda. And there is a growing intrusion of corporate and business interests into public policy decision-making to the exclusion of other regional interests. Citizens and their public officials must not permit research and public policy to be co-opted by and viewed solely through the lens of business interests.
Pittsburgh suffers, too, from a manipulation of public perspective by the use of public and private funds spent simply to produce "happy'" reports rather than tackling critical issues. The foundation community should seek more rigorous standards for the research it supports. Citizens should demand greater accountability of all agencies and agency research that is supported by public dollars.
Progress in addressing Pittsburgh's economic and social ills will not rapidly advance until credible information and research with integrity is generated to inform our elected public officials. Foundations and other funding sources should demand higher standards for delivering better information.
Here's what should be done. The process should be transparent and subject to public review. The public should be informed from the outset of a study's research objectives to avoid the possibility of burying the results or shifting its focus.
The best analysis begins with a literature review and a pre-analysis report to assist in identifying what is already known and what is unknown about an issue. Based on that information, a discussion among a small group of experts often occurs to define what questions need to be answered, or can be answered.
The research team must be independent and objective, the process and participants untainted by politics and vested interests.
Research methodologies must be sound, executed with precision and thoroughly documented. University research routinely submits to scrutiny by peer review. Likewise, private consultants should submit their analysis to independent reviewers who are experts. Oversight of the research -whether public or private -should be guided by independent persons of integrity, who keep their hands off the research itself.
Finally, the region needs to encourage a more open environment for dialogue in which valid criticisms are not dismissed by politicians and the media as the irrelevant input of naysayers. Membership in the club of subscribers to the status quo policy of the moment does not convey the level of loyalty, good intentions or credibility that a person may possess.
Pittsburgh's universities can help fill this role, if the individual faculty and researchers are themselves independent and the public policy issue has not already been decided, but allows the project team to frame the research.
Ideally, the region's universities could provide many basic research needs for this region if a dedicated funding source was available that allowed university faculty to help frame the region's research agenda and complete the work.
Until such changes occur in how our region's economic research is produced, Pittsburgh will continue to proceed largely on faith alone. And sadly, our faith is destined to be rewarded by slipping further behind more progressive regions who face their future, their past and their present with honesty.
|
||||||||||||||||||||
