The recession officially ended more than five years ago and ought to be a distant memory. These should be boom times with widespread optimism and robust spending. Yet consumers are gloomy and the economy is limping along at subpar levels of growth.

It’s becoming clear why: While jobs have returned, incomes have not. The latest evidence is a study by the U.S. Conference of Mayors that highlights stark disparities between the jobs lost during the recession and jobs gained since. The types of jobs lost paid nearly $62,000 per year, on average. The jobs gained during the past six years pay only about $47,000. That 23% shortfall adds up to about $93 billion in lost wages per year — money not being spent because it vanished from the economy.

That startling wage gap reflects the demise of well-paying jobs that don’t require a college degree, which may be the single-biggest challenge facing families trying to uphold a middle-class lifestyle. The two sectors that lost the most jobs during the recession are manufacturing, with average pay of about $63,000, and construction, at about $58,000. Employment in those two fields is still about 3 million workers short of where it was at the start of 2008.

Employers, meanwhile, have hired more hotel and restaurant staff (average pay $21,000), healthcare workers ($47,000) and administrative professionals ($37,000). So while jobs have returned, many Americans are working for considerably less than they used to, as Aaron Task and I discuss in the video above.

This is a pernicious problem that can’t be easily fixed by policy prescriptions or Federal Reserve maneuvers. The Fed has said repeatedly it sees “slack” in the economy, and the income shortfall could be a prime example of what the Fed means by slack. Yet even if the Fed continues to hold interest rates at super-low levels, it’s not clear that would help boost pay or living standards for ordinary people.

Workers who lose jobs in one sector don’t necessarily go to work in another. It’s not as if there are a lot of former assembly-line workers now manning the registers in stores or waiting tables. Many workers who lose decent-paying jobs basically wait for those jobs to return, drawing unemployment insurance as long as they can and then pulling out of the labor force. This could help explain why the labor-force participation rate — the portion of adult Americans either working or looking for work — has hit the lowest levels since the 1970s.

Many of today’s economic dropouts are men, because the two most devastated sectors — construction and manufacturing — are both male-dominated. Hiring is picking up in those sectors, but total employment might never reach the levels it was at before the recession. The process of adjusting to an economy with fewer high-paying jobs is a long one that could permanently lower living standards for some families.

Many workers with skills no longer in demand eventually find something more productive to do. That’s how the economy revives itself amid financial turmoil and technological disruption. The process of adjustment, however, can be painful and even cruel. And when the available jobs pay considerably less than what you earned a few years ago, it’s hard to have much confidence in a so-called recovery.

Rick Newman’s latest book is Rebounders: How Winners Pivot From Setback To Success. Follow him on Twitter: @rickjnewman.