Retirees are known for playing bingo in Florida, watching Jeopardy with religious fervor, and voting at every election. They’re less known for returning to the workforce. But right now, that’s exactly what some forlorn economists are hoping for.

During the early pandemic, layoffs, health concerns, and shifting work attitudes prompted an unprecedented number of baby boomers to leave their jobs as part of the Great Retirement. But many of them began to reconsider their choices and unretired amid forty-year high inflation, recession fears, and a volatile stock market. Yet, new data from the St. Louis Fed finds that there’s still a lot more retirees than the Fed predicted: about 2 million, to be exact.

Now, that doesn’t mean the unretirement narrative isn’t true. Unretirement rates rebounded to pre-pandemic levels as the economy and society began to normalize again. The majority (68%) of pandemic-era retirees reported they’d consider returning to work, per a 2022 CNBC survey. And as the cost of living and life expectancy rises, 20% of retirees continue to work full- or part- time and 7% are job searching, per T. Rowe Price’s recent Retirement Saving & Spending Study. Nearly half of working retirees said they're doing so for either financial reasons or for socio-emotional ones.

“It’s extremely scary. We thought that we would be doing some traveling this year, but that’s come to an abrupt halt. You want your money to last,” Anita Cowles told Fortune of her retirement journey in 2022, adding that her husband was considering going back to work.

The number of retirees is down from 2.8 million last year, per the Fed data, a sign of those returning to work. But it’s picked up from 1.7 million in June. “While the gap seemed to be closing earlier in the year, it seems to have widened slightly since then,” Miguel Faria-e-Castro, economic policy adviser at the Federal Reserve Bank of St. Louis, told Bloomberg. The remaining 1.98 million surplus of retirees could pose a problem for the economy, if you ask some experts.

For one, the declining birth rate coupled with our aging nation has left demographers and experts worried about a shrinking workforce that could make the labor shortage last for years. A mass wave of retirees from a generation as large as baby boomers could also pose a threat to programs like Social Security and Medicare, which are already bending under the weight of the aging population and a lack of funding. That could put greater demand on the care industry, a sector that’s already struggling to keep its head above water. As the massive cohort of older workers leave the workforce it becomes abundantly clear: the nation hasn’t prepared itself for baby boomer to retire.

Baby boomers need a new cradle

The baby boom burst has long been predicted. Decades ago, Brookings’ researcher William G. Gale spoke of an impending trainwreck: “Now that the leading edge of the generation has turned 50, the impending collision between the boomers and the nation’s retirement system is naturally catching the eye of policymakers and the boomers themselves,” he wrote in 1997. One year later, Dean Baker projected for the Economic Policy Institute that the peak impact of baby boomers’ retirement would be realized in 2030. Claiming that the mass retirement would not pose an issue to workers as much as rising inequality would, he stressed that “protect[ing] the well-being of future generations” meant curbing healthcare costs.

But not all baby boomers are growing up and fully leaving the nest of the office any time soon. By the end of the decade, employees over 55 will make up 25% of the workforce, per Bain & Company. And, as the cost to retire comfortably exceeds $1 million, many boomers expect to delay their retirement. Younger boomers especially have struggled to reach the same retirement goals as those slightly older than them. Those born between 1960 and 1965 had 19% less retirement wealth than older boomers did when they were in their 50s due to the Great Recession, which hit them during their spending years, per the Center for Retirement Research at Boston College. Indeed, the average retirement age has increased as people live longer and work longer.

Regardless, a shrinking workforce may not be the problem some demographers fear if we put the right systems in place. Christine Percheski, associate professor of sociology at Northwestern University, told Business Insider that the country would need to make structural adjustments like creating new policies that accommodate changes in population size.

That might also include adjusting America’s retirement system, which has been on crutches for some time; it recently received a C+ rating from consulting firm Mercer, alongside countries like Croatia and Spain.

It seems as if the 2 million retiree surplus is a sign that it’s time to create these structures and systems. “We’ve known for 75 years now that we had a really large birth cohort,” economist Kathryn Anne Edwards told the Washington Post. “Many of the quote-unquote problems related to that aren’t from their numbers. It’s problems that come from us not making policy to address what those numbers would mean.”

This story was originally featured on Fortune.com