If you feel like your Social Security benefits aren’t going very far despite big cost-of-living increases the past few years, there’s a reason for it, and inflation’s only part of the explanation.

The government regularly adjusts many things for inflation: federal tax brackets; contribution limits for retirement accounts; the size of the standard deduction on taxes; Social Security’s cost-of-living adjustment, or COLA, among others.

But one thing that has never been adjusted for inflation is the federal income threshold to determine if you’ll have to pay taxes on your Social Security benefit. That means with each passing year, an increasing proportion of seniors have been reaching those low thresholds and having to pay taxes on their benefits.

“This is a stealth tax,” said Jordan Gilberti, senior lead planner and certified financial planner at financial advisory firm Facet. "Everyone knows Social Security gets taxed, but rarely do they see how it’s taxed. People’s jaws would fall to the ground.”

How is Social Security taxed and what are the tax thresholds?

Depending on your so-called provisional income, up to 85% of your Social Security benefits can be taxed. Provisional income includes your gross income, tax-free interest you received, like from a municipal bond, and 50% of your Social Security benefits.

If you’re single and this total is less than $25,000, or if you're filing jointly and it's less than $32,000, none of your Social Security is federally taxed.

If it’s between $25,000 and $34,000 for single filers or $32,000 and $44,00 for joint filers, up to half your Social Security is taxed. These thresholds have remained the same since taxes on Social Security benefits were introduced in 1984.

And up to 85% is taxed for anything above $34,000 for single filers and $44,000 for joint filers. These thresholds were added in 1993.

For example, if you have $50,000 in income and get $1,500 a month from Social Security, you'll pay taxes on 85% of your $18,000 in annual benefits, or $15,300.

You can file quarterly estimated tax returns with the IRS or ask Social Security to withhold federal taxes from your benefit payment.

What’s wrong with these Social Security tax thresholds?

With such low-income thresholds, a larger proportion of beneficiaries owe taxes on Social Security every year. In 1984, the average monthly check for an individual was $314 and $472 for joint filers. In 2023, it’s $914 and $1,371, respectively.

The percentage of all tax returns with taxable Social Security benefits grew to 33% in 2017 from 7.4% in 1999, and the Congressional Budget Office predicts that it will grow to more than 50% by 2046. And, since 1984, the proportion of beneficiary families whose benefits are taxed has risen over time from less than 1 in 10 to more than half, the Social Security Administration says.

If Social Security income thresholds were indexed to inflation, The Senior Citizens League estimates that the first thresholds of $25,000 for individuals and $32,000 for joint filers would rise to $73,000 and $93,200, respectively. At those levels, many fewer Social Security beneficiaries would probably owe tax on their benefits, says Mary Johnson, a policy analyst for the advocacy group.

Scorching hot inflation from the past couple of years made things worse. Many seniors had to tap retirement funds or take part-time jobs to make ends meet, which boosted their income and meant they likely had to pay taxes on their Social Security.

“Seniors view this as discriminatory, double taxation,” Johnson said.

A recent survey by the league showed 58% of seniors want these thresholds adjusted for inflation, “and quite a few are in favor of getting rid of the tax altogether.”

“When they’re considering changes to Social Security and Medicare, they’ve never, ever turned to senior constituents or advocates as individuals to sit on commissions or in on negotiations,” Johnson said. “We’ve never been invited to the table.”

How it works: Is Social Security income taxable by the IRS? Here's what you might owe on your benefits

Why hasn’t the government adjusted these thresholds for inflation?

“To be responsible about it, there would be a need to find revenues to replace what would be lost,” Johnson said. This year, she estimates Social Security would receive $48.8 billion in revenue from taxing Social Security benefits.

The Social Security and Medicare Boards of Trustees already predict Social Security trust fund reserves will become exhausted in 2034, and though President Joe Biden has called for “protecting and strengthening” Social Security, he's offered no plan to do so.

A Social Security overhaul is also particularly difficult because of a divided Congress. Any overhaul of Social Security would require bipartisan support.

Do states also tax Social Security?

Yes, but only some. While each state has different rules on what or how they will tax your Social Security, money, age and income usually determine if you pay. It’s best to check with your state’s rules.

States that may tax some of your benefits include:

• Colorado: People under age 65 may owe taxes on Social Security benefits but older people generally don't.

• Connecticut: If your income exceeds $75,000 as a single filer or $100,000 filing jointly, 75% of your Social Security benefits are tax-exempt. Those with income under those thresholds pay no tax on Social Security benefits.

• Kansas: People with adjusted gross income above $75,000, regardless of status, pay at the income tax rate.

• Minnesota: The state taxes the same amount of Social Security benefits that are taxed federally, but you may qualify for a subtraction from Minnesota taxable income depending on your income. Single filers who have provisional income above $85,970 or joint filers above $110,020, aren't eligible for any subtractions.

• Missouri: Those people 62 and older with an AGI of less than $85,000 (single) or $100,000 (married, filing jointly) aren't taxed on Social Security. If you earn more, you might only get a partial deduction.

• Montana: High earners may have to pay tax on some of Social Security benefits and should use the state's tax form to determine what they may owe.

• Nebraska: In tax year 2022, married filing jointly taxpayers with a federal AGI of more than $61,760 and individuals with a federal AGI above $45,790 paid tax on part of their Social Security benefits, but the share declines annually as the state phases out taxation of benefits by 2025.

• New Mexico" Social Security income is fully deductible for residents with AGIs below $100,000 for an individual and $150,000 for a couple filing jointly so most people won't pay tax. However, those above those thresholds might.

• Rhode Island: Only people who have reached full retirement age as defined by the Social Security Administration (between 66 and 67, depending on year of birth) and have an AGI below certain thresholds aren't taxed on their benefits.

• Utah: People above a certain income threshold only get a partial tax break on Social Security.

• Vermont: Single filers earning at least $60,000 (joint filers making $75,000 or more) pay some tax on their benefits.

What can Social Security beneficiaries do to avoid taxes?

Plan to reduce your provisional income.

“Provisional income includes every type of income, from a job, IRA distribution, rental income, dividends, interest,” Gilberti of Facet said. “One of the only things that doesn’t count is Roth IRA distributions.”

If you don’t already have a Roth IRA, consider converting some of your 401(k) or traditional IRA if your tax bracket is low to a Roth IRA before retiring. You would have to pay ordinary income tax on those conversions, but it might be worth it.

“We also recommend doing those Roth conversions, if you can, by 63 (years old) because your Medicare premium, which will be taken out of your Social Security check, depends on your income from the last two years,” Gilberti said.

Medora Lee is a money, markets, and personal finance reporter at USA TODAY. You can reach her at [email protected] and subscribe to our free Daily Money newsletter for personal finance tips and business news every Monday through Friday morning.

This article originally appeared on USA TODAY: Is Social Security taxable? Every year, more people are paying taxes