If you happen to be one of the owners of the 2 million or more accounts that Wells Fargo fraudulently created, the bank will be getting in touch asking if you need or want your credit cards.

For many, the natural response probably would be disbelief at the sheer audacity of the question, prompting some variation of “I didn’t want it then and I sure as heck don’t want it now!” Sure enough, one inadvertent Wells Fargo credit card “customer” told Yahoo Finance the question was “laughable at best.”

Unfortunately, many Wells Fargo victims will have to face a cruel dilemma: choosing between their own credit score and getting a measure of satisfaction. Twisted as it sounds, breaking up with a credit card you didn’t know you were in a relationship with could damage your credit.

When Wells Fargo secretly opened credit cards for people in order to meet aggressive sales goals, it inadvertently raised their credit scores. “The new card increased your overall credit line because now, it appears, banks trust you with more money,” says Sean McQuay, a credit card and banking expert at NerdWallet. “Secondly, it shrunk your credit utilization ratio, because now it appears you’re using less of your available credit than you were before. Both of these changes were good for your score.” (Your credit utilization ratio is the percentage of available credit you’re using; a lower the rate typically means a higher score.)

On top of that, if the fraudulently opened card was one of your oldest cards, it would have boosted your score further.

“So here’s the unfortunate part of this whole situation,” says McQuay. “Closing the card that Wells Fargo fraudulently opened for you will most likely hurt your credit score.”

If Wells Fargo opened a card in your name with a $30,000 limit, as was the case with one person Yahoo Finance interviewed, closing it would likely significantly lower your available credit, unless you have an enormous amount of high-limit credit cards to make a $30,000 drop look small. Closing credit card accounts “decreases your overall credit line and lifts your credit utilization ratio,” says McQuay. “Unfortunately, there’s no way to really avoid this loss as the inflated metrics disappear from the scoring algorithm.”

So what should you do?

Well, if you aren’t applying for any credit or loans, you might have some leeway to burn bridges with the bank and close the account you never asked for in the first place. But for people who need good credit to get a favorable interest rate – say you’re shopping for a mortgage – that satisfaction might be expensive.

If the Wells Fargo card doesn’t have a fee and contributes significantly to your overall credit line, you might want to swallow your pride and keep it, says McQuay. “Keeping a relic of Wells Fargo’s misconduct on your credit report might feel weird, but if there’s a silver lining I’d take it.”

If you want to check your credit report for errors or fraudulent accounts you didn’t open, you can visit AnnualCreditReport.com, a government-approved site run by the major credit reporting agencies. On that site, you can also get your credit score for a fee, although many credit card companies offer scores for free.

Ethan Wolff-Mann is a writer at Yahoo Finance focusing on consumerism, tech, and personal finance. Follow him on Twitter @ewolffmann.

Read more:

Amazon’s review-for-free-stuff policy wrecks its reviews

This four-letter word will get you better customer service on the phone

Wells Fargo customers recall their shock upon discovering fraudulent accounts

Americans don’t understand ‘premium’ gas, and it’s costing them billions