Prevent previous bash history entries from being overwritten

I very frequently retrieve commands from my bash history (^R) and start modifying it to fit my current needs, but if I change my mind and break (^C) to cancel it altogether, whatever changes I made end up overwriting that original line in my ~/.bash_history.

How do I turn this feature off? I’ve accidentally overwritten a number of important entries with this before and want to put an end to that once and for all.

The bash versions I’m using are 5.2.37(2)-release on Arch GNU+Linux and 3.2.57(1)-release on OS X El Capitan (and it affects both).