I had one of those customers years ago and I was a bit nervous when I said I won't go down in price, it was already a low price. Best thing I ever did, the work I lost let me do more profitable work and less of a pain at the same time.
I raise my prices several time a year now, in small increments.