"One simple argument I've heard is that: we see the effect of optical brightners - so why would you cut it?"
The problem is that devices see the effect of the optical brighteners differently than our eyes do. We see the paper as brighter and whiter, the spectrophotometer sees it as more blue...