Especially when you are writing libraries, as multiple people will be using it so optimisation makes even more of an effect. For the main program, or most games, yes, it is most likely pointless, but it's still advisable.
Actually, that's a perfect example, for me, of when optimization can be bad. If I'm using a 3rd party library, I want it readable so I can see what's going on. If it's optimized up the wazoo for an imperceptible speed increase, and I can see what's going on, that's no help.
Optmisation is good, but like John said earlier, optimize where it's going to make a difference.