To fix this, I suggest you have a row and column drag system. This means all the user needs to do is click any row or any column, then hold the mouse down and drag either left or right or up and down. The mouse will lock to that AXIS and then allow free dragging along that axis. It's intuitive and you can pretty much control it with the mouse WITHOUT any of that silly clicking the arrow business.
Yes, yes, yes, yes, *YES*!