So the final angle of the ball merely depends on which line I test first for collision
Calculate the collision normal by taking the angle from the point of collision to the center of the ball, and not from the lines surface normal. That way it doesn't matter which of the two lines you test, you'll always get the correct collision normal. To add a little to this, the point of collision will be the point on the line that is closest to the balls center. There are a couple of functions in the code archive for calculating this.