I don't see how many more corners i can cut with this. I suppose only using a triangle instead of a quad may boost the speed of it.
Wrong. It increases the fillrate as you need to have a much larger surface area in order to correctly display a texture.
To speed it up, try ignoring rotations behind the camera and other simple tricks.