Cross / Dot Product question

Oh, I know what you have to do. This is the same problem as generating vertex normals over a surface.

What you need to do is generate the normal for lines V4V1 and V1V2. I put code in the code archives to calculate the normal of a line a while back...

Ah here we go:
http://www.blitzbasic.com/codearcs/codearcs.php?code=450

Now that you have the normal of each of the lines... And do note that the order you put the points into the equation determines if the normals will point towards the inside of your angle (correct) or the outside of your angle in the diagrams above...

You can now simply add the normals of the two lines, Nx1+Nx2, Ny1+Ny2, and normalize the result:

; Normalize the new vertex normal.
			; (Normalizing is scaling the vertex normal down so that it's length = 1)

				Nl# = Sqr(Nx#^2 + Ny#^2 + Nz#^2)

				; Avoid a divide by zero error if by some freak accident, the vectors add up to 0.
				; If Nl# = 0 Then Nl# = 0.1

				Nx# = Nx# / Nl#
				Ny# = Ny# / Nl#
				Nz# = Nz# / Nl#

Just delete all references to Nz in the above, as it is 0.

And voila, you are finished.

In example 2 above, where V4 V1 and V2 lie on a line, the normals of both lines would point downard. You would then add them together, and renormalize. That would result in a normal pointing downward as well, line V1V3.

So that's it. I think that's what you wanted to do, right?