To be honest i havent really looked into this much before and im just googling for answers for you.
there is a fair bit of information out there on self intersection of silhouette loops and how this can cause the problem you are seeing.
http://graphics.cs.lth.se/theses/projects/shadows/so_thesis_lq.pdf
is someones thesis i found detailing some info on shadow volumes that may be of some use.
There are some idea's i've seen while googling around about projecting your volume onto a 2d plane that faces the light sorce, then using faster 2d techniques to check for self intersections of the loop. But alot of the maths involed on those pages were beyond me and i got bored fast and moved onto other pages.
Sorry I couldnt be more help but im just googling around and understanding little(im more into creating 2d platformers etc, i havent reached the 3d age yet)
there is a fair bit of information out there on self intersection of silhouette loops and how this can cause the problem you are seeing.
http://graphics.cs.lth.se/theses/projects/shadows/so_thesis_lq.pdf
is someones thesis i found detailing some info on shadow volumes that may be of some use.
There are some idea's i've seen while googling around about projecting your volume onto a 2d plane that faces the light sorce, then using faster 2d techniques to check for self intersections of the loop. But alot of the maths involed on those pages were beyond me and i got bored fast and moved onto other pages.
Sorry I couldnt be more help but im just googling around and understanding little(im more into creating 2d platformers etc, i havent reached the 3d age yet)