You can use the stencil buffer if you know how to write your own OpenGL or DX programs. In OpenGL you have full support for writing to the stencil buffer using any geometry you like, and using the stencil to test/discard incoming pixels against based on the stencil values.