I outlined some of the key points a while back on Blitzcoder. Here it is again:
First of all, I have to say it took me a little while to get everything set up and working satisfactorily, before I could really get down to making a game at all. So be prepared for a bit of not very exciting coding at the very start of the project... I'm not going to go into much technical detail here, just give a brief overview about the kinds of issues I faced and what I did about them; hopefully some of this is helpful.
First of all, I don't recommend mixing direct-draw style 2d with 3d-as-2d. It messes up when you change res, it messes up on certain cards and drivers, and it's just messy (and slower) in general. Once you have a working set of routines for 2d-in-3d, just use that for everything; much nicer.
Also I didn't use any sprites. Originally, it was because I couldn't apply lighting effects to them, and later it was mostly because of the need to combine things on a single surface - handmade quad meshes are definitely the way to go for maximum flexibility. Faster too.
I had to ditch the ortho camera as well, unfortunately... it was messing with the z-buffer a little on my card. I substituted a camera that was 100000 units away, and zoomed right in and restricted to a very narrow z-band (999900-100000) - a bit of a nuisance, but it fixed the problem, and gives virtually the same result.
Next, I knew I was going to need a thousand sprites or more on screen at once, and there was no way I was going to get away with having a separate surface for each of them, so it meant I had to go with a single surface system. More about that in a minute.
Now, the power-of-two textures was causing me trouble too. First off, I had to deal with a main character whose arms and legs were all over the place - some frames needed to be tall and thin, some short and wide. Others just plain big. To accomodate all of this in a power of 2 square at any kind of acceptable res, would have required a 128x128 square, with lots of wasted space on most frames. With over 150 animation frames needed for the little guy, this would have chewed through the video memory in no time. So what I did was, I took a 256x256 square (this being the maximum size that is guaranteed to be supported by virtually all 3d cards), and I squeezed as many graphics onto this as I could. All the graphics in the game are arranged on tiles of 256x256, and arranged according to themes. The main character takes up 8 tiles. The baddies take up 7. Collectable items are all crammed onto 1. Now, of course a single surface system requires that all the graphics be on a single texture, so I created a big 1024x2048 texture at runtime, and all the 256x256 tiles are loaded in one by one and copied onto this big texture, like a big bank of 4x8 tiles. Doing it like this means I can load and free up different 256x256 tiles according to what graphics set I need for any given level. Of course this raises another issue... only some cards can support a 1024x2048 texture... more on THIS later...
Oh, by the way - never EVER load in textures that aren't powers of 2 to start off with. Yes, blitz will scale them up, but not only is this wasteful, but far more seriously, it looks UGLY. Blitz does NOT do a good job of scaling them up, all it does is double some of the rows of pixels... end result is you get some rows of pixels that are twice as fat as others, and it looks really bad. Avoid like the plague.
Okay, next thing I did was make a little editor that would load up a single 256x256 tile and display it full screen. You could then stretch bounding boxes all over it to define where all the different sprites are. The program would translate the corners of the boxes on screen into UV coordinates on the texture. It then saved a data file that went along with that tile, like, this tile has 34 sprites on it, and these are their UV coordinates. Since the bounding boxes can be of any sizes within that 256x256 tile, this solves the power of 2 problem, and means I get to use my odd rectangular sprites of arbitrary pixel dimensions, and minimise wasted video RAM... providing I can pack them all nice and tightly onto the tiles, which I have more or less managed to do.
I could now start writing the single surface system for the game.
When the program starts, the big texture is created, the tiles are loaded onto it, along with the uv data that goes along with the tiles (Of course the program also has to remember where each tile is on the big texture... and offset its UV coordinates accordingly). A blank surface is created using this texture.
Then I made my draw3dsprite function. This function takes as input which tile you want to draw from, which image in the tile, the x and y co-ordinates, size, rotation, colour, opacity, and other game-specific stuff like whether it should be affected by lights or shadows. What it basically does is it creates 4 new vertexes and 2 new triangles on the big surface, centered around the x and y coordinates you give it. It textures the trianges according to the uv coordinates in the data file, referenced by the tile and frame number. It scales it according to the size, but also taking into account the uv coordinates, so that long skinny frames still come out long and skinny, etc. The colour of the sprite is set using vertex colours, and the opacity is set using vertex alpha (brilliant; I was waiting for this function!). Rotation is a bit trickier, I had to get a bit more maths-y than I usually like in order to rotate it properly, but I think there's stuff in the code archives for this...
I don't keep track of any of the vertices created. Every frame, I do a clearsurface and scrap all of it, and rebuild them all again from scratch. Some speed tests I did showed this was only a tiny, tiny bit slower than moving preexisting vertices around, and it is a lot easier. Plus it has some other advantages, as you'll see... Basically, clearsurface becomes my CLS, and the draw3dsprite function becomes my drawimage.
Right, other issues...
All the sprites in my game use an alpha channel for transparency (which was a big part of why I wanted to use 3d in the first place). This causes problems since, due to the nature of transparent textures, they can't use the z-buffer properly. In other words, don't expect to be able to sort the priority of sprites by changing the z-value, if you're using alpha channels (note that my draw3dsprite command doesn't ask for a z-value!). Instead, you must be aware that all triangles in the surface will be rendered in the order you added them to the surface. The last thing you draw will go on the top. Just like using drawimage, really. Also, this method is only good for triangles that are part of that big, merged surface. If a triangle isn't in that big surface, you CAN'T make it appear, say, behind one object that is in the big surface, and in front of another that is in the big surface. It's either behind them all or in front of them all. Um, did that make sense?
Anyway, that's why I needed to get all the sprites onto the one 1024x2048 surface. Actually, what I do is, the background, like the sky and the mountains, is made up of several surfaces, the font is made up pf another surface, but all the main action stuff, the things that have to go in front and behind each other freely, MUST be part of that same surface.
There are some other issues with using a single surface though - since so many effects are applied on a per-entity basis - you can't apply different blend modes for different sprites... my laser beams can't use an additive blend, sadly, unless they were given their own single surface - and if I did that, then they wouldn't be able to go in front and behind objects in the main surface (they'd either have to be in front of all of them, like the font, or behind all of them, like the background). This is a bit of a bummer, but I just have to work with it. You can't make things full bright either though. This WAS a problem, as I wanted certain things (like laser beams, explosions, etc) to be full-bright even if it was a night-time scene. So I ditched the DX lights altogether, and wrote a custom lighting function that simulated DX lights by manipulating the vertex colours, and was just as flexible as I cared to make it. So I get my full bright objects. And thank you so much, Mark, for adding vertex alpha - before that there was no way to control the opacity of different sprites, except by making them semi-transparent in the source file. As soon as vertex alpha was added, I added another parameter to my draw3dsprite command.
Now, everything's working sweet, it's fast, it's functional, it's more or less just like using a supercharged drawimage command... but there's one problem... remember the cards that don't support 1024x2048 textures? Oops, they're screwed...
Some people would probably say at this point, "Ah, too bad, let them get a new card, I don't want those luddites holding me back, blah blah blah...) - but I didn't want to do that. I wanted to make it work for as many machines as possible... so, one of the first things the game does is create a 1024x2048 texture, and then checks its dimensions... if they come back as anything other than 1024x2048, you know the card won't do it, so it kicks into plan B:
Remember that all the graphics are stored as 256x256 tiles. This is a pretty safe size, even a voodoo 1 can handle it. So plan B involves loading up all these tiles, but instead of merging them into one big texture, it keeps them seperate and creates a surface for each of them. Texture size problem solved - but it raises another problem...
Remember I said that you could only z-order the triangles WITHIN a surface, not between surfaces? But in this case, all of these tile surfaces have to be able to intermingle for the game to look right. Can't be done, so I have to ditch the alpha transparency (ouch!) - the game reverts to using masked transparency. Using masks means that you can use the z-buffer for z-ordering again. So in this way, I can get the sprites to intermingle, at the expense of losing the cool alpha effects - ah well, it's better than having it not working at all. In this case, I have a variable, z_order, which gets set to 0 at the start of each game loop. Then, when the first draw3dimage function is called, it creates the first quad at the z-position of the z_order variable, and then it increments z_order by .01 - meaning the next quad will be drawn slightly in front of it. In this way, the plan B drawing method works just like the plan A one - the last sprite drawn goes on the top.
Whew! Okay, that's the basics of what I did.
Oh, one more thing - since all uv coords are stored as a value between 0 and 1, regardless of the size of the texture, it wasn't too difficult to add an option to load all the graphics in at half resolution - every second pixel, onto a texture half the size - the UV data stays the same. The big advantage here, of course, is saving memory - it doesn't look pretty, but it reduces memory requirements from 16 megs to 4. The game will do this as a last-ditch effort to provide a playable framerate on cards with very little memory.
Sheesh, now that is one long post! Just as well I didn't go into technical details! I hope this is somehow informative or helpful, since even though I was trying to be brief, it sure took a long time to explain...
First of all, I have to say it took me a little while to get everything set up and working satisfactorily, before I could really get down to making a game at all. So be prepared for a bit of not very exciting coding at the very start of the project... I'm not going to go into much technical detail here, just give a brief overview about the kinds of issues I faced and what I did about them; hopefully some of this is helpful.
First of all, I don't recommend mixing direct-draw style 2d with 3d-as-2d. It messes up when you change res, it messes up on certain cards and drivers, and it's just messy (and slower) in general. Once you have a working set of routines for 2d-in-3d, just use that for everything; much nicer.
Also I didn't use any sprites. Originally, it was because I couldn't apply lighting effects to them, and later it was mostly because of the need to combine things on a single surface - handmade quad meshes are definitely the way to go for maximum flexibility. Faster too.
I had to ditch the ortho camera as well, unfortunately... it was messing with the z-buffer a little on my card. I substituted a camera that was 100000 units away, and zoomed right in and restricted to a very narrow z-band (999900-100000) - a bit of a nuisance, but it fixed the problem, and gives virtually the same result.
Next, I knew I was going to need a thousand sprites or more on screen at once, and there was no way I was going to get away with having a separate surface for each of them, so it meant I had to go with a single surface system. More about that in a minute.
Now, the power-of-two textures was causing me trouble too. First off, I had to deal with a main character whose arms and legs were all over the place - some frames needed to be tall and thin, some short and wide. Others just plain big. To accomodate all of this in a power of 2 square at any kind of acceptable res, would have required a 128x128 square, with lots of wasted space on most frames. With over 150 animation frames needed for the little guy, this would have chewed through the video memory in no time. So what I did was, I took a 256x256 square (this being the maximum size that is guaranteed to be supported by virtually all 3d cards), and I squeezed as many graphics onto this as I could. All the graphics in the game are arranged on tiles of 256x256, and arranged according to themes. The main character takes up 8 tiles. The baddies take up 7. Collectable items are all crammed onto 1. Now, of course a single surface system requires that all the graphics be on a single texture, so I created a big 1024x2048 texture at runtime, and all the 256x256 tiles are loaded in one by one and copied onto this big texture, like a big bank of 4x8 tiles. Doing it like this means I can load and free up different 256x256 tiles according to what graphics set I need for any given level. Of course this raises another issue... only some cards can support a 1024x2048 texture... more on THIS later...
Oh, by the way - never EVER load in textures that aren't powers of 2 to start off with. Yes, blitz will scale them up, but not only is this wasteful, but far more seriously, it looks UGLY. Blitz does NOT do a good job of scaling them up, all it does is double some of the rows of pixels... end result is you get some rows of pixels that are twice as fat as others, and it looks really bad. Avoid like the plague.
Okay, next thing I did was make a little editor that would load up a single 256x256 tile and display it full screen. You could then stretch bounding boxes all over it to define where all the different sprites are. The program would translate the corners of the boxes on screen into UV coordinates on the texture. It then saved a data file that went along with that tile, like, this tile has 34 sprites on it, and these are their UV coordinates. Since the bounding boxes can be of any sizes within that 256x256 tile, this solves the power of 2 problem, and means I get to use my odd rectangular sprites of arbitrary pixel dimensions, and minimise wasted video RAM... providing I can pack them all nice and tightly onto the tiles, which I have more or less managed to do.
I could now start writing the single surface system for the game.
When the program starts, the big texture is created, the tiles are loaded onto it, along with the uv data that goes along with the tiles (Of course the program also has to remember where each tile is on the big texture... and offset its UV coordinates accordingly). A blank surface is created using this texture.
Then I made my draw3dsprite function. This function takes as input which tile you want to draw from, which image in the tile, the x and y co-ordinates, size, rotation, colour, opacity, and other game-specific stuff like whether it should be affected by lights or shadows. What it basically does is it creates 4 new vertexes and 2 new triangles on the big surface, centered around the x and y coordinates you give it. It textures the trianges according to the uv coordinates in the data file, referenced by the tile and frame number. It scales it according to the size, but also taking into account the uv coordinates, so that long skinny frames still come out long and skinny, etc. The colour of the sprite is set using vertex colours, and the opacity is set using vertex alpha (brilliant; I was waiting for this function!). Rotation is a bit trickier, I had to get a bit more maths-y than I usually like in order to rotate it properly, but I think there's stuff in the code archives for this...
I don't keep track of any of the vertices created. Every frame, I do a clearsurface and scrap all of it, and rebuild them all again from scratch. Some speed tests I did showed this was only a tiny, tiny bit slower than moving preexisting vertices around, and it is a lot easier. Plus it has some other advantages, as you'll see... Basically, clearsurface becomes my CLS, and the draw3dsprite function becomes my drawimage.
Right, other issues...
All the sprites in my game use an alpha channel for transparency (which was a big part of why I wanted to use 3d in the first place). This causes problems since, due to the nature of transparent textures, they can't use the z-buffer properly. In other words, don't expect to be able to sort the priority of sprites by changing the z-value, if you're using alpha channels (note that my draw3dsprite command doesn't ask for a z-value!). Instead, you must be aware that all triangles in the surface will be rendered in the order you added them to the surface. The last thing you draw will go on the top. Just like using drawimage, really. Also, this method is only good for triangles that are part of that big, merged surface. If a triangle isn't in that big surface, you CAN'T make it appear, say, behind one object that is in the big surface, and in front of another that is in the big surface. It's either behind them all or in front of them all. Um, did that make sense?
Anyway, that's why I needed to get all the sprites onto the one 1024x2048 surface. Actually, what I do is, the background, like the sky and the mountains, is made up of several surfaces, the font is made up pf another surface, but all the main action stuff, the things that have to go in front and behind each other freely, MUST be part of that same surface.
There are some other issues with using a single surface though - since so many effects are applied on a per-entity basis - you can't apply different blend modes for different sprites... my laser beams can't use an additive blend, sadly, unless they were given their own single surface - and if I did that, then they wouldn't be able to go in front and behind objects in the main surface (they'd either have to be in front of all of them, like the font, or behind all of them, like the background). This is a bit of a bummer, but I just have to work with it. You can't make things full bright either though. This WAS a problem, as I wanted certain things (like laser beams, explosions, etc) to be full-bright even if it was a night-time scene. So I ditched the DX lights altogether, and wrote a custom lighting function that simulated DX lights by manipulating the vertex colours, and was just as flexible as I cared to make it. So I get my full bright objects. And thank you so much, Mark, for adding vertex alpha - before that there was no way to control the opacity of different sprites, except by making them semi-transparent in the source file. As soon as vertex alpha was added, I added another parameter to my draw3dsprite command.
Now, everything's working sweet, it's fast, it's functional, it's more or less just like using a supercharged drawimage command... but there's one problem... remember the cards that don't support 1024x2048 textures? Oops, they're screwed...
Some people would probably say at this point, "Ah, too bad, let them get a new card, I don't want those luddites holding me back, blah blah blah...) - but I didn't want to do that. I wanted to make it work for as many machines as possible... so, one of the first things the game does is create a 1024x2048 texture, and then checks its dimensions... if they come back as anything other than 1024x2048, you know the card won't do it, so it kicks into plan B:
Remember that all the graphics are stored as 256x256 tiles. This is a pretty safe size, even a voodoo 1 can handle it. So plan B involves loading up all these tiles, but instead of merging them into one big texture, it keeps them seperate and creates a surface for each of them. Texture size problem solved - but it raises another problem...
Remember I said that you could only z-order the triangles WITHIN a surface, not between surfaces? But in this case, all of these tile surfaces have to be able to intermingle for the game to look right. Can't be done, so I have to ditch the alpha transparency (ouch!) - the game reverts to using masked transparency. Using masks means that you can use the z-buffer for z-ordering again. So in this way, I can get the sprites to intermingle, at the expense of losing the cool alpha effects - ah well, it's better than having it not working at all. In this case, I have a variable, z_order, which gets set to 0 at the start of each game loop. Then, when the first draw3dimage function is called, it creates the first quad at the z-position of the z_order variable, and then it increments z_order by .01 - meaning the next quad will be drawn slightly in front of it. In this way, the plan B drawing method works just like the plan A one - the last sprite drawn goes on the top.
Whew! Okay, that's the basics of what I did.
Oh, one more thing - since all uv coords are stored as a value between 0 and 1, regardless of the size of the texture, it wasn't too difficult to add an option to load all the graphics in at half resolution - every second pixel, onto a texture half the size - the UV data stays the same. The big advantage here, of course, is saving memory - it doesn't look pretty, but it reduces memory requirements from 16 megs to 4. The game will do this as a last-ditch effort to provide a playable framerate on cards with very little memory.
Sheesh, now that is one long post! Just as well I didn't go into technical details! I hope this is somehow informative or helpful, since even though I was trying to be brief, it sure took a long time to explain...