No .bmp is a lossless format. .bmp format is an exact representation of the image with no compression (thus tha large file sizes).
.png is s better alternative compressed representaion of the image but without any loss of data.
.jpg (in certain circumstances, such as photos) is an even better compression format but some image clarity is lost and artifacts in the image can appear.
If you are having trouble with .bmp files, then perhaps your problem is the second suggestion I had (similar pixels not matching the mask color)...
.png is s better alternative compressed representaion of the image but without any loss of data.
.jpg (in certain circumstances, such as photos) is an even better compression format but some image clarity is lost and artifacts in the image can appear.
If you are having trouble with .bmp files, then perhaps your problem is the second suggestion I had (similar pixels not matching the mask color)...