I'm certain this can be simplified. I'm also certain it could be made more elegant - sorry, I didn't get much sleep last night. Also, please excuse my use of a keyword as a type name. ;)
There are further optimizations to be done. Sometimes a third rectangle isn't necessary. Hell, sometimes a second rectangle isn't necessary! Hopefully this should get you going though! :)
Type Rect Field x1, y1, x2, y2 Field r, g, b End Type Function rect_new.Rect(x1=0, y1=0, x2=0, y2=0, red=0, g=1, b=1) r.Rect = New Rect If x1 > x2 Then temp = x2 : x2 = x1 : x1 = temp If y1 > y2 Then temp = y2 : y2 = y1 : y1 = temp r\x1 = x1 : r\y1 = y1 : r\x2 = x2 : r\y2 = y2 r\r = red : r\g = g : r\b = b Return r End Function Function math_max(a, b) If a > b Then Return a Else Return b End Function Function math_min(a, b) If a < b Then Return a Else Return b End Function Function optimize_rects.Rect(r1.Rect, r2.Rect) If Not((r1\x1 < r2\x2) And (r1\x2 > r2\x1) And (r1\y1 < r2\y2) And (r1\y2 > r2\y1)) Then Return Null If r2\x1 < r1\x1 Then temp.Rect = r2 : r2 = r1 : r1 = temp ; r2 is horizontally inset in r1 If r2\x2 < r1\x2 Then r3.Rect = rect_new(r2\x2, r1\y1, r1\x2, r1\y2) r1\x2 = r2\x1 r2\y1 = math_min(r1\y1, r2\y1) r2\y2 = math_max(r1\y2, r2\y2) Return r3 EndIf If r2\y1 < r1\y1 Then temp.Rect = r2 : r2 = r1 : r1 = temp ; r2 is vertically inset in r1 If r2\y2 < r1\y2 Then r3.Rect = rect_new(r1\x1, r2\y2, r1\x2, r1\y2) r1\y2 = r2\y1 r2\x1 = math_min(r1\x1, r2\x1) r2\x2 = math_max(r1\x2, r2\x2) Return r3 EndIf ; diag solution If r2\y1 < r1\y1 Then temp.Rect = r2 : r2 = r1 : r1 = temp r3.Rect = rect_new(math_min(r1\x1, r2\x1), math_max(r1\y1, r2\y1), math_max(r1\x2, r2\x2), math_min(r1\y2, r2\y2)) r1\y2 = r3\y1 r2\y1 = r3\y2 Return r3 End Function Function show_rects(solid) If solid = True Then brightness = 63 Else brightness = 255 For r.Rect = Each Rect Color brightness*r\r, brightness*r\g, brightness*r\b Rect(r\x1, r\y1, r\x2-r\x1, r\y2-r\y1, solid) Next End Function Graphics 800, 600 While Not(KeyHit(1)) r1.Rect = rect_new(Rand(0, 800), Rand(0, 600), Rand(0, 800), Rand(0, 600), 0, 0, 1) r2.Rect = rect_new(Rand(0, 800), Rand(0, 600), Rand(0, 800), Rand(0, 600), 0, 1, 0) show_rects(True) r3.Rect = optimize_rects(r1, r2) show_rects(False) WaitKey Cls Delete r1 Delete r2 If r3 <> Null Then Delete r3 Wend
There are further optimizations to be done. Sometimes a third rectangle isn't necessary. Hell, sometimes a second rectangle isn't necessary! Hopefully this should get you going though! :)