vga: defer V86-mode VBE disable until a legacy mode-set is observed

When a Win9x windowed DOS VM calls INT 10h, vgabios runs in V86 mode and
its mode-set begins by writing dispi[4]=0. The Win9x VDD virtualises the
standard VGA ports (3B0-3DF) for windowed VMs but passes 1CE/1CF straight
through, so the VBE disable reaches the hardware while the rest of the
mode-set (CRTC/seq/gfx/attribute writes) is captured into the VM's
virtual register file. v86 then drops out of LFB rendering with the
legacy registers still holding the SVGA values, and the screen shows
planar garbage until the user manages to Alt+Enter back.

Defer clearing svga_enabled when dispi[4] is cleared from V86 mode
(EFLAGS.VM set); commit it only when an attribute_mode write actually
reaches us. A real passthrough mode-set (fullscreen DOS, display-driver
mode change, X.org int10) writes attribute_mode immediately afterwards,
so the disable lands one register later. The windowed-VM leak never
follows up, so the LFB stays live and the desktop keeps rendering with
the DOS box in its window.

The enable path now also reapplies set_size_graphical whenever dispi[4]
is written with bit 0 set, so a deferred-disable -> reconfigure -> enable
sequence still resizes (set_size_graphical is a no-op when nothing
changed). dispi_enable_value is updated either way, so guest read-back
is unchanged.

Ring-0 / real-mode dispi writes (Linux bochs_drm, vesafb, bare DOS) have
EFLAGS.VM clear and are unaffected.

Author: felixrieseberg

src/vga.js

@@ -1418,6 +1418,13 @@ VGAScreen.prototype.port3C0_write = function(value)
                     var previous_mode = this.attribute_mode;
                     this.attribute_mode = value;
 
+                    if(this.svga_enabled && !(this.dispi_enable_value & 1))
+                    {
+                        // Commit the deferred VBE disable (see port1CF_write case 4)
+                        this.svga_enabled = false;
+                        this.svga_bank_offset = 0;
+                    }
+
                     const is_graphical = (value & 0x1) !== 0;
                     if(!this.svga_enabled && this.graphical_mode !== is_graphical)
                     {
@@ -2165,6 +2172,17 @@ VGAScreen.prototype.port1CF_write = function(value)
             break;
         case 4:
             // enable, options
+            if(!(value & 1) && this.svga_enabled && (this.cpu.flags[0] & (1 << 17)))
+            {
+                // Win9x's VDD virtualises the legacy VGA ports for a windowed
+                // DOS VM but not the dispi ports, so vgabios's VBE disable
+                // leaks through while the rest of its mode-set is virtualised.
+                // Defer the actual disable until a legacy mode register write
+                // reaches us (see port3C0_write); if it never does, the
+                // protected-mode display driver still owns the framebuffer.
+                this.dispi_enable_value = value;
+                break;
+            }
             this.svga_enabled = (value & 1) === 1;
             if(this.svga_enabled && (value & 0x80) === 0)
             {
@@ -2221,11 +2239,14 @@ VGAScreen.prototype.port1CF_write = function(value)
         dbg_log("SVGA: disabled", LOG_VGA);
     }
 
-    if(this.svga_enabled && !was_enabled)
+    if(this.svga_enabled && this.dispi_index === 4)
     {
-        this.svga_offset = 0;
-        this.svga_offset_x = 0;
-        this.svga_offset_y = 0;
+        if(!was_enabled)
+        {
+            this.svga_offset = 0;
+            this.svga_offset_x = 0;
+            this.svga_offset_y = 0;
+        }
 
         this.graphical_mode = true;
         this.screen.set_mode(this.graphical_mode);