Content by: johnfine@erols.com
HTML formatting by: volunteer_needed
This page discusses the major difference between real mode and protected mode, which is the behavior of segment registers.
In real mode the CPU shifts the segment register value left by four places (multiplying it by 16) and adds the 16 bit offset to get a 20 bit physical address.
In pmode the value in a segment register is not a segment number. It is a "selector". Bit 2 of the selector tells the CPU whether to use the GDT or LDT. Bits 3 to 15 of the selector index to one of the descriptors in the GDT or IDT.
The CPU adds the 16-bit or 32-bit offset (from the address portion of the instruction) to the "base" value from the descriptor. The result is a 32-bit "linear" address.
The descriptor also contains a "limit" (length minus one) for the segment. The CPU will generate a fault if you attempt to access beyond the limit of a segment.
Bits 0 and 1 of the selector are known as the "RPL". The current priviledge level is known as the "CPL". Those values are checked against the descriptor priviledge "DPL" to see if the access should be permitted.
In all modes, every access to memory that uses a segment register uses the base, limit, and attribute portions of the segment register and does not use the selector portion.
Every direct access to a segment register (PUSHing it on the stack, MOVing it to a general register etc.) uses only the selector portion. The base, limit, and attribute portions are either very hard or impossible to read (depending on CPU type). They are often called the "hidden" part of the segment register because they are so hard to read.
Intel documentation refers to the hidden part of the segment register as a "descriptor cache". This name obscures the actual behavior of the "hidden" part.
In real mode (or V86 mode), when you write any 16-bit value to a segment register, the value you write goes into the selector and 16 times that value goes into the base. The limit and attribute are not changed.
In pmode, any write to a segment register causes a descriptor to be fetched from the GDT or LDT and unpacked into the base, limit and attribute portion of the segment register. (Special exception for the NULL Selector).
When the CPU switchs between real mode and pmode, the segment registers do not automatically change. The selectors still contain the exact bit pattern that was loaded into them in the previous mode. The hidden parts still contain the values they contained before, so the segment registers can still be used to access whatever segments they refered to before the switch.
When the CPU switches between V86 mode and pmode it updates all the segment registers to have values which are consistent with the new mode. This is the reason that tricks like flat real mode don't work in V86 mode.
When the CPU switches to "System Manangement Mode" from any other
mode it stores the entire contents of the segment register (hidden
as well as selectors) in an SMM work area. On some CPU's, this is
the only way to read the hidden part of the segment registers.
When the CPU switches from SMM to any other mode it reads the entire
contents of the segment registers from that work area. It does not
perform many of the consistency checks that would occur under other
circumstances, so it is possible to create something like flat real
mode for V86, but there are many further problems.
If you think of the hidden part as a "descriptor cache", you will find it very hard to work with these situations. With a cache, you would have no idea when the values will pop back into sync, or what you can/can't safely do while they are out of sync.
Knowing that they are a hidden portion of the segment registers, means that you know they will change as described above, when you write to the segment registers. Clearly you need to have interrupts disabled whenever the segment registers are out of sync (except for stable cases like flat real mode), because any interrupt routine might save and "restore" a segment register and that "restored" segment register will have only the same selector as was saved, not the same hidden part.
mov eax, cr0 inc ax mov cr0, eaxIt is also real hard, because when you first switch to pmode, the slightest mistake will cause your CPU to "triple fault". That makes your system reboot. It doesn't tell you much about what you did wrong, just that you did SOMETHING wrong.
When you first switch to pmode, your CS register still has the same base, limit and attribute values it had before. Don't worry about the fact that the CS selector is now "incorrect" for pmode. As long as you don't have any interrupts or far CALLs, the CS selector doesn't matter. You can execute lots of code, even do near JMPs and CALLs and RETs, all without loading a pmode compatible CS selector.
Similarly, your data, stack, etc. segment registers retain their base, limit and attributes. This brings me to a great debugging aid for mode switching routines:
I will assume that your early pmode code does not use GS, and that your display is in text mode and that your display regen buffer is at B8000. (Make the obvious change if that last item is wrong). Put the following code well before you switch to pmode:
mov ax, 0B800h mov gs, ax mov byte [gs:0], '1' ;NASM SyntaxNow take lines like these:
mov byte [gs:2], '2' mov byte [gs:4], '3' mov byte [gs:6], '4' etc.and distribute them through the following code. DO NOT WORRY about the fact that GS has a selector that is incompatible with pmode. DO NOT load a pmode compatible selector into GS (until you are sure your GDT etc. work well enough that you probably don't need this debugging aid anymore anyway).
When your program triple faults, watch the display carefully. The sequence of characters will be visible for a moment (even after the triple fault), so you can see how far your code got before it died.
Before you can change any segment register within pmode, you need to have a valid GDT. You can do the LGDT instruction before or after switching to pmode. It is safe before switching because the value in the GDT register is ignored in real mode; It doesn't affect the loading of segment registers; It doesn't even need to be valid. Doing the LGDT after switching to pmode is safe because all your segment registers still have valid base, limit and attributes, so memory references (like loading the GDT psuedo-descriptor into the GDT register) are still OK.
Many people misunderstand the extra level of indirection involved in pointing to the GDT. The LGDT instruction encodes an address (usual memory addressing rules). That address is the offset (within the indicated or implicit segment) of the psuedo-descriptor. The psuedo-descriptor in turn contains the address of the GDT. THAT address is a linear address and is not based on the contents of any segment register.
In all my examples I put that psuedo-descriptor in the same location as the GDT itself. This is possible because the CPU never uses the first eight bytes of the GDT. (See NULL Selector).
If your code is a DOS program, or is loaded in some other form in which you cannot know linear addresses until run time, then you must patch the psuedo-descriptor to get the correct linear address.
xor eax, eax
mov ax, ds
shl eax, 4 ;eax = linear address of data segment
add [GDT+2], eax ;patch psuedo-descriptor
lgdt [GDT]
. . .
GDT dw GDT_length - 1
dd GDT
dw 0
;First descriptor goes here
Before enabling or using any interrupts, remember to set up a pmode compatible
Interrupt Descriptor Table. Like LGDT, LIDT may be executed before or after
entering pmode; However, unlike the GDT, the IDT does affect the CPU within
real mode. Interrupts should remain disabled from before the earlier of
the LIDT or the switch to pmode, until after the later of the LIDT or loading
pmode compatible selectors into all segment registers touched (including save/
restore) by any interrupt.WARNING: A 386 needs a very tiny delay (any instruction would be more than enough) after switching to pmode, before it can correctly load a selector into a segment register. In one version of my switch to flat real mode I had the selector value in a general register before switching to pmode, and the very first instruction after switching to pmode was a fast instruction to MOV that selector to a segment register. Depending on instruction alignment, it could corrupt the hidden part of the segment register (on a 386 only). You can safely write to a segment register with the very first instruction after switching to pmode if it is a slow instruction like a POP or a far JMP, but not if it is a fast instruction like "MOV DS,BX". Normally you wouldn't even notice this problem because it is more natural to move the selector to a register right before you move it to the segment register.
Before switching from pmode you must put real mode compatible values in the limit and attribute part of each segment register. You cannot change the limit or attribute while you are in real mode, so you must set the values you want before you exit pmode.
After switching from pmode you must put real mode compatible values in the segment selectors. Immediately after you exit pmode the segment registers still have the selector and base values that were loaded when you fixed the limit and attribute values (as described above). As with the switch TO pmode, you can use the segment registers with those base values to execute code and access data as long as you want after switching. However, you should understand the consequences of having the selector and base values out of sync. As soon as any interrupt service routine saves and restores a segment register, the pmode base will be lost and the base will become 16 times the selector.
There is a default size bit in each of the CS and SS attribute parts. It comes from bit 54 in the descriptor (that is the D_BIG bit, for those using my gdt.inc definitions). In the CS register the default size has a major effect. It sets the default operand and address size to 16 or 32. If you exited to real mode with that bit still set, you would get very strange results (and not very useful, because the BIOS wouldn't be usable and the IVT would still function in 16-bit real mode).
If you exit to real mode with the default size bit still set in SS (a common mistake), you get a much subtler problem. AFAIK, the sole effect of the default size bit in SS is to cause all implicit uses of SP to use ESP instead. Implicit uses of SP are PUSH, POP, CALL, RET, INT, etc.; If the upper half of ESP is cleared, you won't immediately notice that it is being used. However, there are several things that programs might do that will crash as a result of that SS bit. My favorite is:
xor sp, sp push somethingNormally that predecrements sp to FFFE and writes the value there. You may not like the results if the value gets written to ss:FFFFFFFE instead.
Flat real mode has the major advantage over PMODE, that it is compatible with the BIOS and DOS. In fact, if you are running HIMEM.SYS without EMM386.SYS you are probably already in flat real mode.
Getting into flat real mode is pretty easy:
cli ;Don't allow interrupts during this
push ds ;Save real mode selectors
push es
xor eax, eax ;Patch the GDT psuedo-descriptor
mov ax, ds ; (assuming ds points the segment containing the GDT)
shl eax, 4
add [GDT+2], eax
lgdt [GDT] ;Load the GDT
mov eax, cr0 ;Switch to pmode
inc ax
mov cr0, eax
mov bx, flat_data ;Our only pmode selector
mov ds, bx ;Install 4Gb limits (warning)
mov es, bx
dec ax ;Switch back to real mode
mov cr0, eax
pop es ;Restore real mode selectors
pop ds
sti ;Notice, I never needed to change the CS while in pmode.
. . .
%include "gdtn.inc"
GDT start_gdt
flat_data desc 0, 0xFFFFF, D_DATA+D_WRITE+D_BIG_LIM
end_gdt
Most people seem to build their GDT's in hex rather than using macros such as
those defined in gdtn.inc.
If you really want to do it that way, the four lines above that create
the GDT could be replaced by:
GDT dw 0xF, GDT, 0, 0, 0xFFFF, 0, 0x9200, 0x8F flat_data equ 8I think the hex version is unreadable and error prone.
A common fallacy about flat real mode is that it is incompatible with programs that rely on segment wraparound.
On an 8086, offsets within a segment are truely and consistently 16 bits. If you access ds:[bx+si+7000h] and bx=7000h and si=7000h, then you are asking for 7000h*3 which is 15000h; But of course you actually get offset 5000h because offsets are 16 bits.
Similarly, if you access a word at offset [0FFFFh], the low byte of the word will come from offset 0FFFFh and the high byte will come from offset 0, not from the byte that physically follows the low byte.
On a 386+, there is only a slight difference. In 16-bit addressing, 7000h + 7000h + 7000h still equals 5000h; However, accesing the word at offset FFFFh now gives you a protection violation rather than a wraparound.
In flat real mode, there is another slight difference. In 16-bit addressing, 7000h + 7000h + 7000h STILL equals 5000h (despite nonsense to the contrary that I have seen written); However, accesing the word at offset FFFFh now gives you the full word at that location rather than either a wraparound or a protection violation.
In the unlikely event that some program relied on the protection violation, it would be incompatible with flat real mode. If it relied on the wraparound, it wouldn't run correctly on a 386+ anyway.
Flat real mode is incompatible with V86 mode, so if you are running EMM386 etc. which put your DOS session in V86 mode, then you can't switch to flat real mode. Also certain BIOS interrupts and other programs, that temporarily switch in and out of pmode, will trash your extended limits and take you back out of flat real mode.
Another common fallacy is that 32-bit addressing modes are possible in flat real mode but impossible in ordinary real mode. In fact, the modes themselves are always valid; Only the limits change. In real mode you could do:
mov al, [ecx]I have even found reasons to do that in ordinary real mode. It allows you to use the cx register for an address (as long as the high bits of ecx are clear). It just doesn't allow that address to be above 64K.
Flat real mode doesn't allow any new addressing modes; It just allows larger offsets (that otherwise would have caused protection violations) in the 32-bit addressing modes that were already available (though rarely useful) in real mode.
The EIP is FFF0 and the CS selector is F000, so it would seem that it starts at physical address FFFF0, just like an 8086 (which starts at FFFF:0).
Actually the hidden base portion of the CS is FFFF0000, not F0000, so the actual starting address is FFFFFFF0.
As long as you don't write to the CS register in any way, that base value will remain and you can do near JMP, CALL and RET, to execute code in FFFFxxxx. As soon as you do the first far JMP, CALL, INT, etc. the base value will be back in sync with the selector and you can't access above 10FFEFh again until you switch to pmode.
These actions do not affect the base, limit or attributes of any segment register. The new descriptor values will only be used when you write to a segment register.
In this sense the "descriptor cache" acts very unlike a cache and very unlike the "TLB" which is the cache for the paging system.
When you change an entry in a page directory or page table, it is very hard to predict how long you can safely use the old "stale" entry in the TLB. In general you can't. When you write a new CR3 value (ananogous to doing another LGDT) the CPU flushes the TLB and immediately gets rid of all the "stale" values. When you do a new LGDT the "descriptor cache" isn't touched.
The CPU first computes an offset. In 16-bit addressing the CPU truncates the offset to the low 16 bits. The CPU then adds the offset to the segment-base provided by the selected segment register. This yields the linear address that will be accessed.
The obvious example is something like:
MOV DS,AXHowever the same rules apply to many other situations, including:
When you are writing code that crosses mode boundaries, you should consider all possible segment register writes. Things like saving and restoring a segment register, that are normally transparent, may change the hidden part of the segment register, when executed after a mode change.
The general rule would say that a selector of zero refers to the zeroth descriptor in the GDT. But this does not occur.
The CPU never refers to the zeroth descriptor in the GDT. The first 8 bytes of the GDT are available for any use you like and their contents will not affect the behavior of the GDT.
When you write a zero to DS, ES, FS or GS in pmode the CPU sets the attributes of that segment register to mark it unusable, but does not generate a fault at that time. If you then use that segment, you will get a fault.
The purpose of the NULL Selector is to cover situations in which you do not have valid contents for a segment register. If you left old invalid contents in the register and simply didn't use that register, you might run into trouble when some routine saves and restores the register. On restoring the register, it might generate a fault.
The NULL selector may be freely saved and restored without causing a fault.
Trying to put the NULL selector into the CS register will generate a fault on the instruction that attempts that. It will NOT load the NULL selector and generate the fault on the next attempt to fetch an instruction. I am not sure whether putting NULL in SS generates an immediate fault or waits until the next use of SS.
If you want to use MASM or TASM etc. I hope the meaning of the NASM instructions is clear enough that you can translate to the assembler you choose.
NASM is free, so you may be better off downloading a copy of NASM and using that, rather than trying to translate syntax.
I have made several enhancements to NASM. You can find both my enhanced version of NASM and links to standard NASM on my web page:
http://www.erols.com/johnfine/