How to enable paging in x86 Protected Mode

Question

I'm progressing along developing my OS, and I recently implemented a virtual memory manager that handles paging. I used 2 Page Tables: (1) to identity map the first 4 MB and (2) to map the kernel (starts at 1MB) to 3GB. I have also both assigned their proper positions within the Page Directory. After I 'jump' into the higher half I then try to delete the identity-mapping page table from the page directory. However, as soon as I do this, my OS boot-loops.

My questions are:

Am I constructing the Page Tables and their respective entries correctly?
Am I correctly 'jumping' into the higher half of kernel memory?

This is the most relevant function along with all macros/enums used within it:

/* PHYSICALLY allocates a block of memory */
void *alloc_page();

typedef enum {
    PAGE_STRUCT_ENTRY_PRESENT       = 0b00000000000000000000000000000001,
    PAGE_STRUCT_ENTRY_WRITEABLE     = 0b00000000000000000000000000000010,
    PAGE_STRUCT_ENTRY_USER_ACCESS   = 0b00000000000000000000000000000100,
    PAGE_STRUCT_ENTRY_WRITE_THROUGH = 0b00000000000000000000000000001000,
    PAGE_STRUCT_ENTRY_CACHE_DISABLE = 0b00000000000000000000000000010000,
    PAGE_STRUCT_ENTRY_ACCESSED      = 0b00000000000000000000000000100000,
    PAGE_STRUCT_ENTRY_DIRTY         = 0b00000000000000000000000001000000,
    PDE_PAGE_SIZE                   = 0b00000000000000000000000010000000,
    PAGE_STRUCT_GLOBAL              = 0b00000000000000000000000100000000,
    PAGE_STRUCT_PAGE_FRAME          = 0b11111111111111111111000000000000,
} PAGE_STRUCT_ENTRY_MASKS;

typedef uint32_t pt_entry_t;
typedef uint32_t pd_entry_t;

typedef uint32_t vaddr_t;
typedef uint32_t paddr_t;

#define ENTRY_ADD_ATTRIBUTE(entry, attrib)\
    ((entry) |= (uint32_t)(attrib))

#define ENTRY_DEL_ATTRIBUTE(entry, attrib)\
    ((entry) &= ~(uint32_t)(attrib))

#define ENTRY_GET_ATTRIBUTE(entry, attrib)\
    ((entry) & (uint32_t)(attrib))

#define ENTRY_SET_FRAME(entry, frame)\
    ((entry) |= (uint32_t)(PAGE_STRUCT_PAGE_FRAME) & (uint32_t)(frame))

#define ENTRIES_PER_STRUCT 1024

#define PAGE_DIR_INDEX(vaddr)\
    (((vaddr) >> 22) & 0x3FF)

#define PAGE_TABLE_INDEX(vaddr)\
    (((vaddr) >> 12) &0x3FF)

#define PTE_FROM_PDIR(page)\
    (*page & ~0xFFF)

#define PTABLE_ADDRESS_SPACE 0x400000
#define DTABLE_ADDRESS_SPACE 0x100000000

#define PAGE_SIZE 4096

typedef struct {
    pt_entry_t entries[ENTRIES_PER_STRUCT];
} ptable_t;

typedef struct {
    pd_entry_t entries[ENTRIES_PER_STRUCT];
} pdirectory_t;

static pdirectory_t *curr_pd;

static void switch_pd(pdirectory_t *new){
    curr_pd = new;
    asm volatile("movl %0, %%cr3" :: "r"(new) : "memory");
}

void VMM_init() {
    /* default pt */
    ptable_t *table1 = alloc_page();
    /* used for identity mapping */
    ptable_t *table2 = alloc_page();

    memset(table1, 0, sizeof(ptable_t));
    memset(table2, 0, sizeof(ptable_t));

    /* linker symbols to denote start and end of kernel */
    extern uint8_t _begin, _end; 
    size_t kernel_size = &_end - &_begin;

    /* identity map the first 4MB */
    for (int i = 0; i < 1024; ++i){
        pt_entry_t *entry = &table2->entries[i];
        ENTRY_ADD_ATTRIBUTE(*entry, PAGE_STRUCT_ENTRY_PRESENT);
        ENTRY_SET_FRAME(*entry, i * 4096);
    }

    /* map physical 1MB to virtual 3GB */
    for (int i = 0, paddr = 0x00100000; i < 1024; ++i, paddr += 4096){
        pt_entry_t *entry = &table1->entries[i];
        ENTRY_ADD_ATTRIBUTE(*entry, PAGE_STRUCT_ENTRY_PRESENT);
        ENTRY_SET_FRAME(*entry, paddr);
    }

    /* default dt */
    pdirectory_t *pdir = alloc_page();
    memset(pdir, 0, sizeof *pdir);

    /* 1MB -> 3GB Page Table */
    pd_entry_t *pd_entry = &pdir->entries[PAGE_DIR_INDEX(0xC0000000)];
    ENTRY_ADD_ATTRIBUTE(*pd_entry, PAGE_STRUCT_ENTRY_PRESENT);
    ENTRY_ADD_ATTRIBUTE(*pd_entry, PAGE_STRUCT_ENTRY_WRITEABLE);
    ENTRY_SET_FRAME(*pd_entry, table1);

    /* 0-4MB -> 0-4MB Page Table */
    pd_entry = &pdir->entries[PAGE_DIR_INDEX(0x0)];
    ENTRY_ADD_ATTRIBUTE(*pd_entry, PAGE_STRUCT_ENTRY_PRESENT);
    ENTRY_ADD_ATTRIBUTE(*pd_entry, PAGE_STRUCT_ENTRY_WRITEABLE);
    ENTRY_SET_FRAME(*pd_entry, table2);

    switch_pd(pdir);

    asm volatile (
        "movl %%cr0, %%eax\n"
        "orl $0b10000000000000000000000000000000, %%eax\n"
        "movl %%eax, %%cr0\n"
        "jmp 1f\n" // where 'jump' occurs
        "1:\n"
    ::: "eax");

    pdir->entries[PAGE_DIR_INDEX(0x0)] = 0;
    switch_pd(pdir); // boot loop
}

Thanks so much for reading. I've been trying to figure this out for hours and I just don't get it, hopefully one of you more experienced developers will be able to help me. Have a great day!

Your inline asm jmp 1f is a no-op, it'll assemble with the rel8 displacement = 0. The assembler + linker don't know that you want 1: to move. You want jmp 1f + 3*1024*1024*1024 or something. Or 1f + 3*... - 1024*1024 since the distance is 3GiB - 1MiB if I'm reading you correctly that your kernel image starts at 1MiB initially, then will start at 3GiB. — Peter Cordes
– Peter Cordes, Commented Jun 1 at 3:31
@PeterCordes Thanks for your response! so you are just adding the 3GiB - 1MiB offset to the jump address? 1f + (3*1024*1024*1024 - 1024*1024) ? But, wouldn't I also have to change the address of the stack frame as that would be using the identity mapped addresses? IF I mentioned what you intended in the jmp block above, it still results in a bootloop. — user29935845
– user29935845, Commented Jun 1 at 3:36
Oh, yeah, you'd need to do something about stack space. Either by enabling paging and relocating your kernel before jumping to C code (including creating a simple page table), or perhaps by copying a few KiB starting at ESP to a new stack allocation in the high half. But if your C is compiled with a debug build that saves/restores EBP frame pointers, it will reset ESP back to the old value when it tries to return, so stack memory contents could be a problem. Prob. best to set up paging in an asm entry point. Maybe you could call C helper functions that return to that asm to make PDEs / PTEs — Peter Cordes
– Peter Cordes, Commented Jun 1 at 3:51
@PeterCordes understood. So it is best that I set up paging before I jump into the C-portion of the kernel? Also, not doubting your correctness here, but I see on many resources (including the osdev wiki) that a simple jump into the higher half is enough. This is obviously not working for me, so I am inclined to believe you. but what do you think? wiki.osdev.org/Higher_Half_x86_Bare_Bones — user29935845
– user29935845, Commented Jun 1 at 3:58
You're talking about the # Jump to higher half with an absolute jump. with lea 4f, %ecx / jmp *%ecx? The jump target is in a different .section; it's not just a simple jump. Look at the linker script. (It's also an indirect jump, using an absolute address. lea 4f, %ecx is using the absolute address the linker thinks the code should be loaded at. Even if they had put everything in the same section, it would work if everything before the jmp was position-independent. It's not, though, they use .bss.) — Peter Cordes
– Peter Cordes, Commented Jun 1 at 4:07

user29935845 · Accepted Answer · 2025-06-12 14:11:43Z

1

I figured it out after 2 weeks, and it turns out I was doing a lot of things wrong.

Paging should be set up BEFORE jumping into the C portion of the kernel
1. This is because after paging is set up and the identity mapping is invalidated, the kernel is mapped at the higher half and it would be a pain to manually switch all the loaded pointers
You have to link the kernel at 0xC0100000 but it should be physically loaded at 0x00100000 - this allows you to virtually “jump” to the higher half (but physically %eip is still pointing to the lower half)
1. The code that enables paging should map 0x00100000 to 0xC0100000, and 0x0 to 0xC0000000

answered Jun 12 at 14:11

user29935845

313 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

How to enable paging in x86 Protected Mode

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related