It's been a while since I last wrote an OS. Today, I'm mainly here to address some issues I've forgotten and the problems encountered while implementing the minimal U mode.

Revisiting Memory Mapping Concepts

The main problems I currently forget about are still related to page tables, memory, and mapping.

High Address vs. Low Address: VA or PA?

In memory mapping, the high address and low address — do these two addresses refer to VA or PA?

It's VA. PA does not have high or low address distinctions. Memory is allocated from around 0x80000000 and gradually increments from there.

However, it should be noted that in the page table, the mapped address must be a real PA, that is, there is no distinction between VA or PA. It's like I am creating a linked list to store memory addresses after raising the entire address (which may have been a wrong choice), but this address is already a high address, so it still needs to be converted to a low address.

Page Table Mapping in the Kernel

Will the Newly Allocated Page Table Be Mapped to the Kernel Page Table?

It is abundantly clear that during the earliest stages of kernel boot when paging was enabled, we cast a wide net. By intertwining the identity mapping and the high-half address mapping, we essentially created a clever "bijection" between physical memory and virtual space.

Since this grand mapping scheme is already deeply imprinted in the kernel page table, every piece of memory we subsequently pull from the freelist already possesses a valid kernel virtual coordinate. In other words, the newly allocated user page table is tenderly "embraced" by the kernel page table from the very moment of its birth.

Implementing U-Mode: A Step-by-Step Guide

The general approach to implementing U-mode is as follows:

First, after correctly configuring the hardware exception delegation and completing the prerequisite initializations in S-mode, we prepare the transition to U-mode.

Step 1: Allocate Physical Memory Pages

Allocate two physical memory pages:

• One to store our user program machine code (here, we inject ecall and j . for a minimal test payload)
• The other to serve as the user execution stack

Step 2: Construct the User-Mode Page Table

Instead of using a xv6-style trampoline page, we adopt a shared-kernel-memory layout similar to Linux. We achieve this by copying the high-half address space (indexes 256 to 511) of the kernel's top-level page table directly into the newly allocated user page table.

Note: You might wonder how we keep this kernel mapping synchronized across page tables; we will implement this synchronization mechanism later.

Step 3: Map Memory Regions

We map the user code and stack regions into the user page table. It is important to place the stack at a relatively high virtual address since the stack grows downwards towards lower addresses in the RISC-V architecture.

Step 4: Configure State Registers for Privilege Switch

We perform the following configurations:

1. Modify the sstatus register to set the target privilege level to U-mode
2. Write the virtual address of the user code into the sepc register
3. Execute sfence_vma
4. Write the new user page table address into the satp register to switch the virtual address space

Step 5: Split the Trap Vectors

Since we must handle system calls and interrupts after entering U-mode, we point stvec to a dedicated uservec routine.

Why can't we reuse kernelvec?

The primary reason is that kernelvec immediately executes addi sp, sp, -256. If the trap originates from U-mode, sp points to the user stack. The S-mode kernel accessing a user stack (which has PTE_U permissions) will trigger a fatal Store Page Fault and a double-fault deadlock.

Therefore, uservec must first execute csrrw sp, sscratch, sp to safely swap in the kernel stack pointer.

Step 6: Execute the Privilege Drop

We use inline assembly to:

1. Back up the current kernel sp into sscratch
2. Overwrite sp with our newly established user stack top

⚠️ Critical Precaution: While writing this segment, do not call any C functions (such as printf) after sp has been modified. C functions implicitly push data onto the stack, and doing so while sp points to a user page in S-mode will instantly trigger a fatal page fault.

Finally, we execute sret to make the leap of faith, successfully dropping into U-mode!

Key Insight: Understanding Trap Vectors

I found an interesting point: both kernelvec and uservec are essentially for handling interrupts in S-mode, and uservec is just a special handling for U-mode.

Addressing the Stack Pointer Mystery

The Question

Why is the sp swapped out by csrrw at the top of the kernel stack? Why is it a blank 4KB page? I obviously didn't allocate a blank page, and while csrrw is saving sp, it's preparing for U mode. How can it be at the top of the stack?

The Problem

Well, the kernel stack pointer we swap out using csrrw is definitely not sitting on some clean, blank page, nor is there a 100% guarantee that it has 256 bytes left to save all the registers. Honestly, it's basically a game of chance, and we're just praying there's enough space left on that stack. 😂

However, we can't build our kernel on top of probabilistic gambling. As any developer knows, if a bug isn't fully reproducible, debugging it is an absolute nightmare.

The Solution

Therefore, the ultimate fix is to:

1. Allocate a dedicated, blank 4KB page specifically for handling traps
2. Point the stack pointer directly to the absolute top of this new page

This gives us a deterministic, absolute guarantee of having at least 4KB of safe space to work with, rather than crossing our fingers and praying the OS has enough headroom left.

Ah, much better! ✨