Computer memory from the ground up (part 1)

• WordPress

Disclaimer This is an introductory post about how memory and CPU work together. Thus I have simplified some concepts and skipped over others in order to keep this post easy to follow. If you are an specialist and work developing OS, bootloaders or BSPs, then you will find this content very basic.

In these days of cloud computing and containers, physical memory seems a very distant concept. However understanding how memory is physically attached and accessed from a CPU gives us a big advantage to understand why some things go wrong.

A modern computer is composed of many layers. Many of these layers exist to simplify the configuration of the system, by for example checking out the memory and counting how much memory is available or by checking out the available devices. Without them, the configuration of the system would require a lot of manual intervention.

For the purpose of this blog entry we will analyze a simple model, with a simple 32 bit CPU directly connected to memory devices. We will look at both ROM and RAM memories, to understand how they differ. At the end of this blog post, I will cover some of the layers that exist to automate the configuration.

The connections

Every CPU communicates with the outside world by using PINs. A modern CPU has more than hundred connection PINs. These PINs can be roughly be classified into the following categories:

• Power and clock
• Control
• Address lines
• Data lines
• General Purpose IO (GPIO)

Power and clock refers to the PINs that are used to power up the CPU and to take the input from the oscillator in the board.

Control lines is a big category that can be subdivided into:

• Chip select lines (CSn), used to select the device the address, control and data lines communicate with.
• Read and Write control, to indicate whether we want to read or write from a device.
• DRAM/SRAM control (where available)
• Specialized lines, such as I2C lines

Address lines are the lines that carry the address information out to devices.

Data lines are used to either read from or write to a device.

GPIO lines are used to support devices that might not respond to the canonical CS/Address/Data combination. In some cases, GPIO can be used to provide extra lines for CSn, address or even data.

Important concepts

One typical source of confusion is the size of the memory and width of the data. There is the perception that if a CPU is 32 bits, then it can address 2³²-1 bytes of 32 bits words.

The size of the addressable memory is called the memory space and is controlled by the available address lines. Usually a CPU has address lines that are the same width as the data, but this is not a requirement.

In addition, the memory space is shared between several devices. This means that the available memory space is usually smaller since some devices might occupy some of the space. The most common example is the hole that exists in X86 cpus after the 640KB mark and up to 1MB.

The configuration and mapping

During CPU startup, the bootstrap sequence has to configure the CPU and the corresponding address mappings. In our system we have one CPU with one ROM memory and one RAM memory connected. Therefore the bootstrap sequence has to configure the CPU to use CS0 to talk to the ROM memory and CS1 to talk to the RAM memory.

The next step is to tell the CPU how to map the memory addresses for the memories. The address mapping refers to the range where the CPU will automatically signal the corresponding CSn line.

For example, let’s assume that our ROM memory contains the code to be run and is 1MB in size. We can then map the ROM memory to start at 0xF0000000 and last for 1MB.

Our RAM memory is 16 MB in size and we will map it to start at 0x00000000 and last for 16MB.

All accesses below 0x00F42400 will therefore select CS1 and read or write from the RAM memory. All accesses above 0xF0000000 will select CS2 and read from the ROM memory.

The need for buses

It is easy to see that just using chip selects, address and data lines is not going to carry us that far. Sure, it covers many cases and if you work on embedded systems, you might not need to use another way of communicating with memory or devices.

In order to extend the communication possibilities, there are several bus types that can be used to communicate with devices hiding the complexity. The CPU is connected to the bus controller by using a chip select and address and data lines, or maybe a specialized communication interface, such as SPI.

By doing this we hide the complexity of handling devices and only communicate with the bus controller. If you do not believe me, think about a world without USB! (for those of us as old as me, we remember the days of Plug n Play and before, when we needed to deal with setting IRQs and other parameters by hand in our computers).

Pages and segments, why do I care?

As programmers we are used to think of memory as a linear array. In real life, memory is more like a collection of clothes patched together.

Each cloth is a piece of memory, and the OS has the responsibility of patching the different clothes together and make it appear as a linear array.

To achieve this a CPU divides the memory in segments of memory that can be handled separately of each other. For example by adding protection bits such as read, write and execute.

Old CPUs, such as the 80286, allowed the management of memory segments of different size. Modern CPUs use a fixed segment size which is a called a memory page.

Supervisor and user modes

CPUs usually have two modes of operation: supervisor and user mode. In supervisor mode, the CPU has access to all registers and can perform instructions that are restricted. Memory in supervisor mode is seen as the hardware presents it, including any gaps between banks.

In user mode, the CPU can only access a subset of the registers, and can only perform non restricted instructions. In addition the memory in user mode is usually seen linearly starting from 0 and upwards.

One word of caution. To present the memory as a linear array, the CPU needs to have what is called a Memory Management Unit (MMU). Many small processors do not have such units and therefore the memory looks the same in supervisor and in user mode.

MMU and virtual memory

A memory management unit is a component that is in charge of mapping the physical memory to a virtual view of the memory. It also enforces the protection levels configured for the memory.

Let’s assume that our CPU is capable of addressing pages of size 64 KiB. The hexadecimal representation for 64 KiB is 0x0000FFFF. Given that our CPU is a 32 bits CPU, then we only need 16 bits to point to a byte in a page. This leaves the 16 upper (or lower depending on the endianess) bits available to select the page we want to access. These bits are usually called the page prefix, page name or page number.

The next diagram is a very simple and naive representation of a page table with a very simple memory layout.

The page table contains two entries. The memory available for the process starts at 0x00000000 and end at 0x0001FFFF. However the physical address for the corresponding memory are completely different and the translation happens automatically when the address is referred by the user space.

For example, if the process refers to the memory at 0x00010000 then the MMU will translate the address to the physical address 0x00D00000 and access the hardware accordingly.

What now?

The next post will cover what happens when a process starts. We will look at the loading of the process code and how memory is managed inside the process.