This article presents my working experience with the MontaVista Linux distribution (MontaVista Professional Edition 4.0 base on the 2.6.10 Linux kernel) running on a PowerPC processor board. It will present the physical memory mapping in Linux and its limitations using examples of a real problem and the suggested implemented solution that involved changes to kernel configuration.

Linux plays a significant role in embedded application development. It proved to be a platform of choice for soft real-time applications and with Linux distributions such as MontaVista Linux is being improved and hardened for mission-critical embedded application that require a hard real-time response.

Standard Linux (from www.kernel.org) supports two kinds of real-time schedulers (SCHED_RR, SCHED_FIFO) and the preemptive kernel. The hard real-time additions to the Linux kernel offered by MontaVista makes Linux ready to be a platform for developing hard real-time embedded applications. However, at this point, our goal is not the real-time capabilities of Linux but to address some general issues.

Our primary task was to bring up a system on a PC board in record time. However, the applications to be ported required more memory than an existing Linux platform could support. In general, the problem wasn't in the availability of the physical RAM but in the fact that the kernel could only see part of it. The board was equipped with more physical memory than the original kernel was able to access. Facing this problem, we had to modify the Linux kernel to get access to the whole available RAM.

The work started with a cross-development environment setting and some kernel modifications described below. However, before we begin, let's look at the Linux addressing.

Linux Address Space
Each process in the 32-bit address space can address 4GB of memory. This 4GB memory is divided into two parts:

• User space      ( from 0x0      to      PAGE_OFFSET - 1 )
• Kernel space      ( from PAGE_OFFSET      to 0xFFFFFFFF      )

Usually, the PAGE_OFFSET define is assigned the value of 0xC0000000. It means that the user address space covers 3GB of addresses starting from address 0x0 and the kernel space covers 1GB of addresses starting from address 0xC0000000 (see Figure 1).

The user can customize the value of PAGE_OFFSET at the kernel compilation time (the Base Address name is used when referring to it, as well).

It's very important to know that the physical RAM memory installed in the system is mapped to the addresses reserved to the kernel space (starting at PAGE_OFFSET). Therefore, if Linux is installed on the system with a huge amount of RAM, there may not be enough addressing space to do the mapping. So, some other mechanisms may be required to make the whole RAM visible.

Problem Description
Following the model described above, theoretically in the 32-bit addressing, the system can access a maximum 1GB of physical memory, assuming that the base address is set to 0xC0000000. So there's the question of what to do when there's more than 1GB of memory available, or some kernel areas are already used (leaving less that 1GB of address space for mapping).

To address this issue, recent Intel x86 microprocessors include a feature called "Physical Address Extension" or PAE that adds an extra four bits to the standard 32-bit physical address. Linux kernel version 2.4 took advantage of PAE to support up to 64GB of RAM. However, the linear address is still composed of 32 bits, so there's no permanent mapping of the whole amount of RAM.

Our board was equipped with the PowerPC processor and there was no need for a huge extension of RAM. In our case, only an extra 256MB of RAM visible to the system was sufficient.

In the next paragraphs, I will describe our solution to get the extra 256MB accessible to the system.

The step-by-step process of kernel configuration will be presented in detail as well.

Solution
When looking at the picture presented in Figure 1, there's one obvious solution. What will happen when the base address is changed from 0xC000000 to 0xA0000000? By moving the base address down by 0x20000000, there will be more address space mapped to the kernel and less to the user space. In this case we'll gain 256MB of memory access. This is exactly what we wanted (having more address space in the kernel to cover the extra RAM).

Figure 2 illustrates the address partition after the base address change. The modifications to the kernel were straightforward and done in no time as you'll see below.

Kernel Configuration
Before making the kernel modifications, the environment has to be set up for access to the proper cross-compilers and linkers. Remember that the embedded kernel will be built, not on the native environment. In our case the embedded platform was a board, running PowerPC processor and the development platform was based on Solaris (a desktop Linux could be used as well).

Figure 3 illustrates the development environment, which consists of:
a)  A Power PC board, where the Linux kernel produced will be executed
b)  A development system where the Linux kernel will be configured and built using MontaVista tools like cross-compilers, linkers, debuggers
c)  A serial connection between the target board (PowerPC) and the development platform (Solaris, Linux) The process of building the new kernel consisted of the following steps:

• Set up the environment
  All the tools and the kernel source code have to be collected from the MontaVista distribution. The right paths also have to be set up to point to the right directories, etc.
• Configure the kernel by executing (from the Linux base directory) % make menuconfig. Executing this command brings the menu seen in Figure 4.

  Next, the "Advance setup" option was chosen to change the default values to the customized values. Figure 5 illustrates the menu of the parameters to be changed.

  The following changes were made to this menu:
  

• "Set low memory size" to be modified to 0x40000000 (represents 1GB of RAM memory)

  This parameter specifies the size of the memory the system can see. In our case, the original value wasn't set to cover the whole range of 1GB addresses because there was already some address space taken for some other purposes.

• "Set custom kernel base address" to be modified to 0xA0000000

  The default base address is set to 0xC0000000.

The last change left to do was to modify the Makefile. Because the base address has been change, we have to adjust the "srec" generation accordingly. In this case, modify the address from 0x40500000 to 0x60500000. (See Figure 6.)

And that's all. What's left is to build the kernel and srec file by running the "make" command from the Linux base directory.

After the new kernel is built and burnt (over the serial connection) into the board, resetting the board will conclude the development process. Now, comes verifying if the kernel can see the whole memory. Before executing any more sophisticated tests the first way to verify the system is to run the "top" command. And in fact, the top display shows that the kernel can see all of the available RAM.

Conclusion
I'm sure that there are many different ways of solving the problem described here. The purpose of this article was to document one possible solution. Usually when a project requires fast results (as in our case) the most efficient way is best. Our example shows how easy the changes were to make following the clear theory behind it. And it shows how powerful open source can be when delivering a solution that requires some platform adjustments.

References
• Daniel P.Bovet and Marco Cesati.
Understanding the Linux Kernel.
• www.mvista.com/
• www.linuxjournal.com/