Colocation to Virtualization

A page is really virtual memory which is managed by the Translation Lookaside Buffers(TLB) in the CPU. The TLB controls the mapping of the virtual memory pages to physical memory addresses. In doing so, it bypasses the kernel virtual memory manager.

Per RedHat,

The TLB is a limited hardware resource, so utilising a huge amount of physical memory with the default page size consumes the TLB and adds processing overhead – many pages of size 4096 Bytes equates to many TLB resources consumed.

This is where Huge Pages come in. Pages are created at a larger size than the default 4096 bytes, and each page will consume only one TLB resource. So you can see this is a huge benefit. Using Huge Pages decrease the number of TLB resources required.

Side Affect
This is great, depending on what you are trying to accomplish. Once the physical memory is mapped to a Huge Page, it can no longer be used for “normal” memory allocation. This is because the memory is no longer mapped by the kernel virtual memory manager. The applications that you want to dedicate the Huge Pages to have to have support for them.

Benefit
So here is the best part of Huge Pages. It is dedicated memory to be used by only applications that request them. This dedicated memory is stored in physical RAM and will NEVER be swapped out! Thus, guaranteeing a level of performance. When memory is swapped to disk, it’s a lot slower than RAM and decreases the performance of the process(s)/program(s) gets pushed there.

Now knowing that Huge Pages are stored in RAM, this also means that the allocated RAM is dedicated. This is a little bit redundant to the above, but I want to make sure this point is clear.

Example: If a server has 8gigs of RAM and 5gigs are allocated to Huge Pages, that only leaves 3gigs for all other processes, programs, and underlining operating system to use.

Below shows my Linux desktop that has the default page size of 4096 set

user@workstation:~$ cat /proc/meminfo | grep Huge
HugePages_Total:       0
HugePages_Free:        0
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       4096 kB

So as you can see, I have no Huge Pages reserved or in use. The next example is from a production Oracle database server

[root@OracleServer1 ~]# cat /proc/meminfo | grep Huge
HugePages_Total: 12200
HugePages_Free:     85
Hugepagesize:     2048 kB

So to calculate the space dedicated to Huge Pages from above, it is 12,200 x 2048 kB which gives us

24 985 600 kilobytes = 23.828125 gigabytes

In the 2.6x Linux kenel, Huge Pages are enabled using the CONFIG_HUGETLB_PAGE feature when compiling the kernel. Most “Enterprise” Linux OSs by default have this enabled. The ones that I know of are RedHat, CentOS, and possibly Fedora from version 4+.

Notes: Again, applications that you want to dedicate Huge Pages to must have support for them. Most memory intensive ones do, but check for this first.

~ by Kevin Goodman on October 13, 2009.

Posted in Filesystems, Linux
Tags: centos, database, fedora, Huge Pages, hugepage, hugepages, LinkedIn, Linux, memory, oracle, performance, ram, redhat, storage, swap, TLB, Translation Lookaside Buffers, ubuntu, virtual memory