2005-04-17 07:20:36 +09:00
|
|
|
|
|
|
|
|
Virtual memory map with 4 level page tables:
|
|
|
|
|
|
2007-02-13 21:26:23 +09:00
|
|
|
0000000000000000 - 00007fffffffffff (=47 bits) user space, different per mm
|
2017-03-30 17:07:27 +09:00
|
|
|
hole caused by [47:63] sign extension
|
2014-09-19 04:56:06 +09:00
|
|
|
ffff800000000000 - ffff87ffffffffff (=43 bits) guard hole, reserved for hypervisor
|
2018-10-26 21:28:54 +09:00
|
|
|
ffff880000000000 - ffff887fffffffff (=39 bits) LDT remap for PTI
|
|
|
|
|
ffff888000000000 - ffffc87fffffffff (=64 TB) direct mapping of all phys. memory
|
|
|
|
|
ffffc88000000000 - ffffc8ffffffffff (=39 bits) hole
|
2009-05-06 06:28:56 +09:00
|
|
|
ffffc90000000000 - ffffe8ffffffffff (=45 bits) vmalloc/ioremap space
|
|
|
|
|
ffffe90000000000 - ffffe9ffffffffff (=40 bits) hole
|
|
|
|
|
ffffea0000000000 - ffffeaffffffffff (=40 bits) virtual memory map (1TB)
|
2015-02-14 07:39:25 +09:00
|
|
|
... unused hole ...
|
2016-08-23 17:00:45 +09:00
|
|
|
ffffec0000000000 - fffffbffffffffff (=44 bits) kasan shadow memory (16TB)
|
2005-04-17 07:20:36 +09:00
|
|
|
... unused hole ...
|
2018-01-04 20:32:03 +09:00
|
|
|
vaddr_end for KASLR
|
2018-01-04 21:01:40 +09:00
|
|
|
fffffe0000000000 - fffffe7fffffffff (=39 bits) cpu_entry_area mapping
|
|
|
|
|
fffffe8000000000 - fffffeffffffffff (=39 bits) LDT remap for PTI
|
x86-64, espfix: Don't leak bits 31:16 of %esp returning to 16-bit stack
The IRET instruction, when returning to a 16-bit segment, only
restores the bottom 16 bits of the user space stack pointer. This
causes some 16-bit software to break, but it also leaks kernel state
to user space. We have a software workaround for that ("espfix") for
the 32-bit kernel, but it relies on a nonzero stack segment base which
is not available in 64-bit mode.
In checkin:
b3b42ac2cbae x86-64, modify_ldt: Ban 16-bit segments on 64-bit kernels
we "solved" this by forbidding 16-bit segments on 64-bit kernels, with
the logic that 16-bit support is crippled on 64-bit kernels anyway (no
V86 support), but it turns out that people are doing stuff like
running old Win16 binaries under Wine and expect it to work.
This works around this by creating percpu "ministacks", each of which
is mapped 2^16 times 64K apart. When we detect that the return SS is
on the LDT, we copy the IRET frame to the ministack and use the
relevant alias to return to userspace. The ministacks are mapped
readonly, so if IRET faults we promote #GP to #DF which is an IST
vector and thus has its own stack; we then do the fixup in the #DF
handler.
(Making #GP an IST exception would make the msr_safe functions unsafe
in NMI/MC context, and quite possibly have other effects.)
Special thanks to:
- Andy Lutomirski, for the suggestion of using very small stack slots
and copy (as opposed to map) the IRET frame there, and for the
suggestion to mark them readonly and let the fault promote to #DF.
- Konrad Wilk for paravirt fixup and testing.
- Borislav Petkov for testing help and useful comments.
Reported-by: Brian Gerst <brgerst@gmail.com>
Signed-off-by: H. Peter Anvin <hpa@linux.intel.com>
Link: http://lkml.kernel.org/r/1398816946-3351-1-git-send-email-hpa@linux.intel.com
Cc: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
Cc: Borislav Petkov <bp@alien8.de>
Cc: Andrew Lutomriski <amluto@gmail.com>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Dirk Hohndel <dirk@hohndel.org>
Cc: Arjan van de Ven <arjan.van.de.ven@intel.com>
Cc: comex <comexk@gmail.com>
Cc: Alexander van Heukelum <heukelum@fastmail.fm>
Cc: Boris Ostrovsky <boris.ostrovsky@oracle.com>
Cc: <stable@vger.kernel.org> # consider after upstream merge
2014-04-30 08:46:09 +09:00
|
|
|
ffffff0000000000 - ffffff7fffffffff (=39 bits) %esp fixup stacks
|
|
|
|
|
... unused hole ...
|
2016-08-23 17:00:45 +09:00
|
|
|
ffffffef00000000 - fffffffeffffffff (=64 GB) EFI region mapping space
|
2015-11-28 06:09:35 +09:00
|
|
|
... unused hole ...
|
2016-08-23 17:00:45 +09:00
|
|
|
ffffffff80000000 - ffffffff9fffffff (=512 MB) kernel text mapping, from phys 0
|
2017-12-13 00:56:43 +09:00
|
|
|
ffffffffa0000000 - [fixmap start] (~1526 MB) module mapping space (variable)
|
|
|
|
|
[fixmap start] - ffffffffff5fffff kernel-internal fixmap range
|
|
|
|
|
ffffffffff600000 - ffffffffff600fff (=4 kB) legacy vsyscall ABI
|
2013-03-05 05:16:18 +09:00
|
|
|
ffffffffffe00000 - ffffffffffffffff (=2 MB) unused hole
|
2005-04-17 07:20:36 +09:00
|
|
|
|
2017-03-30 17:07:27 +09:00
|
|
|
Virtual memory map with 5 level page tables:
|
|
|
|
|
|
|
|
|
|
0000000000000000 - 00ffffffffffffff (=56 bits) user space, different per mm
|
|
|
|
|
hole caused by [56:63] sign extension
|
|
|
|
|
ff00000000000000 - ff0fffffffffffff (=52 bits) guard hole, reserved for hypervisor
|
2018-10-26 21:28:54 +09:00
|
|
|
ff10000000000000 - ff10ffffffffffff (=48 bits) LDT remap for PTI
|
|
|
|
|
ff11000000000000 - ff90ffffffffffff (=55 bits) direct mapping of all phys. memory
|
|
|
|
|
ff91000000000000 - ff9fffffffffffff (=3840 TB) hole
|
2017-12-13 00:56:44 +09:00
|
|
|
ffa0000000000000 - ffd1ffffffffffff (=54 bits) vmalloc/ioremap space (12800 TB)
|
2017-03-30 17:07:27 +09:00
|
|
|
ffd2000000000000 - ffd3ffffffffffff (=49 bits) hole
|
|
|
|
|
ffd4000000000000 - ffd5ffffffffffff (=49 bits) virtual memory map (512TB)
|
|
|
|
|
... unused hole ...
|
2017-09-29 23:08:18 +09:00
|
|
|
ffdf000000000000 - fffffc0000000000 (=53 bits) kasan shadow memory (8PB)
|
2017-03-30 17:07:27 +09:00
|
|
|
... unused hole ...
|
2018-01-04 20:32:03 +09:00
|
|
|
vaddr_end for KASLR
|
2018-01-04 21:01:40 +09:00
|
|
|
fffffe0000000000 - fffffe7fffffffff (=39 bits) cpu_entry_area mapping
|
|
|
|
|
... unused hole ...
|
2017-03-30 17:07:27 +09:00
|
|
|
ffffff0000000000 - ffffff7fffffffff (=39 bits) %esp fixup stacks
|
|
|
|
|
... unused hole ...
|
|
|
|
|
ffffffef00000000 - fffffffeffffffff (=64 GB) EFI region mapping space
|
|
|
|
|
... unused hole ...
|
|
|
|
|
ffffffff80000000 - ffffffff9fffffff (=512 MB) kernel text mapping, from phys 0
|
2017-12-29 01:06:20 +09:00
|
|
|
ffffffffa0000000 - fffffffffeffffff (1520 MB) module mapping space
|
2017-12-13 00:56:43 +09:00
|
|
|
[fixmap start] - ffffffffff5fffff kernel-internal fixmap range
|
|
|
|
|
ffffffffff600000 - ffffffffff600fff (=4 kB) legacy vsyscall ABI
|
2017-03-30 17:07:27 +09:00
|
|
|
ffffffffffe00000 - ffffffffffffffff (=2 MB) unused hole
|
|
|
|
|
|
|
|
|
|
Architecture defines a 64-bit virtual address. Implementations can support
|
|
|
|
|
less. Currently supported are 48- and 57-bit virtual addresses. Bits 63
|
2017-12-05 21:34:54 +09:00
|
|
|
through to the most-significant implemented bit are sign extended.
|
|
|
|
|
This causes hole between user space and kernel addresses if you interpret them
|
|
|
|
|
as unsigned.
|
2017-03-30 17:07:27 +09:00
|
|
|
|
2007-02-13 21:26:23 +09:00
|
|
|
The direct mapping covers all memory in the system up to the highest
|
2005-11-06 01:25:54 +09:00
|
|
|
memory address (this means in some cases it can also include PCI memory
|
2007-02-13 21:26:23 +09:00
|
|
|
holes).
|
2005-11-06 01:25:54 +09:00
|
|
|
|
2017-03-30 17:07:27 +09:00
|
|
|
vmalloc space is lazily synchronized into the different PML4/PML5 pages of
|
|
|
|
|
the processes using the page fault handler, with init_top_pgt as
|
2005-04-17 07:20:36 +09:00
|
|
|
reference.
|
|
|
|
|
|
2015-11-28 06:09:35 +09:00
|
|
|
We map EFI runtime services in the 'efi_pgd' PGD in a 64Gb large virtual
|
|
|
|
|
memory window (this size is arbitrary, it can be raised later if needed).
|
|
|
|
|
The mappings are not part of any other kernel PGD and are only available
|
|
|
|
|
during EFI runtime calls.
|
2013-11-01 01:25:08 +09:00
|
|
|
|
x86/mm: Implement ASLR for kernel memory regions
Randomizes the virtual address space of kernel memory regions for
x86_64. This first patch adds the infrastructure and does not randomize
any region. The following patches will randomize the physical memory
mapping, vmalloc and vmemmap regions.
This security feature mitigates exploits relying on predictable kernel
addresses. These addresses can be used to disclose the kernel modules
base addresses or corrupt specific structures to elevate privileges
bypassing the current implementation of KASLR. This feature can be
enabled with the CONFIG_RANDOMIZE_MEMORY option.
The order of each memory region is not changed. The feature looks at the
available space for the regions based on different configuration options
and randomizes the base and space between each. The size of the physical
memory mapping is the available physical memory. No performance impact
was detected while testing the feature.
Entropy is generated using the KASLR early boot functions now shared in
the lib directory (originally written by Kees Cook). Randomization is
done on PGD & PUD page table levels to increase possible addresses. The
physical memory mapping code was adapted to support PUD level virtual
addresses. This implementation on the best configuration provides 30,000
possible virtual addresses in average for each memory region. An
additional low memory page is used to ensure each CPU can start with a
PGD aligned virtual address (for realmode).
x86/dump_pagetable was updated to correctly display each region.
Updated documentation on x86_64 memory layout accordingly.
Performance data, after all patches in the series:
Kernbench shows almost no difference (-+ less than 1%):
Before:
Average Optimal load -j 12 Run (std deviation): Elapsed Time 102.63 (1.2695)
User Time 1034.89 (1.18115) System Time 87.056 (0.456416) Percent CPU 1092.9
(13.892) Context Switches 199805 (3455.33) Sleeps 97907.8 (900.636)
After:
Average Optimal load -j 12 Run (std deviation): Elapsed Time 102.489 (1.10636)
User Time 1034.86 (1.36053) System Time 87.764 (0.49345) Percent CPU 1095
(12.7715) Context Switches 199036 (4298.1) Sleeps 97681.6 (1031.11)
Hackbench shows 0% difference on average (hackbench 90 repeated 10 times):
attemp,before,after 1,0.076,0.069 2,0.072,0.069 3,0.066,0.066 4,0.066,0.068
5,0.066,0.067 6,0.066,0.069 7,0.067,0.066 8,0.063,0.067 9,0.067,0.065
10,0.068,0.071 average,0.0677,0.0677
Signed-off-by: Thomas Garnier <thgarnie@google.com>
Signed-off-by: Kees Cook <keescook@chromium.org>
Cc: Alexander Kuleshov <kuleshovmail@gmail.com>
Cc: Alexander Popov <alpopov@ptsecurity.com>
Cc: Andrew Morton <akpm@linux-foundation.org>
Cc: Andy Lutomirski <luto@kernel.org>
Cc: Aneesh Kumar K.V <aneesh.kumar@linux.vnet.ibm.com>
Cc: Baoquan He <bhe@redhat.com>
Cc: Boris Ostrovsky <boris.ostrovsky@oracle.com>
Cc: Borislav Petkov <bp@alien8.de>
Cc: Borislav Petkov <bp@suse.de>
Cc: Brian Gerst <brgerst@gmail.com>
Cc: Christian Borntraeger <borntraeger@de.ibm.com>
Cc: Dan Williams <dan.j.williams@intel.com>
Cc: Dave Hansen <dave.hansen@linux.intel.com>
Cc: Dave Young <dyoung@redhat.com>
Cc: Denys Vlasenko <dvlasenk@redhat.com>
Cc: Dmitry Vyukov <dvyukov@google.com>
Cc: H. Peter Anvin <hpa@zytor.com>
Cc: Jan Beulich <JBeulich@suse.com>
Cc: Joerg Roedel <jroedel@suse.de>
Cc: Jonathan Corbet <corbet@lwn.net>
Cc: Josh Poimboeuf <jpoimboe@redhat.com>
Cc: Juergen Gross <jgross@suse.com>
Cc: Kirill A. Shutemov <kirill.shutemov@linux.intel.com>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Lv Zheng <lv.zheng@intel.com>
Cc: Mark Salter <msalter@redhat.com>
Cc: Martin Schwidefsky <schwidefsky@de.ibm.com>
Cc: Matt Fleming <matt@codeblueprint.co.uk>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Stephen Smalley <sds@tycho.nsa.gov>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Toshi Kani <toshi.kani@hpe.com>
Cc: Xiao Guangrong <guangrong.xiao@linux.intel.com>
Cc: Yinghai Lu <yinghai@kernel.org>
Cc: kernel-hardening@lists.openwall.com
Cc: linux-doc@vger.kernel.org
Link: http://lkml.kernel.org/r/1466556426-32664-6-git-send-email-keescook@chromium.org
Signed-off-by: Ingo Molnar <mingo@kernel.org>
2016-06-22 09:47:02 +09:00
|
|
|
Note that if CONFIG_RANDOMIZE_MEMORY is enabled, the direct mapping of all
|
|
|
|
|
physical memory, vmalloc/ioremap space and virtual memory map are randomized.
|
|
|
|
|
Their order is preserved but their base will be offset early at boot time.
|
2018-01-04 20:32:03 +09:00
|
|
|
|
|
|
|
|
Be very careful vs. KASLR when changing anything here. The KASLR address
|
|
|
|
|
range must not overlap with anything except the KASAN shadow area, which is
|
|
|
|
|
correct as KASAN disables KASLR.
|