This series is a follow-up to the discussion that has started here [1]. While standalone, it also provides primitives reusable for the VPR DMA heap.
When memory is lent to the Secure world via FF-A, CPU speculative accesses from NS to the lent pages can still occur as long as it retains a cacheable mapping to it.
Ideally, lent memory would be "no-map" but that would mean giving up MiBs of useful memory, so let's try to do better with the help of a CMA pool.
On arm64, modifying the direct map at runtime is generally restricted because the linear map defaults to block mapping and splitting blocks at runtime may trigger fatal page fault, unless the CPU implements BBML3 or the entire direct map was mapped at page granularity from boot. Forcing last-level mappings system-wide incurs a severe penalty we want to avoid. Instead, this series introduces targeted last-level mappings for designated memory regions, along with the "arm,ffa-lend-pool" CMA driver to manage unmapping and remapping on lend/reclaim transitions:
1. memblock & OF reserved memory ("ll-map"): - Introduce MEMBLOCK_LLMAP and the DT "ll-map" property for reserved-memory nodes to force last-level (PTE) mappings only for a specific region.
2. set_memory infrastructure: - Introduce can_set_direct_map_range() to check if a specific address range is mapped with last-level entries and can be modified safely. - Introduce __set_direct_map_*() variants that bypass redundant checks when the caller has already validated the range.
3. "arm,ffa-lend-pool" driver - Introduce the "arm,ffa-lend-pool" CMA reserved-memory driver, which unmaps pages prior to lending (ffa_prepare_lend()) and restores them when reclaimed (ffa_lend_reclaimed()).
4. Optee support - Hook OP-TEE dynamic protected memory pools to "arm,ffa-lend-pool" for both SMC (via DT memory-region phandle) and FF-A (via ffa_lend_pool_attach()) transports.
Testing: ========
Tested with QEMU v8 using OP-TEE OS (built with CFG_CORE_DYN_PROTMEM=y) under both SMC and FF-A transports [2]
static void dump_direct_map(const char *label) { printf("\n=== %s ===\n", label); fflush(stdout); system("sed -n '/Linear Mapping start/,/Linear Mapping end/p' /sys/kernel/debug/kernel_page_tables"); fflush(stdout); }
int main(int argc, char *argv[]) { int heap_fd; int dmabuf_fd; struct dma_heap_allocation_data data = { 0 }; size_t size = 1024 * 1024; /* 1MB */
if (argc > 1) size = strtoul(argv[1], NULL, 0);
dump_direct_map("BEFORE ALLOCATION");
heap_fd = open("/dev/dma_heap/protected,secure-video", O_RDWR); if (heap_fd < 0) { perror("open /dev/dma_heap/protected,secure-video"); return 1; }
printf("\nOpened /dev/dma_heap/protected,secure-video\n"); printf("Allocating %zu bytes of protected memory via DMA heap...\n", size);
data.len = size; data.fd_flags = O_RDWR | O_CLOEXEC; if (ioctl(heap_fd, DMA_HEAP_IOCTL_ALLOC, &data) < 0) { perror("ioctl DMA_HEAP_IOCTL_ALLOC"); close(heap_fd); return 1; }
dmabuf_fd = data.fd; printf("Successfully allocated %zu bytes! dmabuf_fd = %d\n", size, dmabuf_fd);
dump_direct_map("DURING LEND (EXPECT HOLE IN DIRECT MAP)");
printf("\nReleasing dmabuf_fd...\n"); close(dmabuf_fd); close(heap_fd);
dump_direct_map("AFTER RECLAIM (RESTORED DIRECT MAP)");
return 0; }
[1] https://lore.kernel.org/all/20260807-tegra-vpr-v4-7-5510d16af89e@nvidia.com/ [2] https://optee.readthedocs.io/en/latest/building/gits/build.html#qemu-v8
Vincent Donnefort (10): memblock: Introduce MEMBLOCK_LLMAP of: reserved_mem: Introduce "ll-map" property set_memory.h: Introduce can_set_direct_map_range() set_memory.h: Introduce __set_direct_map*() arm64: can_set_direct_map() if BBML3 arm64: Implement can_set_direct_map_range() arm64: Implement __set_direct_map*() arm64: Add support for MEMBLOCK_LLMAP firmware: arm_ffa: Introduce ffa-lend-pool optee: Add support for arm,ffa-lend-pool
arch/arm64/include/asm/set_memory.h | 7 + arch/arm64/mm/mmu.c | 23 ++- arch/arm64/mm/pageattr.c | 67 +++++++- drivers/firmware/arm_ffa/Kconfig | 5 + drivers/firmware/arm_ffa/Makefile | 1 + drivers/firmware/arm_ffa/lend_pool.c | 223 +++++++++++++++++++++++++++ drivers/of/of_reserved_mem.c | 104 ++++++++++--- drivers/tee/optee/ffa_abi.c | 13 +- drivers/tee/optee/protmem.c | 8 - drivers/tee/optee/smc_abi.c | 17 +- drivers/tee/tee_shm.c | 11 +- include/linux/arm_ffa.h | 21 +++ include/linux/memblock.h | 9 ++ include/linux/set_memory.h | 39 +++++ mm/memblock.c | 50 ++++++ 15 files changed, 545 insertions(+), 53 deletions(-) create mode 100644 drivers/firmware/arm_ffa/lend_pool.c
base-commit: cee9395acd8043be0644b25c34bfa86623f2b935
Keeping last-level mappings is interesting on some architectures as it allows mapping/unmapping pages from the kernel direct map without the risk of splitting blocks which, under the break-before-make rule, may trigger page-faults the kernel can't handle.
However, mapping the entire direct map at PTE-level is costly. So instead, create a new memblock flag MEMBLOCK_LLMAP to enable the system to decide which region must be covered by mappings up to the last-level.
Signed-off-by: Vincent Donnefort vdonnefort@google.com
diff --git a/include/linux/memblock.h b/include/linux/memblock.h index d62db9e776cf..d40a5ded188d 100644 --- a/include/linux/memblock.h +++ b/include/linux/memblock.h @@ -52,6 +52,7 @@ extern unsigned long long max_possible_pfn; * kernel that we know is good to use. It is the only memory that * allocations may happen from in this phase. * @MEMBLOCK_RSRV_HUGETLB: memory is reserved for hugetlb pages + * @MEMBLOCK_LLMAP: memory region to be mapped using last-level mapping */ enum memblock_flags { MEMBLOCK_NONE = 0x0, /* No special request */ @@ -63,6 +64,7 @@ enum memblock_flags { MEMBLOCK_RSRV_KERN = 0x20, /* memory reserved for kernel use */ MEMBLOCK_KHO_SCRATCH = 0x40, /* scratch memory for kexec handover */ MEMBLOCK_RSRV_HUGETLB = 0x80, /* memory reserved for hugetlb pages */ + MEMBLOCK_LLMAP = 0x100,/* last-level mapping */ };
/** @@ -160,6 +162,8 @@ int memblock_reserved_mark_noinit(phys_addr_t base, phys_addr_t size); int memblock_reserved_mark_kern(phys_addr_t base, phys_addr_t size); int memblock_mark_kho_scratch(phys_addr_t base, phys_addr_t size); int memblock_clear_kho_scratch(phys_addr_t base, phys_addr_t size); +int memblock_mark_llmap(phys_addr_t base, phys_addr_t size); +int memblock_clear_llmap(phys_addr_t base, phys_addr_t size);
void memblock_free(void *ptr, size_t size); void reset_all_zones_managed_pages(void); @@ -306,6 +310,11 @@ static inline bool memblock_is_kho_scratch(struct memblock_region *m) return m->flags & MEMBLOCK_KHO_SCRATCH; }
+static inline bool memblock_is_llmap(struct memblock_region *m) +{ + return m->flags & MEMBLOCK_LLMAP; +} + int memblock_search_pfn_nid(unsigned long pfn, unsigned long *start_pfn, unsigned long *end_pfn); void __next_mem_pfn_range(int *idx, int nid, unsigned long *out_start_pfn, diff --git a/mm/memblock.c b/mm/memblock.c index 9ce86349a29f..1591b50503ed 100644 --- a/mm/memblock.c +++ b/mm/memblock.c @@ -1119,6 +1119,16 @@ int __init_memblock memblock_mark_mirror(phys_addr_t base, phys_addr_t size) */ int __init_memblock memblock_mark_nomap(phys_addr_t base, phys_addr_t size) { + struct memblock_region *r; + + memblock_cap_size(base, &size); + + for_each_mem_region(r) { + if (memblock_is_llmap(r) && + memblock_addrs_overlap(base, size, r->base, r->size)) + return -EINVAL; + } + return memblock_setclr_flag(&memblock.memory, base, size, 1, MEMBLOCK_NOMAP); }
@@ -1204,6 +1214,45 @@ __init int memblock_clear_kho_scratch(phys_addr_t base, phys_addr_t size) MEMBLOCK_KHO_SCRATCH); }
+/** + * memblock_mark_llmap - Mark a memory region with flag MEMBLOCK_LLMAP. + * @base: the base phys addr of the region + * @size: the size of the region + * + * If supported by the architecture, such region is mapped at the last-level in + * the kernel direct map. + * + * Return: 0 on success, -errno on failure. + */ +int __init_memblock memblock_mark_llmap(phys_addr_t base, phys_addr_t size) +{ + struct memblock_region *r; + + memblock_cap_size(base, &size); + + for_each_mem_region(r) { + if (memblock_is_nomap(r) && + memblock_addrs_overlap(base, size, r->base, r->size)) + return -EINVAL; + } + + return memblock_setclr_flag(&memblock.memory, base, size, 1, + MEMBLOCK_LLMAP); +} + +/** + * memblock_clear_llmap - Clear flag MEMBLOCK_LLMAP for a specified region. + * @base: the base phys addr of the region + * @size: the size of the region + * + * Return: 0 on success, -errno on failure. + */ +int __init_memblock memblock_clear_llmap(phys_addr_t base, phys_addr_t size) +{ + return memblock_setclr_flag(&memblock.memory, base, size, 0, + MEMBLOCK_LLMAP); +} + static bool should_skip_region(struct memblock_type *type, struct memblock_region *m, int nid, int flags) @@ -2886,6 +2935,7 @@ static const char * const flagname[] = { [ilog2(MEMBLOCK_RSRV_NOINIT)] = "RSV_NIT", [ilog2(MEMBLOCK_RSRV_KERN)] = "RSV_KERN", [ilog2(MEMBLOCK_KHO_SCRATCH)] = "KHO_SCRATCH", + [ilog2(MEMBLOCK_LLMAP)] = "LLMAP", };
static int memblock_debug_show(struct seq_file *m, void *private)
On Wed, Sep 02, 2026 at 11:47:03AM +0100, Vincent Donnefort wrote:
Keeping last-level mappings is interesting on some architectures as it allows mapping/unmapping pages from the kernel direct map without the risk of splitting blocks which, under the break-before-make rule, may trigger page-faults the kernel can't handle.
However, mapping the entire direct map at PTE-level is costly. So instead, create a new memblock flag MEMBLOCK_LLMAP to enable the system
I believe MEMBLOCK_PTE_MAP sounds more descriptive.
to decide which region must be covered by mappings up to the last-level.
Signed-off-by: Vincent Donnefort vdonnefort@google.com
diff --git a/include/linux/memblock.h b/include/linux/memblock.h index d62db9e776cf..d40a5ded188d 100644 --- a/include/linux/memblock.h +++ b/include/linux/memblock.h @@ -52,6 +52,7 @@ extern unsigned long long max_possible_pfn;
- kernel that we know is good to use. It is the only memory that
- allocations may happen from in this phase.
- @MEMBLOCK_RSRV_HUGETLB: memory is reserved for hugetlb pages
*/
- @MEMBLOCK_LLMAP: memory region to be mapped using last-level mapping
enum memblock_flags { MEMBLOCK_NONE = 0x0, /* No special request */ @@ -63,6 +64,7 @@ enum memblock_flags { MEMBLOCK_RSRV_KERN = 0x20, /* memory reserved for kernel use */ MEMBLOCK_KHO_SCRATCH = 0x40, /* scratch memory for kexec handover */ MEMBLOCK_RSRV_HUGETLB = 0x80, /* memory reserved for hugetlb pages */
- MEMBLOCK_LLMAP = 0x100,/* last-level mapping */
}; /** @@ -160,6 +162,8 @@ int memblock_reserved_mark_noinit(phys_addr_t base, phys_addr_t size); int memblock_reserved_mark_kern(phys_addr_t base, phys_addr_t size); int memblock_mark_kho_scratch(phys_addr_t base, phys_addr_t size); int memblock_clear_kho_scratch(phys_addr_t base, phys_addr_t size); +int memblock_mark_llmap(phys_addr_t base, phys_addr_t size); +int memblock_clear_llmap(phys_addr_t base, phys_addr_t size); void memblock_free(void *ptr, size_t size); void reset_all_zones_managed_pages(void); @@ -306,6 +310,11 @@ static inline bool memblock_is_kho_scratch(struct memblock_region *m) return m->flags & MEMBLOCK_KHO_SCRATCH; } +static inline bool memblock_is_llmap(struct memblock_region *m) +{
- return m->flags & MEMBLOCK_LLMAP;
+}
int memblock_search_pfn_nid(unsigned long pfn, unsigned long *start_pfn, unsigned long *end_pfn); void __next_mem_pfn_range(int *idx, int nid, unsigned long *out_start_pfn, diff --git a/mm/memblock.c b/mm/memblock.c index 9ce86349a29f..1591b50503ed 100644 --- a/mm/memblock.c +++ b/mm/memblock.c @@ -1119,6 +1119,16 @@ int __init_memblock memblock_mark_mirror(phys_addr_t base, phys_addr_t size) */ int __init_memblock memblock_mark_nomap(phys_addr_t base, phys_addr_t size) {
- struct memblock_region *r;
- memblock_cap_size(base, &size);
- for_each_mem_region(r) {
if (memblock_is_llmap(r) &&memblock_addrs_overlap(base, size, r->base, r->size))return -EINVAL;- }
I'm not very fond of implicit skips here. memblock has no idea what's the caller intention, maybe it actually wants to change the memory from nomap to pte-mapped.
I'd rather warn in memblock_is_nomap() and memblock_is_llmap() if they both are set and let the caller deal with making sure they are not.
- return memblock_setclr_flag(&memblock.memory, base, size, 1, MEMBLOCK_NOMAP);
} @@ -1204,6 +1214,45 @@ __init int memblock_clear_kho_scratch(phys_addr_t base, phys_addr_t size) MEMBLOCK_KHO_SCRATCH); } +/**
- memblock_mark_llmap - Mark a memory region with flag MEMBLOCK_LLMAP.
- @base: the base phys addr of the region
- @size: the size of the region
- If supported by the architecture, such region is mapped at the last-level in
- the kernel direct map.
- Return: 0 on success, -errno on failure.
- */
+int __init_memblock memblock_mark_llmap(phys_addr_t base, phys_addr_t size) +{
- struct memblock_region *r;
- memblock_cap_size(base, &size);
- for_each_mem_region(r) {
if (memblock_is_nomap(r) &&memblock_addrs_overlap(base, size, r->base, r->size))return -EINVAL;- }
same here
- return memblock_setclr_flag(&memblock.memory, base, size, 1,
MEMBLOCK_LLMAP);+}
+/**
- memblock_clear_llmap - Clear flag MEMBLOCK_LLMAP for a specified region.
- @base: the base phys addr of the region
- @size: the size of the region
- Return: 0 on success, -errno on failure.
- */
+int __init_memblock memblock_clear_llmap(phys_addr_t base, phys_addr_t size) +{
- return memblock_setclr_flag(&memblock.memory, base, size, 0,
MEMBLOCK_LLMAP);+}
static bool should_skip_region(struct memblock_type *type, struct memblock_region *m, int nid, int flags) @@ -2886,6 +2935,7 @@ static const char * const flagname[] = { [ilog2(MEMBLOCK_RSRV_NOINIT)] = "RSV_NIT", [ilog2(MEMBLOCK_RSRV_KERN)] = "RSV_KERN", [ilog2(MEMBLOCK_KHO_SCRATCH)] = "KHO_SCRATCH",
- [ilog2(MEMBLOCK_LLMAP)] = "LLMAP",
}; static int memblock_debug_show(struct seq_file *m, void *private) -- 2.55.0.970.g62bdec98f9-goog
On Sun, Sep 06, 2026 at 10:33:11PM +0300, Mike Rapoport wrote:
On Wed, Sep 02, 2026 at 11:47:03AM +0100, Vincent Donnefort wrote:
Keeping last-level mappings is interesting on some architectures as it allows mapping/unmapping pages from the kernel direct map without the risk of splitting blocks which, under the break-before-make rule, may trigger page-faults the kernel can't handle.
However, mapping the entire direct map at PTE-level is costly. So instead, create a new memblock flag MEMBLOCK_LLMAP to enable the system
I believe MEMBLOCK_PTE_MAP sounds more descriptive.
The idea was to have something close from NOMAP, to emphasis it is one or the other. But PTE_MAP sounds good too.
to decide which region must be covered by mappings up to the last-level.
Signed-off-by: Vincent Donnefort vdonnefort@google.com
diff --git a/include/linux/memblock.h b/include/linux/memblock.h index d62db9e776cf..d40a5ded188d 100644 --- a/include/linux/memblock.h +++ b/include/linux/memblock.h @@ -52,6 +52,7 @@ extern unsigned long long max_possible_pfn;
- kernel that we know is good to use. It is the only memory that
- allocations may happen from in this phase.
- @MEMBLOCK_RSRV_HUGETLB: memory is reserved for hugetlb pages
*/
- @MEMBLOCK_LLMAP: memory region to be mapped using last-level mapping
enum memblock_flags { MEMBLOCK_NONE = 0x0, /* No special request */ @@ -63,6 +64,7 @@ enum memblock_flags { MEMBLOCK_RSRV_KERN = 0x20, /* memory reserved for kernel use */ MEMBLOCK_KHO_SCRATCH = 0x40, /* scratch memory for kexec handover */ MEMBLOCK_RSRV_HUGETLB = 0x80, /* memory reserved for hugetlb pages */
- MEMBLOCK_LLMAP = 0x100,/* last-level mapping */
}; /** @@ -160,6 +162,8 @@ int memblock_reserved_mark_noinit(phys_addr_t base, phys_addr_t size); int memblock_reserved_mark_kern(phys_addr_t base, phys_addr_t size); int memblock_mark_kho_scratch(phys_addr_t base, phys_addr_t size); int memblock_clear_kho_scratch(phys_addr_t base, phys_addr_t size); +int memblock_mark_llmap(phys_addr_t base, phys_addr_t size); +int memblock_clear_llmap(phys_addr_t base, phys_addr_t size); void memblock_free(void *ptr, size_t size); void reset_all_zones_managed_pages(void); @@ -306,6 +310,11 @@ static inline bool memblock_is_kho_scratch(struct memblock_region *m) return m->flags & MEMBLOCK_KHO_SCRATCH; } +static inline bool memblock_is_llmap(struct memblock_region *m) +{
- return m->flags & MEMBLOCK_LLMAP;
+}
int memblock_search_pfn_nid(unsigned long pfn, unsigned long *start_pfn, unsigned long *end_pfn); void __next_mem_pfn_range(int *idx, int nid, unsigned long *out_start_pfn, diff --git a/mm/memblock.c b/mm/memblock.c index 9ce86349a29f..1591b50503ed 100644 --- a/mm/memblock.c +++ b/mm/memblock.c @@ -1119,6 +1119,16 @@ int __init_memblock memblock_mark_mirror(phys_addr_t base, phys_addr_t size) */ int __init_memblock memblock_mark_nomap(phys_addr_t base, phys_addr_t size) {
- struct memblock_region *r;
- memblock_cap_size(base, &size);
- for_each_mem_region(r) {
if (memblock_is_llmap(r) &&memblock_addrs_overlap(base, size, r->base, r->size))return -EINVAL;- }
I'm not very fond of implicit skips here. memblock has no idea what's the caller intention, maybe it actually wants to change the memory from nomap to pte-mapped.
I'd rather warn in memblock_is_nomap() and memblock_is_llmap() if they both are set and let the caller deal with making sure they are not.
ack.
- return memblock_setclr_flag(&memblock.memory, base, size, 1, MEMBLOCK_NOMAP);
} @@ -1204,6 +1214,45 @@ __init int memblock_clear_kho_scratch(phys_addr_t base, phys_addr_t size) MEMBLOCK_KHO_SCRATCH); } +/**
- memblock_mark_llmap - Mark a memory region with flag MEMBLOCK_LLMAP.
- @base: the base phys addr of the region
- @size: the size of the region
- If supported by the architecture, such region is mapped at the last-level in
- the kernel direct map.
- Return: 0 on success, -errno on failure.
- */
+int __init_memblock memblock_mark_llmap(phys_addr_t base, phys_addr_t size) +{
- struct memblock_region *r;
- memblock_cap_size(base, &size);
- for_each_mem_region(r) {
if (memblock_is_nomap(r) &&memblock_addrs_overlap(base, size, r->base, r->size))return -EINVAL;- }
same here
ack
- return memblock_setclr_flag(&memblock.memory, base, size, 1,
MEMBLOCK_LLMAP);+}
+/**
- memblock_clear_llmap - Clear flag MEMBLOCK_LLMAP for a specified region.
- @base: the base phys addr of the region
- @size: the size of the region
- Return: 0 on success, -errno on failure.
- */
+int __init_memblock memblock_clear_llmap(phys_addr_t base, phys_addr_t size) +{
- return memblock_setclr_flag(&memblock.memory, base, size, 0,
MEMBLOCK_LLMAP);+}
static bool should_skip_region(struct memblock_type *type, struct memblock_region *m, int nid, int flags) @@ -2886,6 +2935,7 @@ static const char * const flagname[] = { [ilog2(MEMBLOCK_RSRV_NOINIT)] = "RSV_NIT", [ilog2(MEMBLOCK_RSRV_KERN)] = "RSV_KERN", [ilog2(MEMBLOCK_KHO_SCRATCH)] = "KHO_SCRATCH",
- [ilog2(MEMBLOCK_LLMAP)] = "LLMAP",
}; static int memblock_debug_show(struct seq_file *m, void *private) -- 2.55.0.970.g62bdec98f9-goog
-- Sincerely yours, Mike.
On Mon, Sep 07, 2026 at 10:50:12AM +0100, Vincent Donnefort wrote:
On Sun, Sep 06, 2026 at 10:33:11PM +0300, Mike Rapoport wrote:
On Wed, Sep 02, 2026 at 11:47:03AM +0100, Vincent Donnefort wrote:
Keeping last-level mappings is interesting on some architectures as it allows mapping/unmapping pages from the kernel direct map without the risk of splitting blocks which, under the break-before-make rule, may trigger page-faults the kernel can't handle.
However, mapping the entire direct map at PTE-level is costly. So instead, create a new memblock flag MEMBLOCK_LLMAP to enable the system
I believe MEMBLOCK_PTE_MAP sounds more descriptive.
The idea was to have something close from NOMAP, to emphasis it is one or the other. But PTE_MAP sounds good too.
Could be PTEMAP if you prefer. My point was that unlike PTE, "LL" is not perceived as last-level, it should be looked up.
to decide which region must be covered by mappings up to the last-level.
Signed-off-by: Vincent Donnefort vdonnefort@google.com
-- Vincent
On Tue, Sep 08, 2026 at 10:40:16AM +0300, Mike Rapoport wrote:
On Mon, Sep 07, 2026 at 10:50:12AM +0100, Vincent Donnefort wrote:
On Sun, Sep 06, 2026 at 10:33:11PM +0300, Mike Rapoport wrote:
On Wed, Sep 02, 2026 at 11:47:03AM +0100, Vincent Donnefort wrote:
Keeping last-level mappings is interesting on some architectures as it allows mapping/unmapping pages from the kernel direct map without the risk of splitting blocks which, under the break-before-make rule, may trigger page-faults the kernel can't handle.
However, mapping the entire direct map at PTE-level is costly. So instead, create a new memblock flag MEMBLOCK_LLMAP to enable the system
I believe MEMBLOCK_PTE_MAP sounds more descriptive.
The idea was to have something close from NOMAP, to emphasis it is one or the other. But PTE_MAP sounds good too.
Could be PTEMAP if you prefer. My point was that unlike PTE, "LL" is not perceived as last-level, it should be looked up.
We were discussing the concept of enabling entire block mappings to be removed at once from the linear map, so at that point "PTE"MAP may no longer be accurate.
I imagine that in some scenarios we might be able to go to PUD mappings for something like VPR (say systems with a fair amount of system memory and we want to carve out 4 GiB for VPR, split into four 1 GiB chunks).
Thierry
On Tue, Sep 08, 2026 at 11:18:53AM +0200, Thierry Reding wrote:
On Tue, Sep 08, 2026 at 10:40:16AM +0300, Mike Rapoport wrote:
On Mon, Sep 07, 2026 at 10:50:12AM +0100, Vincent Donnefort wrote:
On Sun, Sep 06, 2026 at 10:33:11PM +0300, Mike Rapoport wrote:
On Wed, Sep 02, 2026 at 11:47:03AM +0100, Vincent Donnefort wrote:
Keeping last-level mappings is interesting on some architectures as it allows mapping/unmapping pages from the kernel direct map without the risk of splitting blocks which, under the break-before-make rule, may trigger page-faults the kernel can't handle.
However, mapping the entire direct map at PTE-level is costly. So instead, create a new memblock flag MEMBLOCK_LLMAP to enable the system
I believe MEMBLOCK_PTE_MAP sounds more descriptive.
The idea was to have something close from NOMAP, to emphasis it is one or the other. But PTE_MAP sounds good too.
Could be PTEMAP if you prefer. My point was that unlike PTE, "LL" is not perceived as last-level, it should be looked up.
We were discussing the concept of enabling entire block mappings to be removed at once from the linear map, so at that point "PTE"MAP may no longer be accurate.
I imagine that in some scenarios we might be able to go to PUD mappings for something like VPR (say systems with a fair amount of system memory and we want to carve out 4 GiB for VPR, split into four 1 GiB chunks).
_LEAFMAP then? ;-)
Thierry
Keeping last-level mappings is interesting on some architectures as it allows mapping/unmapping pages from the kernel direct map without the risk of splitting blocks which, under the break-before-make rule, may trigger page-faults the kernel can't handle.
Add an "ll-map" property for reserved-memory regions. When set, it splits the underlying memblock and sets the MEMBLOCK_LLMAP flag.
Signed-off-by: Vincent Donnefort vdonnefort@google.com
diff --git a/drivers/of/of_reserved_mem.c b/drivers/of/of_reserved_mem.c index 8c9d6395d6a3..9ff2d02cb149 100644 --- a/drivers/of/of_reserved_mem.c +++ b/drivers/of/of_reserved_mem.c @@ -32,9 +32,32 @@ static struct reserved_mem *reserved_mem __refdata = reserved_mem_array; static int total_reserved_mem_cnt = MAX_RESERVED_REGIONS; static int reserved_mem_count;
+enum of_rmem_flags { + OF_RMEM_NONE = 0, + OF_RMEM_NOMAP = BIT(0), + OF_RMEM_LLMAP = BIT(1), +}; + +static int __init of_reserved_mem_flags(unsigned long node, enum of_rmem_flags *flags) +{ + *flags = OF_RMEM_NONE; + + if (of_get_flat_dt_prop(node, "no-map", NULL)) + *flags |= OF_RMEM_NOMAP; + if (of_get_flat_dt_prop(node, "ll-map", NULL)) + *flags |= OF_RMEM_LLMAP; + + if ((*flags & OF_RMEM_NOMAP) && (*flags & OF_RMEM_LLMAP)) { + pr_err("Reserved memory: no-map and ll-map are mutually exclusive\n"); + return -EINVAL; + } + + return 0; +} + static int __init early_init_dt_alloc_reserved_memory_arch(phys_addr_t size, - phys_addr_t align, phys_addr_t start, phys_addr_t end, bool nomap, - phys_addr_t *res_base) + phys_addr_t align, phys_addr_t start, phys_addr_t end, + enum of_rmem_flags flags, phys_addr_t *res_base) { phys_addr_t base; int err = 0; @@ -46,10 +69,21 @@ static int __init early_init_dt_alloc_reserved_memory_arch(phys_addr_t size, return -ENOMEM;
*res_base = base; - if (nomap) { - err = memblock_mark_nomap(base, size); - if (err) + if (flags & OF_RMEM_LLMAP) { + err = memblock_mark_llmap(base, size); + if (err) { memblock_phys_free(base, size); + return err; + } + } + + if (flags & OF_RMEM_NOMAP) { + err = memblock_mark_nomap(base, size); + if (err) { + if (flags & OF_RMEM_LLMAP) + memblock_clear_llmap(base, size); + memblock_phys_free(base, size); + } }
if (!err) @@ -119,17 +153,29 @@ static int fdt_fixup_reserved_mem_node(unsigned long node, phys_addr_t base, phys_addr_t size);
static int __init early_init_dt_reserve_memory(phys_addr_t base, - phys_addr_t size, bool nomap) + phys_addr_t size, + enum of_rmem_flags flags) { - if (nomap) { + int err; + + if (flags & OF_RMEM_LLMAP) { + err = memblock_mark_llmap(base, size); + if (err) + return err; + } + + if (flags & OF_RMEM_NOMAP) { /* * If the memory is already reserved (by another region), we * should not allow it to be marked nomap, but don't worry * if the region isn't memory as it won't be mapped. */ if (memblock_overlaps_region(&memblock.memory, base, size) && - memblock_is_region_reserved(base, size)) + memblock_is_region_reserved(base, size)) { + if (flags & OF_RMEM_LLMAP) + memblock_clear_llmap(base, size); return -EBUSY; + }
return memblock_mark_nomap(base, size); } @@ -143,10 +189,10 @@ static int __init early_init_dt_reserve_memory(phys_addr_t base, static int __init __reserved_mem_reserve_reg(unsigned long node, const char *uname) { + enum of_rmem_flags flags; phys_addr_t base, size; - int len, err; const __be32 *prop; - bool nomap; + int len, err; u64 b, s;
prop = of_flat_dt_get_addr_size_prop(node, "reg", &len); @@ -157,7 +203,9 @@ static int __init __reserved_mem_reserve_reg(unsigned long node, pr_warn("Reserved memory: node '%s' has %d <base size> entries, only the first is used\n", uname, len);
- nomap = of_get_flat_dt_prop(node, "no-map", NULL) != NULL; + err = of_reserved_mem_flags(node, &flags); + if (err) + return err;
err = fdt_validate_reserved_mem_node(node, NULL); if (err && err != -ENODEV) @@ -167,7 +215,7 @@ static int __init __reserved_mem_reserve_reg(unsigned long node, base = b; size = s;
- if (size && early_init_dt_reserve_memory(base, size, nomap) == 0) { + if (size && early_init_dt_reserve_memory(base, size, flags) == 0) { fdt_fixup_reserved_mem_node(node, base, size); pr_debug("Reserved memory: reserved region for node '%s': base %pa, size %lu MiB\n", uname, &base, (unsigned long)(size / SZ_1M)); @@ -399,8 +447,8 @@ int __init fdt_scan_reserved_mem(void) * reserved regions to keep the reserved memory contiguous if possible. */ static int __init __reserved_mem_alloc_in_range(phys_addr_t size, - phys_addr_t align, phys_addr_t start, phys_addr_t end, bool nomap, - phys_addr_t *res_base) + phys_addr_t align, phys_addr_t start, phys_addr_t end, + enum of_rmem_flags flags, phys_addr_t *res_base) { bool prev_bottom_up = memblock_bottom_up(); bool bottom_up = false, top_down = false; @@ -435,7 +483,7 @@ static int __init __reserved_mem_alloc_in_range(phys_addr_t size, memblock_set_bottom_up(bottom_up);
ret = early_init_dt_alloc_reserved_memory_arch(size, align, - start, end, nomap, res_base); + start, end, flags, res_base);
/* Restore old setting if needed */ if (bottom_up != top_down) @@ -452,9 +500,9 @@ static int __init __reserved_mem_alloc_size(unsigned long node, const char *unam { phys_addr_t start = 0, end = 0; phys_addr_t base = 0, align = 0, size; + enum of_rmem_flags flags; int i, len; const __be32 *prop; - bool nomap; int ret;
prop = of_get_flat_dt_prop(node, "size", &len); @@ -477,7 +525,9 @@ static int __init __reserved_mem_alloc_size(unsigned long node, const char *unam align = dt_mem_next_cell(dt_root_addr_cells, &prop); }
- nomap = of_get_flat_dt_prop(node, "no-map", NULL) != NULL; + ret = of_reserved_mem_flags(node, &flags); + if (ret) + return ret;
ret = fdt_validate_reserved_mem_node(node, &align); if (ret && ret != -ENODEV) @@ -495,7 +545,7 @@ static int __init __reserved_mem_alloc_size(unsigned long node, const char *unam
base = 0; ret = __reserved_mem_alloc_in_range(size, align, - start, end, nomap, &base); + start, end, flags, &base); if (ret == 0) { pr_debug("allocated memory for '%s' node: base %pa, size %lu MiB\n", uname, &base, @@ -505,7 +555,7 @@ static int __init __reserved_mem_alloc_size(unsigned long node, const char *unam } } else { ret = early_init_dt_alloc_reserved_memory_arch(size, align, - 0, 0, nomap, &base); + 0, 0, flags, &base); if (ret == 0) pr_debug("allocated memory for '%s' node: base %pa, size %lu MiB\n", uname, &base, (unsigned long)(size / SZ_1M)); @@ -635,8 +685,8 @@ static int __init __reserved_mem_init_node(struct reserved_mem *rmem, static void __init fdt_init_reserved_mem_node(unsigned long node, const char *uname, phys_addr_t base, phys_addr_t size) { + enum of_rmem_flags flags; int err = 0; - bool nomap;
struct reserved_mem *rmem = &reserved_mem[reserved_mem_count];
@@ -650,14 +700,19 @@ static void __init fdt_init_reserved_mem_node(unsigned long node, const char *un rmem->base = base; rmem->size = size;
- nomap = of_get_flat_dt_prop(node, "no-map", NULL) != NULL; + err = of_reserved_mem_flags(node, &flags); + if (err) + return;
err = __reserved_mem_init_node(rmem, node); if (err != 0 && err != -ENODEV) { pr_info("node %s compatible matching fail\n", rmem->name); rmem->name = NULL;
- if (nomap) + if (flags & OF_RMEM_LLMAP) + memblock_clear_llmap(rmem->base, rmem->size); + + if (flags & OF_RMEM_NOMAP) memblock_clear_nomap(rmem->base, rmem->size); else memblock_phys_free(rmem->base, rmem->size); @@ -667,9 +722,10 @@ static void __init fdt_init_reserved_mem_node(unsigned long node, const char *un bool reusable = (of_get_flat_dt_prop(node, "reusable", NULL)) != NULL;
- pr_info("%pa..%pa (%lu KiB) %s %s %s\n", + pr_info("%pa..%pa (%lu KiB) %s %s %s %s\n", &rmem->base, &end, (unsigned long)(rmem->size / SZ_1K), - nomap ? "nomap" : "map", + (flags & OF_RMEM_LLMAP) ? "last-level " : "", + (flags & OF_RMEM_NOMAP) ? "nomap" : "map", reusable ? "reusable" : "non-reusable", rmem->name ? rmem->name : "unknown"); }
On Wed, Sep 02, 2026 at 11:47:04AM +0100, Vincent Donnefort wrote:
Keeping last-level mappings is interesting on some architectures as it allows mapping/unmapping pages from the kernel direct map without the risk of splitting blocks which, under the break-before-make rule, may trigger page-faults the kernel can't handle.
Add an "ll-map" property for reserved-memory regions. When set, it splits the underlying memblock and sets the MEMBLOCK_LLMAP flag.
Where is this documented? Any DT property must have a schema.
Though I'm not thrilled about more flag properties. Handling all the combinations of properties has proven to be painful. Can this be implied from the compatible string instead?
Rob
On Wed, Sep 02, 2026 at 12:24:36PM -0500, Rob Herring wrote:
On Wed, Sep 02, 2026 at 11:47:04AM +0100, Vincent Donnefort wrote:
Keeping last-level mappings is interesting on some architectures as it allows mapping/unmapping pages from the kernel direct map without the risk of splitting blocks which, under the break-before-make rule, may trigger page-faults the kernel can't handle.
Add an "ll-map" property for reserved-memory regions. When set, it splits the underlying memblock and sets the MEMBLOCK_LLMAP flag.
Where is this documented? Any DT property must have a schema.
Ack.
Though I'm not thrilled about more flag properties. Handling all the combinations of properties has proven to be painful. Can this be implied from the compatible string instead?
Rob
We need this before the kernel direct map has been created (paging_init()).
I would have moved that directly into the driver lend_pool.c, but then it is probed too late (unflatten_device_tree()).
On Thu, Sep 03, 2026 at 11:03:12AM +0100, Vincent Donnefort wrote:
On Wed, Sep 02, 2026 at 12:24:36PM -0500, Rob Herring wrote:
On Wed, Sep 02, 2026 at 11:47:04AM +0100, Vincent Donnefort wrote:
Keeping last-level mappings is interesting on some architectures as it allows mapping/unmapping pages from the kernel direct map without the risk of splitting blocks which, under the break-before-make rule, may trigger page-faults the kernel can't handle.
Add an "ll-map" property for reserved-memory regions. When set, it splits the underlying memblock and sets the MEMBLOCK_LLMAP flag.
Where is this documented? Any DT property must have a schema.
Ack.
Though I'm not thrilled about more flag properties. Handling all the combinations of properties has proven to be painful. Can this be implied from the compatible string instead?
Rob
We need this before the kernel direct map has been created (paging_init()).
I would have moved that directly into the driver lend_pool.c, but then it is probed too late (unflatten_device_tree()).
Any of the node_*() callbacks from struct reserved_mem_ops get called prior to paging_init(), so these could be used to manually set the memblock flags in the corresponding drivers.
Thierry
On Mon, Sep 07, 2026 at 04:00:19PM +0200, Thierry Reding wrote:
On Thu, Sep 03, 2026 at 11:03:12AM +0100, Vincent Donnefort wrote:
On Wed, Sep 02, 2026 at 12:24:36PM -0500, Rob Herring wrote:
On Wed, Sep 02, 2026 at 11:47:04AM +0100, Vincent Donnefort wrote:
Keeping last-level mappings is interesting on some architectures as it allows mapping/unmapping pages from the kernel direct map without the risk of splitting blocks which, under the break-before-make rule, may trigger page-faults the kernel can't handle.
Add an "ll-map" property for reserved-memory regions. When set, it splits the underlying memblock and sets the MEMBLOCK_LLMAP flag.
Where is this documented? Any DT property must have a schema.
Ack.
Though I'm not thrilled about more flag properties. Handling all the combinations of properties has proven to be painful. Can this be implied from the compatible string instead?
Rob
We need this before the kernel direct map has been created (paging_init()).
I would have moved that directly into the driver lend_pool.c, but then it is probed too late (unflatten_device_tree()).
Any of the node_*() callbacks from struct reserved_mem_ops get called prior to paging_init(), so these could be used to manually set the memblock flags in the corresponding drivers.
Thierry
Ha yes my bad, early_init_fdt_scan_reserved_mem() is calling ->node_init()
So I should be able to drop this patch entirely to do set memblock_set_llmap() (or memblock_set_pte_map()) directly ino the that new FF-A lend_pool driver.
While can_set_direct_map() tells whether the direct map can be modified globally, can_set_direct_map_range() checks if a specific range can be modified. e.g. if mapped at PTE-level.
Signed-off-by: Vincent Donnefort vdonnefort@google.com
diff --git a/include/linux/set_memory.h b/include/linux/set_memory.h index 3030d9245f5a..88175c3fa751 100644 --- a/include/linux/set_memory.h +++ b/include/linux/set_memory.h @@ -44,6 +44,12 @@ static inline bool kernel_page_present(struct page *page) { return true; } + +static inline bool can_set_direct_map_range(struct page *page, + unsigned long nr_pages) +{ + return false; +} #else /* CONFIG_ARCH_HAS_SET_DIRECT_MAP */ /* * Some architectures, e.g. ARM64 can disable direct map modifications at @@ -56,6 +62,14 @@ static inline bool can_set_direct_map(void) } #define can_set_direct_map can_set_direct_map #endif + +#ifndef can_set_direct_map_range +static inline bool can_set_direct_map_range(struct page *page, unsigned long nr_pages) +{ + return can_set_direct_map(); +} +#define can_set_direct_map_range can_set_direct_map_range +#endif #endif /* CONFIG_ARCH_HAS_SET_DIRECT_MAP */
#ifdef CONFIG_X86_64
On Wed, Sep 02, 2026 at 11:47:05AM +0100, Vincent Donnefort wrote:
While can_set_direct_map() tells whether the direct map can be modified globally, can_set_direct_map_range() checks if a specific range can be modified. e.g. if mapped at PTE-level.
set_direct_map are going to have number of pages parameter soon:
https://lore.kernel.org/all/20260903-execmem-set-vm-perms-v0-2-v3-1-949b64a9...
so this check can be a part of arm64::set_direct_map
I wouldn't expose can_set_direct_map_range() as a public API and rely on the callers to "Do The Right Thing".
Signed-off-by: Vincent Donnefort vdonnefort@google.com
diff --git a/include/linux/set_memory.h b/include/linux/set_memory.h index 3030d9245f5a..88175c3fa751 100644 --- a/include/linux/set_memory.h +++ b/include/linux/set_memory.h @@ -44,6 +44,12 @@ static inline bool kernel_page_present(struct page *page) { return true; }
+static inline bool can_set_direct_map_range(struct page *page,
unsigned long nr_pages)+{
- return false;
+} #else /* CONFIG_ARCH_HAS_SET_DIRECT_MAP */ /*
- Some architectures, e.g. ARM64 can disable direct map modifications at
@@ -56,6 +62,14 @@ static inline bool can_set_direct_map(void) } #define can_set_direct_map can_set_direct_map #endif
+#ifndef can_set_direct_map_range +static inline bool can_set_direct_map_range(struct page *page, unsigned long nr_pages) +{
- return can_set_direct_map();
+} +#define can_set_direct_map_range can_set_direct_map_range +#endif #endif /* CONFIG_ARCH_HAS_SET_DIRECT_MAP */
#ifdef CONFIG_X86_64
2.55.0.970.g62bdec98f9-goog
On Sun, Sep 06, 2026 at 10:39:02PM +0300, Mike Rapoport wrote:
On Wed, Sep 02, 2026 at 11:47:05AM +0100, Vincent Donnefort wrote:
While can_set_direct_map() tells whether the direct map can be modified globally, can_set_direct_map_range() checks if a specific range can be modified. e.g. if mapped at PTE-level.
set_direct_map are going to have number of pages parameter soon:
https://lore.kernel.org/all/20260903-execmem-set-vm-perms-v0-2-v3-1-949b64a9...
so this check can be a part of arm64::set_direct_map
ack.
I wouldn't expose can_set_direct_map_range() as a public API and rely on the callers to "Do The Right Thing".
ack.
Thanks for having a look at the series.
set_direct_map_*() implementations verify wheter the direct map can be effectively set. This verification can be costly. Add an "advanced" API __set_direct_map_*() where the direct map is modified without any check. The caller must have verified it is safe earlier with one of the can_set_direct_map() or can_set_direct_map_range() functions.
Signed-off-by: Vincent Donnefort vdonnefort@google.com
diff --git a/include/linux/set_memory.h b/include/linux/set_memory.h index 88175c3fa751..5a1917a3fd52 100644 --- a/include/linux/set_memory.h +++ b/include/linux/set_memory.h @@ -34,6 +34,15 @@ static inline int set_direct_map_default_noflush(struct page *page) return 0; }
+static inline int __set_direct_map_invalid_noflush(struct page *page) +{ + return 0; +} +static inline int __set_direct_map_default_noflush(struct page *page) +{ + return 0; +} + static inline int set_direct_map_valid_noflush(struct page *page, unsigned nr, bool valid) { @@ -70,6 +79,22 @@ static inline bool can_set_direct_map_range(struct page *page, unsigned long nr_ } #define can_set_direct_map_range can_set_direct_map_range #endif + +#ifndef __set_direct_map_invalid_noflush +static inline int __set_direct_map_invalid_noflush(struct page *page) +{ + return set_direct_map_invalid_noflush(page); +} +#define __set_direct_map_invalid_noflush __set_direct_map_invalid_noflush +#endif + +#ifndef __set_direct_map_default_noflush +static inline int __set_direct_map_default_noflush(struct page *page) +{ + return set_direct_map_default_noflush(page); +} +#define __set_direct_map_default_noflush __set_direct_map_default_noflush +#endif #endif /* CONFIG_ARCH_HAS_SET_DIRECT_MAP */
#ifdef CONFIG_X86_64
Systems supporting BBML3 can split blocks without risking unhandled page-faults. It is therefore safe for them to set the direct map unconditionally.
Signed-off-by: Vincent Donnefort vdonnefort@google.com
diff --git a/arch/arm64/mm/pageattr.c b/arch/arm64/mm/pageattr.c index bbe98ac9ad8c..58898ae53f5d 100644 --- a/arch/arm64/mm/pageattr.c +++ b/arch/arm64/mm/pageattr.c @@ -99,7 +99,8 @@ bool can_set_direct_map(void) * Realms need to make pages shared/protected at page granularity. */ return rodata_full || debug_pagealloc_enabled() || - arm64_kfence_can_set_direct_map() || is_realm_world(); + arm64_kfence_can_set_direct_map() || is_realm_world() || + system_supports_bbml3(); }
static int update_range_prot(unsigned long start, unsigned long size,
For systems where can_set_direct_map() is false, it is still possible that a subregion of the direct map can be modified if it is mapped at the last-level. Add an implementation for can_set_direct_map_range(). It falls back to a page table walk to verify the mapping level if can_set_direct_map() is false.
Signed-off-by: Vincent Donnefort vdonnefort@google.com
diff --git a/arch/arm64/include/asm/set_memory.h b/arch/arm64/include/asm/set_memory.h index 90f61b17275e..1a37f4ef130b 100644 --- a/arch/arm64/include/asm/set_memory.h +++ b/arch/arm64/include/asm/set_memory.h @@ -9,6 +9,9 @@ bool can_set_direct_map(void); #define can_set_direct_map can_set_direct_map
+bool can_set_direct_map_range(struct page *page, unsigned long nr_pages); +#define can_set_direct_map_range can_set_direct_map_range + int set_memory_valid(unsigned long addr, int numpages, int enable);
int set_direct_map_invalid_noflush(struct page *page); diff --git a/arch/arm64/mm/pageattr.c b/arch/arm64/mm/pageattr.c index 58898ae53f5d..c59ef17eb0d0 100644 --- a/arch/arm64/mm/pageattr.c +++ b/arch/arm64/mm/pageattr.c @@ -103,6 +103,48 @@ bool can_set_direct_map(void) system_supports_bbml3(); }
+bool can_set_direct_map_range(struct page *page, unsigned long nr_pages) +{ + unsigned long addr = (unsigned long)page_address(page); + unsigned long end = addr + nr_pages * PAGE_SIZE; + + if (can_set_direct_map()) + return true; + + /* + * If !can_set_direct_map() then no one can split blocks and it is safe + * to walk the page-table lockless. + */ + while (addr < end) { + pud_t *pudp, pud; + pmd_t *pmdp, pmd; + pgd_t *pgdp; + p4d_t *p4dp; + + pgdp = pgd_offset_k(addr); + if (pgd_none(READ_ONCE(*pgdp))) + return false; + + p4dp = p4d_offset(pgdp, addr); + if (p4d_none(READ_ONCE(*p4dp))) + return false; + + pudp = pud_offset(p4dp, addr); + pud = READ_ONCE(*pudp); + if (pud_none(pud) || pud_leaf(pud)) + return false; + + pmdp = pmd_offset(pudp, addr); + pmd = READ_ONCE(*pmdp); + if (pmd_none(pmd) || pmd_leaf(pmd)) + return false; + + addr = pmd_addr_end(addr, end); + } + + return true; +} + static int update_range_prot(unsigned long start, unsigned long size, pgprot_t set_mask, pgprot_t clear_mask) {
Add implementation for the unsafe functions __set_direct_map*(). They do not verify for can_set_direct_map() and expect the caller to do so beforehand.
Signed-off-by: Vincent Donnefort vdonnefort@google.com
diff --git a/arch/arm64/include/asm/set_memory.h b/arch/arm64/include/asm/set_memory.h index 1a37f4ef130b..080a40846bbe 100644 --- a/arch/arm64/include/asm/set_memory.h +++ b/arch/arm64/include/asm/set_memory.h @@ -16,6 +16,10 @@ int set_memory_valid(unsigned long addr, int numpages, int enable);
int set_direct_map_invalid_noflush(struct page *page); int set_direct_map_default_noflush(struct page *page); +int __set_direct_map_invalid_noflush(struct page *page); +#define __set_direct_map_invalid_noflush __set_direct_map_invalid_noflush +int __set_direct_map_default_noflush(struct page *page); +#define __set_direct_map_default_noflush __set_direct_map_default_noflush int set_direct_map_valid_noflush(struct page *page, unsigned nr, bool valid); bool kernel_page_present(struct page *page);
diff --git a/arch/arm64/mm/pageattr.c b/arch/arm64/mm/pageattr.c index c59ef17eb0d0..28adab0edace 100644 --- a/arch/arm64/mm/pageattr.c +++ b/arch/arm64/mm/pageattr.c @@ -294,28 +294,38 @@ int set_memory_valid(unsigned long addr, int numpages, int enable) __pgprot(PTE_PRESENT_VALID_KERNEL)); }
-int set_direct_map_invalid_noflush(struct page *page) +int __set_direct_map_invalid_noflush(struct page *page) { pgprot_t clear_mask = __pgprot(PTE_PRESENT_VALID_KERNEL); pgprot_t set_mask = __pgprot(PTE_PRESENT_INVALID);
+ return update_range_prot((unsigned long)page_address(page), + PAGE_SIZE, set_mask, clear_mask); +} + +int set_direct_map_invalid_noflush(struct page *page) +{ if (!can_set_direct_map()) return 0;
+ return __set_direct_map_invalid_noflush(page); +} + +int __set_direct_map_default_noflush(struct page *page) +{ + pgprot_t set_mask = __pgprot(PTE_PRESENT_VALID_KERNEL | PTE_WRITE); + pgprot_t clear_mask = __pgprot(PTE_PRESENT_INVALID | PTE_RDONLY); + return update_range_prot((unsigned long)page_address(page), PAGE_SIZE, set_mask, clear_mask); }
int set_direct_map_default_noflush(struct page *page) { - pgprot_t set_mask = __pgprot(PTE_PRESENT_VALID_KERNEL | PTE_WRITE); - pgprot_t clear_mask = __pgprot(PTE_PRESENT_INVALID | PTE_RDONLY); - if (!can_set_direct_map()) return 0;
- return update_range_prot((unsigned long)page_address(page), - PAGE_SIZE, set_mask, clear_mask); + return __set_direct_map_default_noflush(page); }
static int __set_memory_enc_dec(unsigned long addr,
On Wed, Sep 02, 2026 at 11:47:09AM +0100, Vincent Donnefort wrote:
Add implementation for the unsafe functions __set_direct_map*(). They do not verify for can_set_direct_map() and expect the caller to do so beforehand.
Signed-off-by: Vincent Donnefort vdonnefort@google.com
diff --git a/arch/arm64/include/asm/set_memory.h b/arch/arm64/include/asm/set_memory.h index 1a37f4ef130b..080a40846bbe 100644 --- a/arch/arm64/include/asm/set_memory.h +++ b/arch/arm64/include/asm/set_memory.h @@ -16,6 +16,10 @@ int set_memory_valid(unsigned long addr, int numpages, int enable); int set_direct_map_invalid_noflush(struct page *page); int set_direct_map_default_noflush(struct page *page); +int __set_direct_map_invalid_noflush(struct page *page); +#define __set_direct_map_invalid_noflush __set_direct_map_invalid_noflush +int __set_direct_map_default_noflush(struct page *page); +#define __set_direct_map_default_noflush __set_direct_map_default_noflush int set_direct_map_valid_noflush(struct page *page, unsigned nr, bool valid); bool kernel_page_present(struct page *page); diff --git a/arch/arm64/mm/pageattr.c b/arch/arm64/mm/pageattr.c index c59ef17eb0d0..28adab0edace 100644 --- a/arch/arm64/mm/pageattr.c +++ b/arch/arm64/mm/pageattr.c @@ -294,28 +294,38 @@ int set_memory_valid(unsigned long addr, int numpages, int enable) __pgprot(PTE_PRESENT_VALID_KERNEL)); } -int set_direct_map_invalid_noflush(struct page *page) +int __set_direct_map_invalid_noflush(struct page *page) { pgprot_t clear_mask = __pgprot(PTE_PRESENT_VALID_KERNEL); pgprot_t set_mask = __pgprot(PTE_PRESENT_INVALID);
- return update_range_prot((unsigned long)page_address(page),
PAGE_SIZE, set_mask, clear_mask);+}
I think these would work quite nicely if you rebase the series onto linux-next, which contains the patches that add the number of pages to these functions.
You can then keep can_set_direct_map_range() from the previous patch and call that from the set_direct_map_*_noflush() functions instead of the simple can_set_direct_map() (which is a shortcut path in the _range check anyway.
I've used that locally in my tree and it seems to be working just fine.
Thierry
With the MEMBLOCK_LLMAP flag, force a last-level mapping.
Signed-off-by: Vincent Donnefort vdonnefort@google.com
diff --git a/arch/arm64/mm/mmu.c b/arch/arm64/mm/mmu.c index 79d90226fd5d..c0fcc76914d1 100644 --- a/arch/arm64/mm/mmu.c +++ b/arch/arm64/mm/mmu.c @@ -1184,9 +1184,8 @@ static void __init map_mem(void) phys_addr_t init_begin = __pa_symbol(__init_begin); phys_addr_t init_end = __pa_symbol(__init_end); phys_addr_t kernel_end = __pa_symbol(__bss_stop); - phys_addr_t start, end; int flags = NO_EXEC_MAPPINGS; - u64 i; + struct memblock_region *r;
/* * Setting hierarchical PXNTable attributes on table entries covering @@ -1226,9 +1225,13 @@ static void __init map_mem(void) flags);
/* map all the memory banks */ - for_each_mem_range(i, &start, &end) { + for_each_mem_region(r) { + phys_addr_t start, end; + + if (memblock_is_nomap(r)) + continue; /* - * for_each_mem_range may return sub-page-aligned boundaries + * for_each_mem_region may return sub-page-aligned boundaries * after memblock_mark_nomap() splits regions at byte precision. * __create_pgd_mapping_locked aligns phys down to PAGE_MASK, * which could accidentally map no-map memory on the boundary. @@ -1237,17 +1240,23 @@ static void __init map_mem(void) * regions. The cost is at most one page of unmapped gap at * each boundary. */ - start = PAGE_ALIGN(start); - end = end & PAGE_MASK; + start = PAGE_ALIGN(r->base); + end = (r->base + r->size) & PAGE_MASK; if (start >= end) continue; + + int rflags = flags; + + if (memblock_is_llmap(r)) + rflags |= NO_BLOCK_MAPPINGS | NO_CONT_MAPPINGS; + /* * The linear map must allow allocation tags reading/writing * if MTE is present. Otherwise, it has the same attributes as * PAGE_KERNEL. */ __map_memblock(start, end, pgprot_tagged(PAGE_KERNEL), - flags); + rflags); } }
When memory is lent to the Secure world via FF-A, fatal CPU speculative reads from Non-Secure can still occur as long as it retains a cacheable mapping to that memory. Introduce the "arm,ffa-lend-pool" reserved-memory CMA driver to unmap pages before lending (ffa_prepare_lend()) and restore them upon reclaim (ffa_lend_reclaimed()). Devices bind to the pool via the "memory-region" DT property or via ffa_lend_pool_attach().
reserved-memory { #address-cells = <0x2>; #size-cells = <0x2>; ranges;
ffa_lend: ffa-lend-pool { compatible = "arm,ffa-lend-pool"; reusable; ll-map; size = <0x0 0x4000000>; }; };
Signed-off-by: Vincent Donnefort vdonnefort@google.com
diff --git a/drivers/firmware/arm_ffa/Kconfig b/drivers/firmware/arm_ffa/Kconfig index 5e3ae5cf82e8..66dbf74c37c3 100644 --- a/drivers/firmware/arm_ffa/Kconfig +++ b/drivers/firmware/arm_ffa/Kconfig @@ -19,3 +19,8 @@ config ARM_FFA_SMCCC bool default ARM_FFA_TRANSPORT depends on ARM64 && HAVE_ARM_SMCCC_DISCOVERY + +config ARM_FFA_LEND_POOL + bool + default y + depends on ARM_FFA_TRANSPORT && CMA && OF_RESERVED_MEM diff --git a/drivers/firmware/arm_ffa/Makefile b/drivers/firmware/arm_ffa/Makefile index 168990a7e792..5ea3019c407b 100644 --- a/drivers/firmware/arm_ffa/Makefile +++ b/drivers/firmware/arm_ffa/Makefile @@ -6,3 +6,4 @@ ffa-core-objs := $(ffa-bus-y) ffa-module-objs := $(ffa-driver-y) $(ffa-transport-y) obj-$(CONFIG_ARM_FFA_TRANSPORT) = ffa-core.o obj-$(CONFIG_ARM_FFA_TRANSPORT) += ffa-module.o +obj-$(CONFIG_ARM_FFA_LEND_POOL) += lend_pool.o diff --git a/drivers/firmware/arm_ffa/lend_pool.c b/drivers/firmware/arm_ffa/lend_pool.c new file mode 100644 index 000000000000..67a36f5d5aa6 --- /dev/null +++ b/drivers/firmware/arm_ffa/lend_pool.c @@ -0,0 +1,223 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Arm FF-A Reserved Memory CMA driver for Memory Lending + * + * Prevents CPU speculative reads to secure memory by unmapping it from the + * kernel direct map. This only works if the reserved-memory is mapped at the + * last-level ("ll-map;" or "rodata=full") or if all CPUs in the system support + * BBML3. + * + * Copyright (C) 2026 Google LLC + * Author: Vincent Donnefort vdonnefort@google.com + */ + +#include <linux/arm_ffa.h> +#include <linux/cleanup.h> +#include <linux/cma.h> +#include <linux/dma-map-ops.h> +#include <linux/init.h> +#include <linux/mm.h> +#include <linux/of.h> +#include <linux/of_fdt.h> +#include <linux/of_reserved_mem.h> +#include <linux/rcupdate.h> +#include <linux/set_memory.h> +#include <linux/xarray.h> + +static DEFINE_XARRAY(ffa_lend_devices); + +static bool ffa_lend_pool_contains(struct device *dev, phys_addr_t addr, size_t size) +{ + phys_addr_t base, end; + + if (!dev) + return false; + + guard(rcu)(); + + if (xa_load(&ffa_lend_devices, (unsigned long)dev) != dev) + return false; + + if (WARN_ON_ONCE(!dev->cma_area)) + return false; + + base = cma_get_base(dev->cma_area); + end = base + cma_get_size(dev->cma_area); + + return addr >= base && (addr + size) <= end; +} + +/** + * ffa_prepare_lend() - Prepare a memory region to be lent in FF-A + * @dev: Device attached to the lend pool + * @addr: Physical start address of the memory region + * @size: Size in bytes + * + * When memory is lent via FF-A, TrustZone transitions it to the secure state. + * As long as Arm CPUs retain a valid mapping to that now-secure memory, they + * can speculatively read it, which is fatal on some systems. + * + * ffa_prepare_lend() prevents this by unmapping the memory range from the + * kernel's direct map. + * + * Return: 0 on success, negative error code otherwise. + */ +int ffa_prepare_lend(struct device *dev, phys_addr_t addr, size_t size) +{ + unsigned long nr_pages = size >> PAGE_SHIFT; + struct page *page; + unsigned long i; + int ret; + + if (!ffa_lend_pool_contains(dev, addr, size)) + return -ENODEV; + + page = pfn_to_page(PHYS_PFN(addr)); + for (i = 0; i < nr_pages; i++) { + ret = __set_direct_map_invalid_noflush(page + i); + + if (ret) { + while (i--) + __set_direct_map_default_noflush(page + i); + + return ret; + } + } + + return 0; +} +EXPORT_SYMBOL_GPL(ffa_prepare_lend); + +/** + * ffa_lend_reclaimed() - Restore a reclaimed FF-A memory region + * @dev: Device attached to the lend pool + * @addr: Physical start address of the memory region + * @size: Size in bytes + * + * Restores a memory range into the kernel's direct mapping. It must be called + * after a successful FF-A memory reclaim invocation. + */ +void ffa_lend_reclaimed(struct device *dev, phys_addr_t addr, size_t size) +{ + unsigned long nr_pages = size >> PAGE_SHIFT; + struct page *page; + unsigned long i; + + if (!ffa_lend_pool_contains(dev, addr, size)) + return; + + page = pfn_to_page(PHYS_PFN(addr)); + for (i = 0; i < nr_pages; i++) + __set_direct_map_default_noflush(page + i); +} +EXPORT_SYMBOL_GPL(ffa_lend_reclaimed); + +/** + * ffa_lend_pool_attach() - Attach a device to the FF-A lend pool + * @dev: Device to attach + * + * FF-A devices are dynamically discovered and might not have an associated + * device tree node with a "memory-region" phandle. In that case, drivers must + * use this function to attach to the "arm,ffa-lend-pool" reserved memory + * region. + * + * Return: 0 on success, negative error code otherwise. + */ +int ffa_lend_pool_attach(struct device *dev) +{ + struct device_node *np; + struct reserved_mem *rmem; + + np = of_find_compatible_node(NULL, NULL, "arm,ffa-lend-pool"); + if (!np) + return -ENODEV; + + rmem = of_reserved_mem_lookup(np); + of_node_put(np); + if (!rmem || !rmem->ops || !rmem->ops->device_init) + return -EINVAL; + + return rmem->ops->device_init(rmem, dev); +} +EXPORT_SYMBOL_GPL(ffa_lend_pool_attach); + +/** + * ffa_lend_pool_detach() - Detach a device from the FF-A lend pool + * @dev: Device to detach + * + * Releases the device from the "arm,ffa-lend-pool" reserved memory region. + */ +void ffa_lend_pool_detach(struct device *dev) +{ + struct device_node *np; + struct reserved_mem *rmem; + + np = of_find_compatible_node(NULL, NULL, "arm,ffa-lend-pool"); + if (!np) + return; + + rmem = of_reserved_mem_lookup(np); + of_node_put(np); + + if (rmem && rmem->ops && rmem->ops->device_release) + rmem->ops->device_release(rmem, dev); +} +EXPORT_SYMBOL_GPL(ffa_lend_pool_detach); + +static int __init ffa_lend_pool_setup(unsigned long node, struct reserved_mem *rmem) +{ + struct cma *cma; + int ret; + + if (!of_get_flat_dt_prop(node, "reusable", NULL) || + of_get_flat_dt_prop(node, "no-map", NULL)) { + pr_err("FF-A lend pool: node must be 'reusable' and not 'no-map'\n"); + return -EINVAL; + } + + if (!IS_ALIGNED(rmem->base | rmem->size, CMA_MIN_ALIGNMENT_BYTES)) { + pr_err("FF-A lend pool: incorrect alignment of CMA region\n"); + return -EINVAL; + } + + ret = cma_init_reserved_mem(rmem->base, rmem->size, 0, rmem->name, &cma); + if (ret) { + pr_err("FF-A lend pool: unable to setup CMA region (%d)\n", ret); + return ret; + } + + rmem->priv = cma; + + return 0; +} + +static int ffa_lend_pool_device_init(struct reserved_mem *rmem, struct device *dev) +{ + int ret; + + if (!can_set_direct_map_range(pfn_to_page(PHYS_PFN(rmem->base)), rmem->size / PAGE_SIZE)) { + pr_err("FF-A lend pool: reserved memory cannot be unmapped in direct map\n"); + return -EINVAL; + } + + dev->cma_area = rmem->priv; + + ret = xa_err(xa_store(&ffa_lend_devices, (unsigned long)dev, dev, GFP_KERNEL)); + if (ret) + dev->cma_area = NULL; + + return ret; +} + +static void ffa_lend_pool_device_release(struct reserved_mem *rmem, struct device *dev) +{ + xa_erase(&ffa_lend_devices, (unsigned long)dev); + dev->cma_area = NULL; +} + +static const struct reserved_mem_ops ffa_lend_pool_ops = { + .node_init = ffa_lend_pool_setup, + .device_init = ffa_lend_pool_device_init, + .device_release = ffa_lend_pool_device_release, +}; +RESERVEDMEM_OF_DECLARE(ffa_lend_pool, "arm,ffa-lend-pool", &ffa_lend_pool_ops); diff --git a/include/linux/arm_ffa.h b/include/linux/arm_ffa.h index e71d83ee0aef..fa3570f85ff7 100644 --- a/include/linux/arm_ffa.h +++ b/include/linux/arm_ffa.h @@ -519,4 +519,25 @@ struct ffa_ops { const struct ffa_notifier_ops *notifier_ops; };
+#if IS_ENABLED(CONFIG_ARM_FFA_LEND_POOL) +int ffa_lend_pool_attach(struct device *dev); +void ffa_lend_pool_detach(struct device *dev); +int ffa_prepare_lend(struct device *dev, phys_addr_t paddr, size_t size); +void ffa_lend_reclaimed(struct device *dev, phys_addr_t paddr, size_t size); +#else +static inline int ffa_lend_pool_attach(struct device *dev) +{ + return -ENODEV; +} +static inline void ffa_lend_pool_detach(struct device *dev) +{ +} +static inline int ffa_prepare_lend(struct device *dev, phys_addr_t paddr, size_t size) +{ + return -ENODEV; +} +static inline void ffa_lend_reclaimed(struct device *dev, phys_addr_t paddr, size_t size) +{ +} +#endif #endif /* _LINUX_ARM_FFA_H */
On Wed, Sep 02, 2026 at 11:47:11AM +0100, Vincent Donnefort wrote:
When memory is lent to the Secure world via FF-A, fatal CPU speculative reads from Non-Secure can still occur as long as it retains a cacheable mapping to that memory. Introduce the "arm,ffa-lend-pool" reserved-memory CMA driver to unmap pages before lending (ffa_prepare_lend()) and restore them upon reclaim (ffa_lend_reclaimed()). Devices bind to the pool via the "memory-region" DT property or via ffa_lend_pool_attach().
reserved-memory { #address-cells = <0x2>; #size-cells = <0x2>; ranges;
ffa_lend: ffa-lend-pool { compatible = "arm,ffa-lend-pool";
Needs a binding schema...
reusable; ll-map; size = <0x0 0x4000000>; };};
Signed-off-by: Vincent Donnefort vdonnefort@google.com
[...]
+static int __init ffa_lend_pool_setup(unsigned long node, struct reserved_mem *rmem) +{
- struct cma *cma;
- int ret;
- if (!of_get_flat_dt_prop(node, "reusable", NULL) ||
of_get_flat_dt_prop(node, "no-map", NULL)) {pr_err("FF-A lend pool: node must be 'reusable' and not 'no-map'\n");return -EINVAL;
Your schema should enforce/check this.
[...]
+static const struct reserved_mem_ops ffa_lend_pool_ops = {
- .node_init = ffa_lend_pool_setup,
- .device_init = ffa_lend_pool_device_init,
- .device_release = ffa_lend_pool_device_release,
+}; +RESERVEDMEM_OF_DECLARE(ffa_lend_pool, "arm,ffa-lend-pool", &ffa_lend_pool_ops);
Do you need this early? If not, you can create a platform driver for "arm,ffa-lend-pool". ramoops is implemented that way for example.
Rob
On Wed, Sep 02, 2026 at 12:38:26PM -0500, Rob Herring wrote:
On Wed, Sep 02, 2026 at 11:47:11AM +0100, Vincent Donnefort wrote:
When memory is lent to the Secure world via FF-A, fatal CPU speculative reads from Non-Secure can still occur as long as it retains a cacheable mapping to that memory. Introduce the "arm,ffa-lend-pool" reserved-memory CMA driver to unmap pages before lending (ffa_prepare_lend()) and restore them upon reclaim (ffa_lend_reclaimed()). Devices bind to the pool via the "memory-region" DT property or via ffa_lend_pool_attach().
reserved-memory { #address-cells = <0x2>; #size-cells = <0x2>; ranges;
ffa_lend: ffa-lend-pool { compatible = "arm,ffa-lend-pool";Needs a binding schema...
Ack
reusable; ll-map; size = <0x0 0x4000000>; };};
Signed-off-by: Vincent Donnefort vdonnefort@google.com
[...]
+static int __init ffa_lend_pool_setup(unsigned long node, struct reserved_mem *rmem) +{
- struct cma *cma;
- int ret;
- if (!of_get_flat_dt_prop(node, "reusable", NULL) ||
of_get_flat_dt_prop(node, "no-map", NULL)) {pr_err("FF-A lend pool: node must be 'reusable' and not 'no-map'\n");return -EINVAL;Your schema should enforce/check this.
[...]
+static const struct reserved_mem_ops ffa_lend_pool_ops = {
- .node_init = ffa_lend_pool_setup,
- .device_init = ffa_lend_pool_device_init,
- .device_release = ffa_lend_pool_device_release,
+}; +RESERVEDMEM_OF_DECLARE(ffa_lend_pool, "arm,ffa-lend-pool", &ffa_lend_pool_ops);
Do you need this early? If not, you can create a platform driver for "arm,ffa-lend-pool". ramoops is implemented that way for example.
Rob
Only memblock_mark_llmap() must be called very early.
The rest just need to be in place before the TEE driver probed. So a platform driver should work.
Thanks!
Hook OP-TEE dynamically allocated protected memory pools to the "arm,ffa-lend-pool" driver. While the SMC transport platform device resolves the pool through its DT "memory-region" property, the FF-A transport lacks a device tree node and binds via ffa_lend_pool_attach().
Signed-off-by: Vincent Donnefort vdonnefort@google.com
diff --git a/drivers/tee/optee/ffa_abi.c b/drivers/tee/optee/ffa_abi.c index 633715b98625..4d6555171918 100644 --- a/drivers/tee/optee/ffa_abi.c +++ b/drivers/tee/optee/ffa_abi.c @@ -979,6 +979,7 @@ static void optee_ffa_remove(struct ffa_device *ffa_dev)
mutex_destroy(&optee->ffa.mutex); rhashtable_free_and_destroy(&optee->ffa.global_ids, rh_free_fn, NULL); + ffa_lend_pool_detach(&optee->teedev->dev);
kfree(optee); } @@ -1042,13 +1043,21 @@ static int optee_ffa_protmem_pool_init(struct optee *optee, u32 sec_caps) int rc = 0;
if (sec_caps & OPTEE_FFA_SEC_CAP_PROTMEM) { + rc = ffa_lend_pool_attach(&optee->teedev->dev); + if (rc && rc != -ENODEV) + return rc; + pool = optee_protmem_alloc_dyn_pool(optee, id); - if (IS_ERR(pool)) + if (IS_ERR(pool)) { + ffa_lend_pool_detach(&optee->teedev->dev); return PTR_ERR(pool); + }
rc = tee_device_register_dma_heap(optee->teedev, id, pool); - if (rc) + if (rc) { pool->ops->destroy_pool(pool); + ffa_lend_pool_detach(&optee->teedev->dev); + } }
return rc; diff --git a/drivers/tee/optee/protmem.c b/drivers/tee/optee/protmem.c index be3abf6e8aa6..9b64db9b4e64 100644 --- a/drivers/tee/optee/protmem.c +++ b/drivers/tee/optee/protmem.c @@ -42,14 +42,6 @@ static int init_dyn_protmem(struct optee_protmem_dyn_pool *rp) goto err_null_protmem; }
- /* - * TODO unmap the memory range since the physical memory will - * become inaccesible after the lend_protmem() call. - * - * If the platform supports a hypervisor at EL2, it will unmap the - * intermediate physical memory for us and stop cache pre-fetch of - * the memory. - */ rc = rp->optee->ops->lend_protmem(rp->optee, rp->protmem, rp->mem_attrs, rp->mem_attr_count, rp->use_case); diff --git a/drivers/tee/optee/smc_abi.c b/drivers/tee/optee/smc_abi.c index b8a2bdac3208..5949aa717cad 100644 --- a/drivers/tee/optee/smc_abi.c +++ b/drivers/tee/optee/smc_abi.c @@ -19,6 +19,7 @@ #include <linux/of.h> #include <linux/of_irq.h> #include <linux/of_platform.h> +#include <linux/of_reserved_mem.h> #include <linux/platform_device.h> #include <linux/rpmb.h> #include <linux/sched.h> @@ -1528,6 +1529,8 @@ static void optee_smc_remove(struct platform_device *pdev) if (optee->smc.memremaped_shm) memunmap(optee->smc.memremaped_shm);
+ of_reserved_mem_device_release(&optee->teedev->dev); + kfree(optee); }
@@ -1712,16 +1715,22 @@ static int optee_protmem_pool_init(struct optee *optee) if (!protm && !dyn_protm) return 0;
+ of_reserved_mem_device_init_by_idx(&optee->teedev->dev, + dev_of_node(optee->teedev->dev.parent), 0); if (protm) pool = static_protmem_pool_init(optee); if (dyn_protm && IS_ERR(pool)) pool = optee_protmem_alloc_dyn_pool(optee, heap_id); - if (IS_ERR(pool)) + if (IS_ERR(pool)) { + of_reserved_mem_device_release(&optee->teedev->dev); return PTR_ERR(pool); + }
rc = tee_device_register_dma_heap(optee->teedev, heap_id, pool); - if (rc) + if (rc) { pool->ops->destroy_pool(pool); + of_reserved_mem_device_release(&optee->teedev->dev); + }
return rc; } @@ -1833,14 +1842,14 @@ static int optee_probe(struct platform_device *pdev) (sec_caps & OPTEE_SMC_SEC_CAP_RPMB_PROBE)) optee->in_kernel_rpmb_routing = true;
- teedev = tee_device_alloc(&optee_clnt_desc, NULL, pool, optee); + teedev = tee_device_alloc(&optee_clnt_desc, &pdev->dev, pool, optee); if (IS_ERR(teedev)) { rc = PTR_ERR(teedev); goto err_free_optee; } optee->teedev = teedev;
- teedev = tee_device_alloc(&optee_supp_desc, NULL, pool, optee); + teedev = tee_device_alloc(&optee_supp_desc, &pdev->dev, pool, optee); if (IS_ERR(teedev)) { rc = PTR_ERR(teedev); goto err_unreg_teedev; diff --git a/drivers/tee/tee_shm.c b/drivers/tee/tee_shm.c index 6742b3579c86..49a9b2993c83 100644 --- a/drivers/tee/tee_shm.c +++ b/drivers/tee/tee_shm.c @@ -3,6 +3,7 @@ * Copyright (c) 2015-2017, 2019-2021 Linaro Limited */ #include <linux/anon_inodes.h> +#include <linux/arm_ffa.h> #include <linux/device.h> #include <linux/dma-buf.h> #include <linux/dma-mapping.h> @@ -43,6 +44,7 @@ static void tee_shm_release(struct tee_device *teedev, struct tee_shm *shm)
dma_mem = container_of(shm, struct tee_shm_dma_mem, shm); p = dma_mem; + ffa_lend_reclaimed(&teedev->dev, shm->paddr, shm->size); dma_free_pages(&teedev->dev, shm->size, dma_mem->page, dma_mem->dma_addr, DMA_BIDIRECTIONAL); #endif @@ -288,6 +290,7 @@ struct tee_shm *tee_shm_alloc_dma_mem(struct tee_context *ctx, struct tee_shm_dma_mem *dma_mem; dma_addr_t dma_addr; struct page *page; + int ret;
if (!tee_device_get(teedev)) return ERR_PTR(-EINVAL); @@ -297,9 +300,13 @@ struct tee_shm *tee_shm_alloc_dma_mem(struct tee_context *ctx, if (!page) goto err_put_teedev;
+ ret = ffa_prepare_lend(&teedev->dev, page_to_phys(page), page_count * PAGE_SIZE); + if (ret && ret != -ENODEV) + goto err_free_pages; + dma_mem = kzalloc_obj(*dma_mem); if (!dma_mem) - goto err_free_pages; + goto err_map_pages;
refcount_set(&dma_mem->shm.refcount, 1); dma_mem->shm.ctx = ctx; @@ -313,6 +320,8 @@ struct tee_shm *tee_shm_alloc_dma_mem(struct tee_context *ctx,
return &dma_mem->shm;
+err_map_pages: + ffa_lend_reclaimed(&teedev->dev, page_to_phys(page), page_count * PAGE_SIZE); err_free_pages: dma_free_pages(&teedev->dev, page_count * PAGE_SIZE, page, dma_addr, DMA_BIDIRECTIONAL);
This is of course v1, not v9...
On Wed, Sep 02, 2026 at 11:47:02AM +0100, Vincent Donnefort wrote:
This series is a follow-up to the discussion that has started here [1]. While standalone, it also provides primitives reusable for the VPR DMA heap.
When memory is lent to the Secure world via FF-A, CPU speculative accesses from NS to the lent pages can still occur as long as it retains a cacheable mapping to it.
Ideally, lent memory would be "no-map" but that would mean giving up MiBs of useful memory, so let's try to do better with the help of a CMA pool.
On arm64, modifying the direct map at runtime is generally restricted because the linear map defaults to block mapping and splitting blocks at runtime may trigger fatal page fault, unless the CPU implements BBML3 or the entire direct map was mapped at page granularity from boot. Forcing last-level mappings system-wide incurs a severe penalty we want to avoid. Instead, this series introduces targeted last-level mappings for designated memory regions, along with the "arm,ffa-lend-pool" CMA driver to manage unmapping and remapping on lend/reclaim transitions:
memblock & OF reserved memory ("ll-map"):
- Introduce MEMBLOCK_LLMAP and the DT "ll-map" property for reserved-memory nodes to force last-level (PTE) mappings only for a specific region.
set_memory infrastructure:
- Introduce can_set_direct_map_range() to check if a specific address range is mapped with last-level entries and can be modified safely.
- Introduce __set_direct_map_*() variants that bypass redundant checks when the caller has already validated the range.
"arm,ffa-lend-pool" driver
- Introduce the "arm,ffa-lend-pool" CMA reserved-memory driver, which unmaps pages prior to lending (ffa_prepare_lend()) and restores them when reclaimed (ffa_lend_reclaimed()).
Optee support
- Hook OP-TEE dynamic protected memory pools to "arm,ffa-lend-pool" for both SMC (via DT memory-region phandle) and FF-A (via ffa_lend_pool_attach()) transports.
Testing:
Tested with QEMU v8 using OP-TEE OS (built with CFG_CORE_DYN_PROTMEM=y) under both SMC and FF-A transports [2]
static void dump_direct_map(const char *label) { printf("\n=== %s ===\n", label); fflush(stdout); system("sed -n '/Linear Mapping start/,/Linear Mapping end/p' /sys/kernel/debug/kernel_page_tables"); fflush(stdout); }
int main(int argc, char *argv[]) { int heap_fd; int dmabuf_fd; struct dma_heap_allocation_data data = { 0 }; size_t size = 1024 * 1024; /* 1MB */
if (argc > 1) size = strtoul(argv[1], NULL, 0);
dump_direct_map("BEFORE ALLOCATION");
heap_fd = open("/dev/dma_heap/protected,secure-video", O_RDWR); if (heap_fd < 0) { perror("open /dev/dma_heap/protected,secure-video"); return 1; }
printf("\nOpened /dev/dma_heap/protected,secure-video\n"); printf("Allocating %zu bytes of protected memory via DMA heap...\n", size);
data.len = size; data.fd_flags = O_RDWR | O_CLOEXEC; if (ioctl(heap_fd, DMA_HEAP_IOCTL_ALLOC, &data) < 0) { perror("ioctl DMA_HEAP_IOCTL_ALLOC"); close(heap_fd); return 1; }
dmabuf_fd = data.fd; printf("Successfully allocated %zu bytes! dmabuf_fd = %d\n", size, dmabuf_fd);
dump_direct_map("DURING LEND (EXPECT HOLE IN DIRECT MAP)");
printf("\nReleasing dmabuf_fd...\n"); close(dmabuf_fd); close(heap_fd);
dump_direct_map("AFTER RECLAIM (RESTORED DIRECT MAP)");
return 0; }
[1] https://lore.kernel.org/all/20260807-tegra-vpr-v4-7-5510d16af89e@nvidia.com/ [2] https://optee.readthedocs.io/en/latest/building/gits/build.html#qemu-v8
Vincent Donnefort (10): memblock: Introduce MEMBLOCK_LLMAP of: reserved_mem: Introduce "ll-map" property set_memory.h: Introduce can_set_direct_map_range() set_memory.h: Introduce __set_direct_map*() arm64: can_set_direct_map() if BBML3 arm64: Implement can_set_direct_map_range() arm64: Implement __set_direct_map*() arm64: Add support for MEMBLOCK_LLMAP firmware: arm_ffa: Introduce ffa-lend-pool optee: Add support for arm,ffa-lend-pool
arch/arm64/include/asm/set_memory.h | 7 + arch/arm64/mm/mmu.c | 23 ++- arch/arm64/mm/pageattr.c | 67 +++++++- drivers/firmware/arm_ffa/Kconfig | 5 + drivers/firmware/arm_ffa/Makefile | 1 + drivers/firmware/arm_ffa/lend_pool.c | 223 +++++++++++++++++++++++++++ drivers/of/of_reserved_mem.c | 104 ++++++++++--- drivers/tee/optee/ffa_abi.c | 13 +- drivers/tee/optee/protmem.c | 8 - drivers/tee/optee/smc_abi.c | 17 +- drivers/tee/tee_shm.c | 11 +- include/linux/arm_ffa.h | 21 +++ include/linux/memblock.h | 9 ++ include/linux/set_memory.h | 39 +++++ mm/memblock.c | 50 ++++++ 15 files changed, 545 insertions(+), 53 deletions(-) create mode 100644 drivers/firmware/arm_ffa/lend_pool.c
base-commit: cee9395acd8043be0644b25c34bfa86623f2b935
2.55.0.970.g62bdec98f9-goog
op-tee@lists.trustedfirmware.org
-
Mike Rapoport -
Rob Herring -
Thierry Reding -
Vincent Donnefort