Hi,
I have been trying to understand the Arm’s FFA by reading Hafnium’s implementation, and recently discovered an inconsistency between the specification and the implementation regarding the sender’s mode of donated pages after an invocation of FFA_MEM_DONATE.
According to item 12.1 of 11.1.1.2 section in https://developer.arm.com/documentation/den0077/latest,
12. If the call executes successfully, the Relayer must:
1. Ensure that the state of the memory region in the participating FF-A components is observed as follows:
1. If the Receiver is a PE endpoint or a SEPID associated with a dependent peripheral device, then:
• Owner-NA for the Owner.
• !Owner-NA for the Receiver.
the sender(owner) is supposed to still own the pages, which is not the case in the implementation(https://git.trustedfirmware.org/hafnium/hafnium.git/tree/src… ).
I don’t know if there is a reason behind this, but I think at least a consequence of it is that the sender would be not able to invoke FFA_MEM_RECLAIM successfully before the receiver retrieves those pages, which implies possible “page leakage” if the receiver refuses to cooperate.
I also wrote a simple test case in which the sender invokes FFA_MEM_RECLAIM right after FFA_MEM_DONATE. As expected, Hafnium returned FFA_DENIED(https://git.trustedfirmware.org/hafnium/hafnium.git/tree/src/ffa… ).
If it is indeed a bug, then a simple fix could be removing “| MM_MODE_UNOWNED” at line 572.
Regards,
Zongyuan Liu
On Mon, 4 Jan 2021 at 17:03, Raghu Krishnamurthy via Hafnium <
hafnium(a)lists.trustedfirmware.org> wrote:
> Hi All,
>
> I was going through src/mm.c in the hafnium repo and noticed a couple of
> things that I think are problem:
>
> 1. mm_vm_defrag and mm_defrag, which eventually call mm_ptable_defrag
> appears to be changing page entries into block entries. Per D5.10.1 of ARM
> ARM, this should use break-before-make, but I don't see it being done in
> code. Am I missing where this is done in code, or is there some reason
> break-before-make is not required during defrag(that is not obvious to me
> from code or comments)?
> 2. In mm_replace_entry, line 278
> (https://git.trustedfirmware.org/hafnium/hafnium.git/tree/src/mm.c#n278)
> is
> not followed by a DSB + ISB as required by the break-before-make. Without
> the DSB, a load/store that uses a VA translated by the PTE being replaced
> could be reordered before line 278 and fault. Once again, this looks like a
> bug to me but is there some reason that is not obvious due to which it does
> not require barriers?
>
It looks like that code was added in
https://hafnium-review.googlesource.com/c/hafnium/+/3260. Maybe Andrew
Scull or Wedson can comment, if they can remember?
Hi By Yinhua,
Your experiment fails for two main reasons:
1/ you're attempting to load OP-TEE as a normal world VM whereas OP-TEE is a TEE meant to run in TZ secure world.
2/ you're using qemu which is not yet capable of emulating S-EL2. At present, you'd need to use Arm's FVP.
Can you confirm you wish to run OP-TEE as TEE on top of Hafnium in the secure world?
Clearer instructions on how to run OP-TEE as a Secure Partition shall follow in coming days from Arm's platform team.
Regards,
Olivier.
________________________________________
From: Hafnium <hafnium-bounces(a)lists.trustedfirmware.org> on behalf of 陈 胤桦 via Hafnium <hafnium(a)lists.trustedfirmware.org>
Sent: 15 January 2021 09:48
To: hafnium(a)lists.trustedfirmware.org
Subject: [Hafnium] load TEE systems(like OPTEE) on hafnium
Hi all :
I follow the "Getting Started" instructions and run linux as the primary VM on hafnium successfully.
Then, I try to load OP-TEE system as a secondary VM on hafnium but fail.
Is there any document which can help me load TEE system on hafnium?
If not, could you give me some guidance and tips?
The following is the efforts I made :
------
1. I write the file "manifest.dts" as follows:
/dts-v1/;
/ {
hypervisor {
compatible = "hafnium,hafnium";
ffa_tee;
vm1 {
debug_name = "primary VM -chenyinhua";
kernel_filename = "vmlinuz";
ramdisk_filename = "initrd.img";
smc_whitelist = <
0x04000000
0x3200ffff
>;
};
vm2 {
debug_name = "secondary VM 1";
kernel_filename = "kernel0";
vcpu_count = <2>;
mem_size = <0x100000>;
smc_whitelist_permissive;
};
};
};
1.
Run prebuilts/linux-x64/dtc/dtc -I dts -O dtb --out-version 17 -o manifest.dtb manifest.dts
2.
Move "manifest.dtb" into "initrd" directory
3.
Follow this blog to build OP-TEE: https://blog.csdn.net/dddddttttt/article/details/80793453, now I can run optee_examples successfully.
4.
In <opentee-root-dir>/optee_os/out/arm/core, copy the file "tee.bin" into <hafnium-root-dir>/initrd
5.
Rename <hafnium-root-dir>/initrd/tee.bin to <hafnium-root-dir>/initrd/kernel0
6.
In <hafnium-root-dir>/initrd/ , run the following command (The linux's vmlinuz and initrd.img are already in the directory)
find . | cpio -o > ../initrd.img
8. In <hafnium-root-dir>, run
qemu-system-aarch64 \
-M virt,gic_version=3 \
-cpu cortex-a57 -nographic -machine virtualization=true \
-kernel out/reference/qemu_aarch64_clang/hafnium.bin \
-initrd initrd.img -append "rdinit=/sbin/init"
9. The result is as follows:
NOTICE: Initialising hafnium
INFO: text: 0x40001000 - 0x4001b000
INFO: rodata: 0x4001b000 - 0x4001f000
INFO: data: 0x4001f000 - 0x400b7000
INFO: Supported bits in physical address: 44
INFO: Stage 2 has 4 page table levels with 1 pages at the root.
INFO: Found PSCI version: 0x2
INFO: Memory range: 0x40000000 - 0x47ffffff
INFO: Ramdisk range: 0x44000000 - 0x45aa8bff
WARNING: Device memory not provided, defaulting to 1 TB.
INFO: Loaded primary VM with 8 vCPUs, entry at 0x40280000.
INFO: Loading VM id 0x2: secondary VM 1.
INFO: Loaded with 2 vCPUs, entry at 0x47f00000.
INFO: Loading VM id 0x3: secondary VM 2.
INFO: Loaded with 4 vCPUs, entry at 0x47d00000.
./qboot.sh: line 5: 3397 Segmentation fault (core dumped) qemu-system-aarch64 -M virt,gic_version=3 -cpu cortex-a57 -nographic -machine virtualization=true -kernel out/reference/qemu_aarch64_clang/hafnium.bin -initrd initrd.img -append "rdinit=/sbin/init"
I appreciate for any possible help!
Thanks.
------
By Yinhua
--
Hafnium mailing list
Hafnium(a)lists.trustedfirmware.org
https://lists.trustedfirmware.org/mailman/listinfo/hafnium
Hi all :
I follow the "Getting Started" instructions and run linux as the primary VM on hafnium successfully.
Then, I try to load OP-TEE system as a secondary VM on hafnium but fail.
Is there any document which can help me load TEE system on hafnium?
If not, could you give me some guidance and tips?
The following is the efforts I made :
------
1. I write the file "manifest.dts" as follows:
/dts-v1/;
/ {
hypervisor {
compatible = "hafnium,hafnium";
ffa_tee;
vm1 {
debug_name = "primary VM -chenyinhua";
kernel_filename = "vmlinuz";
ramdisk_filename = "initrd.img";
smc_whitelist = <
0x04000000
0x3200ffff
>;
};
vm2 {
debug_name = "secondary VM 1";
kernel_filename = "kernel0";
vcpu_count = <2>;
mem_size = <0x100000>;
smc_whitelist_permissive;
};
};
};
1.
Run prebuilts/linux-x64/dtc/dtc -I dts -O dtb --out-version 17 -o manifest.dtb manifest.dts
2.
Move "manifest.dtb" into "initrd" directory
3.
Follow this blog to build OP-TEE: https://blog.csdn.net/dddddttttt/article/details/80793453, now I can run optee_examples successfully.
4.
In <opentee-root-dir>/optee_os/out/arm/core, copy the file "tee.bin" into <hafnium-root-dir>/initrd
5.
Rename <hafnium-root-dir>/initrd/tee.bin to <hafnium-root-dir>/initrd/kernel0
6.
In <hafnium-root-dir>/initrd/ , run the following command (The linux's vmlinuz and initrd.img are already in the directory)
find . | cpio -o > ../initrd.img
8. In <hafnium-root-dir>, run
qemu-system-aarch64 \
-M virt,gic_version=3 \
-cpu cortex-a57 -nographic -machine virtualization=true \
-kernel out/reference/qemu_aarch64_clang/hafnium.bin \
-initrd initrd.img -append "rdinit=/sbin/init"
9. The result is as follows:
NOTICE: Initialising hafnium
INFO: text: 0x40001000 - 0x4001b000
INFO: rodata: 0x4001b000 - 0x4001f000
INFO: data: 0x4001f000 - 0x400b7000
INFO: Supported bits in physical address: 44
INFO: Stage 2 has 4 page table levels with 1 pages at the root.
INFO: Found PSCI version: 0x2
INFO: Memory range: 0x40000000 - 0x47ffffff
INFO: Ramdisk range: 0x44000000 - 0x45aa8bff
WARNING: Device memory not provided, defaulting to 1 TB.
INFO: Loaded primary VM with 8 vCPUs, entry at 0x40280000.
INFO: Loading VM id 0x2: secondary VM 1.
INFO: Loaded with 2 vCPUs, entry at 0x47f00000.
INFO: Loading VM id 0x3: secondary VM 2.
INFO: Loaded with 4 vCPUs, entry at 0x47d00000.
./qboot.sh: line 5: 3397 Segmentation fault (core dumped) qemu-system-aarch64 -M virt,gic_version=3 -cpu cortex-a57 -nographic -machine virtualization=true -kernel out/reference/qemu_aarch64_clang/hafnium.bin -initrd initrd.img -append "rdinit=/sbin/init"
I appreciate for any possible help!
Thanks.
------
By Yinhua
Hi All,
I was going through src/mm.c in the hafnium repo and noticed a couple of
things that I think are problem:
1. mm_vm_defrag and mm_defrag, which eventually call mm_ptable_defrag
appears to be changing page entries into block entries. Per D5.10.1 of ARM
ARM, this should use break-before-make, but I don't see it being done in
code. Am I missing where this is done in code, or is there some reason
break-before-make is not required during defrag(that is not obvious to me
from code or comments)?
2. In mm_replace_entry, line 278
(https://git.trustedfirmware.org/hafnium/hafnium.git/tree/src/mm.c#n278) is
not followed by a DSB + ISB as required by the break-before-make. Without
the DSB, a load/store that uses a VA translated by the PTE being replaced
could be reordered before line 278 and fault. Once again, this looks like a
bug to me but is there some reason that is not obvious due to which it does
not require barriers?
Thanks
Raghu
Hi Raghu,
one more
5/ maybe answer to 2/3/4 is that it requires an EL1-shim embedded into Hafnium which itself ERETs to a S-EL0 partition?
BTW notice my questions are obviously oriented towards the secure implementation.
Regards,
Olivier.
________________________________________
From: Hafnium <hafnium-bounces(a)lists.trustedfirmware.org> on behalf of Olivier Deprez via Hafnium <hafnium(a)lists.trustedfirmware.org>
Sent: 15 December 2020 09:10
To: hafnium(a)lists.trustedfirmware.org; raghu.ncstate(a)icloud.com
Subject: Re: [Hafnium] VHE support
Hi Raghu,
Thanks for sharing this work.
Few thoughts...
1/ I guess it requires an additional change in project/reference to eventually enable the feature for a platform (enable_vhe=1 in the gn build flow)?
Sorry if my questions below sound obvious, I may miss bits of the VHE architecture.
2/ The changes are effectively toggling HCR_EL2.E2H=1. Currently when Hafnium ERETs resuming a SP, this happens with the secure EL1&0 translation regime.
Though what's the next step forward?
Does this require a host OS in secure world? (like linux does when booting under a Hypervisor with VHE enabled?) Would this be a VHE-enabled TOS?
3/ Is the later goal to enable the secure EL2&0 translation regime?
4/ Is there anything to do with HCR_EL2.TGE?
Regards,
Olivier.
________________________________________
From: Hafnium <hafnium-bounces(a)lists.trustedfirmware.org> on behalf of Raghu Krishnamurthy via Hafnium <hafnium(a)lists.trustedfirmware.org>
Sent: 15 December 2020 04:57
To: hafnium(a)lists.trustedfirmware.org
Subject: [Hafnium] VHE support
Hi All,
I have a series of patches pushed to Gerrit at
https://review.trustedfirmware.org/c/hafnium/hafnium/+/7599 with topic
"vhe_enable". The goal of this patch series is to enable VM's in both secure
and normal world to run with VHE enabled(hcr_el2.e2h=1), without breaking
any existing functionality. This is expected to be the first step in the
long term goal of enabling S-EL0 partitions(and optionally EL0 partitions),
that require VHE support, per the FF-A 1.0 Spec. I'd appreciate feedback on
the patches and approach taken to nominally enabling VHE. Note that the FF-A
1.0 spec(AFAIK) does not expect VHE support in the normal world but this
patch series enables it anyway due to the wealth of available tests in the
hafnium test suite to help with providing confidence in the implementation.
The patch series has been tested as follows:
Hafnium tests using QEMU(prebuilt in the hafnium repo) - Without VHE, since
the prebuilt QEMU does not support VHE.
Hafnium tests using QEMU(5.2-RC4, built from source) - With and without VHE,
this version of QEMU supports VHE.
Hafnium tests using FVP 11.12.28 - With and Without VHE.
TFTF tests for secure hafnium using FVP 11.12.28 - With and without VHE.
Thanks
Raghu
--
Hafnium mailing list
Hafnium(a)lists.trustedfirmware.org
https://lists.trustedfirmware.org/mailman/listinfo/hafnium
--
Hafnium mailing list
Hafnium(a)lists.trustedfirmware.org
https://lists.trustedfirmware.org/mailman/listinfo/hafnium
Hi Achin,
> HCR_EL2.{E2H,TGE} = {1,1} reduces the number of translation stages managed by
> the SPMC for S-EL0 SPs. It allows them to be managed like apps instead of VMs
> which is the case pre-v8.4. I think this is the better configuration for a
> system that only deploys S-EL0 SPs.
If the system only deploys S-EL0 SPs in this configuration, what's the benefit of using Hafnium in place of a S-EL1 TEE?
(appart maybe from product / SW scalability perspective?)
Regards,
Olivier.
________________________________________
From: Hafnium <hafnium-bounces(a)lists.trustedfirmware.org> on behalf of Achin Gupta via Hafnium <hafnium(a)lists.trustedfirmware.org>
Sent: 19 December 2020 15:24
To: raghu.ncstate(a)icloud.com
Cc: hafnium(a)lists.trustedfirmware.org; 'Mayur Gudmeti'; 'Joakim Bech'
Subject: Re: [Hafnium] VHE support
Hi Raghu,
On Fri, Dec 18, 2020 at 02:39:44PM -0800, raghu.ncstate(a)icloud.com wrote:
> Thanks Achin.
>
> >> Now if a S-EL0 SP is run in this configuration, irrespective of how TGE is configured, the SPMC would have to manage both the Stage 1 and 2 translations on behalf of the S-EL0 SP in the Secure EL1&0 translation regime,
> [RK] Not sure I understand the statement "irrespective of how TGE is configured". If HCR_EL2.E2H,TGE=1,1, then SPMC would manage Secure EL2&0 translation regime for the S-EL0 SP and crucially, no stage 2 translations. Your statement applies only when HCR_EL2.{E2H,TGE} = {1,1}. The statement right after this contradicts "irrespective of how TGE is configured".
Apologies. I meant E2H and not TGE i.e. if HCR_EL2.TGE=0, then the SPMC has to
manage both the Stage 1 and 2 translations on behalf of a S-EL0 SP in the
Secure EL1&0 translation regime,
>
> >> Management of a S-EL0 SP in the earlier configuration is expected to be easier.
> [RK] Sorry, which configuration? Earlier is vague.
HCR_EL2.{E2H,TGE} = {1,1}
>
> >> Does this seem reasonable?
> Yes, this is reasonable. However, what configuration of HCR_EL2.{E2H, TGE} should hafnium use to manage S-EL0 SP's? Should it use {0, 1} (not in spec) or {1, 1}( Section 2.2.1, Bullet 1)?
HCR_EL2.{E2H,TGE} = {1,1} reduces the number of translation stages managed by
the SPMC for S-EL0 SPs. It allows them to be managed like apps instead of VMs
which is the case pre-v8.4. I think this is the better configuration for a
system that only deploys S-EL0 SPs.
The spec does not specify the values of E2H or TGE. So {0, 1} is not disallowed
by the spec either.
Looks like that the choices we have for Hafnium are:
1. {1, 1} for S-EL0 SPs and {1, 0} for S-EL1 SPs
2. {0, 1} for S-EL0 SPs and {0, 0} for S-EL1 SPs
We would need to understand the impact of these choices on the Hafnium
implementation before making a call.
The v1.1 spec can recommend the sensible choices to avoid confusion.
cheers,
Achin
>
> Thanks
> Raghu
>
> -----Original Message-----
> From: Achin Gupta <achin.gupta(a)arm.com>
> Sent: Friday, December 18, 2020 1:57 PM
> To: François Ozog <francois.ozog(a)linaro.org>; raghu.ncstate(a)icloud.com
> Cc: raghu.ncstate(a)icloud.com; hafnium(a)lists.trustedfirmware.org; Mayur Gudmeti <mgudmeti(a)nvidia.com>; Ruchika Gupta <ruchika.gupta(a)linaro.org>; Joakim Bech <joakim.bech(a)linaro.org>
> Subject: Re: [Hafnium] VHE support
>
> Hi Francois and Raghu,
>
> Some comments inline...
>
> On Fri, Dec 18, 2020 at 04:12:55PM +0100, François Ozog wrote:
> >
> >
> > On Fri, 18 Dec 2020 at 15:59, <raghu.ncstate(a)icloud.com> wrote:
> >
> >
> > Hi Francois,
> >
> >
> >
> > OPTEE should work with and without VHE enabled and be able to communicate
> > with normal world. VHE should be entirely transparent to S-EL1 partitions/
> > VM’s. Is there a reason you think VHE may not be transparent?
> >
> >
> >
> > that's what I thought when I red "With VHE, the S-EL1 exception level
> > disappears." earlier in the thread.
>
> Raghu is right that the presence or absence of VHE does not matter to OP-TEE or any S-EL1 SP. It is an S-EL2 knob.
>
> Just to be clear, "With VHE, the S-EL1 exception level disappears" is the configuration when HCR_EL2.{E2H, TGE} == {1, 1}.
>
> In this configuration, the S-EL0 SPs are controlled by the SPMC just like Host Apps are controlled by the Host OS in the link from Raghu. OP-TEE or any S-EL1 SP cannot run in this setup.
>
> The key point from the spec's perspective is that the SPMC manages the Secure
> EL2&0 translation regime. I will come back to this.
>
> The configuration in which OP-TEE runs as a VM is when HCR_EL2.{E2H, TGE} == {1, 0} or {0, 0}. The key point is that TGE=0. Whether E2H is set or not is a Hypervisor choice. It does not matter whether VHE is being used or not from the
> S-EL1 SP's perspective.
>
> In this configuration, the SPMC manages the Stage 2 translations on behalf of OP-TEE in the Secure EL1&0 translation regime,
>
> Now if a S-EL0 SP is run in this configuration, irrespective of how TGE is configured, the SPMC would have to manage both the Stage 1 and 2 translations on behalf of the S-EL0 SP in the Secure EL1&0 translation regime,
>
> Going back to the key point above, in the earlier configuration (HCR_EL2.{E2H, TGE} == {1, 1}), the SPMC would have to manage only the Stage 1 translations in Secure EL2&0 translation regime.
>
> The spec is trying to highlight this difference. Management of a S-EL0 SP in the earlier configuration is expected to be easier. Also, this is the closest we get to the FF-A configuration where the SPMC in EL3 manages S-EL0 SPs directly. The SPMC manages the Stage 1 translations in Secure EL1&0 translation regime in this case.
>
> In short, in Section 2.2.1,
>
> - In bullet 1, the spec has HCR_EL2.{E2H, TGE} == {1, 1} in mind.
>
> - In bullet 2, the spec has HCR_EL2.TGE == 0 in mind.
>
> Does this seem reasonable?
>
> >
> >
> >
> > The third picture in the link below helped me visualize the system with
> > VHE:
> >
> > https://developer.arm.com/architectures/learn-the-architecture/
> > aarch64-virtualization/virtualization-host-extensions
> >
> >
> >
> > Hypervisor/Host OS is Hafnium, Guest OS will be OPTEE, Guest Apps with be
> > TA’s. Host App’s with be S-EL0 partitions.
>
> Completely agree.
>
> cheers,
> Achin
>
> >
> >
> >
> > Thanks
> >
> > Raghu
> >
> >
> >
> > From: François Ozog <francois.ozog(a)linaro.org>
> > Sent: Friday, December 18, 2020 4:03 AM
> > To: Achin Gupta <achin.gupta(a)arm.com>
> > Cc: raghu.ncstate(a)icloud.com; hafnium(a)lists.trustedfirmware.org; Mayur
> > Gudmeti <mgudmeti(a)nvidia.com>; Ruchika Gupta <ruchika.gupta(a)linaro.org>;
> > Joakim Bech <joakim.bech(a)linaro.org>
> > Subject: Re: [Hafnium] VHE support
> >
> >
> >
> >
> >
> >
> >
> > On Thu, 17 Dec 2020 at 17:44, Achin Gupta via Hafnium <
> > hafnium(a)lists.trustedfirmware.org> wrote:
> >
> > Hi Raghu,
> >
> > On Tue, Dec 15, 2020 at 09:41:05AM -0800, raghu.ncstate(a)icloud.com
> > wrote:
> > > Hi Olivier,
> > >
> >
> > [snip]
> >
> > >
> > > Hi Achin,
> > >
> > > Question for you. I interpreted the FF-A 1.0 spec as "requiring" VHE
> > for
> > > S-EL0 partitions, based on options 1 and 2 in section 2.2.1, since
> > option 2
> > > does not mention S-EL0 partitions. Can you confirm this is the case?
> > As I
> > > was thinking through this, it seems like we might be able to pull off
> > EL0
> > > partitions without using VHE, i.e by using HCR_EL2.E2H = 0 and
> > > HCR_EL2.TGE=1.
> > > HCR_EL2.TGE is present even on ARMv8.0 architecture and removes
> > dependence
> > > on VHE, and effectively removes the necessity for this patch series.
> > > However, I'd like to understand your thoughts on why VHE is suggested
> > in the
> > > spec and if hafnium should use VHE for S-EL0 partitions or if hafnium
> > can
> > > use the approach suggested above.
> >
> > The intent in the spec is not to mandate VHE for S-EL0 SPs. We can add
> > a
> > clarification if that helps.
> >
> > The intent in the spec is to highlight that VHE is applicable to S-EL0
> > SPs only.
> >
> > With VHE, the S-EL1 exception level disappears. The SPMC can only have
> > awareness
> > of S-EL0 SPs.
> >
> > Without WHE, the S-EL1 exception level is present. As you state above,
> > there are
> > knobs in the architecture that reduce the role of S-EL1. But based upon
> > my
> > current understanding, the SPMC will see S-EL1 as the next lower
> > exception
> > level.
> >
> > If we want OPTEE to be running a number of Apps in one Secure world VM, we
> > wouldn't need WHE right?
> >
> > Can you confirm that without enabling VHE, an OPTEE VM, and its containing
> > apps, is possible and can communicate with normal world VMs?
> >
> >
> >
> > A S-EL0 SP can be "wrapped" as a S-EL1 SP such that the role of S-EL1
> > is
> > reduced. This could be done in SW (e.g. a shim layer in S-EL1), HW
> > (e.g. the TGE
> > bit above) or both. But architecturally, the SPMC will still see S-EL1.
> >
> > I hope this clarifies. That said, I do need to double check the
> > original intent
> > and impact of TGE. So please do correct me if I have misinterpreted
> > anything.
> >
> > cheers,
> > Achin
> >
> > >
> > >
> > > Thanks
> > > Raghu
> > >
> > >
> > > -----Original Message-----
> > > From: Olivier Deprez <Olivier.Deprez(a)arm.com>
> > > Sent: Tuesday, December 15, 2020 12:18 AM
> > > To: hafnium(a)lists.trustedfirmware.org; raghu.ncstate(a)icloud.com;
> > Olivier
> > > Deprez <Olivier.Deprez(a)arm.com>
> > > Subject: Re: [Hafnium] VHE support
> > >
> > > Hi Raghu,
> > >
> > > one more
> > >
> > > 5/ maybe answer to 2/3/4 is that it requires an EL1-shim embedded
> > into
> > > Hafnium which itself ERETs to a S-EL0 partition?
> > >
> > > BTW notice my questions are obviously oriented towards the secure
> > > implementation.
> > >
> > > Regards,
> > > Olivier.
> > >
> > > ________________________________________
> > > From: Hafnium <hafnium-bounces(a)lists.trustedfirmware.org> on behalf
> > of
> > > Olivier Deprez via Hafnium <hafnium(a)lists.trustedfirmware.org>
> > > Sent: 15 December 2020 09:10
> > > To: hafnium(a)lists.trustedfirmware.org; raghu.ncstate(a)icloud.com
> > > Subject: Re: [Hafnium] VHE support
> > >
> > > Hi Raghu,
> > >
> > > Thanks for sharing this work.
> > >
> > > Few thoughts...
> > >
> > > 1/ I guess it requires an additional change in project/reference to
> > > eventually enable the feature for a platform (enable_vhe=1 in the gn
> > build
> > > flow)?
> > >
> > > Sorry if my questions below sound obvious, I may miss bits of the VHE
> > > architecture.
> > >
> > > 2/ The changes are effectively toggling HCR_EL2.E2H=1. Currently when
> > > Hafnium ERETs resuming a SP, this happens with the secure EL1&0
> > translation
> > > regime.
> > > Though what's the next step forward?
> > > Does this require a host OS in secure world? (like linux does when
> > booting
> > > under a Hypervisor with VHE enabled?) Would this be a VHE-enabled
> > TOS?
> > >
> > > 3/ Is the later goal to enable the secure EL2&0 translation regime?
> > >
> > > 4/ Is there anything to do with HCR_EL2.TGE?
> > >
> > > Regards,
> > > Olivier.
> > >
> > >
> > > ________________________________________
> > > From: Hafnium <hafnium-bounces(a)lists.trustedfirmware.org> on behalf
> > of Raghu
> > > Krishnamurthy via Hafnium <hafnium(a)lists.trustedfirmware.org>
> > > Sent: 15 December 2020 04:57
> > > To: hafnium(a)lists.trustedfirmware.org
> > > Subject: [Hafnium] VHE support
> > >
> > > Hi All,
> > >
> > >
> > >
> > > I have a series of patches pushed to Gerrit at
> > > https://review.trustedfirmware.org/c/hafnium/hafnium/+/7599 with
> > topic
> > > "vhe_enable". The goal of this patch series is to enable VM's in both
> > secure
> > > and normal world to run with VHE enabled(hcr_el2.e2h=1), without
> > breaking
> > > any existing functionality. This is expected to be the first step in
> > the
> > > long term goal of enabling S-EL0 partitions(and optionally EL0
> > partitions),
> > > that require VHE support, per the FF-A 1.0 Spec. I'd appreciate
> > feedback on
> > > the patches and approach taken to nominally enabling VHE. Note that
> > the FF-A
> > > 1.0 spec(AFAIK) does not expect VHE support in the normal world but
> > this
> > > patch series enables it anyway due to the wealth of available tests
> > in the
> > > hafnium test suite to help with providing confidence in the
> > implementation.
> > >
> > >
> > >
> > > The patch series has been tested as follows:
> > >
> > >
> > >
> > > Hafnium tests using QEMU(prebuilt in the hafnium repo) - Without VHE,
> > since
> > > the prebuilt QEMU does not support VHE.
> > >
> > > Hafnium tests using QEMU(5.2-RC4, built from source) - With and
> > without VHE,
> > > this version of QEMU supports VHE.
> > >
> > > Hafnium tests using FVP 11.12.28 - With and Without VHE.
> > >
> > > TFTF tests for secure hafnium using FVP 11.12.28 - With and without
> > VHE.
> > >
> > >
> > >
> > >
> > >
> > > Thanks
> > >
> > > Raghu
> > >
> > > --
> > > Hafnium mailing list
> > > Hafnium(a)lists.trustedfirmware.org
> > > https://lists.trustedfirmware.org/mailman/listinfo/hafnium
> > > --
> > > Hafnium mailing list
> > > Hafnium(a)lists.trustedfirmware.org
> > > https://lists.trustedfirmware.org/mailman/listinfo/hafnium
> > >
> > --
> > Hafnium mailing list
> > Hafnium(a)lists.trustedfirmware.org
> > https://lists.trustedfirmware.org/mailman/listinfo/hafnium
> >
> >
> >
> >
> > --
> >
> > [uc] François-Frédéric Ozog | Director Linaro Edge & Fog Computing
> > Group
> >
> > T: +33.67221.6485
> > francois.ozog(a)linaro.org | Skype: ffozog
> >
> >
> >
> >
> >
> >
> > --
> > [uc] François-Frédéric Ozog | Director Linaro Edge & Fog Computing Group
> > T: +33.67221.6485
> > francois.ozog(a)linaro.org | Skype: ffozog
> >
>
--
Hafnium mailing list
Hafnium(a)lists.trustedfirmware.org
https://lists.trustedfirmware.org/mailman/listinfo/hafnium
On Thu, 17 Dec 2020 at 17:44, Achin Gupta via Hafnium <
hafnium(a)lists.trustedfirmware.org> wrote:
> Hi Raghu,
>
> On Tue, Dec 15, 2020 at 09:41:05AM -0800, raghu.ncstate(a)icloud.com wrote:
> > Hi Olivier,
> >
>
> [snip]
>
> >
> > Hi Achin,
> >
> > Question for you. I interpreted the FF-A 1.0 spec as "requiring" VHE for
> > S-EL0 partitions, based on options 1 and 2 in section 2.2.1, since
> option 2
> > does not mention S-EL0 partitions. Can you confirm this is the case? As I
> > was thinking through this, it seems like we might be able to pull off EL0
> > partitions without using VHE, i.e by using HCR_EL2.E2H = 0 and
> > HCR_EL2.TGE=1.
> > HCR_EL2.TGE is present even on ARMv8.0 architecture and removes
> dependence
> > on VHE, and effectively removes the necessity for this patch series.
> > However, I'd like to understand your thoughts on why VHE is suggested in
> the
> > spec and if hafnium should use VHE for S-EL0 partitions or if hafnium can
> > use the approach suggested above.
>
> The intent in the spec is not to mandate VHE for S-EL0 SPs. We can add a
> clarification if that helps.
>
> The intent in the spec is to highlight that VHE is applicable to S-EL0 SPs
> only.
>
> With VHE, the S-EL1 exception level disappears. The SPMC can only have
> awareness
> of S-EL0 SPs.
>
> Without WHE, the S-EL1 exception level is present. As you state above,
> there are
> knobs in the architecture that reduce the role of S-EL1. But based upon my
> current understanding, the SPMC will see S-EL1 as the next lower exception
> level.
>
> If we want OPTEE to be running a number of Apps in one Secure world VM, we
wouldn't need WHE right?
Can you confirm that without enabling VHE, an OPTEE VM, and its containing
apps, is possible and can communicate with normal world VMs?
A S-EL0 SP can be "wrapped" as a S-EL1 SP such that the role of S-EL1 is
> reduced. This could be done in SW (e.g. a shim layer in S-EL1), HW (e.g.
> the TGE
> bit above) or both. But architecturally, the SPMC will still see S-EL1.
>
> I hope this clarifies. That said, I do need to double check the original
> intent
> and impact of TGE. So please do correct me if I have misinterpreted
> anything.
>
> cheers,
> Achin
>
> >
> >
> > Thanks
> > Raghu
> >
> >
> > -----Original Message-----
> > From: Olivier Deprez <Olivier.Deprez(a)arm.com>
> > Sent: Tuesday, December 15, 2020 12:18 AM
> > To: hafnium(a)lists.trustedfirmware.org; raghu.ncstate(a)icloud.com; Olivier
> > Deprez <Olivier.Deprez(a)arm.com>
> > Subject: Re: [Hafnium] VHE support
> >
> > Hi Raghu,
> >
> > one more
> >
> > 5/ maybe answer to 2/3/4 is that it requires an EL1-shim embedded into
> > Hafnium which itself ERETs to a S-EL0 partition?
> >
> > BTW notice my questions are obviously oriented towards the secure
> > implementation.
> >
> > Regards,
> > Olivier.
> >
> > ________________________________________
> > From: Hafnium <hafnium-bounces(a)lists.trustedfirmware.org> on behalf of
> > Olivier Deprez via Hafnium <hafnium(a)lists.trustedfirmware.org>
> > Sent: 15 December 2020 09:10
> > To: hafnium(a)lists.trustedfirmware.org; raghu.ncstate(a)icloud.com
> > Subject: Re: [Hafnium] VHE support
> >
> > Hi Raghu,
> >
> > Thanks for sharing this work.
> >
> > Few thoughts...
> >
> > 1/ I guess it requires an additional change in project/reference to
> > eventually enable the feature for a platform (enable_vhe=1 in the gn
> build
> > flow)?
> >
> > Sorry if my questions below sound obvious, I may miss bits of the VHE
> > architecture.
> >
> > 2/ The changes are effectively toggling HCR_EL2.E2H=1. Currently when
> > Hafnium ERETs resuming a SP, this happens with the secure EL1&0
> translation
> > regime.
> > Though what's the next step forward?
> > Does this require a host OS in secure world? (like linux does when
> booting
> > under a Hypervisor with VHE enabled?) Would this be a VHE-enabled TOS?
> >
> > 3/ Is the later goal to enable the secure EL2&0 translation regime?
> >
> > 4/ Is there anything to do with HCR_EL2.TGE?
> >
> > Regards,
> > Olivier.
> >
> >
> > ________________________________________
> > From: Hafnium <hafnium-bounces(a)lists.trustedfirmware.org> on behalf of
> Raghu
> > Krishnamurthy via Hafnium <hafnium(a)lists.trustedfirmware.org>
> > Sent: 15 December 2020 04:57
> > To: hafnium(a)lists.trustedfirmware.org
> > Subject: [Hafnium] VHE support
> >
> > Hi All,
> >
> >
> >
> > I have a series of patches pushed to Gerrit at
> > https://review.trustedfirmware.org/c/hafnium/hafnium/+/7599 with topic
> > "vhe_enable". The goal of this patch series is to enable VM's in both
> secure
> > and normal world to run with VHE enabled(hcr_el2.e2h=1), without breaking
> > any existing functionality. This is expected to be the first step in the
> > long term goal of enabling S-EL0 partitions(and optionally EL0
> partitions),
> > that require VHE support, per the FF-A 1.0 Spec. I'd appreciate feedback
> on
> > the patches and approach taken to nominally enabling VHE. Note that the
> FF-A
> > 1.0 spec(AFAIK) does not expect VHE support in the normal world but this
> > patch series enables it anyway due to the wealth of available tests in
> the
> > hafnium test suite to help with providing confidence in the
> implementation.
> >
> >
> >
> > The patch series has been tested as follows:
> >
> >
> >
> > Hafnium tests using QEMU(prebuilt in the hafnium repo) - Without VHE,
> since
> > the prebuilt QEMU does not support VHE.
> >
> > Hafnium tests using QEMU(5.2-RC4, built from source) - With and without
> VHE,
> > this version of QEMU supports VHE.
> >
> > Hafnium tests using FVP 11.12.28 - With and Without VHE.
> >
> > TFTF tests for secure hafnium using FVP 11.12.28 - With and without VHE.
> >
> >
> >
> >
> >
> > Thanks
> >
> > Raghu
> >
> > --
> > Hafnium mailing list
> > Hafnium(a)lists.trustedfirmware.org
> > https://lists.trustedfirmware.org/mailman/listinfo/hafnium
> > --
> > Hafnium mailing list
> > Hafnium(a)lists.trustedfirmware.org
> > https://lists.trustedfirmware.org/mailman/listinfo/hafnium
> >
> --
> Hafnium mailing list
> Hafnium(a)lists.trustedfirmware.org
> https://lists.trustedfirmware.org/mailman/listinfo/hafnium
>
--
François-Frédéric Ozog | *Director Linaro Edge & Fog Computing Group*
T: +33.67221.6485
francois.ozog(a)linaro.org | Skype: ffozog
Hi Raghu,
Thanks for sharing this work.
Few thoughts...
1/ I guess it requires an additional change in project/reference to eventually enable the feature for a platform (enable_vhe=1 in the gn build flow)?
Sorry if my questions below sound obvious, I may miss bits of the VHE architecture.
2/ The changes are effectively toggling HCR_EL2.E2H=1. Currently when Hafnium ERETs resuming a SP, this happens with the secure EL1&0 translation regime.
Though what's the next step forward?
Does this require a host OS in secure world? (like linux does when booting under a Hypervisor with VHE enabled?) Would this be a VHE-enabled TOS?
3/ Is the later goal to enable the secure EL2&0 translation regime?
4/ Is there anything to do with HCR_EL2.TGE?
Regards,
Olivier.
________________________________________
From: Hafnium <hafnium-bounces(a)lists.trustedfirmware.org> on behalf of Raghu Krishnamurthy via Hafnium <hafnium(a)lists.trustedfirmware.org>
Sent: 15 December 2020 04:57
To: hafnium(a)lists.trustedfirmware.org
Subject: [Hafnium] VHE support
Hi All,
I have a series of patches pushed to Gerrit at
https://review.trustedfirmware.org/c/hafnium/hafnium/+/7599 with topic
"vhe_enable". The goal of this patch series is to enable VM's in both secure
and normal world to run with VHE enabled(hcr_el2.e2h=1), without breaking
any existing functionality. This is expected to be the first step in the
long term goal of enabling S-EL0 partitions(and optionally EL0 partitions),
that require VHE support, per the FF-A 1.0 Spec. I'd appreciate feedback on
the patches and approach taken to nominally enabling VHE. Note that the FF-A
1.0 spec(AFAIK) does not expect VHE support in the normal world but this
patch series enables it anyway due to the wealth of available tests in the
hafnium test suite to help with providing confidence in the implementation.
The patch series has been tested as follows:
Hafnium tests using QEMU(prebuilt in the hafnium repo) - Without VHE, since
the prebuilt QEMU does not support VHE.
Hafnium tests using QEMU(5.2-RC4, built from source) - With and without VHE,
this version of QEMU supports VHE.
Hafnium tests using FVP 11.12.28 - With and Without VHE.
TFTF tests for secure hafnium using FVP 11.12.28 - With and without VHE.
Thanks
Raghu
--
Hafnium mailing list
Hafnium(a)lists.trustedfirmware.org
https://lists.trustedfirmware.org/mailman/listinfo/hafnium
Hi All,
I have a series of patches pushed to Gerrit at
https://review.trustedfirmware.org/c/hafnium/hafnium/+/7599 with topic
"vhe_enable". The goal of this patch series is to enable VM's in both secure
and normal world to run with VHE enabled(hcr_el2.e2h=1), without breaking
any existing functionality. This is expected to be the first step in the
long term goal of enabling S-EL0 partitions(and optionally EL0 partitions),
that require VHE support, per the FF-A 1.0 Spec. I'd appreciate feedback on
the patches and approach taken to nominally enabling VHE. Note that the FF-A
1.0 spec(AFAIK) does not expect VHE support in the normal world but this
patch series enables it anyway due to the wealth of available tests in the
hafnium test suite to help with providing confidence in the implementation.
The patch series has been tested as follows:
Hafnium tests using QEMU(prebuilt in the hafnium repo) - Without VHE, since
the prebuilt QEMU does not support VHE.
Hafnium tests using QEMU(5.2-RC4, built from source) - With and without VHE,
this version of QEMU supports VHE.
Hafnium tests using FVP 11.12.28 - With and Without VHE.
TFTF tests for secure hafnium using FVP 11.12.28 - With and without VHE.
Thanks
Raghu
