> -----Original Message-----
> From: Soby Mathew [mailto:Soby.Mathew@arm.com]
> Sent: Monday, June 08, 2020 3:36 PM
> To: Sandeep Tripathy; Olivier Deprez; tf-a(a)lists.trustedfirmware.org
> Cc: nd
> Subject: RE: [TF-A] GICV3: system interface EOI ordering RFC
>
> Just some inputs from my side:
> I agree we need at least a dsb() after the write to MMIO region to silence
> the
> peripheral and before the EOI at GIC sysreg interface. Adding it to the
> GIC EOI
Thanks for the confirmation. Now it’s about choosing the right place.
> API seems the logical thing to do but as Olivier mentions, there are
> interrupt
> handler which do not deal with MMIO (eg: the Systimer interrupt) so adding
> it
> to GICv3 API might be arduous for such handlers.
>
> So there is a choice here to let the interrupt handlers to deal with
> ensuring
> completeness before EOI at sysreg interface or adding it to GICv3 EOI API
> and
> take the overhead for interrupt handlers which do not have to deal with
> MMIO.
>
Yes I feel either of these is a must to guarantee functionality
architecturally though
both approach end up with some unnecessary overhead.
If GICv3 api takes care then it is an overhead for some ISRs dealing with
non-MMIO.
At present I do not see an active use case in reference implementation where
sys timer
ISR is in a performance intensive path where one additional DSB will be
perceivable.
But there may be some I could be totally wrong in this assumption (pmu/debug
or.. not sure).
Whereas I can certainly imagine some MMIO ISRs in performance critical path
where unnecessary
DSB is not acceptable at all.
If the interrupt handler needs to ensure then it will generically add 'DSB'
as I think
the driver cannot and should not make assumptions about how EOI is done
afterwards.
That will be overhead for the ISRs dealing with MMIO peripherals and non
GIC-v3.
If we consider only GICv3+ then good. Otherwise I would prefer the
'plat_ic_end_of_interrupt'
like Olivier mentioned with a #if GICv3 instead of each ISRs dealing with
it.
> The GICv3 legacy MMIO CPU interface is deprecated for TF-A and the sys
> interface is the only one GICv3 driver in TF-A supports.
Right we can ignore the GICv3 legacy mode but GICv2 will still remain ?
>
> Best Regards
> Soby Mathew
>
Thanks
Sandeep
> > -----Original Message-----
> > From: TF-A <tf-a-bounces(a)lists.trustedfirmware.org> On Behalf Of Sandeep
> > Tripathy via TF-A
> > Sent: 08 June 2020 10:34
> > To: Olivier Deprez <Olivier.Deprez(a)arm.com>; tf-
> a(a)lists.trustedfirmware.org
> > Subject: Re: [TF-A] GICV3: system interface EOI ordering RFC
> >
> > Hi Olivier,
> > > -----Original Message-----
> > > From: Olivier Deprez [mailto:Olivier.Deprez@arm.com]
> > > Sent: Monday, June 08, 2020 1:14 PM
> > > To: tf-a(a)lists.trustedfirmware.org; Sandeep Tripathy
> > > Subject: Re: [TF-A] GICV3: system interface EOI ordering RFC
> > >
> > > Hi Sandeep,
> > >
> > > gicv3_end_of_interrupt_sel1 is a static access helper macro. Its
> > > naming precisely tells what it does at gicv3 module level. It is
> > > called from
> > the weak
> > > plat_ic_end_of_interrupt function for BL32 image.
> > >
> > > I tend to think it is the driver responsibility to ensure the module
> > interrupt
> > > acknowledge register write is reaching HW in order (or "be visible to
> > all
> > > observers").
> >
> > The driver should be agnostic of what interrupt controller is used and
> > its
> > behavior.
> > And since 'all' writes were to mmio ranges mapped Device(nGnRE)
> > /strongly-ordered(nGnRnE)
> > there was no such need. This is a special case for GICv3 system
> > interface only.
> > Adding at driver level a DSB is unnecessary for other interrupt
> > controllers.
> >
> > > Also I suspect adding a dsb might not be required generically for all
> > kind of IP.
> >
> > Here are you referring to the peripheral IP or interrupt controller IP ?
> > The issue is reordering at arm core itself (STR to device address Vs
> > msr(sysreg
> > write)).
> > So I think Its applicable for all IP.
> >
> > > Adding a barrier in generic code might incur an unnecessary
> > > bottleneck.
> >
> > But if there is a need to *ensure* presence of at least one DSB between
> > the
> > write transfer from core to device clear and gicv3 eoi icc register
> > write in a
> > generic way then what other option we have.
> > >
> > > Thus wouldn't it be better to add the barrier to the overridden
> > > platform function rather than in generic gicv3 code?
> >
> > That can be done but I feel this is more to do with gicv3 system
> > interface only.
> > Inside plat_xxx one has to check #if GICV3 ...and system interface.
> > >
> > > I have a put a comment in the review, we can continue the discussion
> > there.
> > >
> > > Regards,
> > > Olivier.
> > >
> > Thanks
> > Sandeep
> > > ________________________________________
> > > From: TF-A <tf-a-bounces(a)lists.trustedfirmware.org> on behalf of
> > > Sandeep Tripathy via TF-A <tf-a(a)lists.trustedfirmware.org>
> > > Sent: 05 June 2020 19:43
> > > To: tf-a(a)lists.trustedfirmware.org
> > > Subject: [TF-A] GICV3: system interface EOI ordering RFC
> > >
> > > Hi,
> > > In a typical interrupt handling sequence we do 1-Read IAR
> > > 2-Do interrupt handling
> > > 3-Clear the interrupt at source , so that the device de-asserts IRQ
> > request to
> > > GIC
> > > >> I am suggesting a need of DSB here in case of GICv3 +.
> > > 4-Write to GIC cpu interface to do EOI.
> > >
> > > Till GICv2 and with GICv3 legacy interface ICC_EOI write is a write
> > > over
> > AMBA
> > > interface. The
> > > Addresses are mapped with (nR) attribute. Hence the write transfers
> > from the
> > > core will be
> > > generated at step 3 and 4 in order. Please ignore the additional
> > buffers/bridges
> > > in path from
> > > core till peripheral which has to be dealt separately as per SOC.
> > >
> > > Query: I understand GICv3 system interface accesses are not over this
> > protocol
> > > and core will not
> > > follow the ordering rule ?
> > >
> > > I have observed spurious interrupt issue/manifestation ( I don't have
> > the
> > > transfers probed) in
> > > RTOS environment where I have a primitive GICv3 driver but I wonder
> > > why things does not fail in Linux or tf-a. If it is working because
> > > from step(3) to
> > step(4) we have
> > > barriers by chace
> > > due to other device register writes then I would suggest to have one
> > > in
> > the EOI
> > > clearing API itself.
> > >
> > > RFC:
> > https://review.trustedfirmware.org/c/TF-A/trusted-firmware-a/+/4454
> > >
> > > Thanks
> > > Sandeep
> > --
> > TF-A mailing list
> > TF-A(a)lists.trustedfirmware.org
> > https://lists.trustedfirmware.org/mailman/listinfo/tf-a
Just some inputs from my side:
I agree we need atleast a dsb() after the write to MMIO region to silence the peripheral and before the EOI at GIC sysreg interface. Adding it to the GIC EOI API seems the logical thing to do but as Olivier mentions, there are interrupt handler which do not deal with MMIO (eg: the Systimer interrupt) so adding it to GICv3 API might be arduous for such handlers.
So there is a choice here to let the interrupt handlers to deal with ensuring completeness before EOI at sysreg interface or adding it to GICv3 EOI API and take the overhead for interrupt handlers which do not have to deal with MMIO.
The GICv3 legacy MMIO CPU interface is deprecated for TF-A and the sys interface is the only one GICv3 driver in TF-A supports.
Best Regards
Soby Mathew
> -----Original Message-----
> From: TF-A <tf-a-bounces(a)lists.trustedfirmware.org> On Behalf Of Sandeep
> Tripathy via TF-A
> Sent: 08 June 2020 10:34
> To: Olivier Deprez <Olivier.Deprez(a)arm.com>; tf-a(a)lists.trustedfirmware.org
> Subject: Re: [TF-A] GICV3: system interface EOI ordering RFC
>
> Hi Olivier,
> > -----Original Message-----
> > From: Olivier Deprez [mailto:Olivier.Deprez@arm.com]
> > Sent: Monday, June 08, 2020 1:14 PM
> > To: tf-a(a)lists.trustedfirmware.org; Sandeep Tripathy
> > Subject: Re: [TF-A] GICV3: system interface EOI ordering RFC
> >
> > Hi Sandeep,
> >
> > gicv3_end_of_interrupt_sel1 is a static access helper macro. Its
> > naming precisely tells what it does at gicv3 module level. It is
> > called from
> the weak
> > plat_ic_end_of_interrupt function for BL32 image.
> >
> > I tend to think it is the driver responsibility to ensure the module
> interrupt
> > acknowledge register write is reaching HW in order (or "be visible to
> all
> > observers").
>
> The driver should be agnostic of what interrupt controller is used and its
> behavior.
> And since 'all' writes were to mmio ranges mapped Device(nGnRE)
> /strongly-ordered(nGnRnE)
> there was no such need. This is a special case for GICv3 system interface only.
> Adding at driver level a DSB is unnecessary for other interrupt controllers.
>
> > Also I suspect adding a dsb might not be required generically for all
> kind of IP.
>
> Here are you referring to the peripheral IP or interrupt controller IP ?
> The issue is reordering at arm core itself (STR to device address Vs msr(sysreg
> write)).
> So I think Its applicable for all IP.
>
> > Adding a barrier in generic code might incur an unnecessary bottleneck.
>
> But if there is a need to *ensure* presence of at least one DSB between the
> write transfer from core to device clear and gicv3 eoi icc register write in a
> generic way then what other option we have.
> >
> > Thus wouldn't it be better to add the barrier to the overridden
> > platform function rather than in generic gicv3 code?
>
> That can be done but I feel this is more to do with gicv3 system interface only.
> Inside plat_xxx one has to check #if GICV3 ...and system interface.
> >
> > I have a put a comment in the review, we can continue the discussion
> there.
> >
> > Regards,
> > Olivier.
> >
> Thanks
> Sandeep
> > ________________________________________
> > From: TF-A <tf-a-bounces(a)lists.trustedfirmware.org> on behalf of
> > Sandeep Tripathy via TF-A <tf-a(a)lists.trustedfirmware.org>
> > Sent: 05 June 2020 19:43
> > To: tf-a(a)lists.trustedfirmware.org
> > Subject: [TF-A] GICV3: system interface EOI ordering RFC
> >
> > Hi,
> > In a typical interrupt handling sequence we do 1-Read IAR
> > 2-Do interrupt handling
> > 3-Clear the interrupt at source , so that the device de-asserts IRQ
> request to
> > GIC
> > >> I am suggesting a need of DSB here in case of GICv3 +.
> > 4-Write to GIC cpu interface to do EOI.
> >
> > Till GICv2 and with GICv3 legacy interface ICC_EOI write is a write
> > over
> AMBA
> > interface. The
> > Addresses are mapped with (nR) attribute. Hence the write transfers
> from the
> > core will be
> > generated at step 3 and 4 in order. Please ignore the additional
> buffers/bridges
> > in path from
> > core till peripheral which has to be dealt separately as per SOC.
> >
> > Query: I understand GICv3 system interface accesses are not over this
> protocol
> > and core will not
> > follow the ordering rule ?
> >
> > I have observed spurious interrupt issue/manifestation ( I don't have
> the
> > transfers probed) in
> > RTOS environment where I have a primitive GICv3 driver but I wonder
> > why things does not fail in Linux or tf-a. If it is working because
> > from step(3) to
> step(4) we have
> > barriers by chace
> > due to other device register writes then I would suggest to have one
> > in
> the EOI
> > clearing API itself.
> >
> > RFC:
> https://review.trustedfirmware.org/c/TF-A/trusted-firmware-a/+/4454
> >
> > Thanks
> > Sandeep
> --
> TF-A mailing list
> TF-A(a)lists.trustedfirmware.org
> https://lists.trustedfirmware.org/mailman/listinfo/tf-a
Hi Sandeep,
gicv3_end_of_interrupt_sel1 is a static access helper macro. Its naming precisely tells what it does at gicv3 module level. It is called from the weak plat_ic_end_of_interrupt function for BL32 image.
I tend to think it is the driver responsibility to ensure the module interrupt acknowledge register write is reaching HW in order (or "be visible to all observers").
Also I suspect adding a dsb might not be required generically for all kind of IP. Adding a barrier in generic code might incur an unnecessary bottleneck.
Thus wouldn't it be better to add the barrier to the overridden platform function rather than in generic gicv3 code?
I have a put a comment in the review, we can continue the discussion there.
Regards,
Olivier.
________________________________________
From: TF-A <tf-a-bounces(a)lists.trustedfirmware.org> on behalf of Sandeep Tripathy via TF-A <tf-a(a)lists.trustedfirmware.org>
Sent: 05 June 2020 19:43
To: tf-a(a)lists.trustedfirmware.org
Subject: [TF-A] GICV3: system interface EOI ordering RFC
Hi,
In a typical interrupt handling sequence we do
1-Read IAR
2-Do interrupt handling
3-Clear the interrupt at source , so that the device de-asserts IRQ request to GIC
>> I am suggesting a need of DSB here in case of GICv3 +.
4-Write to GIC cpu interface to do EOI.
Till GICv2 and with GICv3 legacy interface ICC_EOI write is a write over AMBA interface. The
Addresses are mapped with (nR) attribute. Hence the write transfers from the core will be
generated at step 3 and 4 in order. Please ignore the additional buffers/bridges in path from
core till peripheral which has to be dealt separately as per SOC.
Query: I understand GICv3 system interface accesses are not over this protocol and core will not
follow the ordering rule ?
I have observed spurious interrupt issue/manifestation ( I don’t have the transfers probed) in
RTOS environment where I have a primitive GICv3 driver but I wonder why things does
not fail in Linux or tf-a. If it is working because from step(3) to step(4) we have barriers by chace
due to other device register writes then I would suggest to have one in the EOI clearing API itself.
RFC: https://review.trustedfirmware.org/c/TF-A/trusted-firmware-a/+/4454
Thanks
Sandeep
Hi Walter,
Am Freitag, 5. Juni 2020, 22:45:25 CEST schrieb Walter Lozano via TF-A:
> I have noticed that default baudrate for rk3399 board RK3399_BAUDRATE
> defined in plat/rockchip/rk3399/rk3399_def.h is 115200 while U-boot
> default value and most documentation points to a value of 1500000 for
> console.
console baudrate on rk3399 differ a lot between boards. There are a
number using the 1.5MHz and another big number using 115200
(including but not limited to the ChromeOS devices from the Gru line).
Hence we have the code that selects the actual baudrate from the
devicetree attached by u-boot when calling TF-A.
So I guess it should stay as it is right now.
Heiko
> This of course means that messages printed by ATF are not visible by
> default in this context.
>
> The change from 1500000 to 115200 was introduced in
> 0c05748bdebfad9fa43a80962186438bb8fbce62,
>
> https://git.trustedfirmware.org/TF-A/trusted-firmware-a.git/commit/?id=0c05…
>
> with the following message
>
> commit 0c05748bdebfad9fa43a80962186438bb8fbce62
> Author: Caesar Wang <wxt(a)rock-chips.com>
> Date: Tue Apr 19 20:42:17 2016 +0800
>
> rockchip: fixes for the required
>
> This patch has the following change for rk3399.
>
> * Set the uart to 115200 since the loader decide to set
> uart baud to 115200Hz. So the ATF also should set uart baud to 115200.
> [..]
>
> However, I'm not sure it this still applies.
>
> I'll be happy to submit a patch to update the value if it is OK.
>
> Regards,
>
> Walter
>
>
I have noticed that default baudrate for rk3399 board RK3399_BAUDRATE
defined in plat/rockchip/rk3399/rk3399_def.h is 115200 while U-boot
default value and most documentation points to a value of 1500000 for
console.
This of course means that messages printed by ATF are not visible by
default in this context.
The change from 1500000 to 115200 was introduced in
0c05748bdebfad9fa43a80962186438bb8fbce62,
https://git.trustedfirmware.org/TF-A/trusted-firmware-a.git/commit/?id=0c05…
with the following message
commit 0c05748bdebfad9fa43a80962186438bb8fbce62
Author: Caesar Wang <wxt(a)rock-chips.com>
Date: Tue Apr 19 20:42:17 2016 +0800
rockchip: fixes for the required
This patch has the following change for rk3399.
* Set the uart to 115200 since the loader decide to set
uart baud to 115200Hz. So the ATF also should set uart baud to 115200.
[..]
However, I'm not sure it this still applies.
I'll be happy to submit a patch to update the value if it is OK.
Regards,
Walter
Hi,
In a typical interrupt handling sequence we do
1-Read IAR
2-Do interrupt handling
3-Clear the interrupt at source , so that the device de-asserts IRQ
request to GIC
>> I am suggesting a need of DSB here in case of GICv3 +.
4-Write to GIC cpu interface to do EOI.
Till GICv2 and with GICv3 legacy interface ICC_EOI write is a write over
AMBA interface. The
Addresses are mapped with (nR) attribute. Hence the write transfers from
the core will be
generated at step 3 and 4 in order. Please ignore the additional
buffers/bridges in path from
core till peripheral which has to be dealt separately as per SOC.
Query: I understand GICv3 system interface accesses are not over this
protocol and core will not
follow the ordering rule ?
I have observed spurious interrupt issue/manifestation ( I don’t have the
transfers probed) in
RTOS environment where I have a primitive GICv3 driver but I wonder why
things does
not fail in Linux or tf-a. If it is working because from step(3) to step(4)
we have barriers by chace
due to other device register writes then I would suggest to have one in the
EOI clearing API itself.
RFC: https://review.trustedfirmware.org/c/TF-A/trusted-firmware-a/+/4454
Thanks
Sandeep
Hi Okash,
These are generally provided on https://www.trustedfirmware.org/meetings/tf-a-technical-forum/ however they are missing from 7th May onwards at the moment. I have an update pending approval on tf.org for adding the details of that session and then I need to add the information from the session on 21st May and the session yesterday.
Joanna
From: TF-A <tf-a-bounces(a)lists.trustedfirmware.org> on behalf of Okash Khawaja via TF-A <tf-a(a)lists.trustedfirmware.org>
Reply to: Okash Khawaja <okash.khawaja(a)gmail.com>
Date: Friday, 5 June 2020 at 11:29
To: "tf-a(a)lists.trustedfirmware.org" <tf-a(a)lists.trustedfirmware.org>
Subject: [TF-A] Slides for recent TF-A Forums
Hi,
Where can I find slides for recent TF-A Forum presentations?
Thanks,
Okash
Hi Bill,
I continue to the error while trying to invoke “Allow-CI” or other votes as part of gerrit review. I tried multiple browsers as well and logged out before attempting again.
Thanks,
Madhu
From: TF-A <tf-a-bounces(a)lists.trustedfirmware.org> On Behalf Of Bill Fletcher via TF-A
Sent: Tuesday, June 2, 2020 12:13 PM
To: Varun Wadekar <vwadekar(a)nvidia.com>
Cc: tf-a(a)lists.trustedfirmware.org
Subject: Re: [TF-A] Gerrit seems to be broken
Hi Varun,
Do you still see the problem? It has been reported before.
Regards
Bill
On Tue, 2 Jun 2020 at 18:01, Varun Wadekar via TF-A <tf-a(a)lists.trustedfirmware.org<mailto:tf-a@lists.trustedfirmware.org>> wrote:
Hello,
The TF gerrit review dashboard seems to be broken this morning. I see “Server error” and it does not allow me to post scores or comments. Tried different browsers, but same issue persists.
Anyone else seeing the same problem? Is this is a known issue? If yes, is anyone working on it?
-Varun
--
TF-A mailing list
TF-A(a)lists.trustedfirmware.org<mailto:TF-A@lists.trustedfirmware.org>
https://lists.trustedfirmware.org/mailman/listinfo/tf-a
--
[Image removed by sender. Linaro]<http://www.linaro.org/>
Bill Fletcher | Field Engineering
T: +44 7833 498336<tel:+44+7833+498336>
bill.fletcher(a)linaro.org<mailto:bill.fletcher@linaro.org> | Skype: billfletcher2020
Hi Varun,
Do you still see the problem? It has been reported before.
Regards
Bill
On Tue, 2 Jun 2020 at 18:01, Varun Wadekar via TF-A <
tf-a(a)lists.trustedfirmware.org> wrote:
> Hello,
>
>
>
> The TF gerrit review dashboard seems to be broken this morning. I see
> “Server error” and it does not allow me to post scores or comments. Tried
> different browsers, but same issue persists.
>
>
>
> Anyone else seeing the same problem? Is this is a known issue? If yes, is
> anyone working on it?
>
>
>
> -Varun
> --
> TF-A mailing list
> TF-A(a)lists.trustedfirmware.org
> https://lists.trustedfirmware.org/mailman/listinfo/tf-a
>
--
[image: Linaro] <http://www.linaro.org/>
*Bill Fletcher* | *Field Engineering*
T: +44 7833 498336 <+44+7833+498336>
bill.fletcher(a)linaro.org | Skype: billfletcher2020
