Hi S*deeps,
Am 10.12.19 um 16:08 schrieb Sudeep Holla via TF-A:
The more we discuss, I think we will get to know all sorts of CENH(as you put) are done all over the place and expecting system is work just fine even when lots of interface/contracts are broken is just .....(fill your own word :))
I promise not to discuss these CENH any further after this email :)
On Tue, Dec 10, 2019 at 03:59:01PM +0530, Sandeep Tripathy wrote:
Hi Sudeep,
On Mon, Dec 9, 2019 at 10:40 PM Sudeep Holla sudeep.holla@arm.com wrote:
The application has to terminate cleanly when SIGTERM is sent(may be using appropriate handler. And can intimate the same to the consumers so that they can consume the data before it's lost.
The DDR is not powered off ever in this scenario. So when to/how to consume the log is up to the (consumer) application design.
CENH#1
Assume its an incrementing log ie. after reboot this (producer) master again will continue to dump more records on to it.
CENH#2
(I see the roles being exchanged, OS was slave + producer and not sure what you are referring has master above. Anyways use KDUMP and features like that if you need RAM dump for portions of memory given to the kernel.
Sorry to barge in, but it sounds like you're not quite understanding each other in this ever-growing thread: Sandeep said his use case was a SmartNIC, so I assume the logging direction is from SoC on the SmartNIC to host SoC via PCIe? Think dmesg-like logs, from one host for another - I fail to see how Sudeep's kdump suggestion could help here.
But maybe I'm not understanding properly, in which case I'll just blame your abstract master/slave discussions. :)
To me and my ignorance however it sounds like this use case has nothing really to do with shutdown? If the log producer does not flush their logs after say each line ending, then also without shutdown you have no guarantee of the logs reaching the consumer in a timely manner through all the producer's caches, no? Think of watchdogs, power loss or other irregular resets that PSCI can do nothing about. Especially for logs, those error conditions would be the most important to receive data of.
Cheers, Andreas
How would you suggest to handle this. In this case both producer and consumer deliberately asked for coherent memory so why it should also consider a possible data loss due to platforms not giving the coherency because it will add some time to flush the core caches.
CENH#3, not sure if such flexibility should be given to applications.
If they get non-cached(coherent) memory range they don't have to do anything isn't it ?
Applications must not try that, kernel mostly provides cached memory from it's memory allocator. I get a sense that this is some magic pre-allocated memory that is either reserved or taken out of kernel memory, but the application (along with its driver) maps it coherent in some magic way.
-- Regards, Sudeep
On Tue, Dec 10, 2019 at 04:36:34PM +0100, Andreas Färber wrote:
Hi S*deeps,
Am 10.12.19 um 16:08 schrieb Sudeep Holla via TF-A:
[...]
(I see the roles being exchanged, OS was slave + producer and not sure what you are referring has master above. Anyways use KDUMP and features like that if you need RAM dump for portions of memory given to the kernel.
Sorry to barge in, but it sounds like you're not quite understanding each other in this ever-growing thread: Sandeep said his use case was a SmartNIC, so I assume the logging direction is from SoC on the SmartNIC to host SoC via PCIe? Think dmesg-like logs, from one host for another - I fail to see how Sudeep's kdump suggestion could help here.
Sorry for my lack of knowledge there. And for sure, you have explained it much better now. Since I was assuming some kind of use-case(like crash logs) and something similar I suggested kdump above. I agree it was irrelevant to the discussion.
But maybe I'm not understanding properly, in which case I'll just blame your abstract master/slave discussions. :)
:)
To me and my ignorance however it sounds like this use case has nothing really to do with shutdown?
Agreed, and I thought I asked this as what happens when the logging process terminates ? It should be handled in similar manner. Sorry if I have missed to ask this aloud, it might have died in my thoughts and didn't come out.
If the log producer does not flush their logs after say each line ending, then also without shutdown you have no guarantee of the logs reaching the consumer in a timely manner through all the producer's caches, no?
Absolutely, I completely agree.
Think of watchdogs, power loss or other irregular resets that PSCI can do nothing about. Especially for logs, those error conditions would be the most important to receive data of.
Indeed. That's the reason I was trying to concentrate this as producer and consumer problem and graceful shutdown/restart can provide opportunity for them to handle it correctly through some sort of notifications. The assumption of coherent memory and the DRAM retaining content after the producer has terminated bother me.
-- Regards, Sudeep
Hi, On Tue, Dec 10, 2019 at 9:25 PM Sudeep Holla sudeep.holla@arm.com wrote:
On Tue, Dec 10, 2019 at 04:36:34PM +0100, Andreas Färber wrote:
Hi S*deeps,
Am 10.12.19 um 16:08 schrieb Sudeep Holla via TF-A:
[...]
(I see the roles being exchanged, OS was slave + producer and not sure what you are referring has master above. Anyways use KDUMP and features like that if you need RAM dump for portions of memory given to the kernel.
Sorry to barge in, but it sounds like you're not quite understanding each other in this ever-growing thread: Sandeep said his use case was a SmartNIC, so I assume the logging direction is from SoC on the SmartNIC to host SoC via PCIe? Think dmesg-like logs, from one host for another -
Thanks! Thats one use case. But I am sure more and more coming up. Hence the generalization in discussion.
I fail to see how Sudeep's kdump suggestion could help here.
Was my fault to not make it clear that DRAM is retained and other masters are not affected while these master go through shutdown/reboot(warm).
Sorry for my lack of knowledge there. And for sure, you have explained it much better now. Since I was assuming some kind of use-case(like crash logs) and something similar I suggested kdump above. I agree it was irrelevant to the discussion.
But maybe I'm not understanding properly, in which case I'll just blame your abstract master/slave discussions. :)
:)
I also want to call "log" as "any-data" :) Otherwise we get diverted to crash dump or alike.
To me and my ignorance however it sounds like this use case has nothing really to do with shutdown?
Agreed, and I thought I asked this as what happens when the logging process terminates ? It should be handled in similar manner. Sorry if I have missed to ask this aloud, it might have died in my thoughts and didn't come out.
If the log producer does not flush their logs after say each line ending, then also without shutdown you have no guarantee of the logs reaching the consumer in a timely manner through all the producer's caches, no?
The updates will get snooped by other masters over ccn when needed provided the processor is alive and still in snoop domain. CCN will guarantee that. (producer->OS->psci) arm core --> L1->L2---- > L3 CCN <-- any other masters (consumer) | DDR psci shutdown/reset/reset2 will power down armcore L1 L2 . L3/CCN, DDR and other masters are not affected. Without flushing the caches in psci sequence or somewhere else content in L1/L2 are lost.
Absolutely, I completely agree.
Think of watchdogs, power loss or other irregular resets that PSCI can do nothing about. Especially for logs, those error conditions would be the most important to receive data of.
Agree. It can do nothing about scenarios where control does not even reach this code sequence. PSCI shutdown/reset/reset2 are in path of graceful shutdown/ reboot(warm) initiated by OS. So in these calls I do not see anything wrong to follow a core power down sequence.
Indeed. That's the reason I was trying to concentrate this as producer and consumer problem and graceful shutdown/restart can provide opportunity for them to handle it correctly through some sort of notifications. The assumption of coherent memory and the DRAM retaining content after the producer has terminated bother me.
I see. sorry if it was not clear that DRAM /CCN/ other masters are functional as usual in this context. But I guess this topology is not unusual.
-- Regards, Sudeep
Thanks Sandeep
tf-a@lists.trustedfirmware.org
-
Andreas Färber -
Sandeep Tripathy -
Sudeep Holla