> Doxygen embeds the documentation structure into the source files and views the
> full documentation as a single "book". Inline comments must define groups and
> similar structure. This does not scale well to situations where files are part
> of multiple projects and hence multiple documentations need to be covered. As
> an example, CMSIS headers forked into TF-M completely ruined the documentation
> structure by adding CMSIS groupings. If the files are changed to fix the
> documentation, then syncing new versions from the original repository become
> difficult. If files are filtered with scripts during documentation build,
> script maintenance becomes an issue (cost depends on frequency of updates to
> filtered source files). If Doxygen is generating XML then further processing
> may solve the issue, but implementing the machinery might be an expensive
> task. Not sure if Breathe can address this issue.
I don't think we need to concern ourselves with multiple projects - aside from the fact that we don't vendor any dependencies that use Doxygen, we could simply exclude them anyway.
> Doxygen supports markdown formatting which is different compared to reST.
> Writing documentation becomes more complex as the writer will need to use
> different syntax for inline and Sphinx documents.
Breathe supports the `@rst`/`@endrst` block for snippets of RST, but I think most in-line source documentation generally does not need the additional complexity of RST in the first
place, so the simplicity of Markdown may be of benefit here.
> Not sure how Breathe manages merging the formatting. Will it remove all
> Doxygen specific formatting, or will it translate to reST? Or is there a
> Sphinx plugin which enables rendering Doxygen markdown and using that solves
> the issue?
Breathe operates on the generated Doxygen XML (which includes formatting information) and is translated into RST directly.
> It would be possible to use Sphinx and Doxygen independently, and to host the
> two documents independent. The two document could still link to each other
> although such inks would have to be sparse as keeping the links active would
> be a manual task.
What would the benefit be of maintaining the documentation outside of ReadTheDocs? Integrating it directly via Breathe would allow us to use the Sphinx reference syntax directly
from the architectural documentation, e.g.
:c:func:`bl31_main()`
> As an example, check the "moderncmakedomain" plugin [1]. This allows using
> inline reST comments in cmake scripts. Something similar could be used for
> C/C++.
This is effectively what Breathe does, only you write your documentation using Doxygen's semantics and they are translated into the Sphinx C domain.
---
I think one place where we might run into trouble with any autogen tool is with preprocessing, as we have a huge amount of conditionally-defined code, a lack of a clear header structure
(e.g. think `<platform_def.h>`), and some function prototypes differ depending on preprocessor definitions.
Doxygen allows us to disable preprocessing entirely, but then I believe it would be unable to document preprocessor definitions (I’m unsure of that – you might still be able to do
it explicitly). Still, that might be a worthwhile tradeoff.
Certainly I think we want to avoid any substantial maintenance burdens here, including any ad-hoc scripts or infrastructure. I’m hoping that prior art (i.e. vanilla Doxygen + Breathe
+ Exhale?) can do the heavy lifting for us, even if it only gets us 75% of the way there.
Chris
From:
Gyorgy Szing <Gyorgy.Szing@arm.com>
Date: Tuesday, 4 April 2023 at 11:11
To: Michal Simek <monstr@monstr.eu>, Chris Kay <Chris.Kay@arm.com>, tf-a@lists.trustedfirmware.org <tf-a@lists.trustedfirmware.org>, Sandrine Bailleux <Sandrine.Bailleux@arm.com>, david.brown@linaro.org <david.brown@linaro.org>
Cc: nd <nd@arm.com>
Subject: RE: [TF-A] Re: TF-A code documentation
Hi,
Sorry for jumping in while not actively working on TF-A. I have some experience
with the topic in relation to TF-M (and other projects).
There are a few problems with Doxygen, although my knowledge is a bit lacking
and perhaps outdated:
- Doxygen embeds the documentation structure into the source files and views
the full documentation as a single "book". Inline comments must define
groups and similar structure. This does not scale well to situations where
files are part of multiple projects and hence multiple documentations need
to be covered. As an example, CMSIS headers forked into TF-M completely
ruined the documentation structure by adding CMSIS groupings. If the files
are changed to fix the documentation, then syncing new versions from the
original repository become difficult. If files are filtered with scripts
during documentation build, script maintenance becomes an issue (cost
depends on frequency of updates to filtered source files). If Doxygen is
generating XML then further processing may solve the issue, but implementing
the machinery might be an expensive task. Not sure if Breathe can address
this issue.
- Doxygen supports markdown formatting which is different compared to reST.
Writing documentation becomes more complex as the writer will need to use
different syntax for inline and Sphinx documents. Not sure how Breathe
manages merging the formatting. Will it remove all Doxygen specific
formatting, or will it translate to reST? Or is there a Sphinx plugin which
enables rendering Doxygen markdown and using that solves the issue?
- It would be possible to use Sphinx and Doxygen independently, and to host
the two documents independent. The two document could still link to each
other although such inks would have to be sparse as keeping the links active
would be a manual task.
Last time I was looking at the documentation topic (a few years back) I planned
to stick to Sphinx and use the C/C++ domain. (See "domains" documentation here
[2]). This solution would not "understand" source code as Doxygen does. It would
not be able to identify functions, macro definitions, etc.. automatically, and
the writer would have to add the proper reST constructs manually. Keeping function
signatures, etc... in sync between documentation and source code would be a manual
task. (With a clang backend developing tools understanding C/C++ code is much more
easy nowadays, and this could be solved longer term.) Undocumented stuff would be
missing from the output.
Documentation could be made inline by using reST comments in source files. As an
example, check the "moderncmakedomain" plugin [1]. This allows using inline reST
comments in cmake scripts. Something similar could be used for C/C++. The way of
implementation is a question:
- Could the "moderncmakedomain" plugin be "misused" for C/C++ files? If it
does nothing more than to filter out inline comments and passing it to
Sphinx, it could work.
- Should a new plugin be developed?
- Or is a python/perl/awk/sed filtering script plus extra documentation build
steps the solution? (The filtered content would be saved to temporary files,
which could be added to the Sphinx document tree.)
As I mentioned my knowledge is a bit dated and I never actually tries Breathe and
Most of my points above might be outdated or simply plain wrong.
Overall, the task to enable inline documentation is not simple. It needs careful
investigation, design and implementation.
/George
1: https://github.com/scikit-build/moderncmakedomain
2: http://www.pythondoc.com/sphinx/domains.html
-----Original Message-----
From: Michal Simek via TF-A <tf-a@lists.trustedfirmware.org>
Sent: Tuesday, April 4, 2023 8:19 AM
To: Chris Kay <Chris.Kay@arm.com>; tf-a@lists.trustedfirmware.org; Sandrine Bailleux <Sandrine.Bailleux@arm.com>; david.brown@linaro.org
Subject: [TF-A] Re: TF-A code documentation
Hi Chris, +David
I think it would be good to make a call on it. Because then we can cleanup at least our platform. If you say it is doxygen I expect there is any tool which can be used for checking. If it is something else that's fine too.
I just want to open this topic to be able to fix our code in a way how it should be.
Thanks,
Michal
On 4/3/23 14:13, Chris Kay wrote:
> Sorry, I sent my response just as Sandrine did so I'll re-submit it
> here for the sake of thread continuity:
>
> Hi Michal,
>
> TF-A doesn't currently employ any source-level documentation tool so
> the documentation style varies across the code-base and we don't
> enforce any particular style. Doxygen is something that we might want
> to consider - particularly because it can be integrated directly into
> the existing Sphinx documentation via Breathe
> (https://breathe.readthedocs.io/en/latest/
> <https://breathe.readthedocs.io/en/latest/>) - but I think we could
> only reasonably enforce it for new code.
>
> Chris
>
> ----------------------------------------------------------------------
> ----------
> *From:* Sandrine Bailleux via TF-A <tf-a@lists.trustedfirmware.org>
> *Sent:* 03 April 2023 13:11
> *To:* Michal Simek <monstr@monstr.eu>; tf-a@lists.trustedfirmware.org
> <tf-a@lists.trustedfirmware.org>
> *Subject:* [TF-A] Re: TF-A code documentation Hi Michal,
>
> Thanks for starting this thread. I'll give my own opinion on this topic.
>
> I think there are areas that could benefit from such documentation
> generation techniques. A prime example would be the porting guide [1].
> I would be favorable to the adoption of some Doxygen-like tool (exact
> tool
> TBD) for documenting platform interfaces and auto-generate the porting
> guide from that.
>
> Maybe generic drivers (GPT, console, I/O, ...) would be good candidates too.
>
> But I would not generalize Doxygen-like comments across the entire
> code base. This sounds too heavyweight on developers to me with no obvious gain.
>
> Also I think this would not replace design documents [2], only
> complement them. Not saying you're suggesting to replace them! but I
> just wanted to point it out.
>
> Regards,
> Sandrine
>
> [1]
>
https://trustedfirmware-a.readthedocs.io/en/latest/getting_started/por
> ting-guide.html
> <https://trustedfirmware-a.readthedocs.io/en/latest/getting_started/po
> rting-guide.html> [2]
>
https://trustedfirmware-a.readthedocs.io/en/latest/components/index.ht
> ml
> <https://trustedfirmware-a.readthedocs.io/en/latest/components/index.h
> tml>
>
> On 4/3/23 13:37, Michal Simek via TF-A wrote:
>> Hi,
>>
>> I have talked to a couple of people to figure out what TF-A project
>> is using for code documentation. Because I see at least in our
>> platform that our documentation is somewhere between doxygen and
>> kernel-doc but actually with a lot of mismatches.
>> Sanbrine mentioned sending an email to the mailing list to start to
>> have discussion about it.
>>
>> That's why I want to know the official code documentation format and
>> how we should be checking that everything matches to make sure that
>> documentation is not out of sync from code itself.
>> When this is clear I will ask my team to fix all these issues.
>>
>> Thanks,
>> Michal
> --
> TF-A mailing list -- tf-a@lists.trustedfirmware.org To unsubscribe
> send an email to tf-a-leave@lists.trustedfirmware.org
--
Michal Simek, Ing. (M.Eng), OpenPGP -> KeyID: FE3D1F91
w: www.monstr.eu p: +42-0-721842854
Maintainer of Linux kernel - Xilinx Microblaze Maintainer of Linux kernel - Xilinx Zynq ARM and ZynqMP/Versal ARM64 SoCs U-Boot custodian - Xilinx Microblaze/Zynq/ZynqMP/Versal/Versal NET SoCs TF-A maintainer - Xilinx ZynqMP/Versal/Versal NET SoCs
--
TF-A mailing list -- tf-a@lists.trustedfirmware.org To unsubscribe send an email to tf-a-leave@lists.trustedfirmware.org