summaryrefslogtreecommitdiff
path: root/scripts/lib
AgeCommit message (Collapse)Author
3 daysMerge tag 'docs-6.17' of git://git.lwn.net/linuxLinus Torvalds
Pull documentation updates from Jonathan Corbet: "It has been a relatively busy cycle for docs, especially the build system: - The Perl kernel-doc script was added to 2.3.52pre1 just after the turn of the millennium. Over the following 25 years, it accumulated a vast amount of cruft, all in a language few people want to deal with anymore. Mauro's Python replacement in 6.16 faithfully reproduced all of the cruft in the hope of avoiding regressions. Now that we have a more reasonable code base, though, we can work on cleaning it up; many of the changes this time around are toward that end. - A reorganization of the ext4 docs into the usual TOC format. - Various Chinese translations and updates. - A new script from Mauro to help with docs-build testing. - A new document for linked lists - A sweep through MAINTAINERS fixing broken GitHub git:// repository links. ...and lots of fixes and updates" * tag 'docs-6.17' of git://git.lwn.net/linux: (147 commits) scripts: add origin commit identification based on specific patterns sphinx: kernel_abi: fix performance regression with O=<dir> Documentation: core-api: entry: Replace deprecated KVM entry/exit functions docs: fault-injection: drop reference to md-faulty docs: document linked lists scripts: kdoc: make it backward-compatible with Python 3.7 docs: kernel-doc: emit warnings for ancient versions of Python Documentation/rtla: Describe exit status Documentation/rtla: Add include common_appendix.rst docs: kernel: Clarify printk_ratelimit_burst reset behavior Documentation: ioctl-number: Don't repeat macro names Documentation: ioctl-number: Shorten macros table Documentation: ioctl-number: Correct full path to papr-physical-attestation.h Documentation: ioctl-number: Extend "Include File" column width Documentation: ioctl-number: Fix linuxppc-dev mailto link overlayfs.rst: fix typos docs: kdoc: emit a warning for ancient versions of Python docs: kdoc: clean up check_sections() docs: kdoc: directly access the always-there KdocItem fields docs: kdoc: straighten up dump_declaration() ...
2025-07-17scripts: kdoc: make it backward-compatible with Python 3.7Mauro Carvalho Chehab
There was a change at kdoc that ended breaking compatibility with Python 3.7: str.removesuffix() was introduced on version 3.9. Restore backward compatibility. Reported-by: Akira Yokosawa <akiyks@gmail.com> Closes: https://lore.kernel.org/linux-doc/57be9f77-9a94-4cde-aacb-184cae111506@gmail.com/ Fixes: 27ad33b6b349 ("kernel-doc: Fix symbol matching for dropped suffixes") Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org> Signed-off-by: Jonathan Corbet <corbet@lwn.net> Link: https://lore.kernel.org/r/d13058d285838ac2bc04c492e60531c013a8a919.1752218291.git.mchehab+huawei@kernel.org
2025-07-15Merge branch 'kdoc-item2' into docs-mwJonathan Corbet
The kerneldoc parsing phase gathers all of the information about the declarations of interest, then passes it through to the output phase as a dict that is an unstructured blob of information; this organization has its origins in the Perl version of the program. It results in an interface that is difficult to reason about, dozen-parameter function calls, and other ills. Introduce a new class (KdocItem) to carry this information between the parser and the output modules, and, step by step, modify the system to use this class in a more structured way. This could be taken further by creating a subclass of KdocItem for each declaration type (function, struct, ...), but that is probably more structure than we need. The result is (I hope) clearer code, the removal of a bunch of boilerplate, and no changes to the generated output.
2025-07-15docs: kdoc: emit a warning for ancient versions of PythonJonathan Corbet
Versions of Python prior to 3.7 do not guarantee to remember the insertion order of dicts; since kernel-doc depends on that guarantee, running with such older versions could result in output with reordered sections. Python 3.9 is the minimum for the kernel as a whole, so this should not be a problem, but put in a warning just in case somebody tries to use something older. Suggested-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org> Reviewed-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org> Signed-off-by: Jonathan Corbet <corbet@lwn.net>
2025-07-15docs: kdoc: clean up check_sections()Jonathan Corbet
entry.sectcheck is just a duplicate of our list of sections that is only passed to check_sections(); its main purpose seems to be to avoid checking the special named sections. Rework check_sections() to not use that field (which is then deleted), tocheck for the known sections directly, and tighten up the logic in general. Reviewed-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org> Signed-off-by: Jonathan Corbet <corbet@lwn.net>
2025-07-15docs: kdoc: directly access the always-there KdocItem fieldsJonathan Corbet
They are part of the interface, so use them directly. This allows the removal of the transitional __dict__ hack in KdocItem. Reviewed-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org> Signed-off-by: Jonathan Corbet <corbet@lwn.net>
2025-07-15docs: kdoc: straighten up dump_declaration()Jonathan Corbet
Get rid of the excess "return" statements in dump_declaration(), along with a line of never-executed dead code. Reviewed-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org> Signed-off-by: Jonathan Corbet <corbet@lwn.net>
2025-07-15docs: kdoc: Regularize the use of the declaration nameJonathan Corbet
Each declaration type passes through the name in a unique field of the "args" blob - even though we have always just passed the name separately. Get rid of all the weird names and just use the common version. Reviewed-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org> Signed-off-by: Jonathan Corbet <corbet@lwn.net>
2025-07-15docs: kdoc: Coalesce parameter-list handlingJonathan Corbet
Callers to output_declaration() always pass the parameter information from self.entry; remove all of the boilerplate arguments and just get at that information directly. Formalize its placement in the KdocItem class. It would be nice to get rid of parameterlist as well, but that has the effect of reordering the output of function parameters and struct fields to match the order in the kerneldoc comment rather than in the declaration. One could argue about which is more correct, but the ordering has been left unchanged for now. Reviewed-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org> Signed-off-by: Jonathan Corbet <corbet@lwn.net>
2025-07-15docs: kdoc: use self.entry.parameterlist directly in check_sections()Jonathan Corbet
Callers of check_sections() join parameterlist into a single string, which is then immediately split back into the original list. Rather than do all that, just use parameterlist directly in check_sections(). Reviewed-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org> Signed-off-by: Jonathan Corbet <corbet@lwn.net>
2025-07-15docs: kdoc: remove the "struct_actual" machineryJonathan Corbet
The code goes out of its way to create a special list of parameters in entry.struct_actual that is just like entry.parameterlist, but with extra junk. The only use of that information, in check_sections(), promptly strips all the extra junk back out. Drop all that extra work and just use parameterlist. No output changes. Reviewed-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org> Signed-off-by: Jonathan Corbet <corbet@lwn.net>
2025-07-15docs: kdoc: Centralize handling of the item section listJonathan Corbet
The section list always comes directly from the under-construction entry and is used uniformly. Formalize section handling in the KdocItem class, and have output_declaration() load the sections directly from the entry, eliminating a lot of duplicated, verbose parameters. Reviewed-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org> Signed-off-by: Jonathan Corbet <corbet@lwn.net>
2025-07-15docs: kdoc: drop "sectionlist"Jonathan Corbet
Python dicts (as of 3.7) are guaranteed to remember the insertion order of items, so we do not need a separate list for that purpose. Drop the per-entry sectionlist variable and just rely on native dict ordering. Reviewed-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org> Signed-off-by: Jonathan Corbet <corbet@lwn.net>
2025-07-08scripts/kernel_doc.py: properly handle VIRTIO_DECLARE_FEATURESPaolo Abeni
The mentioned macro introduce by the next patch will foul kdoc; fully expand the mentioned macro to avoid the issue. Signed-off-by: Paolo Abeni <pabeni@redhat.com>
2025-07-08docs: kdoc: pretty up dump_enum()Jonathan Corbet
Add some comments to dump_enum to help the next person who has to figure out what it is actually doing. Reviewed-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org> Tested-by: Akira Yokosawa <akiyks@gmail.com> Signed-off-by: Jonathan Corbet <corbet@lwn.net> Link: https://lore.kernel.org/r/20250703184403.274408-8-corbet@lwn.net
2025-07-08docs: kdoc: some tweaks to process_proto_function()Jonathan Corbet
Add a set of comments to process_proto_function and reorganize the logic slightly; no functional change. Reviewed-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org> Tested-by: Akira Yokosawa <akiyks@gmail.com> Signed-off-by: Jonathan Corbet <corbet@lwn.net> Link: https://lore.kernel.org/r/20250703184403.274408-6-corbet@lwn.net
2025-07-08docs: kdoc: rework type prototype parsingJonathan Corbet
process_proto_type() is using a complex regex and a "while True" loop to split a declaration into chunks and, in the end, count brackets. Switch to using a simpler regex to just do the split directly, and handle each chunk as it comes. The result is, IMO, easier to understand and reason about. The old algorithm would occasionally elide the space between function parameters; see struct rng_alg->generate(), foe example. The only output difference is to not elide that space, which is more correct. Reviewed-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org> Tested-by: Akira Yokosawa <akiyks@gmail.com> Signed-off-by: Jonathan Corbet <corbet@lwn.net> Link: https://lore.kernel.org/r/20250703184403.274408-5-corbet@lwn.net
2025-07-08docs: kdoc: remove the brcount floor in process_proto_type()Jonathan Corbet
Putting the floor under brcount does not change the output in any way, just remove it. Change the termination test from ==0 to <=0 to prevent infinite loops in case somebody does something truly wacko in the code. Reviewed-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org> Tested-by: Akira Yokosawa <akiyks@gmail.com> Signed-off-by: Jonathan Corbet <corbet@lwn.net> Link: https://lore.kernel.org/r/20250703184403.274408-4-corbet@lwn.net
2025-07-08docs: kdoc: micro-optimize KernReJonathan Corbet
Rework _add_regex() to avoid doing the lookup twice for the (hopefully common) cache-hit case. Reviewed-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org> Tested-by: Akira Yokosawa <akiyks@gmail.com> Signed-off-by: Jonathan Corbet <corbet@lwn.net> Link: https://lore.kernel.org/r/20250703184403.274408-3-corbet@lwn.net
2025-07-08docs: kdoc: don't reinvent string.strip()Jonathan Corbet
process_proto_type() and process_proto_function() reinventing the strip() string method with a whole series of separate regexes; take all that out and just use strip(). The previous implementation also (in process_proto_type()) removed C++ comments *after* the above dance, leaving trailing whitespace in that case; now we do the stripping afterward. This results in exactly one output change: the removal of a spurious space in the definition of BACKLIGHT_POWER_REDUCED - see https://docs.kernel.org/gpu/backlight.html#c.backlight_properties. I note that we are putting semicolons after #define lines that really shouldn't be there - a task for another day. Reviewed-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org> Tested-by: Akira Yokosawa <akiyks@gmail.com> Signed-off-by: Jonathan Corbet <corbet@lwn.net> Link: https://lore.kernel.org/r/20250703184403.274408-2-corbet@lwn.net
2025-07-02docs: kdoc: simplify the output-item passingJonathan Corbet
Since our output items contain their name, we don't need to pass it separately. Reviewed-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org> Signed-off-by: Jonathan Corbet <corbet@lwn.net>
2025-07-02docs: kdoc; Add a rudimentary class to represent output itemsJonathan Corbet
This class is intended to replace the unstructured dict used to accumulate an entry to pass to an output module. For now, it remains unstructured, but it works well enough that the output classes don't notice the difference. Reviewed-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org> Signed-off-by: Jonathan Corbet <corbet@lwn.net>
2025-07-01docs: kdoc: pretty up dump_enum()Jonathan Corbet
Add some comments to dump_enum to help the next person who has to figure out what it is actually doing. Signed-off-by: Jonathan Corbet <corbet@lwn.net>
2025-07-01docs: kdoc: some tweaks to process_proto_function()Jonathan Corbet
Add a set of comments to process_proto_function and reorganize the logic slightly; no functional change. Signed-off-by: Jonathan Corbet <corbet@lwn.net>
2025-07-01docs: kdoc: rework type prototype parsingJonathan Corbet
process_proto_type() is using a complex regex and a "while True" loop to split a declaration into chunks and, in the end, count brackets. Switch to using a simpler regex to just do the split directly, and handle each chunk as it comes. The result is, IMO, easier to understand and reason about. The old algorithm would occasionally elide the space between function parameters; see struct rng_alg->generate(), foe example. The only output difference is to not elide that space, which is more correct. Signed-off-by: Jonathan Corbet <corbet@lwn.net>
2025-07-01docs: kdoc: split the processing of the two remaining inline statesJonathan Corbet
Now that "inline_*" are just ordinary parser states, split them into two separate functions, getting rid of some nested conditional logic. The original process_inline() would simply ignore lines that didn't match any of the regexes (those lacking the initial " * " marker). I have preserved that behavior, but we should perhaps emit a warning instead. Reviewed-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org> Signed-off-by: Jonathan Corbet <corbet@lwn.net> Link: https://lore.kernel.org/r/20250627184000.132291-9-corbet@lwn.net
2025-07-01docs: kdoc: remove the inline states-within-a-stateJonathan Corbet
The processing of inline kerneldoc comments is a state like the rest, but it was implemented as a set of separate substates. Just remove the substate logic and make the inline states normal ones like the rest. INLINE_ERROR was never actually used for anything, so just take it out. No changes to the generated output. Reviewed-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org> Signed-off-by: Jonathan Corbet <corbet@lwn.net> Link: https://lore.kernel.org/r/20250627184000.132291-8-corbet@lwn.net
2025-07-01docs: kdoc: remove the INLINE_END stateJonathan Corbet
It is never used, so just get rid of it. Reviewed-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org> Signed-off-by: Jonathan Corbet <corbet@lwn.net> Link: https://lore.kernel.org/r/20250627184000.132291-7-corbet@lwn.net
2025-07-01docs: kdoc: rework process_export() slightlyJonathan Corbet
Reorganize process_export() to eliminate duplicated code, don't look for exports in states where we don't expect them, and don't bother with normal state-machine processing if an export declaration has been found. Reviewed-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org> Signed-off-by: Jonathan Corbet <corbet@lwn.net> Link: https://lore.kernel.org/r/20250627184000.132291-6-corbet@lwn.net
2025-07-01docs: kdoc: remove KernelEntry::functionJonathan Corbet
This member is unused, to take it out. Reviewed-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org> Signed-off-by: Jonathan Corbet <corbet@lwn.net> Link: https://lore.kernel.org/r/20250627184000.132291-5-corbet@lwn.net
2025-07-01docs: kdoc: remove a bit of dead codeJonathan Corbet
The type_param regex matches "@..." just fine, so the special-case branch for that in dump_section() is never executed. Just remove it. Reviewed-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org> Signed-off-by: Jonathan Corbet <corbet@lwn.net> Link: https://lore.kernel.org/r/20250627184000.132291-4-corbet@lwn.net
2025-07-01docs: kdoc: Move content handling into KernelEntryJonathan Corbet
Rather than having other code mucking around with this bit of internal state, encapsulate it internally. Accumulate the description as a list of strings, joining them at the end, which is a more efficient way of building the text. Reviewed-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org> Signed-off-by: Jonathan Corbet <corbet@lwn.net> Link: https://lore.kernel.org/r/20250627184000.132291-3-corbet@lwn.net
2025-07-01docs: kdoc: remove KernelEntry::in_doc_sectJonathan Corbet
This field is not used for anything, just get rid of it. Reviewed-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org> Signed-off-by: Jonathan Corbet <corbet@lwn.net> Link: https://lore.kernel.org/r/20250627184000.132291-2-corbet@lwn.net
2025-06-30docs: kdoc: remove the brcount floor in process_proto_type()Jonathan Corbet
Putting the floor under brcount does not change the output in any way, just remove it. Change the termination test from ==0 to <=0 to prevent infinite loops in case somebody does something truly wacko in the code. Signed-off-by: Jonathan Corbet <corbet@lwn.net>
2025-06-30docs: kdoc: micro-optimize KernReJonathan Corbet
Switch KernRe::add_regex() to a try..except block to avoid looking up each regex twice. Signed-off-by: Jonathan Corbet <corbet@lwn.net>
2025-06-27docs: kdoc: don't reinvent string.strip()Jonathan Corbet
process_proto_type() and process_proto_function() reinventing the strip() string method with a whole series of separate regexes; take all that out and just use strip(). The previous implementation also (in process_proto_type()) removed C++ comments *after* the above dance, leaving trailing whitespace in that case; now we do the stripping afterward. This results in exactly one output change: the removal of a spurious space in the definition of BACKLIGHT_POWER_REDUCED - see https://docs.kernel.org/gpu/backlight.html#c.backlight_properties. I note that we are putting semicolons after #define lines that really shouldn't be there - a task for another day. Signed-off-by: Jonathan Corbet <corbet@lwn.net>
2025-06-27docs: kdoc: split the processing of the two remaining inline statesJonathan Corbet
Now that "inline_*" are just ordinary parser states, split them into two separate functions, getting rid of some nested conditional logic. The original process_inline() would simply ignore lines that didn't match any of the regexes (those lacking the initial " * " marker). I have preserved that behavior, but we should perhaps emit a warning instead. Signed-off-by: Jonathan Corbet <corbet@lwn.net>
2025-06-27docs: kdoc: remove the inline states-within-a-stateJonathan Corbet
The processing of inline kerneldoc comments is a state like the rest, but it was implemented as a set of separate substates. Just remove the substate logic and make the inline states normal ones like the rest. INLINE_ERROR was never actually used for anything, so just take it out. No changes to the generated output. Signed-off-by: Jonathan Corbet <corbet@lwn.net>
2025-06-26docs: kdoc: remove the INLINE_END stateJonathan Corbet
It is never used, so just get rid of it. Signed-off-by: Jonathan Corbet <corbet@lwn.net>
2025-06-25docs: kdoc: rework process_export() slightlyJonathan Corbet
Reorganize process_export() to eliminate duplicated code, don't look for exports in states where we don't expect them, and don't bother with normal state-machine processing if an export declaration has been found. Signed-off-by: Jonathan Corbet <corbet@lwn.net>
2025-06-25docs: kdoc: remove KernelEntry::functionJonathan Corbet
This member is unused, to take it out. Signed-off-by: Jonathan Corbet <corbet@lwn.net>
2025-06-25docs: kdoc: remove a bit of dead codeJonathan Corbet
The type_param regex matches "@..." just fine, so the special-case branch for that in dump_section() is never executed. Just remove it. Signed-off-by: Jonathan Corbet <corbet@lwn.net>
2025-06-25docs: kdoc: Move content handling into KernelEntryJonathan Corbet
Rather than having other code mucking around with this bit of internal state, encapsulate it internally. Accumulate the description as a list of strings, joining them at the end, which is a more efficient way of building the text. Signed-off-by: Jonathan Corbet <corbet@lwn.net>
2025-06-25docs: kdoc: remove KernelEntry::in_doc_sectJonathan Corbet
This field is not used for anything, just get rid of it. Signed-off-by: Jonathan Corbet <corbet@lwn.net>
2025-06-25docs: kdoc: finish disentangling the BODY and SPECIAL_SECTION statesJonathan Corbet
Move the last SPECIAL_SECTION special case into the proper handler function, getting rid of more if/then/else logic. The leading-space tracking was tightened up a bit in the move. Add some comments describing what is going on. No changes to the generated output. Reviewed-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org> Signed-off-by: Jonathan Corbet <corbet@lwn.net> Link: https://lore.kernel.org/r/20250621203512.223189-10-corbet@lwn.net
2025-06-25docs: kdoc: Add some comments to process_decl()Jonathan Corbet
Now that the function can actually fit into a human brain, add a few comments. While I was at it, I switched to the trim_whitespace() helper rather than open-coding it. Reviewed-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org> Signed-off-by: Jonathan Corbet <corbet@lwn.net> Link: https://lore.kernel.org/r/20250621203512.223189-9-corbet@lwn.net
2025-06-25docs: kdoc: coalesce the end-of-comment processingJonathan Corbet
Separate out the end-of-comment logic into its own helper and remove the duplicated code introduced earlier. Reviewed-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org> Signed-off-by: Jonathan Corbet <corbet@lwn.net> Link: https://lore.kernel.org/r/20250621203512.223189-8-corbet@lwn.net
2025-06-25docs: kdoc: rework the handling of SPECIAL_SECTIONJonathan Corbet
Move the recognition of this state to when we enter it, rather than when we exit, eliminating some twisty logic along the way. Some changes in output do result from this shift, generally for kerneldoc comments that do not quite fit the format. See, for example, struct irqdomain. As far as I can tell, the new behavior is more correct in each case. Reviewed-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org> Signed-off-by: Jonathan Corbet <corbet@lwn.net> Link: https://lore.kernel.org/r/20250621203512.223189-7-corbet@lwn.net
2025-06-25docs: kdoc: coalesce the new-section handlingJonathan Corbet
Merge the duplicated code back into a single implementation. Code movement only, no logic changes. Reviewed-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org> Signed-off-by: Jonathan Corbet <corbet@lwn.net> Link: https://lore.kernel.org/r/20250621203512.223189-6-corbet@lwn.net
2025-06-25docs: kdoc: split out the special-section stateJonathan Corbet
The state known as BODY_WITH_BLANK_LINE really, in a convoluted way, indicates a "special section" that is terminated by a blank line or the beginning of a new section. That is either "@param: desc" sections, or the weird "context" section that plays by the same rules. Rename the state to SPECIAL_SECTION and split its processing into a separate function; no real changes to the logic yet. Reviewed-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org> Signed-off-by: Jonathan Corbet <corbet@lwn.net> Link: https://lore.kernel.org/r/20250621203512.223189-5-corbet@lwn.net