summaryrefslogtreecommitdiff
path: root/scripts/lib/kdoc/kdoc_parser.py
AgeCommit message (Collapse)Author
2025-09-18docs: kdoc: a few more dump_typedef() tweaksJonathan Corbet
Merge "typedef" into the typedef_type pattern rather than repeating it later, and add some comments. Signed-off-by: Jonathan Corbet <corbet@lwn.net>
2025-09-18docs: kdoc: remove redundant comment stripping in dump_typedef()Jonathan Corbet
By the time we get here, comments have long since been stripped out; there is no need to do it again. Signed-off-by: Jonathan Corbet <corbet@lwn.net>
2025-09-18docs: kdoc: remove some dead code in dump_typedef()Jonathan Corbet
The regex in this block of code makes no sense, and a quick test shows that it never matches anything; simply delete the code. No output changes. Signed-off-by: Jonathan Corbet <corbet@lwn.net>
2025-09-18docs: kdoc: final dump_function() cleanupsJonathan Corbet
Add some more comments to dump_function(), add some comments, and trim out an unneeded duplicate output_declaration() call. Signed-off-by: Jonathan Corbet <corbet@lwn.net>
2025-09-18docs: kdoc: consolidate some of the macro-processing logicJonathan Corbet
The logic to handle macros is split in dump_function(); bring it all together into a single place and add a comment saying what's going on. Remove the unneeded is_define_proto variable, and tighten up the code a bit. Signed-off-by: Jonathan Corbet <corbet@lwn.net>
2025-09-18docs: kdoc: Simplify the dump_function() prototype regexesJonathan Corbet
The regexes for the parsing of function prototypes were more complicated than they needed to be and difficult to understand -- at least, I spent a fair amount of time bashing my head against them. Simplify them, and add some documentation comments as well. Signed-off-by: Jonathan Corbet <corbet@lwn.net>
2025-09-18docs: kdoc: remove a useless empty capture groupJonathan Corbet
The is_define_proto case in dump_function() uses a regex with an empty capture group - () - that has no use; just take it out. Signed-off-by: Jonathan Corbet <corbet@lwn.net>
2025-09-18docs: kdoc: remove a couple of spurious regex charactersJonathan Corbet
The "name" regex in dump_function() includes both the tilde and colon characters, but neither has any place in function prototypes. Remove the characters, after which the regex simplifies to "\w+" No output changes. Signed-off-by: Jonathan Corbet <corbet@lwn.net>
2025-09-18doc: kdoc: unify transform handlingJonathan Corbet
Both functions and structs are passed through a set of regex-based transforms, but the two were structured differently, despite being the same thing. Create a utility function to apply transformations and use it in both cases. Signed-off-by: Jonathan Corbet <corbet@lwn.net>
2025-09-18docs: kdoc: move the function transform patterns out of dump_function()Jonathan Corbet
Move these definitions to file level, where they are executed once, and don't clutter the function itself. Signed-off-by: Jonathan Corbet <corbet@lwn.net>
2025-09-18docs: kdoc: remove a single-use variableJonathan Corbet
struct_attribute is only used once, so just put its value there directly and drop the name. Signed-off-by: Jonathan Corbet <corbet@lwn.net>
2025-09-18docs: kdoc: tighten up the push_parameter() no-type caseJonathan Corbet
The handling of untyped parameters involved a number of redundant tests; restructure the code to remove them and be more compact. No output changes. Signed-off-by: Jonathan Corbet <corbet@lwn.net>
2025-09-18docs: kdoc: trim __cacheline_group_* with the other annotationsJonathan Corbet
The special case for __cacheline_group_begin/end() can be handled by just adding another pattern to the struct_prefixes, eliminating the need for a special case in push_parameter(). One change is that these annotations no longer appear in the rendered output, just like all the other annotations that we clean out. Signed-off-by: Jonathan Corbet <corbet@lwn.net>
2025-08-18docs: kdoc: remove redundant comment strippingJonathan Corbet
By the time stuff gets to create_parameter_list(), comments have long since been stripped out, so we do not need to do it again here. Acked-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org> Signed-off-by: Jonathan Corbet <corbet@lwn.net> Link: https://lore.kernel.org/r/20250814154035.328769-8-corbet@lwn.net
2025-08-18docs: kdoc: tighten up the pointer-to-function caseJonathan Corbet
Tighten up the code and remove an unneeded regex operation. Reviewed-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org> Signed-off-by: Jonathan Corbet <corbet@lwn.net> Link: https://lore.kernel.org/r/20250814154035.328769-7-corbet@lwn.net
2025-08-18docs: kdoc: tighten up the array-of-pointers caseJonathan Corbet
Simplify one gnarly regex and remove another altogether; add a comment describing what is going on. There will be no #-substituted commas in this case, so don't bother trying to put them back. Reviewed-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org> Signed-off-by: Jonathan Corbet <corbet@lwn.net> Link: https://lore.kernel.org/r/20250814154035.328769-6-corbet@lwn.net
2025-08-18docs: kdoc: add a couple more comments in create_parameter_list()Jonathan Corbet
Make what the final code is doing a bit more clear to slow readers like me. Reviewed-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org> Signed-off-by: Jonathan Corbet <corbet@lwn.net> Link: https://lore.kernel.org/r/20250814154035.328769-5-corbet@lwn.net
2025-08-18docs: kdoc: clean up the create_parameter_list() "first arg" logicJonathan Corbet
The logic for finding the name of the first in a series of variable names is somewhat convoluted and, in the use of .extend(), actively buggy. Document what is happening and simplify the logic. Acked-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org> Signed-off-by: Jonathan Corbet <corbet@lwn.net> Link: https://lore.kernel.org/r/20250814154035.328769-4-corbet@lwn.net
2025-08-18docs: kdoc: tidy up space removal in create_parameter_list()Jonathan Corbet
Remove a redundant test and add a comment describing what the space removal is doing. Reviewed-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org> Signed-off-by: Jonathan Corbet <corbet@lwn.net> Link: https://lore.kernel.org/r/20250814154035.328769-3-corbet@lwn.net
2025-08-18docs: kdoc: remove dead codeJonathan Corbet
create_parameter_list() tests an argument against the same regex twice, in two different locations; remove the pointless extra tests and the never-executed error cases that go with them. Reviewed-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org> Signed-off-by: Jonathan Corbet <corbet@lwn.net> Link: https://lore.kernel.org/r/20250814154035.328769-2-corbet@lwn.net
2025-08-11Merge branch 'dump-struct' into docs-mwJonathan Corbet
In my ongoing effort to truly understand our new kernel-doc, I continue to make changes to improve the code, and to try to make the understanding task easier for the next person. These patches focus on dump_struct() in particular, which starts out at nearly 300 lines long - to much to fit into my little brain anyway. Hopefully the result is easier to manage. There are no changes in the rendered docs.
2025-08-11docs: kdoc: a few final dump_struct() touchesJonathan Corbet
Add a couple more comments so that each phase of the process is now clearly marked. Reviewed-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org> Signed-off-by: Jonathan Corbet <corbet@lwn.net> Link: https://lore.kernel.org/r/20250807211639.47286-13-corbet@lwn.net
2025-08-11docs: kdoc: extract output formatting from dump_struct()Jonathan Corbet
The last thing done in dump_struct() is to format the structure for printing. That, too, is a separate activity; split it out into its own function. dump_struct() now fits in a single, full-hight editor screen. Reviewed-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org> Signed-off-by: Jonathan Corbet <corbet@lwn.net> Link: https://lore.kernel.org/r/20250807211639.47286-12-corbet@lwn.net
2025-08-11docs: kdoc: further rewrite_struct_members() cleanupJonathan Corbet
Get rid of some redundant checks, and generally tighten up the code; no logical change. Reviewed-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org> Signed-off-by: Jonathan Corbet <corbet@lwn.net> Link: https://lore.kernel.org/r/20250807211639.47286-11-corbet@lwn.net
2025-08-11docs: kdoc: Some rewrite_struct_members() commentingJonathan Corbet
Add comments to rewrite_struct_members() describing what it is actually doing, and reformat/comment the main struct_members regex so that it is (more) comprehensible to humans. Reviewed-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org> Signed-off-by: Jonathan Corbet <corbet@lwn.net> Link: https://lore.kernel.org/r/20250807211639.47286-10-corbet@lwn.net
2025-08-11docs: kdoc: remove an extraneous strip() callJonathan Corbet
...the variable in question was already strip()ed at the top of the loop. Reviewed-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org> Signed-off-by: Jonathan Corbet <corbet@lwn.net> Link: https://lore.kernel.org/r/20250807211639.47286-9-corbet@lwn.net
2025-08-11docs: kdoc: rework the rewrite_struct_members() main loopJonathan Corbet
Adopt a more Pythonic form for the main loop of this function, getting rid of the "while True:" construction and making the actual loop invariant explicit. Reviewed-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org> Signed-off-by: Jonathan Corbet <corbet@lwn.net> Link: https://lore.kernel.org/r/20250807211639.47286-8-corbet@lwn.net
2025-08-11docs: kdoc: split struct-member rewriting out of dump_struct()Jonathan Corbet
The massive loop that massages struct members shares no data with the rest of dump_struct(); split it out into its own function. Code movement only, no other changes. Reviewed-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org> Signed-off-by: Jonathan Corbet <corbet@lwn.net> Link: https://lore.kernel.org/r/20250807211639.47286-7-corbet@lwn.net
2025-08-11docs: kdoc: split top-level prototype parsing out of dump_struct()Jonathan Corbet
Move the initial split of the prototype into its own function in the ongoing effort to cut dump_struct() down to size. Reviewed-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org> Signed-off-by: Jonathan Corbet <corbet@lwn.net> Link: https://lore.kernel.org/r/20250807211639.47286-6-corbet@lwn.net
2025-08-11docs: kdoc: move the prefix transforms out of dump_struct()Jonathan Corbet
dump_struct is one of the longest functions in the kdoc_parser class, making it hard to read and reason about. Move the definition of the prefix transformations out of the function, join them with the definition of "attribute" (which was defined at the top of the file but only used here), and reformat the code slightly for shorter line widths. Just code movement in the end. Reviewed-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org> Signed-off-by: Jonathan Corbet <corbet@lwn.net> Link: https://lore.kernel.org/r/20250807211639.47286-5-corbet@lwn.net
2025-08-11docs: kdoc: backslashectomy in kdoc_parserJonathan Corbet
A lot of the regular expressions in this file have extraneous backslashes that may have been needed in Perl, but aren't helpful here. Take them out to reduce slightly the visual noise. Escaping of (){}[] has been left in place, even when unnecessary, for visual clarity. Reviewed-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org> Signed-off-by: Jonathan Corbet <corbet@lwn.net> Link: https://lore.kernel.org/r/20250807211639.47286-4-corbet@lwn.net
2025-08-11docs: kdoc: Move a regex line in dump_struct()Jonathan Corbet
The complex struct_members regex was defined far from its use; bring the two together. Remove some extraneous backslashes while making the move. Reviewed-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org> Signed-off-by: Jonathan Corbet <corbet@lwn.net> Link: https://lore.kernel.org/r/20250807211639.47286-3-corbet@lwn.net
2025-08-11docs: kdoc: consolidate the stripping of private struct/union membersJonathan Corbet
There were two locations duplicating the logic of stripping private members and associated comments; coalesce them into one, and add some comments describing what's going on. Output change: we now no longer add extraneous white space around macro definitions. Reviewed-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org> Signed-off-by: Jonathan Corbet <corbet@lwn.net> Link: https://lore.kernel.org/r/20250807211639.47286-2-corbet@lwn.net
2025-07-31Merge tag 'docs-6.17' of git://git.lwn.net/linuxLinus Torvalds
Pull documentation updates from Jonathan Corbet: "It has been a relatively busy cycle for docs, especially the build system: - The Perl kernel-doc script was added to 2.3.52pre1 just after the turn of the millennium. Over the following 25 years, it accumulated a vast amount of cruft, all in a language few people want to deal with anymore. Mauro's Python replacement in 6.16 faithfully reproduced all of the cruft in the hope of avoiding regressions. Now that we have a more reasonable code base, though, we can work on cleaning it up; many of the changes this time around are toward that end. - A reorganization of the ext4 docs into the usual TOC format. - Various Chinese translations and updates. - A new script from Mauro to help with docs-build testing. - A new document for linked lists - A sweep through MAINTAINERS fixing broken GitHub git:// repository links. ...and lots of fixes and updates" * tag 'docs-6.17' of git://git.lwn.net/linux: (147 commits) scripts: add origin commit identification based on specific patterns sphinx: kernel_abi: fix performance regression with O=<dir> Documentation: core-api: entry: Replace deprecated KVM entry/exit functions docs: fault-injection: drop reference to md-faulty docs: document linked lists scripts: kdoc: make it backward-compatible with Python 3.7 docs: kernel-doc: emit warnings for ancient versions of Python Documentation/rtla: Describe exit status Documentation/rtla: Add include common_appendix.rst docs: kernel: Clarify printk_ratelimit_burst reset behavior Documentation: ioctl-number: Don't repeat macro names Documentation: ioctl-number: Shorten macros table Documentation: ioctl-number: Correct full path to papr-physical-attestation.h Documentation: ioctl-number: Extend "Include File" column width Documentation: ioctl-number: Fix linuxppc-dev mailto link overlayfs.rst: fix typos docs: kdoc: emit a warning for ancient versions of Python docs: kdoc: clean up check_sections() docs: kdoc: directly access the always-there KdocItem fields docs: kdoc: straighten up dump_declaration() ...
2025-07-17scripts: kdoc: make it backward-compatible with Python 3.7Mauro Carvalho Chehab
There was a change at kdoc that ended breaking compatibility with Python 3.7: str.removesuffix() was introduced on version 3.9. Restore backward compatibility. Reported-by: Akira Yokosawa <akiyks@gmail.com> Closes: https://lore.kernel.org/linux-doc/57be9f77-9a94-4cde-aacb-184cae111506@gmail.com/ Fixes: 27ad33b6b349 ("kernel-doc: Fix symbol matching for dropped suffixes") Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org> Signed-off-by: Jonathan Corbet <corbet@lwn.net> Link: https://lore.kernel.org/r/d13058d285838ac2bc04c492e60531c013a8a919.1752218291.git.mchehab+huawei@kernel.org
2025-07-15docs: kdoc: emit a warning for ancient versions of PythonJonathan Corbet
Versions of Python prior to 3.7 do not guarantee to remember the insertion order of dicts; since kernel-doc depends on that guarantee, running with such older versions could result in output with reordered sections. Python 3.9 is the minimum for the kernel as a whole, so this should not be a problem, but put in a warning just in case somebody tries to use something older. Suggested-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org> Reviewed-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org> Signed-off-by: Jonathan Corbet <corbet@lwn.net>
2025-07-15docs: kdoc: clean up check_sections()Jonathan Corbet
entry.sectcheck is just a duplicate of our list of sections that is only passed to check_sections(); its main purpose seems to be to avoid checking the special named sections. Rework check_sections() to not use that field (which is then deleted), tocheck for the known sections directly, and tighten up the logic in general. Reviewed-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org> Signed-off-by: Jonathan Corbet <corbet@lwn.net>
2025-07-15docs: kdoc: straighten up dump_declaration()Jonathan Corbet
Get rid of the excess "return" statements in dump_declaration(), along with a line of never-executed dead code. Reviewed-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org> Signed-off-by: Jonathan Corbet <corbet@lwn.net>
2025-07-15docs: kdoc: Regularize the use of the declaration nameJonathan Corbet
Each declaration type passes through the name in a unique field of the "args" blob - even though we have always just passed the name separately. Get rid of all the weird names and just use the common version. Reviewed-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org> Signed-off-by: Jonathan Corbet <corbet@lwn.net>
2025-07-15docs: kdoc: Coalesce parameter-list handlingJonathan Corbet
Callers to output_declaration() always pass the parameter information from self.entry; remove all of the boilerplate arguments and just get at that information directly. Formalize its placement in the KdocItem class. It would be nice to get rid of parameterlist as well, but that has the effect of reordering the output of function parameters and struct fields to match the order in the kerneldoc comment rather than in the declaration. One could argue about which is more correct, but the ordering has been left unchanged for now. Reviewed-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org> Signed-off-by: Jonathan Corbet <corbet@lwn.net>
2025-07-15docs: kdoc: use self.entry.parameterlist directly in check_sections()Jonathan Corbet
Callers of check_sections() join parameterlist into a single string, which is then immediately split back into the original list. Rather than do all that, just use parameterlist directly in check_sections(). Reviewed-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org> Signed-off-by: Jonathan Corbet <corbet@lwn.net>
2025-07-15docs: kdoc: remove the "struct_actual" machineryJonathan Corbet
The code goes out of its way to create a special list of parameters in entry.struct_actual that is just like entry.parameterlist, but with extra junk. The only use of that information, in check_sections(), promptly strips all the extra junk back out. Drop all that extra work and just use parameterlist. No output changes. Reviewed-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org> Signed-off-by: Jonathan Corbet <corbet@lwn.net>
2025-07-15docs: kdoc: Centralize handling of the item section listJonathan Corbet
The section list always comes directly from the under-construction entry and is used uniformly. Formalize section handling in the KdocItem class, and have output_declaration() load the sections directly from the entry, eliminating a lot of duplicated, verbose parameters. Reviewed-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org> Signed-off-by: Jonathan Corbet <corbet@lwn.net>
2025-07-15docs: kdoc: drop "sectionlist"Jonathan Corbet
Python dicts (as of 3.7) are guaranteed to remember the insertion order of items, so we do not need a separate list for that purpose. Drop the per-entry sectionlist variable and just rely on native dict ordering. Reviewed-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org> Signed-off-by: Jonathan Corbet <corbet@lwn.net>
2025-07-08scripts/kernel_doc.py: properly handle VIRTIO_DECLARE_FEATURESPaolo Abeni
The mentioned macro introduce by the next patch will foul kdoc; fully expand the mentioned macro to avoid the issue. Signed-off-by: Paolo Abeni <pabeni@redhat.com>
2025-07-02docs: kdoc: simplify the output-item passingJonathan Corbet
Since our output items contain their name, we don't need to pass it separately. Reviewed-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org> Signed-off-by: Jonathan Corbet <corbet@lwn.net>
2025-07-02docs: kdoc; Add a rudimentary class to represent output itemsJonathan Corbet
This class is intended to replace the unstructured dict used to accumulate an entry to pass to an output module. For now, it remains unstructured, but it works well enough that the output classes don't notice the difference. Reviewed-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org> Signed-off-by: Jonathan Corbet <corbet@lwn.net>
2025-07-01docs: kdoc: pretty up dump_enum()Jonathan Corbet
Add some comments to dump_enum to help the next person who has to figure out what it is actually doing. Signed-off-by: Jonathan Corbet <corbet@lwn.net>
2025-07-01docs: kdoc: some tweaks to process_proto_function()Jonathan Corbet
Add a set of comments to process_proto_function and reorganize the logic slightly; no functional change. Signed-off-by: Jonathan Corbet <corbet@lwn.net>
2025-07-01docs: kdoc: rework type prototype parsingJonathan Corbet
process_proto_type() is using a complex regex and a "while True" loop to split a declaration into chunks and, in the end, count brackets. Switch to using a simpler regex to just do the split directly, and handle each chunk as it comes. The result is, IMO, easier to understand and reason about. The old algorithm would occasionally elide the space between function parameters; see struct rng_alg->generate(), foe example. The only output difference is to not elide that space, which is more correct. Signed-off-by: Jonathan Corbet <corbet@lwn.net>