diff options
Diffstat (limited to 'Documentation/process')
44 files changed, 5529 insertions, 1753 deletions
diff --git a/Documentation/process/1.Intro.rst b/Documentation/process/1.Intro.rst index c3d0270bbfb3..25ca49f7ae4d 100644 --- a/Documentation/process/1.Intro.rst +++ b/Documentation/process/1.Intro.rst @@ -251,12 +251,12 @@ there is no prospect of a migration to version 3 of the GPL in the foreseeable future. It is imperative that all code contributed to the kernel be legitimately -free software. For that reason, code from anonymous (or pseudonymous) -contributors will not be accepted. All contributors are required to "sign -off" on their code, stating that the code can be distributed with the -kernel under the GPL. Code which has not been licensed as free software by -its owner, or which risks creating copyright-related problems for the -kernel (such as code which derives from reverse-engineering efforts lacking +free software. For that reason, code from contributors without a known +identity or anonymous contributors will not be accepted. All contributors are +required to "sign off" on their code, stating that the code can be distributed +with the kernel under the GPL. Code which has not been licensed as free +software by its owner, or which risks creating copyright-related problems for +the kernel (such as code which derives from reverse-engineering efforts lacking proper safeguards) cannot be contributed. Questions about copyright-related issues are common on Linux development diff --git a/Documentation/process/2.Process.rst b/Documentation/process/2.Process.rst index 6a919cffcbfd..7bd41838a546 100644 --- a/Documentation/process/2.Process.rst +++ b/Documentation/process/2.Process.rst @@ -13,24 +13,19 @@ how the process works is required in order to be an effective part of it. The big picture --------------- -The kernel developers use a loosely time-based release process, with a new -major kernel release happening every two or three months. The recent -release history looks like this: - - ====== ================= - 5.0 March 3, 2019 - 5.1 May 5, 2019 - 5.2 July 7, 2019 - 5.3 September 15, 2019 - 5.4 November 24, 2019 - 5.5 January 6, 2020 - ====== ================= - -Every 5.x release is a major kernel release with new features, internal -API changes, and more. A typical release can contain about 13,000 -changesets with changes to several hundred thousand lines of code. 5.x is -the leading edge of Linux kernel development; the kernel uses a -rolling development model which is continually integrating major changes. +The Linux kernel uses a loosely time-based, rolling release development +model. A new major kernel release (which we will call, as an example, 9.x) +[1]_ happens every two or three months, which comes with new features, +internal API changes, and more. A typical release can contain about 13,000 +changesets with changes to several hundred thousand lines of code. Recent +releases, along with their dates, can be found at `Wikipedia +<https://en.wikipedia.org/wiki/Linux_kernel_version_history>`_. + +.. [1] Strictly speaking, the Linux kernel does not use semantic versioning + number scheme, but rather the 9.x pair identifies major release + version as a whole number. For each release, x is incremented, + but 9 is incremented only if x is deemed large enough (e.g. + Linux 5.0 is released following Linux 4.20). A relatively straightforward discipline is followed with regard to the merging of patches for each release. At the beginning of each development @@ -48,9 +43,9 @@ detail later on). The merge window lasts for approximately two weeks. At the end of this time, Linus Torvalds will declare that the window is closed and release the -first of the "rc" kernels. For the kernel which is destined to be 5.6, +first of the "rc" kernels. For the kernel which is destined to be 9.x, for example, the release which happens at the end of the merge window will -be called 5.6-rc1. The -rc1 release is the signal that the time to +be called 9.x-rc1. The -rc1 release is the signal that the time to merge new features has passed, and that the time to stabilize the next kernel has begun. @@ -99,13 +94,15 @@ release is made. In the real world, this kind of perfection is hard to achieve; there are just too many variables in a project of this size. There comes a point where delaying the final release just makes the problem worse; the pile of changes waiting for the next merge window will grow -larger, creating even more regressions the next time around. So most 5.x -kernels go out with a handful of known regressions though, hopefully, none -of them are serious. +larger, creating even more regressions the next time around. So most kernels +go out with a handful of known regressions, though, hopefully, none of them +are serious. Once a stable release is made, its ongoing maintenance is passed off to the -"stable team," currently Greg Kroah-Hartman. The stable team will release -occasional updates to the stable release using the 5.x.y numbering scheme. +"stable team," currently consists of Greg Kroah-Hartman and Sasha Levin. The +stable team will release occasional updates to the stable release using the +9.x.y numbering scheme. + To be considered for an update release, a patch must (1) fix a significant bug, and (2) already be merged into the mainline for the next development kernel. Kernels will typically receive stable updates for a little more @@ -392,13 +389,13 @@ represent a potential hazard to developers, who risk getting buried under a load of electronic mail, running afoul of the conventions used on the Linux lists, or both. -Most kernel mailing lists are run on vger.kernel.org; the master list can +Most kernel mailing lists are hosted at kernel.org; the master list can be found at: - http://vger.kernel.org/vger-lists.html + https://subspace.kernel.org -There are lists hosted elsewhere, though; a number of them are at -redhat.com/mailman/listinfo. +There are lists hosted elsewhere; please check the MAINTAINERS file for +the list relevant for any particular subsystem. The core mailing list for kernel development is, of course, linux-kernel. This list is an intimidating place to be; volume can reach 500 messages per @@ -434,9 +431,10 @@ There are a few hints which can help with linux-kernel survival: questions. Some developers can get impatient with people who clearly have not done their homework. -- Avoid top-posting (the practice of putting your answer above the quoted - text you are responding to). It makes your response harder to read and - makes a poor impression. +- Use interleaved ("inline") replies, which makes your response easier to + read. (i.e. avoid top-posting -- the practice of putting your answer above + the quoted text you are responding to.) For more details, see + :ref:`Documentation/process/submitting-patches.rst <interleaved_replies>`. - Ask on the correct mailing list. Linux-kernel may be the general meeting point, but it is not the best place to find developers from all diff --git a/Documentation/process/4.Coding.rst b/Documentation/process/4.Coding.rst index 1f0d81f44e14..80bcc1cabc23 100644 --- a/Documentation/process/4.Coding.rst +++ b/Documentation/process/4.Coding.rst @@ -63,9 +63,13 @@ these rules, to quickly re-format parts of your code automatically, and to review full files in order to spot coding style mistakes, typos and possible improvements. It is also handy for sorting ``#includes``, for aligning variables/macros, for reflowing text and other similar tasks. -See the file :ref:`Documentation/process/clang-format.rst <clangformat>` +See the file :ref:`Documentation/dev-tools/clang-format.rst <clangformat>` for more details. +Some basic editor settings, such as indentation and line endings, will be +set automatically if you are using an editor that is compatible with +EditorConfig. See the official EditorConfig website for more information: +https://editorconfig.org/ Abstraction layers ****************** diff --git a/Documentation/process/5.Posting.rst b/Documentation/process/5.Posting.rst index d87f1fee4cbc..9999bcbdccc9 100644 --- a/Documentation/process/5.Posting.rst +++ b/Documentation/process/5.Posting.rst @@ -191,11 +191,6 @@ change to a revision control system. It will be followed by: option to diff will associate function names with changes, making the resulting patch easier for others to read. -You should avoid including changes to irrelevant files (those generated by -the build process, for example, or editor backup files) in the patch. The -file "dontdiff" in the Documentation directory can help in this regard; -pass it to diff with the "-X" option. - The tags already briefly mentioned above are used to provide insights how the patch came into being. They are described in detail in the :ref:`Documentation/process/submitting-patches.rst <submittingpatches>` @@ -207,17 +202,26 @@ the patch:: Fixes: 1f2e3d4c5b6a ("The first line of the commit specified by the first 12 characters of its SHA-1 ID") Another tag is used for linking web pages with additional backgrounds or -details, for example a report about a bug fixed by the patch or a document -with a specification implemented by the patch:: +details, for example an earlier discussion which leads to the patch or a +document with a specification implemented by the patch:: Link: https://example.com/somewhere.html optional-other-stuff -Many maintainers when applying a patch also add this tag to link to the -latest public review posting of the patch; often this is automatically done -by tools like b4 or a git hook like the one described in -'Documentation/maintainer/configure-git.rst'. +As per guidance from the Chief Penguin, a Link: tag should only be added to +a commit if it leads to useful information that is not found in the commit +itself. + +If the URL points to a public bug report being fixed by the patch, use the +"Closes:" tag instead:: + + Closes: https://example.com/issues/1234 optional-other-stuff -A third kind of tag is used to document who was involved in the development of +Some bug trackers have the ability to close issues automatically when a +commit with such a tag is applied. Some bots monitoring mailing lists can +also track such tags and take certain actions. Private bug trackers and +invalid URLs are forbidden. + +Another kind of tag is used to document who was involved in the development of the patch. Each of these uses this format:: tag: Full Name <email address> optional-other-stuff @@ -251,15 +255,27 @@ The tags in common use are: - Reported-by: names a user who reported a problem which is fixed by this patch; this tag is used to give credit to the (often underappreciated) people who test our code and let us know when things do not work - correctly. + correctly. Note, this tag should be followed by a Closes: tag pointing to + the report, unless the report is not available on the web. The Link: tag + can be used instead of Closes: if the patch fixes a part of the issue(s) + being reported. + + - A Suggested-by: tag indicates that the patch idea is suggested by the person + named and ensures credit to the person for the idea. This will, hopefully, + inspire them to help us again in the future. - Cc: the named person received a copy of the patch and had the opportunity to comment on it. -Be careful in the addition of tags to your patches, as only Cc: is appropriate -for addition without the explicit permission of the person named; using -Reported-by: is fine most of the time as well, but ask for permission if -the bug was reported in private. +Be careful in the addition of the aforementioned tags to your patches, as all +except for Cc:, Reported-by:, and Suggested-by: need explicit permission of the +person named. For those three implicit permission is sufficient if the person +contributed to the Linux kernel using that name and email address according +to the lore archives or the commit history -- and in case of Reported-by: +and Suggested-by: did the reporting or suggestion in public. Note, +bugzilla.kernel.org is a public place in this sense, but email addresses +used there are private; so do not expose them in tags, unless the person +used them in earlier contributions. Sending the patch diff --git a/Documentation/process/6.Followthrough.rst b/Documentation/process/6.Followthrough.rst index a173cd5f93d2..66fa400c6d94 100644 --- a/Documentation/process/6.Followthrough.rst +++ b/Documentation/process/6.Followthrough.rst @@ -51,6 +51,13 @@ mind: working toward the creation of the best kernel they can; they are not trying to create discomfort for their employers' competitors. + - Be prepared for seemingly silly requests for coding style changes + and requests to factor out some of your code to shared parts of + the kernel. One job the maintainers do is to keep things looking + the same. Sometimes this means that the clever hack in your driver + to get around a problem actually needs to become a generalized + kernel feature ready for next time. + What all of this comes down to is that, when reviewers send you comments, you need to pay attention to the technical observations that they are making. Do not let their form of expression or your own pride keep that diff --git a/Documentation/process/7.AdvancedTopics.rst b/Documentation/process/7.AdvancedTopics.rst index bf7cbfb4caa5..43291704338e 100644 --- a/Documentation/process/7.AdvancedTopics.rst +++ b/Documentation/process/7.AdvancedTopics.rst @@ -146,6 +146,7 @@ pull. The git request-pull command can be helpful in this regard; it will format the request as other developers expect, and will also check to be sure that you have remembered to push those changes to the public server. +.. _development_advancedtopics_reviews: Reviewing patches ----------------- @@ -167,6 +168,12 @@ comments as questions rather than criticisms. Asking "how does the lock get released in this path?" will always work better than stating "the locking here is wrong." +Another technique that is useful in case of a disagreement is to ask for others +to chime in. If a discussion reaches a stalemate after a few exchanges, +then call for opinions of other reviewers or maintainers. Often those in +agreement with a reviewer remain silent unless called upon. +The opinion of multiple people carries exponentially more weight. + Different developers will review code from different points of view. Some are mostly concerned with coding style and whether code lines have trailing white space. Others will focus primarily on whether the change implemented @@ -176,3 +183,14 @@ security issues, duplication of code found elsewhere, adequate documentation, adverse effects on performance, user-space ABI changes, etc. All types of review, if they lead to better code going into the kernel, are welcome and worthwhile. + +There is no strict requirement to use specific tags like ``Reviewed-by``. +In fact reviews in plain English are more informative and encouraged +even when a tag is provided, e.g. "I looked at aspects A, B and C of this +submission and it looks good to me." +Some form of a review message or reply is obviously necessary otherwise +maintainers will not know that the reviewer has looked at the patch at all! + +Last but not least patch review may become a negative process, focused +on pointing out problems. Please throw in a compliment once in a while, +particularly for newbies! diff --git a/Documentation/process/adding-syscalls.rst b/Documentation/process/adding-syscalls.rst index 906c47f1a9e5..fc0b0bbcd34d 100644 --- a/Documentation/process/adding-syscalls.rst +++ b/Documentation/process/adding-syscalls.rst @@ -248,6 +248,52 @@ To summarize, you need a commit that includes: - fallback stub in ``kernel/sys_ni.c`` +.. _syscall_generic_6_11: + +Since 6.11 +~~~~~~~~~~ + +Starting with kernel version 6.11, general system call implementation for the +following architectures no longer requires modifications to +``include/uapi/asm-generic/unistd.h``: + + - arc + - arm64 + - csky + - hexagon + - loongarch + - nios2 + - openrisc + - riscv + +Instead, you need to update ``scripts/syscall.tbl`` and, if applicable, adjust +``arch/*/kernel/Makefile.syscalls``. + +As ``scripts/syscall.tbl`` serves as a common syscall table across multiple +architectures, a new entry is required in this table:: + + 468 common xyzzy sys_xyzzy + +Note that adding an entry to ``scripts/syscall.tbl`` with the "common" ABI +also affects all architectures that share this table. For more limited or +architecture-specific changes, consider using an architecture-specific ABI or +defining a new one. + +If a new ABI, say ``xyz``, is introduced, the corresponding updates should be +made to ``arch/*/kernel/Makefile.syscalls`` as well:: + + syscall_abis_{32,64} += xyz (...) + +To summarize, you need a commit that includes: + + - ``CONFIG`` option for the new function, normally in ``init/Kconfig`` + - ``SYSCALL_DEFINEn(xyzzy, ...)`` for the entry point + - corresponding prototype in ``include/linux/syscalls.h`` + - new entry in ``scripts/syscall.tbl`` + - (if needed) Makefile updates in ``arch/*/kernel/Makefile.syscalls`` + - fallback stub in ``kernel/sys_ni.c`` + + x86 System Call Implementation ------------------------------ @@ -353,6 +399,41 @@ To summarize, you need: ``include/uapi/asm-generic/unistd.h`` +Since 6.11 +~~~~~~~~~~ + +This applies to all the architectures listed in :ref:`Since 6.11<syscall_generic_6_11>` +under "Generic System Call Implementation", except arm64. See +:ref:`Compatibility System Calls (arm64)<compat_arm64>` for more information. + +You need to extend the entry in ``scripts/syscall.tbl`` with an extra column +to indicate that a 32-bit userspace program running on a 64-bit kernel should +hit the compat entry point:: + + 468 common xyzzy sys_xyzzy compat_sys_xyzzy + +To summarize, you need: + + - ``COMPAT_SYSCALL_DEFINEn(xyzzy, ...)`` for the compat entry point + - corresponding prototype in ``include/linux/compat.h`` + - modification of the entry in ``scripts/syscall.tbl`` to include an extra + "compat" column + - (if needed) 32-bit mapping struct in ``include/linux/compat.h`` + + +.. _compat_arm64: + +Compatibility System Calls (arm64) +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +On arm64, there is a dedicated syscall table for compatibility system calls +targeting 32-bit (AArch32) userspace: ``arch/arm64/tools/syscall_32.tbl``. +You need to add an additional line to this table specifying the compat +entry point:: + + 468 common xyzzy sys_xyzzy compat_sys_xyzzy + + Compatibility System Calls (x86) -------------------------------- @@ -575,3 +656,6 @@ References and Sources - Recommendation from Linus Torvalds that x32 system calls should prefer compatibility with 64-bit versions rather than 32-bit versions: https://lore.kernel.org/r/CA+55aFxfmwfB7jbbrXxa=K7VBYPfAvmu3XOkGrLbB1UFjX1+Ew@mail.gmail.com + - Patch series revising system call table infrastructure to use + scripts/syscall.tbl across multiple architectures: + https://lore.kernel.org/lkml/20240704143611.2979589-1-arnd@kernel.org diff --git a/Documentation/process/backporting.rst b/Documentation/process/backporting.rst new file mode 100644 index 000000000000..c42779fbcd33 --- /dev/null +++ b/Documentation/process/backporting.rst @@ -0,0 +1,604 @@ +.. SPDX-License-Identifier: GPL-2.0 + +=================================== +Backporting and conflict resolution +=================================== + +:Author: Vegard Nossum <vegard.nossum@oracle.com> + +.. contents:: + :local: + :depth: 3 + :backlinks: none + +Introduction +============ + +Some developers may never really have to deal with backporting patches, +merging branches, or resolving conflicts in their day-to-day work, so +when a merge conflict does pop up, it can be daunting. Luckily, +resolving conflicts is a skill like any other, and there are many useful +techniques you can use to make the process smoother and increase your +confidence in the result. + +This document aims to be a comprehensive, step-by-step guide to +backporting and conflict resolution. + +Applying the patch to a tree +============================ + +Sometimes the patch you are backporting already exists as a git commit, +in which case you just cherry-pick it directly using +``git cherry-pick``. However, if the patch comes from an email, as it +often does for the Linux kernel, you will need to apply it to a tree +using ``git am``. + +If you've ever used ``git am``, you probably already know that it is +quite picky about the patch applying perfectly to your source tree. In +fact, you've probably had nightmares about ``.rej`` files and trying to +edit the patch to make it apply. + +It is strongly recommended to instead find an appropriate base version +where the patch applies cleanly and *then* cherry-pick it over to your +destination tree, as this will make git output conflict markers and let +you resolve conflicts with the help of git and any other conflict +resolution tools you might prefer to use. For example, if you want to +apply a patch that just arrived on LKML to an older stable kernel, you +can apply it to the most recent mainline kernel and then cherry-pick it +to your older stable branch. + +It's generally better to use the exact same base as the one the patch +was generated from, but it doesn't really matter that much as long as it +applies cleanly and isn't too far from the original base. The only +problem with applying the patch to the "wrong" base is that it may pull +in more unrelated changes in the context of the diff when cherry-picking +it to the older branch. + +A good reason to prefer ``git cherry-pick`` over ``git am`` is that git +knows the precise history of an existing commit, so it will know when +code has moved around and changed the line numbers; this in turn makes +it less likely to apply the patch to the wrong place (which can result +in silent mistakes or messy conflicts). + +If you are using `b4`_. and you are applying the patch directly from an +email, you can use ``b4 am`` with the options ``-g``/``--guess-base`` +and ``-3``/``--prep-3way`` to do some of this automatically (see the +`b4 presentation`_ for more information). However, the rest of this +article will assume that you are doing a plain ``git cherry-pick``. + +.. _b4: https://people.kernel.org/monsieuricon/introducing-b4-and-patch-attestation +.. _b4 presentation: https://youtu.be/mF10hgVIx9o?t=2996 + +Once you have the patch in git, you can go ahead and cherry-pick it into +your source tree. Don't forget to cherry-pick with ``-x`` if you want a +written record of where the patch came from! + +Note that if you are submitting a patch for stable, the format is +slightly different; the first line after the subject line needs to be +either:: + + commit <upstream commit> upstream + +or:: + + [ Upstream commit <upstream commit> ] + +Resolving conflicts +=================== + +Uh-oh; the cherry-pick failed with a vaguely threatening message:: + + CONFLICT (content): Merge conflict + +What to do now? + +In general, conflicts appear when the context of the patch (i.e., the +lines being changed and/or the lines surrounding the changes) doesn't +match what's in the tree you are trying to apply the patch *to*. + +For backports, what likely happened was that the branch you are +backporting from contains patches not in the branch you are backporting +to. However, the reverse is also possible. In any case, the result is a +conflict that needs to be resolved. + +If your attempted cherry-pick fails with a conflict, git automatically +edits the files to include so-called conflict markers showing you where +the conflict is and how the two branches have diverged. Resolving the +conflict typically means editing the end result in such a way that it +takes into account these other commits. + +Resolving the conflict can be done either by hand in a regular text +editor or using a dedicated conflict resolution tool. + +Many people prefer to use their regular text editor and edit the +conflict directly, as it may be easier to understand what you're doing +and to control the final result. There are definitely pros and cons to +each method, and sometimes there's value in using both. + +We will not cover using dedicated merge tools here beyond providing some +pointers to various tools that you could use: + +- `Emacs Ediff mode <https://www.emacswiki.org/emacs/EdiffMode>`__ +- `vimdiff/gvimdiff <https://linux.die.net/man/1/vimdiff>`__ +- `KDiff3 <http://kdiff3.sourceforge.net/>`__ +- `TortoiseMerge <https://tortoisesvn.net/TortoiseMerge.html>`__ +- `Meld <https://meldmerge.org/help/>`__ +- `P4Merge <https://www.perforce.com/products/helix-core-apps/merge-diff-tool-p4merge>`__ +- `Beyond Compare <https://www.scootersoftware.com/>`__ +- `IntelliJ <https://www.jetbrains.com/help/idea/resolve-conflicts.html>`__ +- `VSCode <https://code.visualstudio.com/docs/editor/versioncontrol>`__ + +To configure git to work with these, see ``git mergetool --help`` or +the official `git-mergetool documentation`_. + +.. _git-mergetool documentation: https://git-scm.com/docs/git-mergetool + +Prerequisite patches +-------------------- + +Most conflicts happen because the branch you are backporting to is +missing some patches compared to the branch you are backporting *from*. +In the more general case (such as merging two independent branches), +development could have happened on either branch, or the branches have +simply diverged -- perhaps your older branch had some other backports +applied to it that themselves needed conflict resolutions, causing a +divergence. + +It's important to always identify the commit or commits that caused the +conflict, as otherwise you cannot be confident in the correctness of +your resolution. As an added bonus, especially if the patch is in an +area you're not that familiar with, the changelogs of these commits will +often give you the context to understand the code and potential problems +or pitfalls with your conflict resolution. + +git log +~~~~~~~ + +A good first step is to look at ``git log`` for the file that has the +conflict -- this is usually sufficient when there aren't a lot of +patches to the file, but may get confusing if the file is big and +frequently patched. You should run ``git log`` on the range of commits +between your currently checked-out branch (``HEAD``) and the parent of +the patch you are picking (``<commit>``), i.e.:: + + git log HEAD..<commit>^ -- <path> + +Even better, if you want to restrict this output to a single function +(because that's where the conflict appears), you can use the following +syntax:: + + git log -L:'\<function\>':<path> HEAD..<commit>^ + +.. note:: + The ``\<`` and ``\>`` around the function name ensure that the + matches are anchored on a word boundary. This is important, as this + part is actually a regex and git only follows the first match, so + if you use ``-L:thread_stack:kernel/fork.c`` it may only give you + results for the function ``try_release_thread_stack_to_cache`` even + though there are many other functions in that file containing the + string ``thread_stack`` in their names. + +Another useful option for ``git log`` is ``-G``, which allows you to +filter on certain strings appearing in the diffs of the commits you are +listing:: + + git log -G'regex' HEAD..<commit>^ -- <path> + +This can also be a handy way to quickly find when something (e.g. a +function call or a variable) was changed, added, or removed. The search +string is a regular expression, which means you can potentially search +for more specific things like assignments to a specific struct member:: + + git log -G'\->index\>.*=' + +git blame +~~~~~~~~~ + +Another way to find prerequisite commits (albeit only the most recent +one for a given conflict) is to run ``git blame``. In this case, you +need to run it against the parent commit of the patch you are +cherry-picking and the file where the conflict appeared, i.e.:: + + git blame <commit>^ -- <path> + +This command also accepts the ``-L`` argument (for restricting the +output to a single function), but in this case you specify the filename +at the end of the command as usual:: + + git blame -L:'\<function\>' <commit>^ -- <path> + +Navigate to the place where the conflict occurred. The first column of +the blame output is the commit ID of the patch that added a given line +of code. + +It might be a good idea to ``git show`` these commits and see if they +look like they might be the source of the conflict. Sometimes there will +be more than one of these commits, either because multiple commits +changed different lines of the same conflict area *or* because multiple +subsequent patches changed the same line (or lines) multiple times. In +the latter case, you may have to run ``git blame`` again and specify the +older version of the file to look at in order to dig further back in +the history of the file. + +Prerequisite vs. incidental patches +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Having found the patch that caused the conflict, you need to determine +whether it is a prerequisite for the patch you are backporting or +whether it is just incidental and can be skipped. An incidental patch +would be one that touches the same code as the patch you are +backporting, but does not change the semantics of the code in any +material way. For example, a whitespace cleanup patch is completely +incidental -- likewise, a patch that simply renames a function or a +variable would be incidental as well. On the other hand, if the function +being changed does not even exist in your current branch then this would +not be incidental at all and you need to carefully consider whether the +patch adding the function should be cherry-picked first. + +If you find that there is a necessary prerequisite patch, then you need +to stop and cherry-pick that instead. If you've already resolved some +conflicts in a different file and don't want to do it again, you can +create a temporary copy of that file. + +To abort the current cherry-pick, go ahead and run +``git cherry-pick --abort``, then restart the cherry-picking process +with the commit ID of the prerequisite patch instead. + +Understanding conflict markers +------------------------------ + +Combined diffs +~~~~~~~~~~~~~~ + +Let's say you've decided against picking (or reverting) additional +patches and you just want to resolve the conflict. Git will have +inserted conflict markers into your file. Out of the box, this will look +something like:: + + <<<<<<< HEAD + this is what's in your current tree before cherry-picking + ======= + this is what the patch wants it to be after cherry-picking + >>>>>>> <commit>... title + +This is what you would see if you opened the file in your editor. +However, if you were to run ``git diff`` without any arguments, the +output would look something like this:: + + $ git diff + [...] + ++<<<<<<<< HEAD + +this is what's in your current tree before cherry-picking + ++======== + + this is what the patch wants it to be after cherry-picking + ++>>>>>>>> <commit>... title + +When you are resolving a conflict, the behavior of ``git diff`` differs +from its normal behavior. Notice the two columns of diff markers +instead of the usual one; this is a so-called "`combined diff`_", here +showing the 3-way diff (or diff-of-diffs) between + +#. the current branch (before cherry-picking) and the current working + directory, and +#. the current branch (before cherry-picking) and the file as it looks + after the original patch has been applied. + +.. _combined diff: https://git-scm.com/docs/diff-format#_combined_diff_format + + +Better diffs +~~~~~~~~~~~~ + +3-way combined diffs include all the other changes that happened to the +file between your current branch and the branch you are cherry-picking +from. While this is useful for spotting other changes that you need to +take into account, this also makes the output of ``git diff`` somewhat +intimidating and difficult to read. You may instead prefer to run +``git diff HEAD`` (or ``git diff --ours``) which shows only the diff +between the current branch before cherry-picking and the current working +directory. It looks like this:: + + $ git diff HEAD + [...] + +<<<<<<<< HEAD + this is what's in your current tree before cherry-picking + +======== + +this is what the patch wants it to be after cherry-picking + +>>>>>>>> <commit>... title + +As you can see, this reads just like any other diff and makes it clear +which lines are in the current branch and which lines are being added +because they are part of the merge conflict or the patch being +cherry-picked. + +Merge styles and diff3 +~~~~~~~~~~~~~~~~~~~~~~ + +The default conflict marker style shown above is known as the ``merge`` +style. There is also another style available, known as the ``diff3`` +style, which looks like this:: + + <<<<<<< HEAD + this is what is in your current tree before cherry-picking + ||||||| parent of <commit> (title) + this is what the patch expected to find there + ======= + this is what the patch wants it to be after being applied + >>>>>>> <commit> (title) + +As you can see, this has 3 parts instead of 2, and includes what git +expected to find there but didn't. It is *highly recommended* to use +this conflict style as it makes it much clearer what the patch actually +changed; i.e., it allows you to compare the before-and-after versions +of the file for the commit you are cherry-picking. This allows you to +make better decisions about how to resolve the conflict. + +To change conflict marker styles, you can use the following command:: + + git config merge.conflictStyle diff3 + +There is a third option, ``zdiff3``, introduced in `Git 2.35`_, +which has the same 3 sections as ``diff3``, but where common lines have +been trimmed off, making the conflict area smaller in some cases. + +.. _Git 2.35: https://github.blog/2022-01-24-highlights-from-git-2-35/ + +Iterating on conflict resolutions +--------------------------------- + +The first step in any conflict resolution process is to understand the +patch you are backporting. For the Linux kernel this is especially +important, since an incorrect change can lead to the whole system +crashing -- or worse, an undetected security vulnerability. + +Understanding the patch can be easy or difficult depending on the patch +itself, the changelog, and your familiarity with the code being changed. +However, a good question for every change (or every hunk of the patch) +might be: "Why is this hunk in the patch?" The answers to these +questions will inform your conflict resolution. + +Resolution process +~~~~~~~~~~~~~~~~~~ + +Sometimes the easiest thing to do is to just remove all but the first +part of the conflict, leaving the file essentially unchanged, and apply +the changes by hand. Perhaps the patch is changing a function call +argument from ``0`` to ``1`` while a conflicting change added an +entirely new (and insignificant) parameter to the end of the parameter +list; in that case, it's easy enough to change the argument from ``0`` +to ``1`` by hand and leave the rest of the arguments alone. This +technique of manually applying changes is mostly useful if the conflict +pulled in a lot of unrelated context that you don't really need to care +about. + +For particularly nasty conflicts with many conflict markers, you can use +``git add`` or ``git add -i`` to selectively stage your resolutions to +get them out of the way; this also lets you use ``git diff HEAD`` to +always see what remains to be resolved or ``git diff --cached`` to see +what your patch looks like so far. + +Dealing with file renames +~~~~~~~~~~~~~~~~~~~~~~~~~ + +One of the most annoying things that can happen while backporting a +patch is discovering that one of the files being patched has been +renamed, as that typically means git won't even put in conflict markers, +but will just throw up its hands and say (paraphrased): "Unmerged path! +You do the work..." + +There are generally a few ways to deal with this. If the patch to the +renamed file is small, like a one-line change, the easiest thing is to +just go ahead and apply the change by hand and be done with it. On the +other hand, if the change is big or complicated, you definitely don't +want to do it by hand. + +As a first pass, you can try something like this, which will lower the +rename detection threshold to 30% (by default, git uses 50%, meaning +that two files need to have at least 50% in common for it to consider +an add-delete pair to be a potential rename):: + + git cherry-pick -strategy=recursive -Xrename-threshold=30 + +Sometimes the right thing to do will be to also backport the patch that +did the rename, but that's definitely not the most common case. Instead, +what you can do is to temporarily rename the file in the branch you're +backporting to (using ``git mv`` and committing the result), restart the +attempt to cherry-pick the patch, rename the file back (``git mv`` and +committing again), and finally squash the result using ``git rebase -i`` +(see the `rebase tutorial`_) so it appears as a single commit when you +are done. + +.. _rebase tutorial: https://medium.com/@slamflipstrom/a-beginners-guide-to-squashing-commits-with-git-rebase-8185cf6e62ec + +Gotchas +------- + +Function arguments +~~~~~~~~~~~~~~~~~~ + +Pay attention to changing function arguments! It's easy to gloss over +details and think that two lines are the same but actually they differ +in some small detail like which variable was passed as an argument +(especially if the two variables are both a single character that look +the same, like i and j). + +Error handling +~~~~~~~~~~~~~~ + +If you cherry-pick a patch that includes a ``goto`` statement (typically +for error handling), it is absolutely imperative to double check that +the target label is still correct in the branch you are backporting to. +The same goes for added ``return``, ``break``, and ``continue`` +statements. + +Error handling is typically located at the bottom of the function, so it +may not be part of the conflict even though could have been changed by +other patches. + +A good way to ensure that you review the error paths is to always use +``git diff -W`` and ``git show -W`` (AKA ``--function-context``) when +inspecting your changes. For C code, this will show you the whole +function that's being changed in a patch. One of the things that often +go wrong during backports is that something else in the function changed +on either of the branches that you're backporting from or to. By +including the whole function in the diff you get more context and can +more easily spot problems that might otherwise go unnoticed. + +Refactored code +~~~~~~~~~~~~~~~ + +Something that happens quite often is that code gets refactored by +"factoring out" a common code sequence or pattern into a helper +function. When backporting patches to an area where such a refactoring +has taken place, you effectively need to do the reverse when +backporting: a patch to a single location may need to be applied to +multiple locations in the backported version. (One giveaway for this +scenario is that a function was renamed -- but that's not always the +case.) + +To avoid incomplete backports, it's worth trying to figure out if the +patch fixes a bug that appears in more than one place. One way to do +this would be to use ``git grep``. (This is actually a good idea to do +in general, not just for backports.) If you do find that the same kind +of fix would apply to other places, it's also worth seeing if those +places exist upstream -- if they don't, it's likely the patch may need +to be adjusted. ``git log`` is your friend to figure out what happened +to these areas as ``git blame`` won't show you code that has been +removed. + +If you do find other instances of the same pattern in the upstream tree +and you're not sure whether it's also a bug, it may be worth asking the +patch author. It's not uncommon to find new bugs during backporting! + +Verifying the result +==================== + +colordiff +--------- + +Having committed a conflict-free new patch, you can now compare your +patch to the original patch. It is highly recommended that you use a +tool such as `colordiff`_ that can show two files side by side and color +them according to the changes between them:: + + colordiff -yw -W 200 <(git diff -W <upstream commit>^-) <(git diff -W HEAD^-) | less -SR + +.. _colordiff: https://www.colordiff.org/ + +Here, ``-y`` means to do a side-by-side comparison; ``-w`` ignores +whitespace, and ``-W 200`` sets the width of the output (as otherwise it +will use 130 by default, which is often a bit too little). + +The ``rev^-`` syntax is a handy shorthand for ``rev^..rev``, essentially +giving you just the diff for that single commit; also see +the official `git rev-parse documentation`_. + +.. _git rev-parse documentation: https://git-scm.com/docs/git-rev-parse#_other_rev_parent_shorthand_notations + +Again, note the inclusion of ``-W`` for ``git diff``; this ensures that +you will see the full function for any function that has changed. + +One incredibly important thing that colordiff does is to highlight lines +that are different. For example, if an error-handling ``goto`` has +changed labels between the original and backported patch, colordiff will +show these side-by-side but highlighted in a different color. Thus, it +is easy to see that the two ``goto`` statements are jumping to different +labels. Likewise, lines that were not modified by either patch but +differ in the context will also be highlighted and thus stand out during +a manual inspection. + +Of course, this is just a visual inspection; the real test is building +and running the patched kernel (or program). + +Build testing +------------- + +We won't cover runtime testing here, but it can be a good idea to build +just the files touched by the patch as a quick sanity check. For the +Linux kernel you can build single files like this, assuming you have the +``.config`` and build environment set up correctly:: + + make path/to/file.o + +Note that this won't discover linker errors, so you should still do a +full build after verifying that the single file compiles. By compiling +the single file first you can avoid having to wait for a full build *in +case* there are compiler errors in any of the files you've changed. + +Runtime testing +--------------- + +Even a successful build or boot test is not necessarily enough to rule +out a missing dependency somewhere. Even though the chances are small, +there could be code changes where two independent changes to the same +file result in no conflicts, no compile-time errors, and runtime errors +only in exceptional cases. + +One concrete example of this was a pair of patches to the system call +entry code where the first patch saved/restored a register and a later +patch made use of the same register somewhere in the middle of this +sequence. Since there was no overlap between the changes, one could +cherry-pick the second patch, have no conflicts, and believe that +everything was fine, when in fact the code was now scribbling over an +unsaved register. + +Although the vast majority of errors will be caught during compilation +or by superficially exercising the code, the only way to *really* verify +a backport is to review the final patch with the same level of scrutiny +as you would (or should) give to any other patch. Having unit tests and +regression tests or other types of automatic testing can help increase +the confidence in the correctness of a backport. + +Submitting backports to stable +============================== + +As the stable maintainers try to cherry-pick mainline fixes onto their +stable kernels, they may send out emails asking for backports when +encountering conflicts, see e.g. +<https://lore.kernel.org/stable/2023101528-jawed-shelving-071a@gregkh/>. +These emails typically include the exact steps you need to cherry-pick +the patch to the correct tree and submit the patch. + +One thing to make sure is that your changelog conforms to the expected +format:: + + <original patch title> + + [ Upstream commit <mainline rev> ] + + <rest of the original changelog> + [ <summary of the conflicts and their resolutions> ] + Signed-off-by: <your name and email> + +The "Upstream commit" line is sometimes slightly different depending on +the stable version. Older version used this format:: + + commit <mainline rev> upstream. + +It is most common to indicate the kernel version the patch applies to +in the email subject line (using e.g. +``git send-email --subject-prefix='PATCH 6.1.y'``), but you can also put +it in the Signed-off-by:-area or below the ``---`` line. + +The stable maintainers expect separate submissions for each active +stable version, and each submission should also be tested separately. + +A few final words of advice +=========================== + +1) Approach the backporting process with humility. +2) Understand the patch you are backporting; this means reading both + the changelog and the code. +3) Be honest about your confidence in the result when submitting the + patch. +4) Ask relevant maintainers for explicit acks. + +Examples +======== + +The above shows roughly the idealized process of backporting a patch. +For a more concrete example, see this video tutorial where two patches +are backported from mainline to stable: +`Backporting Linux Kernel Patches`_. + +.. _Backporting Linux Kernel Patches: https://youtu.be/sBR7R1V2FeA diff --git a/Documentation/process/botching-up-ioctls.rst b/Documentation/process/botching-up-ioctls.rst index ba4667ab396b..a05e8401de1c 100644 --- a/Documentation/process/botching-up-ioctls.rst +++ b/Documentation/process/botching-up-ioctls.rst @@ -41,7 +41,7 @@ will need to add a 32-bit compat layer: structures to the kernel, or if the kernel checks the structure size, which e.g. the drm core does. - * Pointers are __u64, cast from/to a uintprt_t on the userspace side and + * Pointers are __u64, cast from/to a uintptr_t on the userspace side and from/to a void __user * in the kernel. Try really hard not to delay this conversion or worse, fiddle the raw __u64 through your code since that diminishes the checking tools like sparse can provide. The macro @@ -208,7 +208,7 @@ Not every problem needs a new ioctl: it's much quicker to push a driver-private interface than engaging in lengthy discussions for a more generic solution. And occasionally doing a private interface to spearhead a new concept is what's required. But in the - end, once the generic interface comes around you'll end up maintainer two + end, once the generic interface comes around you'll end up maintaining two interfaces. Indefinitely. * Consider other interfaces than ioctls. A sysfs attribute is much better for diff --git a/Documentation/process/changes.rst b/Documentation/process/changes.rst index 9844ca3a71a6..62951cdb13ad 100644 --- a/Documentation/process/changes.rst +++ b/Documentation/process/changes.rst @@ -29,24 +29,23 @@ you probably needn't concern yourself with pcmciautils. ====================== =============== ======================================== Program Minimal version Command to check the version ====================== =============== ======================================== -GNU C 5.1 gcc --version -Clang/LLVM (optional) 11.0.0 clang --version -Rust (optional) 1.62.0 rustc --version -bindgen (optional) 0.56.0 bindgen --version -GNU make 3.82 make --version +GNU C 8.1 gcc --version +Clang/LLVM (optional) 15.0.0 clang --version +Rust (optional) 1.78.0 rustc --version +bindgen (optional) 0.65.1 bindgen --version +GNU make 4.0 make --version bash 4.2 bash --version -binutils 2.23 ld -v +binutils 2.30 ld -v flex 2.5.35 flex --version bison 2.0 bison --version pahole 1.16 pahole --version -util-linux 2.10o fdformat --version +util-linux 2.10o mount --version kmod 13 depmod -V e2fsprogs 1.41.4 e2fsck -V jfsutils 1.1.3 fsck.jfs -V -reiserfsprogs 3.6.3 reiserfsck -V xfsprogs 2.6.0 xfs_db -V squashfs-tools 4.0 mksquashfs -version -btrfs-progs 0.18 btrfsck +btrfs-progs 0.18 btrfs --version pcmciautils 004 pccardctl -V quota-tools 3.09 quota -V PPP 2.4.0 pppd --version @@ -58,8 +57,12 @@ mcelog 0.6 mcelog --version iptables 1.4.2 iptables -V openssl & libcrypto 1.0.0 openssl version bc 1.06.95 bc --version -Sphinx\ [#f1]_ 1.7 sphinx-build --version -cpio any cpio --version +Sphinx\ [#f1]_ 3.4.3 sphinx-build --version +GNU tar 1.28 tar --version +gtags (optional) 6.6.5 gtags --version +mkimage (optional) 2017.01 mkimage --version +Python 3.9.x python3 --version +GNU AWK (optional) 5.1.0 gawk --version ====================== =============== ======================================== .. [#f1] Sphinx is needed only to build the Kernel documentation @@ -85,14 +88,7 @@ docs on :ref:`Building Linux with Clang/LLVM <kbuild_llvm>`. Rust (optional) --------------- -A particular version of the Rust toolchain is required. Newer versions may or -may not work because the kernel depends on some unstable Rust features, for -the moment. - -Each Rust toolchain comes with several "components", some of which are required -(like ``rustc``) and some that are optional. The ``rust-src`` component (which -is optional) needs to be installed to build the kernel. Other components are -useful for developing. +A recent version of the Rust compiler is required. Please see Documentation/rust/quick-start.rst for instructions on how to satisfy the build requirements of Rust support. In particular, the ``Makefile`` @@ -108,7 +104,7 @@ It depends on ``libclang``. Make ---- -You will need GNU make 3.82 or later to build the kernel. +You will need GNU make 4.0 or later to build the kernel. Bash ---- @@ -119,7 +115,7 @@ Bash 4.2 or newer is needed. Binutils -------- -Binutils 2.23 or newer is needed to build the kernel. +Binutils 2.30 or newer is needed to build the kernel. pkg-config ---------- @@ -142,8 +138,8 @@ Bison Since Linux 4.16, the build system generates parsers during build. This requires bison 2.0 or later. -pahole: -------- +pahole +------ Since Linux 5.2, if CONFIG_DEBUG_INFO_BTF is selected, the build system generates BTF (BPF Type Format) from DWARF in vmlinux, a bit later from kernel @@ -158,6 +154,13 @@ Perl You will need perl 5 and the following modules: ``Getopt::Long``, ``Getopt::Std``, ``File::Basename``, and ``File::Find`` to build the kernel. +Python +------ + +Several config options require it: it is required for arm/arm64 +default configs, CONFIG_LTO_CLANG, some DRM optional configs, +the kernel-doc tool, and docs build (Sphinx), among others. + BC -- @@ -174,6 +177,32 @@ You will need openssl to build kernels 3.7 and higher if module signing is enabled. You will also need openssl development packages to build kernels 4.3 and higher. +Tar +--- + +GNU tar is needed if you want to enable access to the kernel headers via sysfs +(CONFIG_IKHEADERS). + +gtags / GNU GLOBAL (optional) +----------------------------- + +The kernel build requires GNU GLOBAL version 6.6.5 or later to generate +tag files through ``make gtags``. This is due to its use of the gtags +``-C (--directory)`` flag. + +mkimage +------- + +This tool is used when building a Flat Image Tree (FIT), commonly used on ARM +platforms. The tool is available via the ``u-boot-tools`` package or can be +built from the U-Boot source code. See the instructions at +https://docs.u-boot.org/en/latest/build/tools.html#building-tools-for-linux + +GNU AWK +------- + +GNU AWK is needed if you want kernel builds to generate address range data for +builtin modules (CONFIG_BUILTIN_MODULE_RANGES). System utilities **************** @@ -199,7 +228,7 @@ Util-linux New versions of util-linux provide ``fdisk`` support for larger disks, support new options to mount, recognize more supported partition -types, have a fdformat which works with 2.4 kernels, and similar goodies. +types, and similar goodies. You'll probably want to upgrade. Ksymoops @@ -239,14 +268,6 @@ The following utilities are available: - other file system utilities are also available in this package. -Reiserfsprogs -------------- - -The reiserfsprogs package should be used for reiserfs-3.6.x -(Linux kernels 2.4.x). It is a combined package and contains working -versions of ``mkreiserfs``, ``resize_reiserfs``, ``debugreiserfs`` and -``reiserfsck``. These utils work on both i386 and alpha platforms. - Xfsprogs -------- @@ -468,12 +489,7 @@ E2fsprogs JFSutils -------- -- <http://jfs.sourceforge.net/> - -Reiserfsprogs -------------- - -- <https://git.kernel.org/pub/scm/linux/kernel/git/jeffm/reiserfsprogs.git/> +- <https://jfs.sourceforge.net/> Xfsprogs -------- @@ -489,7 +505,7 @@ Pcmciautils Quota-tools ----------- -- <http://sourceforge.net/projects/linuxquota/> +- <https://sourceforge.net/projects/linuxquota/> Intel P6 microcode @@ -510,12 +526,7 @@ FUSE mcelog ------ -- <http://www.mcelog.org/> - -cpio ----- - -- <https://www.gnu.org/software/cpio/> +- <https://www.mcelog.org/> Networking ********** @@ -530,7 +541,8 @@ PPP NFS-utils --------- -- <http://sourceforge.net/project/showfiles.php?group_id=14> +- <https://sourceforge.net/project/showfiles.php?group_id=14> +- <https://nfs.sourceforge.net/> Iptables -------- @@ -545,12 +557,7 @@ Ip-route2 OProfile -------- -- <http://oprofile.sf.net/download/> - -NFS-Utils ---------- - -- <http://nfs.sourceforge.net/> +- <https://oprofile.sf.net/download/> Kernel documentation ******************** diff --git a/Documentation/process/clang-format.rst b/Documentation/process/clang-format.rst deleted file mode 100644 index 1d089a847c1b..000000000000 --- a/Documentation/process/clang-format.rst +++ /dev/null @@ -1,184 +0,0 @@ -.. _clangformat: - -clang-format -============ - -``clang-format`` is a tool to format C/C++/... code according to -a set of rules and heuristics. Like most tools, it is not perfect -nor covers every single case, but it is good enough to be helpful. - -``clang-format`` can be used for several purposes: - - - Quickly reformat a block of code to the kernel style. Specially useful - when moving code around and aligning/sorting. See clangformatreformat_. - - - Spot style mistakes, typos and possible improvements in files - you maintain, patches you review, diffs, etc. See clangformatreview_. - - - Help you follow the coding style rules, specially useful for those - new to kernel development or working at the same time in several - projects with different coding styles. - -Its configuration file is ``.clang-format`` in the root of the kernel tree. -The rules contained there try to approximate the most common kernel -coding style. They also try to follow :ref:`Documentation/process/coding-style.rst <codingstyle>` -as much as possible. Since not all the kernel follows the same style, -it is possible that you may want to tweak the defaults for a particular -subsystem or folder. To do so, you can override the defaults by writing -another ``.clang-format`` file in a subfolder. - -The tool itself has already been included in the repositories of popular -Linux distributions for a long time. Search for ``clang-format`` in -your repositories. Otherwise, you can either download pre-built -LLVM/clang binaries or build the source code from: - - https://releases.llvm.org/download.html - -See more information about the tool at: - - https://clang.llvm.org/docs/ClangFormat.html - - https://clang.llvm.org/docs/ClangFormatStyleOptions.html - - -.. _clangformatreview: - -Review files and patches for coding style ------------------------------------------ - -By running the tool in its inline mode, you can review full subsystems, -folders or individual files for code style mistakes, typos or improvements. - -To do so, you can run something like:: - - # Make sure your working directory is clean! - clang-format -i kernel/*.[ch] - -And then take a look at the git diff. - -Counting the lines of such a diff is also useful for improving/tweaking -the style options in the configuration file; as well as testing new -``clang-format`` features/versions. - -``clang-format`` also supports reading unified diffs, so you can review -patches and git diffs easily. See the documentation at: - - https://clang.llvm.org/docs/ClangFormat.html#script-for-patch-reformatting - -To avoid ``clang-format`` formatting some portion of a file, you can do:: - - int formatted_code; - // clang-format off - void unformatted_code ; - // clang-format on - void formatted_code_again; - -While it might be tempting to use this to keep a file always in sync with -``clang-format``, specially if you are writing new files or if you are -a maintainer, please note that people might be running different -``clang-format`` versions or not have it available at all. Therefore, -you should probably refrain yourself from using this in kernel sources; -at least until we see if ``clang-format`` becomes commonplace. - - -.. _clangformatreformat: - -Reformatting blocks of code ---------------------------- - -By using an integration with your text editor, you can reformat arbitrary -blocks (selections) of code with a single keystroke. This is specially -useful when moving code around, for complex code that is deeply intended, -for multi-line macros (and aligning their backslashes), etc. - -Remember that you can always tweak the changes afterwards in those cases -where the tool did not do an optimal job. But as a first approximation, -it can be very useful. - -There are integrations for many popular text editors. For some of them, -like vim, emacs, BBEdit and Visual Studio you can find support built-in. -For instructions, read the appropriate section at: - - https://clang.llvm.org/docs/ClangFormat.html - -For Atom, Eclipse, Sublime Text, Visual Studio Code, XCode and other -editors and IDEs you should be able to find ready-to-use plugins. - -For this use case, consider using a secondary ``.clang-format`` -so that you can tweak a few options. See clangformatextra_. - - -.. _clangformatmissing: - -Missing support ---------------- - -``clang-format`` is missing support for some things that are common -in kernel code. They are easy to remember, so if you use the tool -regularly, you will quickly learn to avoid/ignore those. - -In particular, some very common ones you will notice are: - - - Aligned blocks of one-line ``#defines``, e.g.:: - - #define TRACING_MAP_BITS_DEFAULT 11 - #define TRACING_MAP_BITS_MAX 17 - #define TRACING_MAP_BITS_MIN 7 - - vs.:: - - #define TRACING_MAP_BITS_DEFAULT 11 - #define TRACING_MAP_BITS_MAX 17 - #define TRACING_MAP_BITS_MIN 7 - - - Aligned designated initializers, e.g.:: - - static const struct file_operations uprobe_events_ops = { - .owner = THIS_MODULE, - .open = probes_open, - .read = seq_read, - .llseek = seq_lseek, - .release = seq_release, - .write = probes_write, - }; - - vs.:: - - static const struct file_operations uprobe_events_ops = { - .owner = THIS_MODULE, - .open = probes_open, - .read = seq_read, - .llseek = seq_lseek, - .release = seq_release, - .write = probes_write, - }; - - -.. _clangformatextra: - -Extra features/options ----------------------- - -Some features/style options are not enabled by default in the configuration -file in order to minimize the differences between the output and the current -code. In other words, to make the difference as small as possible, -which makes reviewing full-file style, as well diffs and patches as easy -as possible. - -In other cases (e.g. particular subsystems/folders/files), the kernel style -might be different and enabling some of these options may approximate -better the style there. - -For instance: - - - Aligning assignments (``AlignConsecutiveAssignments``). - - - Aligning declarations (``AlignConsecutiveDeclarations``). - - - Reflowing text in comments (``ReflowComments``). - - - Sorting ``#includes`` (``SortIncludes``). - -They are typically useful for block re-formatting, rather than full-file. -You might want to create another ``.clang-format`` file and use that one -from your editor/IDE instead. diff --git a/Documentation/process/code-of-conduct-interpretation.rst b/Documentation/process/code-of-conduct-interpretation.rst index 922e0b547bc3..4cdef8360698 100644 --- a/Documentation/process/code-of-conduct-interpretation.rst +++ b/Documentation/process/code-of-conduct-interpretation.rst @@ -51,7 +51,7 @@ the Technical Advisory Board (TAB) or other maintainers if you're uncertain how to handle situations that come up. It will not be considered a violation report unless you want it to be. If you are uncertain about approaching the TAB or any other maintainers, please -reach out to our conflict mediator, Joanna Lee <joanna.lee@gesmer.com>. +reach out to our conflict mediator, Joanna Lee <jlee@linuxfoundation.org>. In the end, "be kind to each other" is really what the end goal is for everybody. We know everyone is human and we all fail at times, but the @@ -145,14 +145,106 @@ kernel community. Any decisions regarding enforcement recommendations will be brought to the TAB for implementation of enforcement with the relevant maintainers -if needed. A decision by the Code of Conduct Committee can be overturned -by the TAB by a two-thirds vote. +if needed. Once the TAB approves one or more of the measures outlined +in the scope of the ban by two-thirds of the members voting for the +measures, the Code of Conduct Committee will enforce the TAB approved +measures. Any Code of Conduct Committee members serving on the TAB will +not vote on the measures. At quarterly intervals, the Code of Conduct Committee and TAB will provide a report summarizing the anonymised reports that the Code of Conduct committee has received and their status, as well details of any -overridden decisions including complete and identifiable voting details. +TAB approved decisions including complete and identifiable voting details. Because how we interpret and enforce the Code of Conduct will evolve over time, this document will be updated when necessary to reflect any changes. + +Enforcement for Unacceptable Behavior Code of Conduct Violations +---------------------------------------------------------------- + +The Code of Conduct committee works to ensure that our community continues +to be inclusive and fosters diverse discussions and viewpoints, and works +to improve those characteristics over time. A majority of the reports the +Code of Conduct Committee receives stem from incorrect understanding regarding +the development process and maintainers' roles, responsibilities, and their +right to make decisions on code acceptance. These are resolved through +clarification of the development process and the scope of the Code of Conduct. + +Unacceptable behaviors could interrupt respectful collaboration for a short +period of time and negatively impact the health of the community longer term. +Unacceptable behaviors often get resolved when individuals acknowledge their +behavior and make amends for it in the setting the violation has taken place. + +The Code of Conduct Committee receives reports about unacceptable behaviors +when they don't get resolved through community discussions. The Code of +Conduct committee takes measures to restore productive and respectful +collaboration when an unacceptable behavior has negatively impacted that +relationship. + +The Code of Conduct Committee has the obligation to keep the reports and +reporters' information private. Reports could come from injured parties +and community members who are observers of unacceptable behaviors. The +Code of Conduct Committee has the responsibility to investigate and resolve +these reports, working with all involved parties. + +The Code of Conduct Committee works with the individual to bring about +change in their understanding of the importance to repair the damage caused +by their behavior to the injured party and the long term negative impact +on the community. + +The goal is to reach a resolution which is agreeable to all parties. If +working with the individual fails to bring about the desired outcome, the +Code of Conduct Committee will evaluate other measures such as seeking +public apology to repair the damage. + +Seek public apology for the violation +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +The Code of Conduct Committee publicly calls out the behavior in the +setting in which the violation has taken place, seeking public apology +for the violation. + +A public apology for the violation is the first step towards rebuilding +the trust. Trust is essential for the continued success and health of the +community which operates on trust and respect. + +Remedial measures if there is no public apology for the violation +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +The Code of Conduct Committee determines the next course of action to restore +the healthy collaboration by recommending remedial measure(s) to the TAB for +approval. + +- Ban violator from participating in the kernel development process for + a period of up to a full kernel development cycle. The Code of Conduct + Committee could require public apology as a condition for lifting the + ban. + +The scope of the ban for a period of time could include: + + a. denying patch contributions and pull requests + b. pausing collaboration with the violator by ignoring their + contributions and/or blocking their email account(s) + c. restricting their ability to communicate via kernel.org platforms, + such as mailing lists and social media sites + +Once the TAB approves one or more of the measures outlined in the scope of +the ban by two-thirds of the members voting for the measures, the Code of +Conduct Committee will enforce the TAB approved measure(s) in collaboration +with the community, maintainers, sub-maintainers, and kernel.org +administrators. Any Code of Conduct Committee members serving on the TAB +will not vote on the measures. + +The Code of Conduct Committee is mindful of the negative impact of seeking +public apology and instituting ban could have on individuals. It is also +mindful of the longer term harm to the community that could result from +not taking action when such serious public violations occur. + +The effectiveness of the remedial measure(s) approved by the TAB depends +on the trust and cooperation from the community, maintainers, sub-maintainers, +and kernel.org administrators in enforcing them. + +The Code of Conduct Committee sincerely hopes that unacceptable behaviors +that require seeking public apologies continue to be exceedingly rare +occurrences in the future. diff --git a/Documentation/process/coding-style.rst b/Documentation/process/coding-style.rst index 007e49ef6cec..2969ca378dbb 100644 --- a/Documentation/process/coding-style.rst +++ b/Documentation/process/coding-style.rst @@ -76,7 +76,7 @@ Don't use commas to avoid using braces: if (condition) do_this(), do_that(); -Always uses braces for multiple statements: +Always use braces for multiple statements: .. code-block:: c @@ -203,7 +203,7 @@ Do not unnecessarily use braces where a single statement will do. and -.. code-block:: none +.. code-block:: c if (condition) do_this(); @@ -586,9 +586,9 @@ fix for this is to split it up into two error labels ``err_free_bar:`` and .. code-block:: c - err_free_bar: + err_free_bar: kfree(foo->bar); - err_free_foo: + err_free_foo: kfree(foo); return ret; @@ -614,7 +614,10 @@ it. When commenting the kernel API functions, please use the kernel-doc format. See the files at :ref:`Documentation/doc-guide/ <doc_guide>` and -``scripts/kernel-doc`` for details. +``scripts/kernel-doc`` for details. Note that the danger of over-commenting +applies to kernel-doc comments all the same. Do not add boilerplate +kernel-doc which simply reiterates what's obvious from the signature +of the function. The preferred style for long (multi-line) comments is: @@ -629,18 +632,6 @@ The preferred style for long (multi-line) comments is: * with beginning and ending almost-blank lines. */ -For files in net/ and drivers/net/ the preferred style for long (multi-line) -comments is a little different. - -.. code-block:: c - - /* The preferred comment style for files in net/ and drivers/net - * looks like this. - * - * It is nearly the same as the generally preferred comment style, - * but there is no initial almost-blank line. - */ - It's also important to comment data, whether they are basic types or derived types. To this end, use just one data declaration per line (no commas for multiple data declarations). This leaves you room for a small comment on each @@ -660,7 +651,7 @@ make a good program). So, you can either get rid of GNU emacs, or change it to use saner values. To do the latter, you can stick the following in your .emacs file: -.. code-block:: none +.. code-block:: elisp (defun c-lineup-arglist-tabs-only (ignored) "Line up argument lists by tabs, not spaces" @@ -679,7 +670,7 @@ values. To do the latter, you can stick the following in your .emacs file: (c-offsets-alist . ( (arglist-close . c-lineup-arglist-tabs-only) (arglist-cont-nonempty . - (c-lineup-gcc-asm-reg c-lineup-arglist-tabs-only)) + (c-lineup-gcc-asm-reg c-lineup-arglist-tabs-only)) (arglist-intro . +) (brace-list-intro . +) (c . c-lineup-C-comments) @@ -732,9 +723,13 @@ these rules, to quickly re-format parts of your code automatically, and to review full files in order to spot coding style mistakes, typos and possible improvements. It is also handy for sorting ``#includes``, for aligning variables/macros, for reflowing text and other similar tasks. -See the file :ref:`Documentation/process/clang-format.rst <clangformat>` +See the file :ref:`Documentation/dev-tools/clang-format.rst <clangformat>` for more details. +Some basic editor settings, such as indentation and line endings, will be +set automatically if you are using an editor that is compatible with +EditorConfig. See the official EditorConfig website for more information: +https://editorconfig.org/ 10) Kconfig configuration files ------------------------------- @@ -823,6 +818,29 @@ Macros with multiple statements should be enclosed in a do - while block: do_this(b, c); \ } while (0) +Function-like macros with unused parameters should be replaced by static +inline functions to avoid the issue of unused variables: + +.. code-block:: c + + static inline void fun(struct foo *foo) + { + } + +Due to historical practices, many files still employ the "cast to (void)" +approach to evaluate parameters. However, this method is not advisable. +Inline functions address the issue of "expression with side effects +evaluated more than once", circumvent unused-variable problems, and +are generally better documented than macros for some reason. + +.. code-block:: c + + /* + * Avoid doing this whenever possible and instead opt for static + * inline functions + */ + #define macrofun(foo) do { (void) (foo); } while (0) + Things to avoid when using macros: 1) macros that affect control flow: @@ -895,7 +913,8 @@ which you should use to make sure messages are matched to the right device and driver, and are tagged with the right level: dev_err(), dev_warn(), dev_info(), and so forth. For messages that aren't associated with a particular device, <linux/printk.h> defines pr_notice(), pr_info(), -pr_warn(), pr_err(), etc. +pr_warn(), pr_err(), etc. When drivers are working properly they are quiet, +so prefer to use dev_dbg/pr_debug unless something is wrong. Coming up with good debugging messages can be quite a challenge; and once you have them, they can be a huge help for remote troubleshooting. However @@ -970,7 +989,7 @@ that can go into these 5 milliseconds. A reasonable rule of thumb is to not put inline at functions that have more than 3 lines of code in them. An exception to this rule are the cases where -a parameter is known to be a compiletime constant, and as a result of this +a parameter is known to be a compile time constant, and as a result of this constantness you *know* the compiler will be able to optimize most of your function away at compile time. For a good example of this later case, see the kmalloc() inline function. @@ -1267,5 +1286,5 @@ gcc internals and indent, all available from https://www.gnu.org/manual/ WG14 is the international standardization working group for the programming language C, URL: http://www.open-std.org/JTC1/SC22/WG14/ -Kernel :ref:`process/coding-style.rst <codingstyle>`, by greg@kroah.com at OLS 2002: +Kernel CodingStyle, by greg@kroah.com at OLS 2002: http://www.kroah.com/linux/talks/ols_2002_kernel_codingstyle_talk/html/ diff --git a/Documentation/process/contribution-maturity-model.rst b/Documentation/process/contribution-maturity-model.rst new file mode 100644 index 000000000000..b87ab34de22c --- /dev/null +++ b/Documentation/process/contribution-maturity-model.rst @@ -0,0 +1,109 @@ +.. SPDX-License-Identifier: GPL-2.0 + +======================================== +Linux Kernel Contribution Maturity Model +======================================== + + +Background +========== + +As a part of the 2021 Linux Kernel Maintainers’ Summit, there was a +`discussion <https://lwn.net/Articles/870581/>`_ about the challenges in +recruiting kernel maintainers as well as maintainer succession. Some of +the conclusions from that discussion included that companies which are a +part of the Linux Kernel community need to allow engineers to be +maintainers as part of their job, so they can grow into becoming +respected leaders and eventually, kernel maintainers. To support a +strong talent pipeline, developers should be allowed and encouraged to +take on upstream contributions such as reviewing other people’s patches, +refactoring kernel infrastructure, and writing documentation. + +To that end, the Linux Foundation Technical Advisory Board (TAB) +proposes this Linux Kernel Contribution Maturity Model. These common +expectations for upstream community engagement aim to increase the +influence of individual developers, increase the collaboration of +organizations, and improve the overall health of the Linux Kernel +ecosystem. + +The TAB urges organizations to continuously evaluate their Open Source +maturity model and commit to improvements to align with this model. To +be effective, this evaluation should incorporate feedback from across +the organization, including management and developers at all seniority +levels. In the spirit of Open Source, we encourage organizations to +publish their evaluations and plans to improve their engagement with the +upstream community. + +Level 0 +======= + +* Software Engineers are not allowed to contribute patches to the Linux + kernel. + + +Level 1 +======= + +* Software Engineers are allowed to contribute patches to the Linux + kernel, either as part of their job responsibilities or on their own + time. + +Level 2 +======= + +* Software Engineers are expected to contribute to the Linux Kernel as + part of their job responsibilities. +* Software Engineers will be supported to attend Linux-related + conferences as a part of their job. +* A Software Engineer’s upstream code contributions will be considered + in promotion and performance reviews. + +Level 3 +======= + +* Software Engineers are expected to review patches (including patches + authored by engineers from other companies) as part of their job + responsibilities +* Contributing presentations or papers to Linux-related or academic + conferences (such those organized by the Linux Foundation, Usenix, + ACM, etc.), are considered part of an engineer’s work. +* A Software Engineer’s community contributions will be considered in + promotion and performance reviews. +* Organizations will regularly report metrics of their open source + contributions and track these metrics over time. These metrics may be + published only internally within the organization, or at the + organization’s discretion, some or all may be published externally. + Metrics that are strongly suggested include: + + * The number of upstream kernel contributions by team or organization + (e.g., all people reporting up to a manager, director, or VP). + * The percentage of kernel developers who have made upstream + contributions relative to the total kernel developers in the + organization. + * The time interval between kernels used in the organization’s servers + and/or products, and the publication date of the upstream kernel + upon which the internal kernel is based. + * The number of out-of-tree commits present in internal kernels. + +Level 4 +======= + +* Software Engineers are encouraged to spend a portion of their work + time focused on Upstream Work, which is defined as reviewing patches, + serving on program committees, improving core project infrastructure + such as writing or maintaining tests, upstream tech debt reduction, + writing documentation, etc. +* Software Engineers are supported in helping to organize Linux-related + conferences. +* Organizations will consider community member feedback in official + performance reviews. + +Level 5 +======= + +* Upstream kernel development is considered a formal job position, with + at least a third of the engineer’s time spent doing Upstream Work. +* Organizations will actively seek out community member feedback as a + factor in official performance reviews. +* Organizations will regularly report internally on the ratio of + Upstream Work to work focused on directly pursuing business goals. diff --git a/Documentation/process/cve.rst b/Documentation/process/cve.rst new file mode 100644 index 000000000000..5e2753eff729 --- /dev/null +++ b/Documentation/process/cve.rst @@ -0,0 +1,121 @@ +==== +CVEs +==== + +Common Vulnerabilities and Exposure (CVE®) numbers were developed as an +unambiguous way to identify, define, and catalog publicly disclosed +security vulnerabilities. Over time, their usefulness has declined with +regards to the kernel project, and CVE numbers were very often assigned +in inappropriate ways and for inappropriate reasons. Because of this, +the kernel development community has tended to avoid them. However, the +combination of continuing pressure to assign CVEs and other forms of +security identifiers, and ongoing abuses by individuals and companies +outside of the kernel community has made it clear that the kernel +community should have control over those assignments. + +The Linux kernel developer team does have the ability to assign CVEs for +potential Linux kernel security issues. This assignment is independent +of the :doc:`normal Linux kernel security bug reporting +process<../process/security-bugs>`. + +A list of all assigned CVEs for the Linux kernel can be found in the +archives of the linux-cve mailing list, as seen on +https://lore.kernel.org/linux-cve-announce/. To get notice of the +assigned CVEs, please `subscribe +<https://subspace.kernel.org/subscribing.html>`_ to that mailing list. + +Process +======= + +As part of the normal stable release process, kernel changes that are +potentially security issues are identified by the developers responsible +for CVE number assignments and have CVE numbers automatically assigned +to them. These assignments are published on the linux-cve-announce +mailing list as announcements on a frequent basis. + +Note, due to the layer at which the Linux kernel is in a system, almost +any bug might be exploitable to compromise the security of the kernel, +but the possibility of exploitation is often not evident when the bug is +fixed. Because of this, the CVE assignment team is overly cautious and +assign CVE numbers to any bugfix that they identify. This +explains the seemingly large number of CVEs that are issued by the Linux +kernel team. + +If the CVE assignment team misses a specific fix that any user feels +should have a CVE assigned to it, please email them at <cve@kernel.org> +and the team there will work with you on it. Note that no potential +security issues should be sent to this alias, it is ONLY for assignment +of CVEs for fixes that are already in released kernel trees. If you +feel you have found an unfixed security issue, please follow the +:doc:`normal Linux kernel security bug reporting +process<../process/security-bugs>`. + +No CVEs will be automatically assigned for unfixed security issues in +the Linux kernel; assignment will only automatically happen after a fix +is available and applied to a stable kernel tree, and it will be tracked +that way by the git commit id of the original fix. If anyone wishes to +have a CVE assigned before an issue is resolved with a commit, please +contact the kernel CVE assignment team at <cve@kernel.org> to get an +identifier assigned from their batch of reserved identifiers. + +No CVEs will be assigned for any issue found in a version of the kernel +that is not currently being actively supported by the Stable/LTS kernel +team. A list of the currently supported kernel branches can be found at +https://kernel.org/releases.html + +Disputes of assigned CVEs +========================= + +The authority to dispute or modify an assigned CVE for a specific kernel +change lies solely with the maintainers of the relevant subsystem +affected. This principle ensures a high degree of accuracy and +accountability in vulnerability reporting. Only those individuals with +deep expertise and intimate knowledge of the subsystem can effectively +assess the validity and scope of a reported vulnerability and determine +its appropriate CVE designation. Any attempt to modify or dispute a CVE +outside of this designated authority could lead to confusion, inaccurate +reporting, and ultimately, compromised systems. + +Invalid CVEs +============ + +If a security issue is found in a Linux kernel that is only supported by +a Linux distribution due to the changes that have been made by that +distribution, or due to the distribution supporting a kernel version +that is no longer one of the kernel.org supported releases, then a CVE +can not be assigned by the Linux kernel CVE team, and must be asked for +from that Linux distribution itself. + +Any CVE that is assigned against the Linux kernel for an actively +supported kernel version, by any group other than the kernel assignment +CVE team should not be treated as a valid CVE. Please notify the +kernel CVE assignment team at <cve@kernel.org> so that they can work to +invalidate such entries through the CNA remediation process. + +Applicability of specific CVEs +============================== + +As the Linux kernel can be used in many different ways, with many +different ways of accessing it by external users, or no access at all, +the applicability of any specific CVE is up to the user of Linux to +determine, it is not up to the CVE assignment team. Please do not +contact us to attempt to determine the applicability of any specific +CVE. + +Also, as the source tree is so large, and any one system only uses a +small subset of the source tree, any users of Linux should be aware that +large numbers of assigned CVEs are not relevant for their systems. + +In short, we do not know your use case, and we do not know what portions +of the kernel that you use, so there is no way for us to determine if a +specific CVE is relevant for your system. + +As always, it is best to take all released kernel changes, as they are +tested together in a unified whole by many community members, and not as +individual cherry-picked changes. Also note that for many bugs, the +solution to the overall problem is not found in a single change, but by +the sum of many fixes on top of each other. Ideally CVEs will be +assigned to all fixes for all issues, but sometimes we will fail to +notice fixes, therefore assume that some changes without a CVE assigned +might be relevant to take. + diff --git a/Documentation/process/debugging/driver_development_debugging_guide.rst b/Documentation/process/debugging/driver_development_debugging_guide.rst new file mode 100644 index 000000000000..aca08f457793 --- /dev/null +++ b/Documentation/process/debugging/driver_development_debugging_guide.rst @@ -0,0 +1,235 @@ +.. SPDX-License-Identifier: GPL-2.0 + +======================================== +Debugging advice for driver development +======================================== + +This document serves as a general starting point and lookup for debugging +device drivers. +While this guide focuses on debugging that requires re-compiling the +module/kernel, the :doc:`userspace debugging guide +</process/debugging/userspace_debugging_guide>` will guide +you through tools like dynamic debug, ftrace and other tools useful for +debugging issues and behavior. +For general debugging advice, see the :doc:`general advice document +</process/debugging/index>`. + +.. contents:: + :depth: 3 + +The following sections show you the available tools. + +printk() & friends +------------------ + +These are derivatives of printf() with varying destinations and support for +being dynamically turned on or off, or lack thereof. + +Simple printk() +~~~~~~~~~~~~~~~ + +The classic, can be used to great effect for quick and dirty development +of new modules or to extract arbitrary necessary data for troubleshooting. + +Prerequisite: ``CONFIG_PRINTK`` (usually enabled by default) + +**Pros**: + +- No need to learn anything, simple to use +- Easy to modify exactly to your needs (formatting of the data (See: + :doc:`/core-api/printk-formats`), visibility in the log) +- Can cause delays in the execution of the code (beneficial to confirm whether + timing is a factor) + +**Cons**: + +- Requires rebuilding the kernel/module +- Can cause delays in the execution of the code (which can cause issues to be + not reproducible) + +For the full documentation see :doc:`/core-api/printk-basics` + +Trace_printk +~~~~~~~~~~~~ + +Prerequisite: ``CONFIG_DYNAMIC_FTRACE`` & ``#include <linux/ftrace.h>`` + +It is a tiny bit less comfortable to use than printk(), because you will have +to read the messages from the trace file (See: :ref:`read_ftrace_log` +instead of from the kernel log, but very useful when printk() adds unwanted +delays into the code execution, causing issues to be flaky or hidden.) + +If the processing of this still causes timing issues then you can try +trace_puts(). + +For the full Documentation see trace_printk() + +dev_dbg +~~~~~~~ + +Print statement, which can be targeted by +:ref:`process/debugging/userspace_debugging_guide:dynamic debug` that contains +additional information about the device used within the context. + +**When is it appropriate to leave a debug print in the code?** + +Permanent debug statements have to be useful for a developer to troubleshoot +driver misbehavior. Judging that is a bit more of an art than a science, but +some guidelines are in the :ref:`Coding style guidelines +<process/coding-style:13) printing kernel messages>`. In almost all cases the +debug statements shouldn't be upstreamed, as a working driver is supposed to be +silent. + +Custom printk +~~~~~~~~~~~~~ + +Example:: + + #define core_dbg(fmt, arg...) do { \ + if (core_debug) \ + printk(KERN_DEBUG pr_fmt("core: " fmt), ## arg); \ + } while (0) + +**When should you do this?** + +It is better to just use a pr_debug(), which can later be turned on/off with +dynamic debug. Additionally, a lot of drivers activate these prints via a +variable like ``core_debug`` set by a module parameter. However, Module +parameters `are not recommended anymore +<https://lore.kernel.org/all/2024032757-surcharge-grime-d3dd@gregkh>`_. + +Ftrace +------ + +Creating a custom Ftrace tracepoint +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +A tracepoint adds a hook into your code that will be called and logged when the +tracepoint is enabled. This can be used, for example, to trace hitting a +conditional branch or to dump the internal state at specific points of the code +flow during a debugging session. + +Here is a basic description of :ref:`how to implement new tracepoints +<trace/tracepoints:usage>`. + +For the full event tracing documentation see :doc:`/trace/events` + +For the full Ftrace documentation see :doc:`/trace/ftrace` + +DebugFS +------- + +Prerequisite: ``CONFIG_DEBUG_FS` & `#include <linux/debugfs.h>`` + +DebugFS differs from the other approaches of debugging, as it doesn't write +messages to the kernel log nor add traces to the code. Instead it allows the +developer to handle a set of files. +With these files you can either store values of variables or make +register/memory dumps or you can make these files writable and modify +values/settings in the driver. + +Possible use-cases among others: + +- Store register values +- Keep track of variables +- Store errors +- Store settings +- Toggle a setting like debug on/off +- Error injection + +This is especially useful, when the size of a data dump would be hard to digest +as part of the general kernel log (for example when dumping raw bitstream data) +or when you are not interested in all the values all the time, but with the +possibility to inspect them. + +The general idea is: + +- Create a directory during probe (``struct dentry *parent = + debugfs_create_dir("my_driver", NULL);``) +- Create a file (``debugfs_create_u32("my_value", 444, parent, &my_variable);``) + + - In this example the file is found in + ``/sys/kernel/debug/my_driver/my_value`` (with read permissions for + user/group/all) + - any read of the file will return the current contents of the variable + ``my_variable`` + +- Clean up the directory when removing the device + (``debugfs_remove(parent);``) + +For the full documentation see :doc:`/filesystems/debugfs`. + +KASAN, UBSAN, lockdep and other error checkers +---------------------------------------------- + +KASAN (Kernel Address Sanitizer) +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Prerequisite: ``CONFIG_KASAN`` + +KASAN is a dynamic memory error detector that helps to find use-after-free and +out-of-bounds bugs. It uses compile-time instrumentation to check every memory +access. + +For the full documentation see :doc:`/dev-tools/kasan`. + +UBSAN (Undefined Behavior Sanitizer) +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Prerequisite: ``CONFIG_UBSAN`` + +UBSAN relies on compiler instrumentation and runtime checks to detect undefined +behavior. It is designed to find a variety of issues, including signed integer +overflow, array index out of bounds, and more. + +For the full documentation see :doc:`/dev-tools/ubsan` + +lockdep (Lock Dependency Validator) +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Prerequisite: ``CONFIG_DEBUG_LOCKDEP`` + +lockdep is a runtime lock dependency validator that detects potential deadlocks +and other locking-related issues in the kernel. +It tracks lock acquisitions and releases, building a dependency graph that is +analyzed for potential deadlocks. +lockdep is especially useful for validating the correctness of lock ordering in +the kernel. + +PSI (Pressure stall information tracking) +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Prerequisite: ``CONFIG_PSI`` + +PSI is a measurement tool to identify excessive overcommits on hardware +resources, that can cause performance disruptions or even OOM kills. + +device coredump +--------------- + +Prerequisite: ``CONFIG_DEV_COREDUMP`` & ``#include <linux/devcoredump.h>`` + +Provides the infrastructure for a driver to provide arbitrary data to userland. +It is most often used in conjunction with udev or similar userland application +to listen for kernel uevents, which indicate that the dump is ready. Udev has +rules to copy that file somewhere for long-term storage and analysis, as by +default, the data for the dump is automatically cleaned up after a default +5 minutes. That data is analyzed with driver-specific tools or GDB. + +A device coredump can be created with a vmalloc area, with read/free +methods, or as a scatter/gather list. + +You can find an example implementation at: +`drivers/media/platform/qcom/venus/core.c +<https://elixir.bootlin.com/linux/v6.11.6/source/drivers/media/platform/qcom/venus/core.c#L30>`__, +in the Bluetooth HCI layer, in several wireless drivers, and in several +DRM drivers. + +devcoredump interfaces +~~~~~~~~~~~~~~~~~~~~~~ + +.. kernel-doc:: include/linux/devcoredump.h + +.. kernel-doc:: drivers/base/devcoredump.c + +**Copyright** ©2024 : Collabora diff --git a/Documentation/process/debugging/gdb-kernel-debugging.rst b/Documentation/process/debugging/gdb-kernel-debugging.rst new file mode 100644 index 000000000000..9475c759c722 --- /dev/null +++ b/Documentation/process/debugging/gdb-kernel-debugging.rst @@ -0,0 +1,175 @@ +.. highlight:: none + +Debugging kernel and modules via gdb +==================================== + +The kernel debugger kgdb, hypervisors like QEMU or JTAG-based hardware +interfaces allow to debug the Linux kernel and its modules during runtime +using gdb. Gdb comes with a powerful scripting interface for python. The +kernel provides a collection of helper scripts that can simplify typical +kernel debugging steps. This is a short tutorial about how to enable and use +them. It focuses on QEMU/KVM virtual machines as target, but the examples can +be transferred to the other gdb stubs as well. + + +Requirements +------------ + +- gdb 7.2+ (recommended: 7.4+) with python support enabled (typically true + for distributions) + + +Setup +----- + +- Create a virtual Linux machine for QEMU/KVM (see www.linux-kvm.org and + www.qemu.org for more details). For cross-development, + https://landley.net/aboriginal/bin keeps a pool of machine images and + toolchains that can be helpful to start from. + +- Build the kernel with CONFIG_GDB_SCRIPTS enabled, but leave + CONFIG_DEBUG_INFO_REDUCED off. If your architecture supports + CONFIG_FRAME_POINTER, keep it enabled. + +- Install that kernel on the guest, turn off KASLR if necessary by adding + "nokaslr" to the kernel command line. + Alternatively, QEMU allows to boot the kernel directly using -kernel, + -append, -initrd command line switches. This is generally only useful if + you do not depend on modules. See QEMU documentation for more details on + this mode. In this case, you should build the kernel with + CONFIG_RANDOMIZE_BASE disabled if the architecture supports KASLR. + +- Build the gdb scripts (required on kernels v5.1 and above):: + + make scripts_gdb + +- Enable the gdb stub of QEMU/KVM, either + + - at VM startup time by appending "-s" to the QEMU command line + + or + + - during runtime by issuing "gdbserver" from the QEMU monitor + console + +- cd /path/to/linux-build + +- Start gdb: gdb vmlinux + + Note: Some distros may restrict auto-loading of gdb scripts to known safe + directories. In case gdb reports to refuse loading vmlinux-gdb.py, add:: + + add-auto-load-safe-path /path/to/linux-build + + to ~/.gdbinit. See gdb help for more details. + +- Attach to the booted guest:: + + (gdb) target remote :1234 + + +Examples of using the Linux-provided gdb helpers +------------------------------------------------ + +- Load module (and main kernel) symbols:: + + (gdb) lx-symbols + loading vmlinux + scanning for modules in /home/user/linux/build + loading @0xffffffffa0020000: /home/user/linux/build/net/netfilter/xt_tcpudp.ko + loading @0xffffffffa0016000: /home/user/linux/build/net/netfilter/xt_pkttype.ko + loading @0xffffffffa0002000: /home/user/linux/build/net/netfilter/xt_limit.ko + loading @0xffffffffa00ca000: /home/user/linux/build/net/packet/af_packet.ko + loading @0xffffffffa003c000: /home/user/linux/build/fs/fuse/fuse.ko + ... + loading @0xffffffffa0000000: /home/user/linux/build/drivers/ata/ata_generic.ko + +- Set a breakpoint on some not yet loaded module function, e.g.:: + + (gdb) b btrfs_init_sysfs + Function "btrfs_init_sysfs" not defined. + Make breakpoint pending on future shared library load? (y or [n]) y + Breakpoint 1 (btrfs_init_sysfs) pending. + +- Continue the target:: + + (gdb) c + +- Load the module on the target and watch the symbols being loaded as well as + the breakpoint hit:: + + loading @0xffffffffa0034000: /home/user/linux/build/lib/libcrc32c.ko + loading @0xffffffffa0050000: /home/user/linux/build/lib/lzo/lzo_compress.ko + loading @0xffffffffa006e000: /home/user/linux/build/lib/zlib_deflate/zlib_deflate.ko + loading @0xffffffffa01b1000: /home/user/linux/build/fs/btrfs/btrfs.ko + + Breakpoint 1, btrfs_init_sysfs () at /home/user/linux/fs/btrfs/sysfs.c:36 + 36 btrfs_kset = kset_create_and_add("btrfs", NULL, fs_kobj); + +- Dump the log buffer of the target kernel:: + + (gdb) lx-dmesg + [ 0.000000] Initializing cgroup subsys cpuset + [ 0.000000] Initializing cgroup subsys cpu + [ 0.000000] Linux version 3.8.0-rc4-dbg+ (... + [ 0.000000] Command line: root=/dev/sda2 resume=/dev/sda1 vga=0x314 + [ 0.000000] e820: BIOS-provided physical RAM map: + [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable + [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved + .... + +- Examine fields of the current task struct(supported by x86 and arm64 only):: + + (gdb) p $lx_current().pid + $1 = 4998 + (gdb) p $lx_current().comm + $2 = "modprobe\000\000\000\000\000\000\000" + +- Make use of the per-cpu function for the current or a specified CPU:: + + (gdb) p $lx_per_cpu(runqueues).nr_running + $3 = 1 + (gdb) p $lx_per_cpu(runqueues, 2).nr_running + $4 = 0 + +- Dig into hrtimers using the container_of helper:: + + (gdb) set $leftmost = $lx_per_cpu(hrtimer_bases).clock_base[0].active.rb_root.rb_leftmost + (gdb) p *$container_of($leftmost, "struct hrtimer", "node") + $5 = { + node = { + node = { + __rb_parent_color = 18446612686384860673, + rb_right = 0xffff888231da8b00, + rb_left = 0x0 + }, + expires = 1228461000000 + }, + _softexpires = 1228461000000, + function = 0xffffffff8137ab20 <tick_nohz_handler>, + base = 0xffff888231d9b4c0, + state = 1 '\001', + is_rel = 0 '\000', + is_soft = 0 '\000', + is_hard = 1 '\001' + } + + +List of commands and functions +------------------------------ + +The number of commands and convenience functions may evolve over the time, +this is just a snapshot of the initial version:: + + (gdb) apropos lx + function lx_current -- Return current task + function lx_module -- Find module by name and return the module variable + function lx_per_cpu -- Return per-cpu variable + function lx_task_by_pid -- Find Linux task by PID and return the task_struct variable + function lx_thread_info -- Calculate Linux thread_info from task variable + lx-dmesg -- Print Linux kernel log buffer + lx-lsmod -- List currently loaded modules + lx-symbols -- (Re-)load symbols of Linux kernel and currently loaded modules + +Detailed help can be obtained via "help <command-name>" for commands and "help +function <function-name>" for convenience functions. diff --git a/Documentation/process/debugging/index.rst b/Documentation/process/debugging/index.rst new file mode 100644 index 000000000000..387d33d16f5e --- /dev/null +++ b/Documentation/process/debugging/index.rst @@ -0,0 +1,80 @@ +.. SPDX-License-Identifier: GPL-2.0 + +============================================ +Debugging advice for Linux Kernel developers +============================================ + +general guides +-------------- + +.. toctree:: + :maxdepth: 1 + + driver_development_debugging_guide + gdb-kernel-debugging + kgdb + userspace_debugging_guide + +.. only:: subproject and html + +subsystem specific guides +------------------------- + +.. toctree:: + :maxdepth: 1 + + media_specific_debugging_guide + +.. only:: subproject and html + + Indices + ======= + + * :ref:`genindex` + +General debugging advice +======================== + +Depending on the issue, a different set of tools is available to track down the +problem or even to realize whether there is one in the first place. + +As a first step you have to figure out what kind of issue you want to debug. +Depending on the answer, your methodology and choice of tools may vary. + +Do I need to debug with limited access? +--------------------------------------- + +Do you have limited access to the machine or are you unable to stop the running +execution? + +In this case your debugging capability depends on built-in debugging support of +provided distribution kernel. +The :doc:`/process/debugging/userspace_debugging_guide` provides a brief +overview over a range of possible debugging tools in that situation. You can +check the capability of your kernel, in most cases, by looking into config file +within the /boot directory. + +Do I have root access to the system? +------------------------------------ + +Are you easily able to replace the module in question or to install a new +kernel? + +In that case your range of available tools is a lot bigger, you can find the +tools in the :doc:`/process/debugging/driver_development_debugging_guide`. + +Is timing a factor? +------------------- + +It is important to understand if the problem you want to debug manifests itself +consistently (i.e. given a set of inputs you always get the same, incorrect +output), or inconsistently. If it manifests itself inconsistently, some timing +factor might be at play. If inserting delays into the code does change the +behavior, then quite likely timing is a factor. + +When timing does alter the outcome of the code execution using a simple +printk() for debugging purposes may not work, a similar alternative is to use +trace_printk() , which logs the debug messages to the trace file instead of the +kernel log. + +**Copyright** ©2024 : Collabora diff --git a/Documentation/process/debugging/kgdb.rst b/Documentation/process/debugging/kgdb.rst new file mode 100644 index 000000000000..b29b0aac2717 --- /dev/null +++ b/Documentation/process/debugging/kgdb.rst @@ -0,0 +1,937 @@ +================================================= +Using kgdb, kdb and the kernel debugger internals +================================================= + +:Author: Jason Wessel + +Introduction +============ + +The kernel has two different debugger front ends (kdb and kgdb) which +interface to the debug core. It is possible to use either of the +debugger front ends and dynamically transition between them if you +configure the kernel properly at compile and runtime. + +Kdb is simplistic shell-style interface which you can use on a system +console with a keyboard or serial console. You can use it to inspect +memory, registers, process lists, dmesg, and even set breakpoints to +stop in a certain location. Kdb is not a source level debugger, although +you can set breakpoints and execute some basic kernel run control. Kdb +is mainly aimed at doing some analysis to aid in development or +diagnosing kernel problems. You can access some symbols by name in +kernel built-ins or in kernel modules if the code was built with +``CONFIG_KALLSYMS``. + +Kgdb is intended to be used as a source level debugger for the Linux +kernel. It is used along with gdb to debug a Linux kernel. The +expectation is that gdb can be used to "break in" to the kernel to +inspect memory, variables and look through call stack information +similar to the way an application developer would use gdb to debug an +application. It is possible to place breakpoints in kernel code and +perform some limited execution stepping. + +Two machines are required for using kgdb. One of these machines is a +development machine and the other is the target machine. The kernel to +be debugged runs on the target machine. The development machine runs an +instance of gdb against the vmlinux file which contains the symbols (not +a boot image such as bzImage, zImage, uImage...). In gdb the developer +specifies the connection parameters and connects to kgdb. The type of +connection a developer makes with gdb depends on the availability of +kgdb I/O modules compiled as built-ins or loadable kernel modules in the +test machine's kernel. + +Compiling a kernel +================== + +- In order to enable compilation of kdb, you must first enable kgdb. + +- The kgdb test compile options are described in the kgdb test suite + chapter. + +Kernel config options for kgdb +------------------------------ + +To enable ``CONFIG_KGDB`` you should look under +:menuselection:`Kernel hacking --> Kernel debugging` and select +:menuselection:`KGDB: kernel debugger`. + +While it is not a hard requirement that you have symbols in your vmlinux +file, gdb tends not to be very useful without the symbolic data, so you +will want to turn on ``CONFIG_DEBUG_INFO`` which is called +:menuselection:`Compile the kernel with debug info` in the config menu. + +It is advised, but not required, that you turn on the +``CONFIG_FRAME_POINTER`` kernel option which is called :menuselection:`Compile +the kernel with frame pointers` in the config menu. This option inserts code +into the compiled executable which saves the frame information in registers +or on the stack at different points which allows a debugger such as gdb to +more accurately construct stack back traces while debugging the kernel. + +If the architecture that you are using supports the kernel option +``CONFIG_STRICT_KERNEL_RWX``, you should consider turning it off. This +option will prevent the use of software breakpoints because it marks +certain regions of the kernel's memory space as read-only. If kgdb +supports it for the architecture you are using, you can use hardware +breakpoints if you desire to run with the ``CONFIG_STRICT_KERNEL_RWX`` +option turned on, else you need to turn off this option. + +Next you should choose one or more I/O drivers to interconnect the debugging +host and debugged target. Early boot debugging requires a KGDB I/O +driver that supports early debugging and the driver must be built into +the kernel directly. Kgdb I/O driver configuration takes place via +kernel or module parameters which you can learn more about in the +section that describes the parameter kgdboc. + +Here is an example set of ``.config`` symbols to enable or disable for kgdb:: + + # CONFIG_STRICT_KERNEL_RWX is not set + CONFIG_FRAME_POINTER=y + CONFIG_KGDB=y + CONFIG_KGDB_SERIAL_CONSOLE=y + +Kernel config options for kdb +----------------------------- + +Kdb is quite a bit more complex than the simple gdbstub sitting on top +of the kernel's debug core. Kdb must implement a shell, and also adds +some helper functions in other parts of the kernel, responsible for +printing out interesting data such as what you would see if you ran +``lsmod``, or ``ps``. In order to build kdb into the kernel you follow the +same steps as you would for kgdb. + +The main config option for kdb is ``CONFIG_KGDB_KDB`` which is called +:menuselection:`KGDB_KDB: include kdb frontend for kgdb` in the config menu. +In theory you would have already also selected an I/O driver such as the +``CONFIG_KGDB_SERIAL_CONSOLE`` interface if you plan on using kdb on a +serial port, when you were configuring kgdb. + +If you want to use a PS/2-style keyboard with kdb, you would select +``CONFIG_KDB_KEYBOARD`` which is called :menuselection:`KGDB_KDB: keyboard as +input device` in the config menu. The ``CONFIG_KDB_KEYBOARD`` option is not +used for anything in the gdb interface to kgdb. The ``CONFIG_KDB_KEYBOARD`` +option only works with kdb. + +Here is an example set of ``.config`` symbols to enable/disable kdb:: + + # CONFIG_STRICT_KERNEL_RWX is not set + CONFIG_FRAME_POINTER=y + CONFIG_KGDB=y + CONFIG_KGDB_SERIAL_CONSOLE=y + CONFIG_KGDB_KDB=y + CONFIG_KDB_KEYBOARD=y + +Kernel Debugger Boot Arguments +============================== + +This section describes the various runtime kernel parameters that affect +the configuration of the kernel debugger. The following chapter covers +using kdb and kgdb as well as providing some examples of the +configuration parameters. + +Kernel parameter: kgdboc +------------------------ + +The kgdboc driver was originally an abbreviation meant to stand for +"kgdb over console". Today it is the primary mechanism to configure how +to communicate from gdb to kgdb as well as the devices you want to use +to interact with the kdb shell. + +For kgdb/gdb, kgdboc is designed to work with a single serial port. It +is intended to cover the circumstance where you want to use a serial +console as your primary console as well as using it to perform kernel +debugging. It is also possible to use kgdb on a serial port which is not +designated as a system console. Kgdboc may be configured as a kernel +built-in or a kernel loadable module. You can only make use of +``kgdbwait`` and early debugging if you build kgdboc into the kernel as +a built-in. + +Optionally you can elect to activate kms (Kernel Mode Setting) +integration. When you use kms with kgdboc and you have a video driver +that has atomic mode setting hooks, it is possible to enter the debugger +on the graphics console. When the kernel execution is resumed, the +previous graphics mode will be restored. This integration can serve as a +useful tool to aid in diagnosing crashes or doing analysis of memory +with kdb while allowing the full graphics console applications to run. + +kgdboc arguments +~~~~~~~~~~~~~~~~ + +Usage:: + + kgdboc=[kms][[,]kbd][[,]serial_device][,baud] + +The order listed above must be observed if you use any of the optional +configurations together. + +Abbreviations: + +- kms = Kernel Mode Setting + +- kbd = Keyboard + +You can configure kgdboc to use the keyboard, and/or a serial device +depending on if you are using kdb and/or kgdb, in one of the following +scenarios. The order listed above must be observed if you use any of the +optional configurations together. Using kms + only gdb is generally not +a useful combination. + +Using loadable module or built-in +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +1. As a kernel built-in: + + Use the kernel boot argument:: + + kgdboc=<tty-device>,[baud] + +2. As a kernel loadable module: + + Use the command:: + + modprobe kgdboc kgdboc=<tty-device>,[baud] + + Here are two examples of how you might format the kgdboc string. The + first is for an x86 target using the first serial port. The second + example is for the ARM Versatile AB using the second serial port. + + 1. ``kgdboc=ttyS0,115200`` + + 2. ``kgdboc=ttyAMA1,115200`` + +Configure kgdboc at runtime with sysfs +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +At run time you can enable or disable kgdboc by writing parameters +into sysfs. Here are two examples: + +1. Enable kgdboc on ttyS0:: + + echo ttyS0 > /sys/module/kgdboc/parameters/kgdboc + +2. Disable kgdboc:: + + echo "" > /sys/module/kgdboc/parameters/kgdboc + +.. note:: + + You do not need to specify the baud if you are configuring the + console on tty which is already configured or open. + +More examples +^^^^^^^^^^^^^ + +You can configure kgdboc to use the keyboard, and/or a serial device +depending on if you are using kdb and/or kgdb, in one of the following +scenarios. + +1. kdb and kgdb over only a serial port:: + + kgdboc=<serial_device>[,baud] + + Example:: + + kgdboc=ttyS0,115200 + +2. kdb and kgdb with keyboard and a serial port:: + + kgdboc=kbd,<serial_device>[,baud] + + Example:: + + kgdboc=kbd,ttyS0,115200 + +3. kdb with a keyboard:: + + kgdboc=kbd + +4. kdb with kernel mode setting:: + + kgdboc=kms,kbd + +5. kdb with kernel mode setting and kgdb over a serial port:: + + kgdboc=kms,kbd,ttyS0,115200 + +.. note:: + + Kgdboc does not support interrupting the target via the gdb remote + protocol. You must manually send a `SysRq-G` unless you have a proxy + that splits console output to a terminal program. A console proxy has a + separate TCP port for the debugger and a separate TCP port for the + "human" console. The proxy can take care of sending the `SysRq-G` + for you. + +When using kgdboc with no debugger proxy, you can end up connecting the +debugger at one of two entry points. If an exception occurs after you +have loaded kgdboc, a message should print on the console stating it is +waiting for the debugger. In this case you disconnect your terminal +program and then connect the debugger in its place. If you want to +interrupt the target system and forcibly enter a debug session you have +to issue a `Sysrq` sequence and then type the letter `g`. Then you +disconnect the terminal session and connect gdb. Your options if you +don't like this are to hack gdb to send the `SysRq-G` for you as well as +on the initial connect, or to use a debugger proxy that allows an +unmodified gdb to do the debugging. + +Kernel parameter: ``kgdboc_earlycon`` +------------------------------------- + +If you specify the kernel parameter ``kgdboc_earlycon`` and your serial +driver registers a boot console that supports polling (doesn't need +interrupts and implements a nonblocking read() function) kgdb will attempt +to work using the boot console until it can transition to the regular +tty driver specified by the ``kgdboc`` parameter. + +Normally there is only one boot console (especially that implements the +read() function) so just adding ``kgdboc_earlycon`` on its own is +sufficient to make this work. If you have more than one boot console you +can add the boot console's name to differentiate. Note that names that +are registered through the boot console layer and the tty layer are not +the same for the same port. + +For instance, on one board to be explicit you might do:: + + kgdboc_earlycon=qcom_geni kgdboc=ttyMSM0 + +If the only boot console on the device was "qcom_geni", you could simplify:: + + kgdboc_earlycon kgdboc=ttyMSM0 + +Kernel parameter: ``kgdbwait`` +------------------------------ + +The Kernel command line option ``kgdbwait`` makes kgdb wait for a +debugger connection during booting of a kernel. You can only use this +option if you compiled a kgdb I/O driver into the kernel and you +specified the I/O driver configuration as a kernel command line option. +The kgdbwait parameter should always follow the configuration parameter +for the kgdb I/O driver in the kernel command line else the I/O driver +will not be configured prior to asking the kernel to use it to wait. + +The kernel will stop and wait as early as the I/O driver and +architecture allows when you use this option. If you build the kgdb I/O +driver as a loadable kernel module kgdbwait will not do anything. + +Kernel parameter: ``kgdbcon`` +----------------------------- + +The ``kgdbcon`` feature allows you to see printk() messages inside gdb +while gdb is connected to the kernel. Kdb does not make use of the kgdbcon +feature. + +Kgdb supports using the gdb serial protocol to send console messages to +the debugger when the debugger is connected and running. There are two +ways to activate this feature. + +1. Activate with the kernel command line option:: + + kgdbcon + +2. Use sysfs before configuring an I/O driver:: + + echo 1 > /sys/module/debug_core/parameters/kgdb_use_con + +.. note:: + + If you do this after you configure the kgdb I/O driver, the + setting will not take effect until the next point the I/O is + reconfigured. + +.. important:: + + You cannot use kgdboc + kgdbcon on a tty that is an + active system console. An example of incorrect usage is:: + + console=ttyS0,115200 kgdboc=ttyS0 kgdbcon + +It is possible to use this option with kgdboc on a tty that is not a +system console. + +Run time parameter: ``kgdbreboot`` +---------------------------------- + +The kgdbreboot feature allows you to change how the debugger deals with +the reboot notification. You have 3 choices for the behavior. The +default behavior is always set to 0. + +.. tabularcolumns:: |p{0.4cm}|p{11.5cm}|p{5.6cm}| + +.. flat-table:: + :widths: 1 10 8 + + * - 1 + - ``echo -1 > /sys/module/debug_core/parameters/kgdbreboot`` + - Ignore the reboot notification entirely. + + * - 2 + - ``echo 0 > /sys/module/debug_core/parameters/kgdbreboot`` + - Send the detach message to any attached debugger client. + + * - 3 + - ``echo 1 > /sys/module/debug_core/parameters/kgdbreboot`` + - Enter the debugger on reboot notify. + +Kernel parameter: ``nokaslr`` +----------------------------- + +If the architecture that you are using enables KASLR by default, +you should consider turning it off. KASLR randomizes the +virtual address where the kernel image is mapped and confuses +gdb which resolves addresses of kernel symbols from the symbol table +of vmlinux. + +Using kdb +========= + +Quick start for kdb on a serial port +------------------------------------ + +This is a quick example of how to use kdb. + +1. Configure kgdboc at boot using kernel parameters:: + + console=ttyS0,115200 kgdboc=ttyS0,115200 nokaslr + + OR + + Configure kgdboc after the kernel has booted; assuming you are using + a serial port console:: + + echo ttyS0 > /sys/module/kgdboc/parameters/kgdboc + +2. Enter the kernel debugger manually or by waiting for an oops or + fault. There are several ways you can enter the kernel debugger + manually; all involve using the `SysRq-G`, which means you must have + enabled ``CONFIG_MAGIC_SYSRQ=y`` in your kernel config. + + - When logged in as root or with a super user session you can run:: + + echo g > /proc/sysrq-trigger + + - Example using minicom 2.2 + + Press: `CTRL-A` `f` `g` + + - When you have telneted to a terminal server that supports sending + a remote break + + Press: `CTRL-]` + + Type in: ``send break`` + + Press: `Enter` `g` + +3. From the kdb prompt you can run the ``help`` command to see a complete + list of the commands that are available. + + Some useful commands in kdb include: + + =========== ================================================================= + ``lsmod`` Shows where kernel modules are loaded + ``ps`` Displays only the active processes + ``ps A`` Shows all the processes + ``summary`` Shows kernel version info and memory usage + ``bt`` Get a backtrace of the current process using dump_stack() + ``dmesg`` View the kernel syslog buffer + ``go`` Continue the system + =========== ================================================================= + +4. When you are done using kdb you need to consider rebooting the system + or using the ``go`` command to resuming normal kernel execution. If you + have paused the kernel for a lengthy period of time, applications + that rely on timely networking or anything to do with real wall clock + time could be adversely affected, so you should take this into + consideration when using the kernel debugger. + +Quick start for kdb using a keyboard connected console +------------------------------------------------------ + +This is a quick example of how to use kdb with a keyboard. + +1. Configure kgdboc at boot using kernel parameters:: + + kgdboc=kbd + + OR + + Configure kgdboc after the kernel has booted:: + + echo kbd > /sys/module/kgdboc/parameters/kgdboc + +2. Enter the kernel debugger manually or by waiting for an oops or + fault. There are several ways you can enter the kernel debugger + manually; all involve using the `SysRq-G`, which means you must have + enabled ``CONFIG_MAGIC_SYSRQ=y`` in your kernel config. + + - When logged in as root or with a super user session you can run:: + + echo g > /proc/sysrq-trigger + + - Example using a laptop keyboard: + + Press and hold down: `Alt` + + Press and hold down: `Fn` + + Press and release the key with the label: `SysRq` + + Release: `Fn` + + Press and release: `g` + + Release: `Alt` + + - Example using a PS/2 101-key keyboard + + Press and hold down: `Alt` + + Press and release the key with the label: `SysRq` + + Press and release: `g` + + Release: `Alt` + +3. Now type in a kdb command such as ``help``, ``dmesg``, ``bt`` or ``go`` to + continue kernel execution. + +Using kgdb / gdb +================ + +In order to use kgdb you must activate it by passing configuration +information to one of the kgdb I/O drivers. If you do not pass any +configuration information kgdb will not do anything at all. Kgdb will +only actively hook up to the kernel trap hooks if a kgdb I/O driver is +loaded and configured. If you unconfigure a kgdb I/O driver, kgdb will +unregister all the kernel hook points. + +All kgdb I/O drivers can be reconfigured at run time, if +``CONFIG_SYSFS`` and ``CONFIG_MODULES`` are enabled, by echo'ing a new +config string to ``/sys/module/<driver>/parameter/<option>``. The driver +can be unconfigured by passing an empty string. You cannot change the +configuration while the debugger is attached. Make sure to detach the +debugger with the ``detach`` command prior to trying to unconfigure a +kgdb I/O driver. + +Connecting with gdb to a serial port +------------------------------------ + +1. Configure kgdboc + + Configure kgdboc at boot using kernel parameters:: + + kgdboc=ttyS0,115200 + + OR + + Configure kgdboc after the kernel has booted:: + + echo ttyS0 > /sys/module/kgdboc/parameters/kgdboc + +2. Stop kernel execution (break into the debugger) + + In order to connect to gdb via kgdboc, the kernel must first be + stopped. There are several ways to stop the kernel which include + using kgdbwait as a boot argument, via a `SysRq-G`, or running the + kernel until it takes an exception where it waits for the debugger to + attach. + + - When logged in as root or with a super user session you can run:: + + echo g > /proc/sysrq-trigger + + - Example using minicom 2.2 + + Press: `CTRL-A` `f` `g` + + - When you have telneted to a terminal server that supports sending + a remote break + + Press: `CTRL-]` + + Type in: ``send break`` + + Press: `Enter` `g` + +3. Connect from gdb + + Example (using a directly connected port):: + + % gdb ./vmlinux + (gdb) set serial baud 115200 + (gdb) target remote /dev/ttyS0 + + + Example (kgdb to a terminal server on TCP port 2012):: + + % gdb ./vmlinux + (gdb) target remote 192.168.2.2:2012 + + + Once connected, you can debug a kernel the way you would debug an + application program. + + If you are having problems connecting or something is going seriously + wrong while debugging, it will most often be the case that you want + to enable gdb to be verbose about its target communications. You do + this prior to issuing the ``target remote`` command by typing in:: + + set debug remote 1 + +Remember if you continue in gdb, and need to "break in" again, you need +to issue an other `SysRq-G`. It is easy to create a simple entry point by +putting a breakpoint at ``sys_sync`` and then you can run ``sync`` from a +shell or script to break into the debugger. + +kgdb and kdb interoperability +============================= + +It is possible to transition between kdb and kgdb dynamically. The debug +core will remember which you used the last time and automatically start +in the same mode. + +Switching between kdb and kgdb +------------------------------ + +Switching from kgdb to kdb +~~~~~~~~~~~~~~~~~~~~~~~~~~ + +There are two ways to switch from kgdb to kdb: you can use gdb to issue +a maintenance packet, or you can blindly type the command ``$3#33``. +Whenever the kernel debugger stops in kgdb mode it will print the +message ``KGDB or $3#33 for KDB``. It is important to note that you have +to type the sequence correctly in one pass. You cannot type a backspace +or delete because kgdb will interpret that as part of the debug stream. + +1. Change from kgdb to kdb by blindly typing:: + + $3#33 + +2. Change from kgdb to kdb with gdb:: + + maintenance packet 3 + + .. note:: + + Now you must kill gdb. Typically you press `CTRL-Z` and issue + the command:: + + kill -9 % + +Change from kdb to kgdb +~~~~~~~~~~~~~~~~~~~~~~~ + +There are two ways you can change from kdb to kgdb. You can manually +enter kgdb mode by issuing the kgdb command from the kdb shell prompt, +or you can connect gdb while the kdb shell prompt is active. The kdb +shell looks for the typical first commands that gdb would issue with the +gdb remote protocol and if it sees one of those commands it +automatically changes into kgdb mode. + +1. From kdb issue the command:: + + kgdb + +2. At the kdb prompt, disconnect the terminal program and connect gdb in + its place. + +Running kdb commands from gdb +----------------------------- + +It is possible to run a limited set of kdb commands from gdb, using the +gdb monitor command. You don't want to execute any of the run control or +breakpoint operations, because it can disrupt the state of the kernel +debugger. You should be using gdb for breakpoints and run control +operations if you have gdb connected. The more useful commands to run +are things like lsmod, dmesg, ps or possibly some of the memory +information commands. To see all the kdb commands you can run +``monitor help``. + +Example:: + + (gdb) monitor ps + 1 idle process (state I) and + 27 sleeping system daemon (state M) processes suppressed, + use 'ps A' to see all. + Task Addr Pid Parent [*] cpu State Thread Command + + 0xc78291d0 1 0 0 0 S 0xc7829404 init + 0xc7954150 942 1 0 0 S 0xc7954384 dropbear + 0xc78789c0 944 1 0 0 S 0xc7878bf4 sh + (gdb) + +kgdb Test Suite +=============== + +When kgdb is enabled in the kernel config you can also elect to enable +the config parameter ``KGDB_TESTS``. Turning this on will enable a special +kgdb I/O module which is designed to test the kgdb internal functions. + +The kgdb tests are mainly intended for developers to test the kgdb +internals as well as a tool for developing a new kgdb architecture +specific implementation. These tests are not really for end users of the +Linux kernel. The primary source of documentation would be to look in +the ``drivers/misc/kgdbts.c`` file. + +The kgdb test suite can also be configured at compile time to run the +core set of tests by setting the kernel config parameter +``KGDB_TESTS_ON_BOOT``. This particular option is aimed at automated +regression testing and does not require modifying the kernel boot config +arguments. If this is turned on, the kgdb test suite can be disabled by +specifying ``kgdbts=`` as a kernel boot argument. + +Kernel Debugger Internals +========================= + +Architecture Specifics +---------------------- + +The kernel debugger is organized into a number of components: + +1. The debug core + + The debug core is found in ``kernel/debugger/debug_core.c``. It + contains: + + - A generic OS exception handler which includes sync'ing the + processors into a stopped state on an multi-CPU system. + + - The API to talk to the kgdb I/O drivers + + - The API to make calls to the arch-specific kgdb implementation + + - The logic to perform safe memory reads and writes to memory while + using the debugger + + - A full implementation for software breakpoints unless overridden + by the arch + + - The API to invoke either the kdb or kgdb frontend to the debug + core. + + - The structures and callback API for atomic kernel mode setting. + + .. note:: kgdboc is where the kms callbacks are invoked. + +2. kgdb arch-specific implementation + + This implementation is generally found in ``arch/*/kernel/kgdb.c``. As + an example, ``arch/x86/kernel/kgdb.c`` contains the specifics to + implement HW breakpoint as well as the initialization to dynamically + register and unregister for the trap handlers on this architecture. + The arch-specific portion implements: + + - contains an arch-specific trap catcher which invokes + kgdb_handle_exception() to start kgdb about doing its work + + - translation to and from gdb specific packet format to struct pt_regs + + - Registration and unregistration of architecture specific trap + hooks + + - Any special exception handling and cleanup + + - NMI exception handling and cleanup + + - (optional) HW breakpoints + +3. gdbstub frontend (aka kgdb) + + The gdbstub is located in ``kernel/debug/gdbstub.c``. It contains: + + - All the logic to implement the gdb serial protocol + +4. kdb frontend + + The kdb debugger shell is broken down into a number of components. + The kdb core is located in kernel/debug/kdb. There are a number of + helper functions in some of the other kernel components to make it + possible for kdb to examine and report information about the kernel + without taking locks that could cause a kernel deadlock. The kdb core + implements the following functionality. + + - A simple shell + + - The kdb core command set + + - A registration API to register additional kdb shell commands. + + - A good example of a self-contained kdb module is the ``ftdump`` + command for dumping the ftrace buffer. See: + ``kernel/trace/trace_kdb.c`` + + - For an example of how to dynamically register a new kdb command + you can build the kdb_hello.ko kernel module from + ``samples/kdb/kdb_hello.c``. To build this example you can set + ``CONFIG_SAMPLES=y`` and ``CONFIG_SAMPLE_KDB=m`` in your kernel + config. Later run ``modprobe kdb_hello`` and the next time you + enter the kdb shell, you can run the ``hello`` command. + + - The implementation for kdb_printf() which emits messages directly + to I/O drivers, bypassing the kernel log. + + - SW / HW breakpoint management for the kdb shell + +5. kgdb I/O driver + + Each kgdb I/O driver has to provide an implementation for the + following: + + - configuration via built-in or module + + - dynamic configuration and kgdb hook registration calls + + - read and write character interface + + - A cleanup handler for unconfiguring from the kgdb core + + - (optional) Early debug methodology + + Any given kgdb I/O driver has to operate very closely with the + hardware and must do it in such a way that does not enable interrupts + or change other parts of the system context without completely + restoring them. The kgdb core will repeatedly "poll" a kgdb I/O + driver for characters when it needs input. The I/O driver is expected + to return immediately if there is no data available. Doing so allows + for the future possibility to touch watchdog hardware in such a way + as to have a target system not reset when these are enabled. + +If you are intent on adding kgdb architecture specific support for a new +architecture, the architecture should define ``HAVE_ARCH_KGDB`` in the +architecture specific Kconfig file. This will enable kgdb for the +architecture, and at that point you must create an architecture specific +kgdb implementation. + +There are a few flags which must be set on every architecture in their +``asm/kgdb.h`` file. These are: + +- ``NUMREGBYTES``: + The size in bytes of all of the registers, so that we + can ensure they will all fit into a packet. + +- ``BUFMAX``: + The size in bytes of the buffer GDB will read into. This must + be larger than NUMREGBYTES. + +- ``CACHE_FLUSH_IS_SAFE``: + Set to 1 if it is always safe to call + flush_cache_range or flush_icache_range. On some architectures, + these functions may not be safe to call on SMP since we keep other + CPUs in a holding pattern. + +There are also the following functions for the common backend, found in +``kernel/kgdb.c``, that must be supplied by the architecture-specific +backend unless marked as (optional), in which case a default function +maybe used if the architecture does not need to provide a specific +implementation. + +.. kernel-doc:: include/linux/kgdb.h + :internal: + +kgdboc internals +---------------- + +kgdboc and uarts +~~~~~~~~~~~~~~~~ + +The kgdboc driver is actually a very thin driver that relies on the +underlying low level to the hardware driver having "polling hooks" to +which the tty driver is attached. In the initial implementation of +kgdboc the serial_core was changed to expose a low level UART hook for +doing polled mode reading and writing of a single character while in an +atomic context. When kgdb makes an I/O request to the debugger, kgdboc +invokes a callback in the serial core which in turn uses the callback in +the UART driver. + +When using kgdboc with a UART, the UART driver must implement two +callbacks in the struct uart_ops. +Example from ``drivers/8250.c``:: + + + #ifdef CONFIG_CONSOLE_POLL + .poll_get_char = serial8250_get_poll_char, + .poll_put_char = serial8250_put_poll_char, + #endif + + +Any implementation specifics around creating a polling driver use the +``#ifdef CONFIG_CONSOLE_POLL``, as shown above. Keep in mind that +polling hooks have to be implemented in such a way that they can be +called from an atomic context and have to restore the state of the UART +chip on return such that the system can return to normal when the +debugger detaches. You need to be very careful with any kind of lock you +consider, because failing here is most likely going to mean pressing the +reset button. + +kgdboc and keyboards +~~~~~~~~~~~~~~~~~~~~~~~~ + +The kgdboc driver contains logic to configure communications with an +attached keyboard. The keyboard infrastructure is only compiled into the +kernel when ``CONFIG_KDB_KEYBOARD=y`` is set in the kernel configuration. + +The core polled keyboard driver for PS/2 type keyboards is in +``drivers/char/kdb_keyboard.c``. This driver is hooked into the debug core +when kgdboc populates the callback in the array called +:c:expr:`kdb_poll_funcs[]`. The kdb_get_kbd_char() is the top-level +function which polls hardware for single character input. + +kgdboc and kms +~~~~~~~~~~~~~~~~~~ + +The kgdboc driver contains logic to request the graphics display to +switch to a text context when you are using ``kgdboc=kms,kbd``, provided +that you have a video driver which has a frame buffer console and atomic +kernel mode setting support. + +Every time the kernel debugger is entered it calls +kgdboc_pre_exp_handler() which in turn calls con_debug_enter() +in the virtual console layer. On resuming kernel execution, the kernel +debugger calls kgdboc_post_exp_handler() which in turn calls +con_debug_leave(). + +Any video driver that wants to be compatible with the kernel debugger +and the atomic kms callbacks must implement the ``mode_set_base_atomic``, +``fb_debug_enter`` and ``fb_debug_leave operations``. For the +``fb_debug_enter`` and ``fb_debug_leave`` the option exists to use the +generic drm fb helper functions or implement something custom for the +hardware. The following example shows the initialization of the +.mode_set_base_atomic operation in +drivers/gpu/drm/i915/intel_display.c:: + + + static const struct drm_crtc_helper_funcs intel_helper_funcs = { + [...] + .mode_set_base_atomic = intel_pipe_set_base_atomic, + [...] + }; + + +Here is an example of how the i915 driver initializes the +fb_debug_enter and fb_debug_leave functions to use the generic drm +helpers in ``drivers/gpu/drm/i915/intel_fb.c``:: + + + static struct fb_ops intelfb_ops = { + [...] + .fb_debug_enter = drm_fb_helper_debug_enter, + .fb_debug_leave = drm_fb_helper_debug_leave, + [...] + }; + + +Credits +======= + +The following people have contributed to this document: + +1. Amit Kale <amitkale@linsyssoft.com> + +2. Tom Rini <trini@kernel.crashing.org> + +In March 2008 this document was completely rewritten by: + +- Jason Wessel <jason.wessel@windriver.com> + +In Jan 2010 this document was updated to include kdb. + +- Jason Wessel <jason.wessel@windriver.com> diff --git a/Documentation/process/debugging/media_specific_debugging_guide.rst b/Documentation/process/debugging/media_specific_debugging_guide.rst new file mode 100644 index 000000000000..c5a93bafaf67 --- /dev/null +++ b/Documentation/process/debugging/media_specific_debugging_guide.rst @@ -0,0 +1,180 @@ +.. SPDX-License-Identifier: GPL-2.0 + +============================================ +Debugging and tracing in the media subsystem +============================================ + +This document serves as a starting point and lookup for debugging device +drivers in the media subsystem and to debug these drivers from userspace. + +.. contents:: + :depth: 3 + +General debugging advice +------------------------ + +For general advice see the :doc:`general advice document +</process/debugging/index>`. + +The following sections show you some of the available tools. + +dev_debug module parameter +-------------------------- + +Every video device provides a ``dev_debug`` parameter, which allows to get +further insights into the IOCTLs in the background.:: + + # cat /sys/class/video4linux/video3/name + rkvdec + # echo 0xff > /sys/class/video4linux/video3/dev_debug + # dmesg -wH + [...] videodev: v4l2_open: video3: open (0) + [ +0.000036] video3: VIDIOC_QUERYCAP: driver=rkvdec, card=rkvdec, + bus=platform:rkvdec, version=0x00060900, capabilities=0x84204000, + device_caps=0x04204000 + +For the full documentation see :ref:`driver-api/media/v4l2-dev:video device +debugging` + +dev_dbg() / v4l2_dbg() +---------------------- + +Two debug print statements, which are specific for devices and for the v4l2 +subsystem, avoid adding these to your final submission unless they have +long-term value for investigations. + +For a general overview please see the +:ref:`process/debugging/driver_development_debugging_guide:printk() & friends` +guide. + +- Difference between both? + + - v4l2_dbg() utilizes v4l2_printk() under the hood, which further uses + printk() directly, thus it cannot be targeted by dynamic debug + - dev_dbg() can be targeted by dynamic debug + - v4l2_dbg() has a more specific prefix format for the media subsystem, while + dev_dbg only highlights the driver name and the location of the log + +Dynamic debug +------------- + +A method to trim down the debug output to your needs. + +For general advice see the +:ref:`process/debugging/userspace_debugging_guide:dynamic debug` guide. + +Here is one example, that enables all available pr_debug()'s within the file:: + + $ alias ddcmd='echo $* > /proc/dynamic_debug/control' + $ ddcmd '-p; file v4l2-h264.c +p' + $ grep =p /proc/dynamic_debug/control + drivers/media/v4l2-core/v4l2-h264.c:372 [v4l2_h264]print_ref_list_b =p + "ref_pic_list_b%u (cur_poc %u%c) %s" + drivers/media/v4l2-core/v4l2-h264.c:333 [v4l2_h264]print_ref_list_p =p + "ref_pic_list_p (cur_poc %u%c) %s\n" + +Ftrace +------ + +An internal kernel tracer that can trace static predefined events, function +calls, etc. Very useful for debugging problems without changing the kernel and +understanding the behavior of subsystems. + +For general advice see the +:ref:`process/debugging/userspace_debugging_guide:ftrace` guide. + +DebugFS +------- + +This tool allows you to dump or modify internal values of your driver to files +in a custom filesystem. + +For general advice see the +:ref:`process/debugging/driver_development_debugging_guide:debugfs` guide. + +Perf & alternatives +------------------- + +Tools to measure the various stats on a running system to diagnose issues. + +For general advice see the +:ref:`process/debugging/userspace_debugging_guide:perf & alternatives` guide. + +Example for media devices: + +Gather statistics data for a decoding job: (This example is on a RK3399 SoC +with the rkvdec codec driver using the `fluster test suite +<https://github.com/fluendo/fluster>`__):: + + perf stat -d python3 fluster.py run -d GStreamer-H.264-V4L2SL-Gst1.0 -ts + JVT-AVC_V1 -tv AUD_MW_E -j1 + ... + Performance counter stats for 'python3 fluster.py run -d + GStreamer-H.264-V4L2SL-Gst1.0 -ts JVT-AVC_V1 -tv AUD_MW_E -j1 -v': + + 7794.23 msec task-clock:u # 0.697 CPUs utilized + 0 context-switches:u # 0.000 /sec + 0 cpu-migrations:u # 0.000 /sec + 11901 page-faults:u # 1.527 K/sec + 882671556 cycles:u # 0.113 GHz (95.79%) + 711708695 instructions:u # 0.81 insn per cycle (95.79%) + 10581935 branches:u # 1.358 M/sec (15.13%) + 6871144 branch-misses:u # 64.93% of all branches (95.79%) + 281716547 L1-dcache-loads:u # 36.144 M/sec (95.79%) + 9019581 L1-dcache-load-misses:u # 3.20% of all L1-dcache accesses (95.79%) + <not supported> LLC-loads:u + <not supported> LLC-load-misses:u + + 11.180830431 seconds time elapsed + + 1.502318000 seconds user + 6.377221000 seconds sys + +The availability of events and metrics depends on the system you are running. + +Error checking & panic analysis +------------------------------- + +Various Kernel configuration options to enhance error detection of the Linux +Kernel with the cost of lowering performance. + +For general advice see the +:ref:`process/debugging/driver_development_debugging_guide:kasan, ubsan, +lockdep and other error checkers` guide. + +Driver verification with v4l2-compliance +---------------------------------------- + +To verify, that a driver adheres to the v4l2 API, the tool v4l2-compliance is +used, which is part of the `v4l_utils +<https://git.linuxtv.org/v4l-utils.git>`__, a suite of userspace tools to work +with the media subsystem. + +To see the detailed media topology (and check it) use:: + + v4l2-compliance -M /dev/mediaX --verbose + +You can also run a full compliance check for all devices referenced in the +media topology with:: + + v4l2-compliance -m /dev/mediaX + +Debugging problems with receiving video +--------------------------------------- + +Implementing vidioc_log_status in the driver: this can log the current status +to the kernel log. It's called by v4l2-ctl --log-status. Very useful for +debugging problems with receiving video (TV/S-Video/HDMI/etc) since the video +signal is external (so unpredictable). Less useful with camera sensor inputs +since you have control over what the camera sensor does. + +Usually you can just assign the default:: + + .vidioc_log_status = v4l2_ctrl_log_status, + +But you can also create your own callback, to create a custom status log. + +You can find an example in the cobalt driver +(`drivers/media/pci/cobalt/cobalt-v4l2.c <https://elixir.bootlin.com/linux/v6.11.6/source/drivers/media/pci/cobalt/cobalt-v4l2.c#L567>`__). + +**Copyright** ©2024 : Collabora diff --git a/Documentation/process/debugging/userspace_debugging_guide.rst b/Documentation/process/debugging/userspace_debugging_guide.rst new file mode 100644 index 000000000000..db7396261e07 --- /dev/null +++ b/Documentation/process/debugging/userspace_debugging_guide.rst @@ -0,0 +1,280 @@ +.. SPDX-License-Identifier: GPL-2.0 + +========================== +Userspace debugging advice +========================== + +This document provides a brief overview of common tools to debug the Linux +Kernel from userspace. +For debugging advice aimed at driver developers go :doc:`here +</process/debugging/driver_development_debugging_guide>`. +For general debugging advice, see :doc:`general advice document +</process/debugging/index>`. + +.. contents:: + :depth: 3 + +The following sections show you the available tools. + +Dynamic debug +------------- + +Mechanism to filter what ends up in the kernel log by dis-/en-abling log +messages. + +Prerequisite: ``CONFIG_DYNAMIC_DEBUG`` + +Dynamic debug is only able to target: + +- pr_debug() +- dev_dbg() +- print_hex_dump_debug() +- print_hex_dump_bytes() + +Therefore the usability of this tool is, as of now, quite limited as there is +no uniform rule for adding debug prints to the codebase, resulting in a variety +of ways these prints are implemented. + +Also, note that most debug statements are implemented as a variation of +dprintk(), which have to be activated via a parameter in respective module, +dynamic debug is unable to do that step for you. + +Here is one example, that enables all available pr_debug()'s within the file:: + + $ alias ddcmd='echo $* > /proc/dynamic_debug/control' + $ ddcmd '-p; file v4l2-h264.c +p' + $ grep =p /proc/dynamic_debug/control + drivers/media/v4l2-core/v4l2-h264.c:372 [v4l2_h264]print_ref_list_b =p + "ref_pic_list_b%u (cur_poc %u%c) %s" + drivers/media/v4l2-core/v4l2-h264.c:333 [v4l2_h264]print_ref_list_p =p + "ref_pic_list_p (cur_poc %u%c) %s\n" + +**When should you use this over Ftrace ?** + +- When the code contains one of the valid print statements (see above) or when + you have added multiple pr_debug() statements during development +- When timing is not an issue, meaning if multiple pr_debug() statements in + the code won't cause delays +- When you care more about receiving specific log messages than tracing the + pattern of how a function is called + +For the full documentation see :doc:`/admin-guide/dynamic-debug-howto` + +Ftrace +------ + +Prerequisite: ``CONFIG_DYNAMIC_FTRACE`` + +This tool uses the tracefs file system for the control files and output files. +That file system will be mounted as a ``tracing`` directory, which can be found +in either ``/sys/kernel/`` or ``/sys/debug/kernel/``. + +Some of the most important operations for debugging are: + +- You can perform a function trace by adding a function name to the + ``set_ftrace_filter`` file (which accepts any function name found within the + ``available_filter_functions`` file) or you can specifically disable certain + functions by adding their names to the ``set_ftrace_notrace`` file (more info + at: :ref:`trace/ftrace:dynamic ftrace`). +- In order to find out where calls originate from you can activate the + ``func_stack_trace`` option under ``options/func_stack_trace``. +- Tracing the children of a function call and showing the return values are + possible by adding the desired function in the ``set_graph_function`` file + (requires config ``FUNCTION_GRAPH_RETVAL``); more info at + :ref:`trace/ftrace:dynamic ftrace with the function graph tracer`. + +For the full Ftrace documentation see :doc:`/trace/ftrace` + +Or you could also trace for specific events by :ref:`using event tracing +<trace/events:2. using event tracing>`, which can be defined as described here: +:ref:`Creating a custom Ftrace tracepoint +<process/debugging/driver_development_debugging_guide:ftrace>`. + +For the full Ftrace event tracing documentation see :doc:`/trace/events` + +.. _read_ftrace_log: + +Reading the ftrace log +~~~~~~~~~~~~~~~~~~~~~~ + +The ``trace`` file can be read just like any other file (``cat``, ``tail``, +``head``, ``vim``, etc.), the size of the file is limited by the +``buffer_size_kb`` (``echo 1000 > buffer_size_kb``). The +:ref:`trace/ftrace:trace_pipe` will behave similarly to the ``trace`` file, but +whenever you read from the file the content is consumed. + +Kernelshark +~~~~~~~~~~~ + +A GUI interface to visualize the traces as a graph and list view from the +output of the `trace-cmd +<https://git.kernel.org/pub/scm/utils/trace-cmd/trace-cmd.git/>`__ application. + +For the full documentation see `<https://kernelshark.org/Documentation.html>`__ + +Perf & alternatives +------------------- + +The tools mentioned above provide ways to inspect kernel code, results, +variable values, etc. Sometimes you have to find out first where to look and +for those cases, a box of performance tracking tools can help you to frame the +issue. + +Why should you do a performance analysis? +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +A performance analysis is a good first step when among other reasons: + +- you cannot define the issue +- you do not know where it occurs +- the running system should not be interrupted or it is a remote system, where + you cannot install a new module/kernel + +How to do a simple analysis with linux tools? +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +For the start of a performance analysis, you can start with the usual tools +like: + +- ``top`` / ``htop`` / ``atop`` (*get an overview of the system load, see + spikes on specific processes*) +- ``mpstat -P ALL`` (*look at the load distribution among CPUs*) +- ``iostat -x`` (*observe input and output devices utilization and performance*) +- ``vmstat`` (*overview of memory usage on the system*) +- ``pidstat`` (*similar to* ``vmstat`` *but per process, to dial it down to the + target*) +- ``strace -tp $PID`` (*once you know the process, you can figure out how it + communicates with the Kernel*) + +These should help to narrow down the areas to look at sufficiently. + +Diving deeper with perf +~~~~~~~~~~~~~~~~~~~~~~~ + +The **perf** tool provides a series of metrics and events to further dial down +on issues. + +Prerequisite: build or install perf on your system + +Gather statistics data for finding all files starting with ``gcc`` in ``/usr``:: + + # perf stat -d find /usr -name 'gcc*' | wc -l + + Performance counter stats for 'find /usr -name gcc*': + + 1277.81 msec task-clock # 0.997 CPUs utilized + 9 context-switches # 7.043 /sec + 1 cpu-migrations # 0.783 /sec + 704 page-faults # 550.943 /sec + 766548897 cycles # 0.600 GHz (97.15%) + 798285467 instructions # 1.04 insn per cycle (97.15%) + 57582731 branches # 45.064 M/sec (2.85%) + 3842573 branch-misses # 6.67% of all branches (97.15%) + 281616097 L1-dcache-loads # 220.390 M/sec (97.15%) + 4220975 L1-dcache-load-misses # 1.50% of all L1-dcache accesses (97.15%) + <not supported> LLC-loads + <not supported> LLC-load-misses + + 1.281746009 seconds time elapsed + + 0.508796000 seconds user + 0.773209000 seconds sys + + + 52 + +The availability of events and metrics depends on the system you are running. + +For the full documentation see +`<https://perf.wiki.kernel.org/index.php/Main_Page>`__ + +Perfetto +~~~~~~~~ + +A set of tools to measure and analyze how well applications and systems perform. +You can use it to: + +* identify bottlenecks +* optimize code +* make software run faster and more efficiently. + +**What is the difference between perfetto and perf?** + +* perf is tool as part of and specialized for the Linux Kernel and has CLI user + interface. +* perfetto cross-platform performance analysis stack, has extended + functionality into userspace and provides a WEB user interface. + +For the full documentation see `<https://perfetto.dev/docs/>`__ + +Kernel panic analysis tools +--------------------------- + + To capture the crash dump please use ``Kdump`` & ``Kexec``. Below you can find + some advice for analysing the data. + + For the full documentation see the :doc:`/admin-guide/kdump/kdump` + + In order to find the corresponding line in the code you can use `faddr2line + <https://elixir.bootlin.com/linux/v6.11.6/source/scripts/faddr2line>`__; note + that you need to enable ``CONFIG_DEBUG_INFO`` for that to work. + + An alternative to using ``faddr2line`` is the use of ``objdump`` (and its + derivatives for the different platforms like ``aarch64-linux-gnu-objdump``). + Take this line as an example: + + ``[ +0.000240] rkvdec_device_run+0x50/0x138 [rockchip_vdec]``. + + We can find the corresponding line of code by executing:: + + aarch64-linux-gnu-objdump -dS drivers/staging/media/rkvdec/rockchip-vdec.ko | grep rkvdec_device_run\>: -A 40 + 0000000000000ac8 <rkvdec_device_run>: + ac8: d503201f nop + acc: d503201f nop + { + ad0: d503233f paciasp + ad4: a9bd7bfd stp x29, x30, [sp, #-48]! + ad8: 910003fd mov x29, sp + adc: a90153f3 stp x19, x20, [sp, #16] + ae0: a9025bf5 stp x21, x22, [sp, #32] + const struct rkvdec_coded_fmt_desc *desc = ctx->coded_fmt_desc; + ae4: f9411814 ldr x20, [x0, #560] + struct rkvdec_dev *rkvdec = ctx->dev; + ae8: f9418015 ldr x21, [x0, #768] + if (WARN_ON(!desc)) + aec: b4000654 cbz x20, bb4 <rkvdec_device_run+0xec> + ret = pm_runtime_resume_and_get(rkvdec->dev); + af0: f943d2b6 ldr x22, [x21, #1952] + ret = __pm_runtime_resume(dev, RPM_GET_PUT); + af4: aa0003f3 mov x19, x0 + af8: 52800081 mov w1, #0x4 // #4 + afc: aa1603e0 mov x0, x22 + b00: 94000000 bl 0 <__pm_runtime_resume> + if (ret < 0) { + b04: 37f80340 tbnz w0, #31, b6c <rkvdec_device_run+0xa4> + dev_warn(rkvdec->dev, "Not good\n"); + b08: f943d2a0 ldr x0, [x21, #1952] + b0c: 90000001 adrp x1, 0 <rkvdec_try_ctrl-0x8> + b10: 91000021 add x1, x1, #0x0 + b14: 94000000 bl 0 <_dev_warn> + *bad = 1; + b18: d2800001 mov x1, #0x0 // #0 + ... + + Meaning, in this line from the crash dump:: + + [ +0.000240] rkvdec_device_run+0x50/0x138 [rockchip_vdec] + + I can take the ``0x50`` as offset, which I have to add to the base address + of the corresponding function, which I find in this line:: + + 0000000000000ac8 <rkvdec_device_run>: + + The result of ``0xac8 + 0x50 = 0xb18`` + And when I search for that address within the function I get the + following line:: + + *bad = 1; + b18: d2800001 mov x1, #0x0 + +**Copyright** ©2024 : Collabora diff --git a/Documentation/process/deprecated.rst b/Documentation/process/deprecated.rst index c8fd53a11a20..1f7f3e6c9cda 100644 --- a/Documentation/process/deprecated.rst +++ b/Documentation/process/deprecated.rst @@ -77,7 +77,7 @@ kzalloc() can be replaced with kcalloc(). If no 2-factor form is available, the saturate-on-overflow helpers should be used:: - bar = vmalloc(array_size(count, size)); + bar = dma_alloc_coherent(dev, array_size(count, size), &dma, GFP_KERNEL); Another common case to avoid is calculating the size of a structure with a trailing array of others structures, as in:: @@ -346,3 +346,29 @@ struct_size() and flex_array_size() helpers:: instance->count = count; memcpy(instance->items, source, flex_array_size(instance, items, instance->count)); + +There are two special cases of replacement where the DECLARE_FLEX_ARRAY() +helper needs to be used. (Note that it is named __DECLARE_FLEX_ARRAY() for +use in UAPI headers.) Those cases are when the flexible array is either +alone in a struct or is part of a union. These are disallowed by the C99 +specification, but for no technical reason (as can be seen by both the +existing use of such arrays in those places and the work-around that +DECLARE_FLEX_ARRAY() uses). For example, to convert this:: + + struct something { + ... + union { + struct type1 one[0]; + struct type2 two[0]; + }; + }; + +The helper must be used:: + + struct something { + ... + union { + DECLARE_FLEX_ARRAY(struct type1, one); + DECLARE_FLEX_ARRAY(struct type2, two); + }; + }; diff --git a/Documentation/process/development-process.rst b/Documentation/process/development-process.rst index 61c627e41ba8..e34d7da58b7f 100644 --- a/Documentation/process/development-process.rst +++ b/Documentation/process/development-process.rst @@ -3,9 +3,17 @@ A guide to the Kernel Development Process ========================================= -Contents: +The purpose of this document is to help developers (and their managers) +work with the development community with a minimum of frustration. It is +an attempt to document how this community works in a way which is +accessible to those who are not intimately familiar with Linux kernel +development (or, indeed, free software development in general). While +there is some technical material here, this is very much a process-oriented +discussion which does not require a deep knowledge of kernel programming to +understand. .. toctree:: + :caption: Contents :numbered: :maxdepth: 2 @@ -17,12 +25,3 @@ Contents: 6.Followthrough 7.AdvancedTopics 8.Conclusion - -The purpose of this document is to help developers (and their managers) -work with the development community with a minimum of frustration. It is -an attempt to document how this community works in a way which is -accessible to those who are not intimately familiar with Linux kernel -development (or, indeed, free software development in general). While -there is some technical material here, this is very much a process-oriented -discussion which does not require a deep knowledge of kernel programming to -understand. diff --git a/Documentation/process/email-clients.rst b/Documentation/process/email-clients.rst index fc2c46f3f82d..84a2450bb6ec 100644 --- a/Documentation/process/email-clients.rst +++ b/Documentation/process/email-clients.rst @@ -86,7 +86,7 @@ In the :menuselection:`Sending Preferences` section: - :menuselection:`Strip Whitespace Before Sending` must be ``disabled`` When composing the message, the cursor should be placed where the patch -should appear, and then pressing :kbd:`CTRL-R` let you specify the patch file +should appear, and then pressing `CTRL-R` let you specify the patch file to insert into the message. Claws Mail (GUI) @@ -94,7 +94,7 @@ Claws Mail (GUI) Works. Some people use this successfully for patches. -To insert a patch use :menuselection:`Message-->Insert File` (:kbd:`CTRL-I`) +To insert a patch use :menuselection:`Message-->Insert File` (`CTRL-I`) or an external editor. If the inserted patch has to be edited in the Claws composition window @@ -108,11 +108,11 @@ Evolution (GUI) Some people use this successfully for patches. When composing mail select: Preformat - from :menuselection:`Format-->Paragraph Style-->Preformatted` (:kbd:`CTRL-7`) + from :menuselection:`Format-->Paragraph Style-->Preformatted` (`CTRL-7`) or the toolbar Then use: -:menuselection:`Insert-->Text File...` (:kbd:`ALT-N x`) +:menuselection:`Insert-->Text File...` (`ALT-N x`) to insert the patch. You can also ``diff -Nru old.c new.c | xclip``, select @@ -216,7 +216,7 @@ Mutt is highly customizable. Here is a minimum configuration to start using Mutt to send patches through Gmail:: # .muttrc - # ================ IMAP ==================== + # ================ IMAP ==================== set imap_user = 'yourusername@gmail.com' set imap_pass = 'yourpassword' set spoolfile = imaps://imap.gmail.com/INBOX @@ -350,3 +350,12 @@ although tab2space problem can be solved with external editor. Another problem is that Gmail will base64-encode any message that has a non-ASCII character. That includes things like European names. + +HacKerMaiL (TUI) +**************** + +HacKerMaiL (hkml) is a public-inbox based simple mails management tool that +doesn't require subscription of mailing lists. It is developed and maintained +by the DAMON maintainer and aims to support simple development workflows for +DAMON and general kernel subsystems. Refer to the README +(https://github.com/sjp38/hackermail/blob/master/README.md) for details. diff --git a/Documentation/process/embargoed-hardware-issues.rst b/Documentation/process/embargoed-hardware-issues.rst index b6b4481e2474..34e00848e0da 100644 --- a/Documentation/process/embargoed-hardware-issues.rst +++ b/Documentation/process/embargoed-hardware-issues.rst @@ -13,9 +13,9 @@ kernel. Hardware issues like Meltdown, Spectre, L1TF etc. must be treated differently because they usually affect all Operating Systems ("OS") and therefore need coordination across different OS vendors, distributions, -hardware vendors and other parties. For some of the issues, software -mitigations can depend on microcode or firmware updates, which need further -coordination. +silicon vendors, hardware integrators, and other parties. For some of the +issues, software mitigations can depend on microcode or firmware updates, +which need further coordination. .. _Contact: @@ -25,15 +25,15 @@ Contact The Linux kernel hardware security team is separate from the regular Linux kernel security team. -The team only handles the coordination of embargoed hardware security -issues. Reports of pure software security bugs in the Linux kernel are not +The team only handles developing fixes for embargoed hardware security +issues. Reports of pure software security bugs in the Linux kernel are not handled by this team and the reporter will be guided to contact the regular Linux kernel security team (:ref:`Documentation/admin-guide/ <securitybugs>`) instead. The team can be contacted by email at <hardware-security@kernel.org>. This -is a private list of security officers who will help you to coordinate an -issue according to our documented process. +is a private list of security officers who will help you coordinate a fix +according to our documented process. The list is encrypted and email to the list can be sent by either PGP or S/MIME encrypted and must be signed with the reporter's PGP key or S/MIME @@ -43,7 +43,7 @@ the following URLs: - PGP: https://www.kernel.org/static/files/hardware-security.asc - S/MIME: https://www.kernel.org/static/files/hardware-security.crt -While hardware security issues are often handled by the affected hardware +While hardware security issues are often handled by the affected silicon vendor, we welcome contact from researchers or individuals who have identified a potential hardware flaw. @@ -65,7 +65,7 @@ of Linux Foundation's IT operations personnel technically have the ability to access the embargoed information, but are obliged to confidentiality by their employment contract. Linux Foundation IT personnel are also responsible for operating and managing the rest of -kernel.org infrastructure. +kernel.org's infrastructure. The Linux Foundation's current director of IT Project infrastructure is Konstantin Ryabitsev. @@ -85,7 +85,7 @@ Memorandum of Understanding The Linux kernel community has a deep understanding of the requirement to keep hardware security issues under embargo for coordination between -different OS vendors, distributors, hardware vendors and other parties. +different OS vendors, distributors, silicon vendors, and other parties. The Linux kernel community has successfully handled hardware security issues in the past and has the necessary mechanisms in place to allow @@ -103,11 +103,11 @@ the issue in the best technical way. All involved developers pledge to adhere to the embargo rules and to keep the received information confidential. Violation of the pledge will lead to immediate exclusion from the current issue and removal from all related -mailing-lists. In addition, the hardware security team will also exclude +mailing lists. In addition, the hardware security team will also exclude the offender from future issues. The impact of this consequence is a highly effective deterrent in our community. In case a violation happens the hardware security team will inform the involved parties immediately. If you -or anyone becomes aware of a potential violation, please report it +or anyone else becomes aware of a potential violation, please report it immediately to the Hardware security officers. @@ -124,19 +124,21 @@ method for these types of issues. Start of Disclosure """"""""""""""""""" -Disclosure starts by contacting the Linux kernel hardware security team by -email. This initial contact should contain a description of the problem and -a list of any known affected hardware. If your organization builds or -distributes the affected hardware, we encourage you to also consider what -other hardware could be affected. +Disclosure starts by emailing the Linux kernel hardware security team per +the Contact section above. This initial contact should contain a +description of the problem and a list of any known affected silicon. If +your organization builds or distributes the affected hardware, we encourage +you to also consider what other hardware could be affected. The disclosing +party is responsible for contacting the affected silicon vendors in a +timely manner. The hardware security team will provide an incident-specific encrypted -mailing-list which will be used for initial discussion with the reporter, -further disclosure and coordination. +mailing list which will be used for initial discussion with the reporter, +further disclosure, and coordination of fixes. The hardware security team will provide the disclosing party a list of developers (domain experts) who should be informed initially about the -issue after confirming with the developers that they will adhere to this +issue after confirming with the developers that they will adhere to this Memorandum of Understanding and the documented process. These developers form the initial response team and will be responsible for handling the issue after initial contact. The hardware security team is supporting the @@ -158,8 +160,8 @@ This serves several purposes: - The disclosed entities can be contacted to name experts who should participate in the mitigation development. - - If an expert which is required to handle an issue is employed by an - listed entity or member of an listed entity, then the response teams can + - If an expert who is required to handle an issue is employed by a listed + entity or member of an listed entity, then the response teams can request the disclosure of that expert from that entity. This ensures that the expert is also part of the entity's response team. @@ -169,8 +171,8 @@ Disclosure The disclosing party provides detailed information to the initial response team via the specific encrypted mailing-list. -From our experience the technical documentation of these issues is usually -a sufficient starting point and further technical clarification is best +From our experience, the technical documentation of these issues is usually +a sufficient starting point, and further technical clarification is best done via email. Mitigation development @@ -179,52 +181,93 @@ Mitigation development The initial response team sets up an encrypted mailing-list or repurposes an existing one if appropriate. -Using a mailing-list is close to the normal Linux development process and -has been successfully used in developing mitigations for various hardware +Using a mailing list is close to the normal Linux development process and +has been successfully used to develop mitigations for various hardware security issues in the past. -The mailing-list operates in the same way as normal Linux development. -Patches are posted, discussed and reviewed and if agreed on applied to a -non-public git repository which is only accessible to the participating +The mailing list operates in the same way as normal Linux development. +Patches are posted, discussed, and reviewed and if agreed upon, applied to +a non-public git repository which is only accessible to the participating developers via a secure connection. The repository contains the main development branch against the mainline kernel and backport branches for stable kernel versions as necessary. The initial response team will identify further experts from the Linux -kernel developer community as needed. Bringing in experts can happen at any -time of the development process and needs to be handled in a timely manner. +kernel developer community as needed. Any involved party can suggest +further experts to be included, each of which will be subject to the same +requirements outlined above. -If an expert is employed by or member of an entity on the disclosure list +Bringing in experts can happen at any time in the development process and +needs to be handled in a timely manner. + +If an expert is employed by or a member of an entity on the disclosure list provided by the disclosing party, then participation will be requested from the relevant entity. -If not, then the disclosing party will be informed about the experts +If not, then the disclosing party will be informed about the experts' participation. The experts are covered by the Memorandum of Understanding -and the disclosing party is requested to acknowledge the participation. In -case that the disclosing party has a compelling reason to object, then this -objection has to be raised within five work days and resolved with the -incident team immediately. If the disclosing party does not react within -five work days this is taken as silent acknowledgement. - -After acknowledgement or resolution of an objection the expert is disclosed -by the incident team and brought into the development process. - +and the disclosing party is requested to acknowledge their participation. +In the case where the disclosing party has a compelling reason to object, +any objection must to be raised within five working days and resolved with +the incident team immediately. If the disclosing party does not react +within five working days this is taken as silent acknowledgment. + +After the incident team acknowledges or resolves an objection, the expert +is disclosed and brought into the development process. + +List participants may not communicate about the issue outside of the +private mailing list. List participants may not use any shared resources +(e.g. employer build farms, CI systems, etc) when working on patches. + +Early access +"""""""""""" + +The patches discussed and developed on the list can neither be distributed +to any individual who is not a member of the response team nor to any other +organization. + +To allow the affected silicon vendors to work with their internal teams and +industry partners on testing, validation, and logistics, the following +exception is provided: + + Designated representatives of the affected silicon vendors are + allowed to hand over the patches at any time to the silicon + vendor’s response team. The representative must notify the kernel + response team about the handover. The affected silicon vendor must + have and maintain their own documented security process for any + patches shared with their response team that is consistent with + this policy. + + The silicon vendor’s response team can distribute these patches to + their industry partners and to their internal teams under the + silicon vendor’s documented security process. Feedback from the + industry partners goes back to the silicon vendor and is + communicated by the silicon vendor to the kernel response team. + + The handover to the silicon vendor’s response team removes any + responsibility or liability from the kernel response team regarding + premature disclosure, which happens due to the involvement of the + silicon vendor’s internal teams or industry partners. The silicon + vendor guarantees this release of liability by agreeing to this + process. Coordinated release """"""""""""""""""" -The involved parties will negotiate the date and time where the embargo -ends. At that point the prepared mitigations are integrated into the -relevant kernel trees and published. +The involved parties will negotiate the date and time when the embargo +ends. At that point, the prepared mitigations are published into the +relevant kernel trees. There is no pre-notification process: the +mitigations are published in public and available to everyone at the same +time. While we understand that hardware security issues need coordinated embargo -time, the embargo time should be constrained to the minimum time which is -required for all involved parties to develop, test and prepare the +time, the embargo time should be constrained to the minimum time that is +required for all involved parties to develop, test, and prepare their mitigations. Extending embargo time artificially to meet conference talk -dates or other non-technical reasons is creating more work and burden for -the involved developers and response teams as the patches need to be kept -up to date in order to follow the ongoing upstream kernel development, -which might create conflicting changes. +dates or other non-technical reasons creates more work and burden for the +involved developers and response teams as the patches need to be kept up to +date in order to follow the ongoing upstream kernel development, which +might create conflicting changes. CVE assignment """""""""""""" @@ -247,13 +290,14 @@ an involved disclosed party. The current ambassadors list: AMD Tom Lendacky <thomas.lendacky@amd.com> Ampere Darren Hart <darren@os.amperecomputing.com> ARM Catalin Marinas <catalin.marinas@arm.com> - IBM Power Anton Blanchard <anton@linux.ibm.com> + IBM Power Madhavan Srinivasan <maddy@linux.ibm.com> IBM Z Christian Borntraeger <borntraeger@de.ibm.com> Intel Tony Luck <tony.luck@intel.com> - Qualcomm Trilok Soni <tsoni@codeaurora.org> + Qualcomm Trilok Soni <quic_tsoni@quicinc.com> + RISC-V Palmer Dabbelt <palmer@dabbelt.com> + Samsung Javier González <javier.gonz@samsung.com> Microsoft James Morris <jamorris@linux.microsoft.com> - VMware Xen Andrew Cooper <andrew.cooper3@citrix.com> Canonical John Johansen <john.johansen@canonical.com> @@ -262,43 +306,42 @@ an involved disclosed party. The current ambassadors list: Red Hat Josh Poimboeuf <jpoimboe@redhat.com> SUSE Jiri Kosina <jkosina@suse.cz> - Amazon Google Kees Cook <keescook@chromium.org> - GCC - LLVM Nick Desaulniers <ndesaulniers@google.com> + LLVM Nick Desaulniers <nick.desaulniers+lkml@gmail.com> ============= ======================================================== If you want your organization to be added to the ambassadors list, please contact the hardware security team. The nominated ambassador has to -understand and support our process fully and is ideally well connected in +understand and support our process fully and is ideally well-connected in the Linux kernel community. Encrypted mailing-lists ----------------------- -We use encrypted mailing-lists for communication. The operating principle +We use encrypted mailing lists for communication. The operating principle of these lists is that email sent to the list is encrypted either with the -list's PGP key or with the list's S/MIME certificate. The mailing-list +list's PGP key or with the list's S/MIME certificate. The mailing list software decrypts the email and re-encrypts it individually for each subscriber with the subscriber's PGP key or S/MIME certificate. Details -about the mailing-list software and the setup which is used to ensure the +about the mailing list software and the setup that is used to ensure the security of the lists and protection of the data can be found here: https://korg.wiki.kernel.org/userdoc/remail. List keys ^^^^^^^^^ -For initial contact see :ref:`Contact`. For incident specific mailing-lists -the key and S/MIME certificate are conveyed to the subscribers by email -sent from the specific list. +For initial contact see the :ref:`Contact` section above. For incident +specific mailing lists, the key and S/MIME certificate are conveyed to the +subscribers by email sent from the specific list. -Subscription to incident specific lists +Subscription to incident-specific lists ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ -Subscription is handled by the response teams. Disclosed parties who want -to participate in the communication send a list of potential subscribers to -the response team so the response team can validate subscription requests. +Subscription to incident-specific lists is handled by the response teams. +Disclosed parties who want to participate in the communication send a list +of potential experts to the response team so the response team can validate +subscription requests. Each subscriber needs to send a subscription request to the response team by email. The email must be signed with the subscriber's PGP key or S/MIME diff --git a/Documentation/process/handling-regressions.rst b/Documentation/process/handling-regressions.rst index abb741b1aeee..1f5ab49c48a4 100644 --- a/Documentation/process/handling-regressions.rst +++ b/Documentation/process/handling-regressions.rst @@ -27,11 +27,11 @@ The important bits (aka "The TL;DR") is optional, but recommended): * For mailed reports, check if the reporter included a line like ``#regzbot - introduced v5.13..v5.14-rc1``. If not, send a reply (with the regressions + introduced: v5.13..v5.14-rc1``. If not, send a reply (with the regressions list in CC) containing a paragraph like the following, which tells regzbot when the issue started to happen:: - #regzbot ^introduced 1f2e3d4c5b6a + #regzbot ^introduced: 1f2e3d4c5b6a * When forwarding reports from a bug tracker to the regressions list (see above), include a paragraph like the following:: @@ -40,10 +40,13 @@ The important bits (aka "The TL;DR") #regzbot from: Some N. Ice Human <some.human@example.com> #regzbot monitor: http://some.bugtracker.example.com/ticket?id=123456789 -#. When submitting fixes for regressions, add "Link:" tags to the patch +#. When submitting fixes for regressions, add "Closes:" tags to the patch description pointing to all places where the issue was reported, as mandated by Documentation/process/submitting-patches.rst and - :ref:`Documentation/process/5.Posting.rst <development_posting>`. + :ref:`Documentation/process/5.Posting.rst <development_posting>`. If you are + only fixing part of the issue that caused the regression, you may use + "Link:" tags instead. regzbot currently makes no distinction between the + two. #. Try to fix regressions quickly once the culprit has been identified; fixes for most regressions should be merged within two weeks, but some need to be @@ -79,7 +82,7 @@ When doing either, consider making the Linux kernel regression tracking bot "regzbot" immediately start tracking the issue: * For mailed reports, check if the reporter included a "regzbot command" like - ``#regzbot introduced 1f2e3d4c5b6a``. If not, send a reply (with the + ``#regzbot introduced: 1f2e3d4c5b6a``. If not, send a reply (with the regressions list in CC) with a paragraph like the following::: #regzbot ^introduced: v5.13..v5.14-rc1 @@ -91,10 +94,10 @@ When doing either, consider making the Linux kernel regression tracking bot Note the caret (^) before the "introduced": it tells regzbot to treat the parent mail (the one you reply to) as the initial report for the regression you want to see tracked; that's important, as regzbot will later look out - for patches with "Link:" tags pointing to the report in the archives on + for patches with "Closes:" tags pointing to the report in the archives on lore.kernel.org. - * When forwarding a regressions reported to a bug tracker, include a paragraph + * When forwarding a regression reported to a bug tracker, include a paragraph with these regzbot commands:: #regzbot introduced: 1f2e3d4c5b6a @@ -102,7 +105,7 @@ When doing either, consider making the Linux kernel regression tracking bot #regzbot monitor: http://some.bugtracker.example.com/ticket?id=123456789 Regzbot will then automatically associate patches with the report that - contain "Link:" tags pointing to your mail or the mentioned ticket. + contain "Closes:" tags pointing to your mail or the mentioned ticket. What's important when fixing regressions ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ @@ -112,10 +115,14 @@ remember to do what Documentation/process/submitting-patches.rst, :ref:`Documentation/process/5.Posting.rst <development_posting>`, and Documentation/process/stable-kernel-rules.rst already explain in more detail: - * Point to all places where the issue was reported using "Link:" tags:: + * Point to all places where the issue was reported using "Closes:" tags:: - Link: https://lore.kernel.org/r/30th.anniversary.repost@klaava.Helsinki.FI/ - Link: https://bugzilla.kernel.org/show_bug.cgi?id=1234567890 + Closes: https://lore.kernel.org/r/30th.anniversary.repost@klaava.Helsinki.FI/ + Closes: https://bugzilla.kernel.org/show_bug.cgi?id=1234567890 + + If you are only fixing part of the issue, you may use "Link:" instead as + described in the first document mentioned above. regzbot currently treats + both of these equivalently and considers the linked reports as resolved. * Add a "Fixes:" tag to specify the commit causing the regression. @@ -126,91 +133,135 @@ All this is expected from you and important when it comes to regression, as these tags are of great value for everyone (you included) that might be looking into the issue weeks, months, or years later. These tags are also crucial for tools and scripts used by other kernel developers or Linux distributions; one of -these tools is regzbot, which heavily relies on the "Link:" tags to associate +these tools is regzbot, which heavily relies on the "Closes:" tags to associate reports for regression with changes resolving them. -Prioritize work on fixing regressions -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -You should fix any reported regression as quickly as possible, to provide -affected users with a solution in a timely manner and prevent more users from -running into the issue; nevertheless developers need to take enough time and -care to ensure regression fixes do not cause additional damage. - -In the end though, developers should give their best to prevent users from -running into situations where a regression leaves them only three options: "run -a kernel with a regression that seriously impacts usage", "continue running an -outdated and thus potentially insecure kernel version for more than two weeks -after a regression's culprit was identified", and "downgrade to a still -supported kernel series that lack required features". - -How to realize this depends a lot on the situation. Here are a few rules of -thumb for you, in order or importance: - - * Prioritize work on handling regression reports and fixing regression over all - other Linux kernel work, unless the latter concerns acute security issues or - bugs causing data loss or damage. - - * Always consider reverting the culprit commits and reapplying them later - together with necessary fixes, as this might be the least dangerous and - quickest way to fix a regression. - - * Developers should handle regressions in all supported kernel series, but are - free to delegate the work to the stable team, if the issue probably at no - point in time occurred with mainline. - - * Try to resolve any regressions introduced in the current development before - its end. If you fear a fix might be too risky to apply only days before a new - mainline release, let Linus decide: submit the fix separately to him as soon - as possible with the explanation of the situation. He then can make a call - and postpone the release if necessary, for example if multiple such changes - show up in his inbox. - - * Address regressions in stable, longterm, or proper mainline releases with - more urgency than regressions in mainline pre-releases. That changes after - the release of the fifth pre-release, aka "-rc5": mainline then becomes as - important, to ensure all the improvements and fixes are ideally tested - together for at least one week before Linus releases a new mainline version. - - * Fix regressions within two or three days, if they are critical for some - reason -- for example, if the issue is likely to affect many users of the - kernel series in question on all or certain architectures. Note, this - includes mainline, as issues like compile errors otherwise might prevent many - testers or continuous integration systems from testing the series. - - * Aim to fix regressions within one week after the culprit was identified, if - the issue was introduced in either: - - * a recent stable/longterm release - - * the development cycle of the latest proper mainline release - - In the latter case (say Linux v5.14), try to address regressions even - quicker, if the stable series for the predecessor (v5.13) will be abandoned - soon or already was stamped "End-of-Life" (EOL) -- this usually happens about - three to four weeks after a new mainline release. - - * Try to fix all other regressions within two weeks after the culprit was - found. Two or three additional weeks are acceptable for performance - regressions and other issues which are annoying, but don't prevent anyone - from running Linux (unless it's an issue in the current development cycle, - as those should ideally be addressed before the release). A few weeks in - total are acceptable if a regression can only be fixed with a risky change - and at the same time is affecting only a few users; as much time is - also okay if the regression is already present in the second newest longterm - kernel series. - -Note: The aforementioned time frames for resolving regressions are meant to -include getting the fix tested, reviewed, and merged into mainline, ideally with -the fix being in linux-next at least briefly. This leads to delays you need to -account for. - -Subsystem maintainers are expected to assist in reaching those periods by doing -timely reviews and quick handling of accepted patches. They thus might have to -send git-pull requests earlier or more often than usual; depending on the fix, -it might even be acceptable to skip testing in linux-next. Especially fixes for -regressions in stable and longterm kernels need to be handled quickly, as fixes -need to be merged in mainline before they can be backported to older series. +Expectations and best practices for fixing regressions +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +As a Linux kernel developer, you are expected to give your best to prevent +situations where a regression caused by a recent change of yours leaves users +only these options: + + * Run a kernel with a regression that impacts usage. + + * Switch to an older or newer kernel series. + + * Continue running an outdated and thus potentially insecure kernel for more + than three weeks after the regression's culprit was identified. Ideally it + should be less than two. And it ought to be just a few days, if the issue is + severe or affects many users -- either in general or in prevalent + environments. + +How to realize that in practice depends on various factors. Use the following +rules of thumb as a guide. + +In general: + + * Prioritize work on regressions over all other Linux kernel work, unless the + latter concerns a severe issue (e.g. acute security vulnerability, data loss, + bricked hardware, ...). + + * Expedite fixing mainline regressions that recently made it into a proper + mainline, stable, or longterm release (either directly or via backport). + + * Do not consider regressions from the current cycle as something that can wait + till the end of the cycle, as the issue might discourage or prevent users and + CI systems from testing mainline now or generally. + + * Work with the required care to avoid additional or bigger damage, even if + resolving an issue then might take longer than outlined below. + +On timing once the culprit of a regression is known: + + * Aim to mainline a fix within two or three days, if the issue is severe or + bothering many users -- either in general or in prevalent conditions like a + particular hardware environment, distribution, or stable/longterm series. + + * Aim to mainline a fix by Sunday after the next, if the culprit made it + into a recent mainline, stable, or longterm release (either directly or via + backport); if the culprit became known early during a week and is simple to + resolve, try to mainline the fix within the same week. + + * For other regressions, aim to mainline fixes before the hindmost Sunday + within the next three weeks. One or two Sundays later are acceptable, if the + regression is something people can live with easily for a while -- like a + mild performance regression. + + * It's strongly discouraged to delay mainlining regression fixes till the next + merge window, except when the fix is extraordinarily risky or when the + culprit was mainlined more than a year ago. + +On procedure: + + * Always consider reverting the culprit, as it's often the quickest and least + dangerous way to fix a regression. Don't worry about mainlining a fixed + variant later: that should be straight-forward, as most of the code went + through review once already. + + * Try to resolve any regressions introduced in mainline during the past + twelve months before the current development cycle ends: Linus wants such + regressions to be handled like those from the current cycle, unless fixing + bears unusual risks. + + * Consider CCing Linus on discussions or patch review, if a regression seems + tangly. Do the same in precarious or urgent cases -- especially if the + subsystem maintainer might be unavailable. Also CC the stable team, when you + know such a regression made it into a mainline, stable, or longterm release. + + * For urgent regressions, consider asking Linus to pick up the fix straight + from the mailing list: he is totally fine with that for uncontroversial + fixes. Ideally though such requests should happen in accordance with the + subsystem maintainers or come directly from them. + + * In case you are unsure if a fix is worth the risk applying just days before + a new mainline release, send Linus a mail with the usual lists and people in + CC; in it, summarize the situation while asking him to consider picking up + the fix straight from the list. He then himself can make the call and when + needed even postpone the release. Such requests again should ideally happen + in accordance with the subsystem maintainers or come directly from them. + +Regarding stable and longterm kernels: + + * You are free to leave regressions to the stable team, if they at no point in + time occurred with mainline or were fixed there already. + + * If a regression made it into a proper mainline release during the past + twelve months, ensure to tag the fix with "Cc: stable@vger.kernel.org", as a + "Fixes:" tag alone does not guarantee a backport. Please add the same tag, + in case you know the culprit was backported to stable or longterm kernels. + + * When receiving reports about regressions in recent stable or longterm kernel + series, please evaluate at least briefly if the issue might happen in current + mainline as well -- and if that seems likely, take hold of the report. If in + doubt, ask the reporter to check mainline. + + * Whenever you want to swiftly resolve a regression that recently also made it + into a proper mainline, stable, or longterm release, fix it quickly in + mainline; when appropriate thus involve Linus to fast-track the fix (see + above). That's because the stable team normally does neither revert nor fix + any changes that cause the same problems in mainline. + + * In case of urgent regression fixes you might want to ensure prompt + backporting by dropping the stable team a note once the fix was mainlined; + this is especially advisable during merge windows and shortly thereafter, as + the fix otherwise might land at the end of a huge patch queue. + +On patch flow: + + * Developers, when trying to reach the time periods mentioned above, remember + to account for the time it takes to get fixes tested, reviewed, and merged by + Linus, ideally with them being in linux-next at least briefly. Hence, if a + fix is urgent, make it obvious to ensure others handle it appropriately. + + * Reviewers, you are kindly asked to assist developers in reaching the time + periods mentioned above by reviewing regression fixes in a timely manner. + + * Subsystem maintainers, you likewise are encouraged to expedite the handling + of regression fixes. Thus evaluate if skipping linux-next is an option for + the particular fix. Also consider sending git pull requests more often than + usual when needed. And try to avoid holding onto regression fixes over + weekends -- especially when the fix is marked for backporting. More aspects regarding regressions developers should be aware of @@ -240,7 +291,7 @@ What else is there to known about regressions? Check out Documentation/admin-guide/reporting-regressions.rst, it covers a lot of other aspects you want might want to be aware of: - * the purpose of the "no regressions rule" + * the purpose of the "no regressions" rule * what issues actually qualify as regression @@ -282,7 +333,7 @@ How does regression tracking work with regzbot? The bot watches for replies to reports of tracked regressions. Additionally, it's looking out for posted or committed patches referencing such reports -with "Link:" tags; replies to such patch postings are tracked as well. +with "Closes:" tags; replies to such patch postings are tracked as well. Combined this data provides good insights into the current state of the fixing process. @@ -294,8 +345,7 @@ take care of that using ``#regzbot ^introduced``. For developers there normally is no extra work involved, they just need to make sure to do something that was expected long before regzbot came to light: add -"Link:" tags to the patch description pointing to all reports about the issue -fixed. +links to the patch description pointing to all reports about the issue fixed. Do I have to use regzbot? ~~~~~~~~~~~~~~~~~~~~~~~~~ @@ -354,9 +404,9 @@ By using a 'regzbot command' in a direct or indirect reply to the mail with the regression report. These commands need to be in their own paragraph (IOW: they need to be separated from the rest of the mail using blank lines). -One such command is ``#regzbot introduced <version or commit>``, which makes +One such command is ``#regzbot introduced: <version or commit>``, which makes regzbot consider your mail as a regressions report added to the tracking, as -already described above; ``#regzbot ^introduced <version or commit>`` is another +already described above; ``#regzbot ^introduced: <version or commit>`` is another such command, which makes regzbot consider the parent mail as a report for a regression which it starts to track. @@ -388,7 +438,7 @@ or itself is a reply to that mail: * Mark a regression as fixed by a commit that is heading upstream or already landed:: - #regzbot fixed-by: 1f2e3d4c5d + #regzbot fix: 1f2e3d4c5d * Mark a regression as a duplicate of another one already tracked by regzbot:: diff --git a/Documentation/process/howto.rst b/Documentation/process/howto.rst index cb6abcb2b6d0..9438e03d6f50 100644 --- a/Documentation/process/howto.rst +++ b/Documentation/process/howto.rst @@ -82,8 +82,7 @@ documentation files are also added which explain how to use the feature. When a kernel change causes the interface that the kernel exposes to userspace to change, it is recommended that you send the information or a patch to the manual pages explaining the change to the manual pages -maintainer at mtk.manpages@gmail.com, and CC the list -linux-api@vger.kernel.org. +maintainer at alx@kernel.org, and CC the list linux-api@vger.kernel.org. Here is a list of files that are in the kernel source tree that are required reading: @@ -138,7 +137,7 @@ required reading: philosophy and is very important for people moving to Linux from development on other Operating Systems. - :ref:`Documentation/admin-guide/security-bugs.rst <securitybugs>` + :ref:`Documentation/process/security-bugs.rst <securitybugs>` If you feel you have found a security problem in the Linux kernel, please follow the steps in this document to help notify the kernel developers, and help solve the issue. @@ -332,7 +331,7 @@ they need to be integration-tested. For this purpose, a special testing repository exists into which virtually all subsystem trees are pulled on an almost daily basis: - https://git.kernel.org/?p=linux/kernel/git/next/linux-next.git + https://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git This way, the linux-next gives a summary outlook onto what will be expected to go into the mainline kernel at the next merge period. @@ -352,8 +351,8 @@ Managing bug reports -------------------- One of the best ways to put into practice your hacking skills is by fixing -bugs reported by other people. Not only you will help to make the kernel -more stable, but you'll also learn to fix real world problems and you will +bugs reported by other people. Not only will you help to make the kernel +more stable, but you'll also learn to fix real-world problems and you will improve your skills, and other developers will be aware of your presence. Fixing bugs is one of the best ways to get merits among other developers, because not many people like wasting time fixing other people's bugs. @@ -374,12 +373,12 @@ As some of the above documents describe, the majority of the core kernel developers participate on the Linux Kernel Mailing list. Details on how to subscribe and unsubscribe from the list can be found at: - http://vger.kernel.org/vger-lists.html#linux-kernel + https://subspace.kernel.org/subscribing.html There are archives of the mailing list on the web in many different places. Use a search engine to find these archives. For example: - https://lore.kernel.org/lkml/ + https://lore.kernel.org/linux-kernel/ It is highly recommended that you search the archives about the topic you want to bring up, before you post it to the list. A lot of things @@ -394,13 +393,13 @@ groups. Many of the lists are hosted on kernel.org. Information on them can be found at: - http://vger.kernel.org/vger-lists.html + https://subspace.kernel.org Please remember to follow good behavioral habits when using the lists. Though a bit cheesy, the following URL has some simple guidelines for interacting with the list (or any list): - http://www.albion.com/netiquette/ + https://subspace.kernel.org/etiquette.html If multiple people respond to your mail, the CC: list of recipients may get pretty large. Don't remove anybody from the CC: list without a good diff --git a/Documentation/process/index.rst b/Documentation/process/index.rst index d4b6217472b0..aa12f2660194 100644 --- a/Documentation/process/index.rst +++ b/Documentation/process/index.rst @@ -15,56 +15,99 @@ to learn about how our community works. Reading these documents will make it much easier for you to get your changes merged with a minimum of trouble. -Below are the essential guides that every developer should read. +An introduction to how kernel development works +----------------------------------------------- + +Read these documents first: an understanding of the material here will ease +your entry into the kernel community. .. toctree:: :maxdepth: 1 - license-rules howto - code-of-conduct - code-of-conduct-interpretation development-process submitting-patches - handling-regressions + submit-checklist + +Tools and technical guides for kernel developers +------------------------------------------------ + +This is a collection of material that kernel developers should be familiar +with. + +.. toctree:: + :maxdepth: 1 + + changes programming-language coding-style - maintainer-handbooks maintainer-pgp-guide email-clients - kernel-enforcement-statement - kernel-driver-statement + applying-patches + backporting + adding-syscalls + volatile-considered-harmful + botching-up-ioctls + +Policy guides and developer statements +-------------------------------------- -Other guides to the community that are of interest to most developers are: +These are the rules that we try to live by in the kernel community (and +beyond). .. toctree:: :maxdepth: 1 - changes + license-rules + code-of-conduct + code-of-conduct-interpretation + contribution-maturity-model + kernel-enforcement-statement + kernel-driver-statement stable-api-nonsense - management-style stable-kernel-rules - submit-checklist - kernel-docs - deprecated + management-style + researcher-guidelines + +Dealing with bugs +----------------- + +Bugs are a fact of life; it is important that we handle them properly. The +documents below provide general advice about debugging and describe our +policies around the handling of a couple of special classes of bugs: +regressions and security problems. + +.. toctree:: + :maxdepth: 1 + + debugging/index + handling-regressions + security-bugs + cve embargoed-hardware-issues + +Maintainer information +---------------------- + +How to find the people who will accept your patches. + +.. toctree:: + :maxdepth: 1 + + maintainer-handbooks maintainers - researcher-guidelines -These are some overall technical guides that have been put here for now for -lack of a better place. +Other material +-------------- + +Here are some other guides to the community that are of interest to most +developers: .. toctree:: :maxdepth: 1 - applying-patches - adding-syscalls - magic-number - volatile-considered-harmful - botching-up-ioctls - clang-format - ../riscv/patch-acceptance - ../core-api/unaligned-memory-access + kernel-docs + deprecated .. only:: subproject and html diff --git a/Documentation/process/kernel-docs.rst b/Documentation/process/kernel-docs.rst index 306ad373a002..c67ac12cf789 100644 --- a/Documentation/process/kernel-docs.rst +++ b/Documentation/process/kernel-docs.rst @@ -3,35 +3,34 @@ Index of Further Kernel Documentation ===================================== -Initial Author: Juan-Mariano de Goyeneche (<jmseyas@dit.upm.es>; -email address is defunct now.) +The need for a document like this one became apparent in the linux-kernel +mailing list as the same questions, asking for pointers to information, +appeared again and again. -The need for a document like this one became apparent in the -linux-kernel mailing list as the same questions, asking for pointers -to information, appeared again and again. +Fortunately, as more and more people get to GNU/Linux, more and more get +interested in the Kernel. But reading the sources is not always enough. It +is easy to understand the code, but miss the concepts, the philosophy and +design decisions behind this code. -Fortunately, as more and more people get to GNU/Linux, more and more -get interested in the Kernel. But reading the sources is not always -enough. It is easy to understand the code, but miss the concepts, the -philosophy and design decisions behind this code. - -Unfortunately, not many documents are available for beginners to -start. And, even if they exist, there was no "well-known" place which -kept track of them. These lines try to cover this lack. +Unfortunately, not many documents are available for beginners to start. +And, even if they exist, there was no "well-known" place which kept track +of them. These lines try to cover this lack. PLEASE, if you know any paper not listed here or write a new document, include a reference to it here, following the kernel's patch submission process. Any corrections, ideas or comments are also welcome. All documents are cataloged with the following fields: the document's -"Title", the "Author"/s, the "URL" where they can be found, some -"Keywords" helpful when searching for specific topics, and a brief -"Description" of the Document. +"Title", the "Author"/s, the "URL" where they can be found, some "Keywords" +helpful when searching for specific topics, and a brief "Description" of +the Document. .. note:: The documents on each section of this document are ordered by its - published date, from the newest to the oldest. + published date, from the newest to the oldest. The maintainer(s) should + periodically retire resources as they become obsolete or outdated; with + the exception of foundational books. Docs at the Linux Kernel tree ----------------------------- @@ -61,24 +60,6 @@ On-line docs a brief description of some of the acronyms and terms you may hear during discussion of the Linux kernel". - * Title: **Tracing the Way of Data in a TCP Connection through the Linux Kernel** - - :Author: Richard Sailer - :URL: https://archive.org/details/linux_kernel_data_flow_short_paper - :Date: 2016 - :Keywords: Linux Kernel Networking, TCP, tracing, ftrace - :Description: A seminar paper explaining ftrace and how to use it for - understanding linux kernel internals, - illustrated at tracing the way of a TCP packet through the kernel. - :Abstract: *This short paper outlines the usage of ftrace a tracing framework - as a tool to understand a running Linux system. - Having obtained a trace-log a kernel hacker can read and understand - source code more determined and with context. - In a detailed example this approach is demonstrated in tracing - and the way of data in a TCP Connection through the kernel. - Finally this trace-log is used as base for more a exact conceptual - exploration and description of the Linux TCP/IP implementation.* - * Title: **The Linux Kernel Module Programming Guide** :Author: Peter Jay Salzman, Michael Burian, Ori Pomerantz, Bob Mottram, @@ -91,379 +72,62 @@ On-line docs programming. Lots of examples. Currently the new version is being actively maintained at https://github.com/sysprog21/lkmpg. - * Title: **On submitting kernel Patches** +Published books +--------------- - :Author: Andi Kleen - :URL: http://halobates.de/on-submitting-kernel-patches.pdf - :Date: 2008 - :Keywords: patches, review process, types of submissions, basic rules, case studies - :Description: This paper gives several experience values on what types of patches - there are and how likely they get merged. - :Abstract: - [...]. This paper examines some common problems for - submitting larger changes and some strategies to avoid problems. + * Title: **The Linux Memory Manager** - * Title: **Linux Device Drivers, Third Edition** + :Author: Lorenzo Stoakes + :Publisher: No Starch Press + :Date: February 2025 + :Pages: 1300 + :ISBN: 978-1718504462 + :Notes: Memory management. Full draft available as early access for + pre-order, full release scheduled for Fall 2025. See + https://nostarch.com/linux-memory-manager for further info. - :Author: Jonathan Corbet, Alessandro Rubini, Greg Kroah-Hartman - :URL: https://lwn.net/Kernel/LDD3/ - :Date: 2005 - :Description: A 600-page book covering the (2.6.10) driver - programming API and kernel hacking in general. Available under the - Creative Commons Attribution-ShareAlike 2.0 license. - :note: You can also :ref:`purchase a copy from O'Reilly or elsewhere <ldd3_published>`. + * Title: **Practical Linux System Administration: A Guide to Installation, Configuration, and Management, 1st Edition** - * Title: **Writing an ALSA Driver** + :Author: Kenneth Hess + :Publisher: O'Reilly Media + :Date: May, 2023 + :Pages: 246 + :ISBN: 978-1098109035 + :Notes: System administration - :Author: Takashi Iwai <tiwai@suse.de> - :URL: https://www.kernel.org/doc/html/latest/sound/kernel-api/writing-an-alsa-driver.html - :Date: 2005 - :Keywords: ALSA, sound, soundcard, driver, lowlevel, hardware. - :Description: Advanced Linux Sound Architecture for developers, - both at kernel and user-level sides. ALSA is the Linux kernel - sound architecture in the 2.6 kernel version. - - * Title: **Linux PCMCIA Programmer's Guide** - - :Author: David Hinds. - :URL: http://pcmcia-cs.sourceforge.net/ftp/doc/PCMCIA-PROG.html - :Date: 2003 - :Keywords: PCMCIA. - :Description: "This document describes how to write kernel device - drivers for the Linux PCMCIA Card Services interface. It also - describes how to write user-mode utilities for communicating with - Card Services. - - * Title: **How NOT to write kernel drivers** - - :Author: Arjan van de Ven. - :URL: https://landley.net/kdocs/ols/2002/ols2002-pages-545-555.pdf - :Date: 2002 - :Keywords: driver. - :Description: Programming bugs and Do-nots in kernel driver development - :Abstract: *Quit a few tutorials, articles and books give an introduction - on how to write Linux kernel drivers. Unfortunately the things one - should NOT do in Linux kernel code is either only a minor appendix - or, more commonly, completely absent. This paper tries to briefly touch - the areas in which the most common and serious bugs and do-nots are - encountered.* - - * Title: **Global spinlock list and usage** - - :Author: Rick Lindsley. - :URL: http://lse.sourceforge.net/lockhier/global-spin-lock - :Date: 2001 - :Keywords: spinlock. - :Description: This is an attempt to document both the existence and - usage of the spinlocks in the Linux 2.4.5 kernel. Comprehensive - list of spinlocks showing when they are used, which functions - access them, how each lock is acquired, under what conditions it - is held, whether interrupts can occur or not while it is held... - - * Title: **A Linux vm README** - - :Author: Kanoj Sarcar. - :URL: http://kos.enix.org/pub/linux-vmm.html - :Date: 2001 - :Keywords: virtual memory, mm, pgd, vma, page, page flags, page - cache, swap cache, kswapd. - :Description: Telegraphic, short descriptions and definitions - relating the Linux virtual memory implementation. - - * Title: **Video4linux Drivers, Part 1: Video-Capture Device** - - :Author: Alan Cox. - :URL: http://www.linux-mag.com/id/406 - :Date: 2000 - :Keywords: video4linux, driver, video capture, capture devices, - camera driver. - :Description: The title says it all. - - * Title: **Video4linux Drivers, Part 2: Video-capture Devices** - - :Author: Alan Cox. - :URL: http://www.linux-mag.com/id/429 - :Date: 2000 - :Keywords: video4linux, driver, video capture, capture devices, - camera driver, control, query capabilities, capability, facility. - :Description: The title says it all. - - * Title: **Linux IP Networking. A Guide to the Implementation and Modification of the Linux Protocol Stack.** - - :Author: Glenn Herrin. - :URL: http://www.cs.unh.edu/cnrg/gherrin - :Date: 2000 - :Keywords: network, networking, protocol, IP, UDP, TCP, connection, - socket, receiving, transmitting, forwarding, routing, packets, - modules, /proc, sk_buff, FIB, tags. - :Description: Excellent paper devoted to the Linux IP Networking, - explaining anything from the kernel's to the user space - configuration tools' code. Very good to get a general overview of - the kernel networking implementation and understand all steps - packets follow from the time they are received at the network - device till they are delivered to applications. The studied kernel - code is from 2.2.14 version. Provides code for a working packet - dropper example. - - * Title: **How To Make Sure Your Driver Will Work On The Power Macintosh** - - :Author: Paul Mackerras. - :URL: http://www.linux-mag.com/id/261 - :Date: 1999 - :Keywords: Mac, Power Macintosh, porting, drivers, compatibility. - :Description: The title says it all. - - * Title: **An Introduction to SCSI Drivers** - - :Author: Alan Cox. - :URL: http://www.linux-mag.com/id/284 - :Date: 1999 - :Keywords: SCSI, device, driver. - :Description: The title says it all. - - * Title: **Advanced SCSI Drivers And Other Tales** - - :Author: Alan Cox. - :URL: http://www.linux-mag.com/id/307 - :Date: 1999 - :Keywords: SCSI, device, driver, advanced. - :Description: The title says it all. - - * Title: **Writing Linux Mouse Drivers** - - :Author: Alan Cox. - :URL: http://www.linux-mag.com/id/330 - :Date: 1999 - :Keywords: mouse, driver, gpm. - :Description: The title says it all. - - * Title: **More on Mouse Drivers** - - :Author: Alan Cox. - :URL: http://www.linux-mag.com/id/356 - :Date: 1999 - :Keywords: mouse, driver, gpm, races, asynchronous I/O. - :Description: The title still says it all. - - * Title: **Writing Video4linux Radio Driver** - - :Author: Alan Cox. - :URL: http://www.linux-mag.com/id/381 - :Date: 1999 - :Keywords: video4linux, driver, radio, radio devices. - :Description: The title says it all. - - * Title: **I/O Event Handling Under Linux** - - :Author: Richard Gooch. - :URL: https://web.mit.edu/~yandros/doc/io-events.html - :Date: 1999 - :Keywords: IO, I/O, select(2), poll(2), FDs, aio_read(2), readiness - event queues. - :Description: From the Introduction: "I/O Event handling is about - how your Operating System allows you to manage a large number of - open files (file descriptors in UNIX/POSIX, or FDs) in your - application. You want the OS to notify you when FDs become active - (have data ready to be read or are ready for writing). Ideally you - want a mechanism that is scalable. This means a large number of - inactive FDs cost very little in memory and CPU time to manage". - - * Title: **(nearly) Complete Linux Loadable Kernel Modules. The definitive guide for hackers, virus coders and system administrators.** - - :Author: pragmatic/THC. - :URL: http://packetstormsecurity.org/docs/hack/LKM_HACKING.html - :Date: 1999 - :Keywords: syscalls, intercept, hide, abuse, symbol table. - :Description: Interesting paper on how to abuse the Linux kernel in - order to intercept and modify syscalls, make - files/directories/processes invisible, become root, hijack ttys, - write kernel modules based virus... and solutions for admins to - avoid all those abuses. - :Notes: For 2.0.x kernels. Gives guidances to port it to 2.2.x - kernels. - - * Name: **Linux Virtual File System** - - :Author: Peter J. Braam. - :URL: http://www.coda.cs.cmu.edu/doc/talks/linuxvfs/ - :Date: 1998 - :Keywords: slides, VFS, inode, superblock, dentry, dcache. - :Description: Set of slides, presumably from a presentation on the - Linux VFS layer. Covers version 2.1.x, with dentries and the - dcache. - - * Title: **The Venus kernel interface** - - :Author: Peter J. Braam. - :URL: http://www.coda.cs.cmu.edu/doc/html/kernel-venus-protocol.html - :Date: 1998 - :Keywords: coda, filesystem, venus, cache manager. - :Description: "This document describes the communication between - Venus and kernel level file system code needed for the operation - of the Coda filesystem. This version document is meant to describe - the current interface (version 1.0) as well as improvements we - envisage". - - * Title: **Design and Implementation of the Second Extended Filesystem** - - :Author: Rémy Card, Theodore Ts'o, Stephen Tweedie. - :URL: https://web.mit.edu/tytso/www/linux/ext2intro.html - :Date: 1998 - :Keywords: ext2, linux fs history, inode, directory, link, devices, - VFS, physical structure, performance, benchmarks, ext2fs library, - ext2fs tools, e2fsck. - :Description: Paper written by three of the top ext2 hackers. - Covers Linux filesystems history, ext2 motivation, ext2 features, - design, physical structure on disk, performance, benchmarks, - e2fsck's passes description... A must read! - :Notes: This paper was first published in the Proceedings of the - First Dutch International Symposium on Linux, ISBN 90-367-0385-9. - - * Title: **The Linux RAID-1, 4, 5 Code** - - :Author: Ingo Molnar, Gadi Oxman and Miguel de Icaza. - :URL: http://www.linuxjournal.com/article.php?sid=2391 - :Date: 1997 - :Keywords: RAID, MD driver. - :Description: Linux Journal Kernel Korner article. - :Abstract: *A description of the implementation of the RAID-1, - RAID-4 and RAID-5 personalities of the MD device driver in the - Linux kernel, providing users with high performance and reliable, - secondary-storage capability using software*. - - * Title: **Linux Kernel Hackers' Guide** - - :Author: Michael K. Johnson. - :URL: https://www.tldp.org/LDP/khg/HyperNews/get/khg.html - :Date: 1997 - :Keywords: device drivers, files, VFS, kernel interface, character vs - block devices, hardware interrupts, scsi, DMA, access to user memory, - memory allocation, timers. - :Description: A guide designed to help you get up to speed on the - concepts that are not intuitively obvious, and to document the internal - structures of Linux. - - * Title: **Dynamic Kernels: Modularized Device Drivers** - - :Author: Alessandro Rubini. - :URL: http://www.linuxjournal.com/article.php?sid=1219 - :Date: 1996 - :Keywords: device driver, module, loading/unloading modules, - allocating resources. - :Description: Linux Journal Kernel Korner article. - :Abstract: *This is the first of a series of four articles - co-authored by Alessandro Rubini and Georg Zezchwitz which present - a practical approach to writing Linux device drivers as kernel - loadable modules. This installment presents an introduction to the - topic, preparing the reader to understand next month's - installment*. - - * Title: **Dynamic Kernels: Discovery** - - :Author: Alessandro Rubini. - :URL: http://www.linuxjournal.com/article.php?sid=1220 - :Date: 1996 - :Keywords: character driver, init_module, clean_up module, - autodetection, mayor number, minor number, file operations, - open(), close(). - :Description: Linux Journal Kernel Korner article. - :Abstract: *This article, the second of four, introduces part of - the actual code to create custom module implementing a character - device driver. It describes the code for module initialization and - cleanup, as well as the open() and close() system calls*. - - * Title: **The Devil's in the Details** - - :Author: Georg v. Zezschwitz and Alessandro Rubini. - :URL: http://www.linuxjournal.com/article.php?sid=1221 - :Date: 1996 - :Keywords: read(), write(), select(), ioctl(), blocking/non - blocking mode, interrupt handler. - :Description: Linux Journal Kernel Korner article. - :Abstract: *This article, the third of four on writing character - device drivers, introduces concepts of reading, writing, and using - ioctl-calls*. - - * Title: **Dissecting Interrupts and Browsing DMA** - - :Author: Alessandro Rubini and Georg v. Zezschwitz. - :URL: https://www.linuxjournal.com/article.php?sid=1222 - :Date: 1996 - :Keywords: interrupts, irqs, DMA, bottom halves, task queues. - :Description: Linux Journal Kernel Korner article. - :Abstract: *This is the fourth in a series of articles about - writing character device drivers as loadable kernel modules. This - month, we further investigate the field of interrupt handling. - Though it is conceptually simple, practical limitations and - constraints make this an ''interesting'' part of device driver - writing, and several different facilities have been provided for - different situations. We also investigate the complex topic of - DMA*. - - * Title: **Device Drivers Concluded** - - :Author: Georg v. Zezschwitz. - :URL: https://www.linuxjournal.com/article.php?sid=1287 - :Date: 1996 - :Keywords: address spaces, pages, pagination, page management, - demand loading, swapping, memory protection, memory mapping, mmap, - virtual memory areas (VMAs), vremap, PCI. - :Description: Finally, the above turned out into a five articles - series. This latest one's introduction reads: "This is the last of - five articles about character device drivers. In this final - section, Georg deals with memory mapping devices, beginning with - an overall description of the Linux memory management concepts". - - * Title: **Network Buffers And Memory Management** - - :Author: Alan Cox. - :URL: https://www.linuxjournal.com/article.php?sid=1312 - :Date: 1996 - :Keywords: sk_buffs, network devices, protocol/link layer - variables, network devices flags, transmit, receive, - configuration, multicast. - :Description: Linux Journal Kernel Korner. - :Abstract: *Writing a network device driver for Linux is fundamentally - simple---most of the complexity (other than talking to the - hardware) involves managing network packets in memory*. - - * Title: **Analysis of the Ext2fs structure** - - :Author: Louis-Dominique Dubeau. - :URL: https://teaching.csse.uwa.edu.au/units/CITS2002/fs-ext2/ - :Date: 1994 - :Keywords: ext2, filesystem, ext2fs. - :Description: Description of ext2's blocks, directories, inodes, - bitmaps, invariants... + * Title: **Linux Kernel Debugging: Leverage proven tools and advanced techniques to effectively debug Linux kernels and kernel modules** -Published books ---------------- + :Author: Kaiwan N Billimoria + :Publisher: Packt Publishing Ltd + :Date: August, 2022 + :Pages: 638 + :ISBN: 978-1801075039 + :Notes: Debugging book - * Title: **Linux Treiber entwickeln** + * Title: **Linux Kernel Programming: A Comprehensive Guide to Kernel Internals, Writing Kernel Modules, and Kernel Synchronization** - :Author: Jürgen Quade, Eva-Katharina Kunst - :Publisher: dpunkt.verlag - :Date: Oct 2015 (4th edition) - :Pages: 688 - :ISBN: 978-3-86490-288-8 - :Note: German. The third edition from 2011 is - much cheaper and still quite up-to-date. + :Author: Kaiwan N Billimoria + :Publisher: Packt Publishing Ltd + :Date: March, 2021 (Second Edition published in 2024) + :Pages: 754 + :ISBN: 978-1789953435 (Second Edition ISBN is 978-1803232225) - * Title: **Linux Kernel Networking: Implementation and Theory** + * Title: **Linux Kernel Programming Part 2 - Char Device Drivers and Kernel Synchronization: Create user-kernel interfaces, work with peripheral I/O, and handle hardware interrupts** - :Author: Rami Rosen - :Publisher: Apress - :Date: December 22, 2013 - :Pages: 648 - :ISBN: 978-1430261964 + :Author: Kaiwan N Billimoria + :Publisher: Packt Publishing Ltd + :Date: March, 2021 + :Pages: 452 + :ISBN: 978-1801079518 - * Title: **Embedded Linux Primer: A practical Real-World Approach, 2nd Edition** + * Title: **Linux System Programming: Talking Directly to the Kernel and C Library** - :Author: Christopher Hallinan - :Publisher: Pearson - :Date: November, 2010 - :Pages: 656 - :ISBN: 978-0137017836 + :Author: Robert Love + :Publisher: O'Reilly Media + :Date: June, 2013 + :Pages: 456 + :ISBN: 978-1449339531 + :Notes: Foundational book * Title: **Linux Kernel Development, 3rd Edition** @@ -472,14 +136,7 @@ Published books :Date: July, 2010 :Pages: 440 :ISBN: 978-0672329463 - - * Title: **Essential Linux Device Drivers** - - :Author: Sreekrishnan Venkateswaran - :Published: Prentice Hall - :Date: April, 2008 - :Pages: 744 - :ISBN: 978-0132396554 + :Notes: Foundational book .. _ldd3_published: @@ -490,68 +147,10 @@ Published books :Date: 2005 :Pages: 636 :ISBN: 0-596-00590-3 - :Notes: Further information in + :Notes: Foundational book. Further information in http://www.oreilly.com/catalog/linuxdrive3/ PDF format, URL: https://lwn.net/Kernel/LDD3/ - * Title: **Linux Kernel Internals** - - :Author: Michael Beck - :Publisher: Addison-Wesley - :Date: 1997 - :ISBN: 0-201-33143-8 (second edition) - - * Title: **Programmation Linux 2.0 API systeme et fonctionnement du noyau** - - :Author: Remy Card, Eric Dumas, Franck Mevel - :Publisher: Eyrolles - :Date: 1997 - :Pages: 520 - :ISBN: 2-212-08932-5 - :Notes: French - - * Title: **The Design and Implementation of the 4.4 BSD UNIX Operating System** - - :Author: Marshall Kirk McKusick, Keith Bostic, Michael J. Karels, - John S. Quarterman - :Publisher: Addison-Wesley - :Date: 1996 - :ISBN: 0-201-54979-4 - - * Title: **Unix internals -- the new frontiers** - - :Author: Uresh Vahalia - :Publisher: Prentice Hall - :Date: 1996 - :Pages: 600 - :ISBN: 0-13-101908-2 - - * Title: **Programming for the real world - POSIX.4** - - :Author: Bill O. Gallmeister - :Publisher: O'Reilly & Associates, Inc - :Date: 1995 - :Pages: 552 - :ISBN: I-56592-074-0 - :Notes: Though not being directly about Linux, Linux aims to be - POSIX. Good reference. - - * Title: **UNIX Systems for Modern Architectures: Symmetric Multiprocessing and Caching for Kernel Programmers** - - :Author: Curt Schimmel - :Publisher: Addison Wesley - :Date: June, 1994 - :Pages: 432 - :ISBN: 0-201-63338-8 - - * Title: **The Design and Implementation of the 4.3 BSD UNIX Operating System** - - :Author: Samuel J. Leffler, Marshall Kirk McKusick, Michael J - Karels, John S. Quarterman - :Publisher: Addison-Wesley - :Date: 1989 (reprinted with corrections on October, 1990) - :ISBN: 0-201-06196-1 - * Title: **The Design of the UNIX Operating System** :Author: Maurice J. Bach @@ -559,6 +158,7 @@ Published books :Date: 1986 :Pages: 471 :ISBN: 0-13-201757-1 + :Notes: Foundational book Miscellaneous ------------- @@ -577,7 +177,7 @@ Miscellaneous :Keywords: latest kernel news. :Description: The title says it all. There's a fixed kernel section summarizing developers' work, bug fixes, new features and versions - produced during the week. Published every Thursday. + produced during the week. * Name: **The home page of Linux-MM** @@ -605,16 +205,177 @@ Miscellaneous * Name: **linux-kernel mailing list archives and search engines** - :URL: http://vger.kernel.org/vger-lists.html - :URL: http://www.uwsg.indiana.edu/hypermail/linux/kernel/index.html - :URL: http://groups.google.com/group/mlist.linux.kernel + :URL: https://subspace.kernel.org + :URL: https://lore.kernel.org :Keywords: linux-kernel, archives, search. :Description: Some of the linux-kernel mailing list archivers. If you have a better/another one, please let me know. + * Name: **The Linux Foundation YouTube channel** + + :URL: https://www.youtube.com/user/thelinuxfoundation + :Keywords: linux, videos, linux-foundation, youtube. + :Description: The Linux Foundation uploads video recordings of their + collaborative events, Linux conferences including LinuxCon, and + other original research and content related to Linux and software + development. + +Rust +---- + + * Title: **Rust for Linux** + + :Author: various + :URL: https://rust-for-linux.com/ + :Date: rolling version + :Keywords: glossary, terms, linux-kernel, rust. + :Description: From the website: "Rust for Linux is the project adding + support for the Rust language to the Linux kernel. This website is + intended as a hub of links, documentation and resources related to + the project". + + * Title: **Learn Rust the Dangerous Way** + + :Author: Cliff L. Biffle + :URL: https://cliffle.com/p/dangerust/ + :Date: Accessed Sep 11 2024 + :Keywords: rust, blog. + :Description: From the website: "LRtDW is a series of articles + putting Rust features in context for low-level C programmers who + maybe don’t have a formal CS background — the sort of people who + work on firmware, game engines, OS kernels, and the like. + Basically, people like me.". It illustrates line-by-line + conversions from C to Rust. + + * Title: **The Rust Book** + + :Author: Steve Klabnik and Carol Nichols, with contributions from the + Rust community + :URL: https://doc.rust-lang.org/book/ + :Date: Accessed Sep 11 2024 + :Keywords: rust, book. + :Description: From the website: "This book fully embraces the + potential of Rust to empower its users. It’s a friendly and + approachable text intended to help you level up not just your + knowledge of Rust, but also your reach and confidence as a + programmer in general. So dive in, get ready to learn—and welcome + to the Rust community!". + + * Title: **Rust for the Polyglot Programmer** + + :Author: Ian Jackson + :URL: https://www.chiark.greenend.org.uk/~ianmdlvl/rust-polyglot/index.html + :Date: December 2022 + :Keywords: rust, blog, tooling. + :Description: From the website: "There are many guides and + introductions to Rust. This one is something different: it is + intended for the experienced programmer who already knows many + other programming languages. I try to be comprehensive enough to be + a starting point for any area of Rust, but to avoid going into too + much detail except where things are not as you might expect. Also + this guide is not entirely free of opinion, including + recommendations of libraries (crates), tooling, etc.". + + * Title: **Fasterthanli.me** + + :Author: Amos Wenger + :URL: https://fasterthanli.me/ + :Date: Accessed Sep 11 2024 + :Keywords: rust, blog, news. + :Description: From the website: "I make articles and videos about how + computers work. My content is long-form, didactic and exploratory + — and often an excuse to teach Rust!". + + * Title: **Comprehensive Rust** + + :Author: Android team at Google + :URL: https://google.github.io/comprehensive-rust/ + :Date: Accessed Sep 13 2024 + :Keywords: rust, blog. + :Description: From the website: "The course covers the full spectrum + of Rust, from basic syntax to advanced topics like generics and + error handling". + + * Title: **The Embedded Rust Book** + + :Author: Multiple contributors, mostly Jorge Aparicio + :URL: https://docs.rust-embedded.org/book/ + :Date: Accessed Sep 13 2024 + :Keywords: rust, blog. + :Description: From the website: "An introductory book about using + the Rust Programming Language on "Bare Metal" embedded systems, + such as Microcontrollers". + + * Title: **Experiment: Improving the Rust Book** + + :Author: Cognitive Engineering Lab at Brown University + :URL: https://rust-book.cs.brown.edu/ + :Date: Accessed Sep 22 2024 + :Keywords: rust, blog. + :Description: From the website: "The goal of this experiment is to + evaluate and improve the content of the Rust Book to help people + learn Rust more effectively.". + + * Title: **New Rustacean** (podcast) + + :Author: Chris Krycho + :URL: https://newrustacean.com/ + :Date: Accessed Sep 22 2024 + :Keywords: rust, podcast. + :Description: From the website: "This is a podcast about learning + the programming language Rust—from scratch! Apart from this spiffy + landing page, all the site content is built with Rust's own + documentation tools.". + + * Title: **Opsem-team** (repository) + + :Author: Operational semantics team + :URL: https://github.com/rust-lang/opsem-team/tree/main + :Date: Accessed Sep 22 2024 + :Keywords: rust, repository. + :Description: From the README: "The opsem team is the successor of + the unsafe-code-guidelines working group and responsible for + answering many of the difficult questions about the semantics of + unsafe Rust". + + * Title: **You Can't Spell Trust Without Rust** + + :Author: Alexis Beingessner + :URL: https://repository.library.carleton.ca/downloads/1j92g820w?locale=en + :Date: 2015 + :Keywords: rust, master, thesis. + :Description: This thesis focuses on Rust's ownership system, which + ensures memory safety by controlling data manipulation and + lifetime, while also highlighting its limitations and comparing it + to similar systems in Cyclone and C++. + + * Name: **Linux Plumbers (LPC) 2024 Rust presentations** + + :Title: Rust microconference + :URL: https://lpc.events/event/18/sessions/186/#20240918 + :Title: Rust for Linux + :URL: https://lpc.events/event/18/contributions/1912/ + :Title: Journey of a C kernel engineer starting a Rust driver project + :URL: https://lpc.events/event/18/contributions/1911/ + :Title: Crafting a Linux kernel scheduler that runs in user-space + using Rust + :URL: https://lpc.events/event/18/contributions/1723/ + :Title: openHCL: A Linux and Rust based paravisor + :URL: https://lpc.events/event/18/contributions/1956/ + :Keywords: rust, lpc, presentations. + :Description: A number of LPC talks related to Rust. + + * Name: **The Rustacean Station Podcast** + + :URL: https://rustacean-station.org/ + :Keywords: rust, podcasts. + :Description: A community project for creating podcast content for + the Rust programming language. + ------- -Document last updated on Tue 2016-Sep-20 +This document was originally based on: -This document is based on: https://www.dit.upm.es/~jmseyas/linux/kernel/hackers-docs.html + +and written by Juan-Mariano de Goyeneche diff --git a/Documentation/process/license-rules.rst b/Documentation/process/license-rules.rst index 2ef44ada3f11..59a7832df7d0 100644 --- a/Documentation/process/license-rules.rst +++ b/Documentation/process/license-rules.rst @@ -471,14 +471,16 @@ _`MODULE_LICENSE` source files. "Proprietary" The module is under a proprietary license. - This string is solely for proprietary third - party modules and cannot be used for modules - which have their source code in the kernel - tree. Modules tagged that way are tainting - the kernel with the 'P' flag when loaded and - the kernel module loader refuses to link such - modules against symbols which are exported - with EXPORT_SYMBOL_GPL(). + "Proprietary" is to be understood only as + "The license is not compatible to GPLv2". + This string is solely for non-GPL2 compatible + third party modules and cannot be used for + modules which have their source code in the + kernel tree. Modules tagged that way are + tainting the kernel with the 'P' flag when + loaded and the kernel module loader refuses + to link such modules against symbols which + are exported with EXPORT_SYMBOL_GPL(). ============================= ============================================= diff --git a/Documentation/process/magic-number.rst b/Documentation/process/magic-number.rst deleted file mode 100644 index 64b5948fc1d4..000000000000 --- a/Documentation/process/magic-number.rst +++ /dev/null @@ -1,85 +0,0 @@ -.. _magicnumbers: - -Linux magic numbers -=================== - -This file is a registry of magic numbers which are in use. When you -add a magic number to a structure, you should also add it to this -file, since it is best if the magic numbers used by various structures -are unique. - -It is a **very** good idea to protect kernel data structures with magic -numbers. This allows you to check at run time whether (a) a structure -has been clobbered, or (b) you've passed the wrong structure to a -routine. This last is especially useful --- particularly when you are -passing pointers to structures via a void * pointer. The tty code, -for example, does this frequently to pass driver-specific and line -discipline-specific structures back and forth. - -The way to use magic numbers is to declare them at the beginning of -the structure, like so:: - - struct tty_ldisc { - int magic; - ... - }; - -Please follow this discipline when you are adding future enhancements -to the kernel! It has saved me countless hours of debugging, -especially in the screwy cases where an array has been overrun and -structures following the array have been overwritten. Using this -discipline, these cases get detected quickly and safely. - -Changelog:: - - Theodore Ts'o - 31 Mar 94 - - The magic table is current to Linux 2.1.55. - - Michael Chastain - <mailto:mec@shout.net> - 22 Sep 1997 - - Now it should be up to date with Linux 2.1.112. Because - we are in feature freeze time it is very unlikely that - something will change before 2.2.x. The entries are - sorted by number field. - - Krzysztof G. Baranowski - <mailto: kgb@knm.org.pl> - 29 Jul 1998 - - Updated the magic table to Linux 2.5.45. Right over the feature freeze, - but it is possible that some new magic numbers will sneak into the - kernel before 2.6.x yet. - - Petr Baudis - <pasky@ucw.cz> - 03 Nov 2002 - - Updated the magic table to Linux 2.5.74. - - Fabian Frederick - <ffrederick@users.sourceforge.net> - 09 Jul 2003 - - -===================== ================ ======================== ========================================== -Magic Name Number Structure File -===================== ================ ======================== ========================================== -PG_MAGIC 'P' pg_{read,write}_hdr ``include/linux/pg.h`` -APM_BIOS_MAGIC 0x4101 apm_user ``arch/x86/kernel/apm_32.c`` -FASYNC_MAGIC 0x4601 fasync_struct ``include/linux/fs.h`` -SLIP_MAGIC 0x5302 slip ``drivers/net/slip.h`` -MGSLPC_MAGIC 0x5402 mgslpc_info ``drivers/char/pcmcia/synclink_cs.c`` -BAYCOM_MAGIC 0x19730510 baycom_state ``drivers/net/baycom_epp.c`` -HDLCDRV_MAGIC 0x5ac6e778 hdlcdrv_state ``include/linux/hdlcdrv.h`` -KV_MAGIC 0x5f4b565f kernel_vars_s ``arch/mips/include/asm/sn/klkernvars.h`` -CODA_MAGIC 0xC0DAC0DA coda_file_info ``fs/coda/coda_fs_i.h`` -YAM_MAGIC 0xF10A7654 yam_port ``drivers/net/hamradio/yam.c`` -CCB_MAGIC 0xf2691ad2 ccb ``drivers/scsi/ncr53c8xx.c`` -QUEUE_MAGIC_FREE 0xf7e1c9a3 queue_entry ``drivers/scsi/arm/queue.c`` -QUEUE_MAGIC_USED 0xf7e1cc33 queue_entry ``drivers/scsi/arm/queue.c`` -NMI_MAGIC 0x48414d4d455201 nmi_s ``arch/mips/include/asm/sn/nmi.h`` -===================== ================ ======================== ========================================== diff --git a/Documentation/process/maintainer-handbooks.rst b/Documentation/process/maintainer-handbooks.rst index d783060b4cc6..976391cec528 100644 --- a/Documentation/process/maintainer-handbooks.rst +++ b/Documentation/process/maintainer-handbooks.rst @@ -15,5 +15,8 @@ Contents: :numbered: :maxdepth: 2 - maintainer-tip maintainer-netdev + maintainer-soc + maintainer-soc-clean-dts + maintainer-tip + maintainer-kvm-x86 diff --git a/Documentation/process/maintainer-kvm-x86.rst b/Documentation/process/maintainer-kvm-x86.rst new file mode 100644 index 000000000000..9183bd449762 --- /dev/null +++ b/Documentation/process/maintainer-kvm-x86.rst @@ -0,0 +1,390 @@ +.. SPDX-License-Identifier: GPL-2.0 + +KVM x86 +======= + +Foreword +-------- +KVM strives to be a welcoming community; contributions from newcomers are +valued and encouraged. Please do not be discouraged or intimidated by the +length of this document and the many rules/guidelines it contains. Everyone +makes mistakes, and everyone was a newbie at some point. So long as you make +an honest effort to follow KVM x86's guidelines, are receptive to feedback, +and learn from any mistakes you make, you will be welcomed with open arms, not +torches and pitchforks. + +TL;DR +----- +Testing is mandatory. Be consistent with established styles and patterns. + +Trees +----- +KVM x86 is currently in a transition period from being part of the main KVM +tree, to being "just another KVM arch". As such, KVM x86 is split across the +main KVM tree, ``git.kernel.org/pub/scm/virt/kvm/kvm.git``, and a KVM x86 +specific tree, ``github.com/kvm-x86/linux.git``. + +Generally speaking, fixes for the current cycle are applied directly to the +main KVM tree, while all development for the next cycle is routed through the +KVM x86 tree. In the unlikely event that a fix for the current cycle is routed +through the KVM x86 tree, it will be applied to the ``fixes`` branch before +making its way to the main KVM tree. + +Note, this transition period is expected to last quite some time, i.e. will be +the status quo for the foreseeable future. + +Branches +~~~~~~~~ +The KVM x86 tree is organized into multiple topic branches. The purpose of +using finer-grained topic branches is to make it easier to keep tabs on an area +of development, and to limit the collateral damage of human errors and/or buggy +commits, e.g. dropping the HEAD commit of a topic branch has no impact on other +in-flight commits' SHA1 hashes, and having to reject a pull request due to bugs +delays only that topic branch. + +All topic branches, except for ``next`` and ``fixes``, are rolled into ``next`` +via a Cthulhu merge on an as-needed basis, i.e. when a topic branch is updated. +As a result, force pushes to ``next`` are common. + +Lifecycle +~~~~~~~~~ +Fixes that target the current release, a.k.a. mainline, are typically applied +directly to the main KVM tree, i.e. do not route through the KVM x86 tree. + +Changes that target the next release are routed through the KVM x86 tree. Pull +requests (from KVM x86 to main KVM) are sent for each KVM x86 topic branch, +typically the week before Linus' opening of the merge window, e.g. the week +following rc7 for "normal" releases. If all goes well, the topic branches are +rolled into the main KVM pull request sent during Linus' merge window. + +The KVM x86 tree doesn't have its own official merge window, but there's a soft +close around rc5 for new features, and a soft close around rc6 for fixes (for +the next release; see above for fixes that target the current release). + +Timeline +~~~~~~~~ +Submissions are typically reviewed and applied in FIFO order, with some wiggle +room for the size of a series, patches that are "cache hot", etc. Fixes, +especially for the current release and or stable trees, get to jump the queue. +Patches that will be taken through a non-KVM tree (most often through the tip +tree) and/or have other acks/reviews also jump the queue to some extent. + +Note, the vast majority of review is done between rc1 and rc6, give or take. +The period between rc6 and the next rc1 is used to catch up on other tasks, +i.e. radio silence during this period isn't unusual. + +Pings to get a status update are welcome, but keep in mind the timing of the +current release cycle and have realistic expectations. If you are pinging for +acceptance, i.e. not just for feedback or an update, please do everything you +can, within reason, to ensure that your patches are ready to be merged! Pings +on series that break the build or fail tests lead to unhappy maintainers! + +Development +----------- + +Base Tree/Branch +~~~~~~~~~~~~~~~~ +Fixes that target the current release, a.k.a. mainline, should be based on +``git://git.kernel.org/pub/scm/virt/kvm/kvm.git master``. Note, fixes do not +automatically warrant inclusion in the current release. There is no singular +rule, but typically only fixes for bugs that are urgent, critical, and/or were +introduced in the current release should target the current release. + +Everything else should be based on ``kvm-x86/next``, i.e. there is no need to +select a specific topic branch as the base. If there are conflicts and/or +dependencies across topic branches, it is the maintainer's job to sort them +out. + +The only exception to using ``kvm-x86/next`` as the base is if a patch/series +is a multi-arch series, i.e. has non-trivial modifications to common KVM code +and/or has more than superficial changes to other architectures' code. Multi- +arch patch/series should instead be based on a common, stable point in KVM's +history, e.g. the release candidate upon which ``kvm-x86 next`` is based. If +you're unsure whether a patch/series is truly multi-arch, err on the side of +caution and treat it as multi-arch, i.e. use a common base. + +Coding Style +~~~~~~~~~~~~ +When it comes to style, naming, patterns, etc., consistency is the number one +priority in KVM x86. If all else fails, match what already exists. + +With a few caveats listed below, follow the tip tree maintainers' preferred +:ref:`maintainer-tip-coding-style`, as patches/series often touch both KVM and +non-KVM x86 files, i.e. draw the attention of KVM *and* tip tree maintainers. + +Using reverse fir tree, a.k.a. reverse Christmas tree or reverse XMAS tree, for +variable declarations isn't strictly required, though it is still preferred. + +Except for a handful of special snowflakes, do not use kernel-doc comments for +functions. The vast majority of "public" KVM functions aren't truly public as +they are intended only for KVM-internal consumption (there are plans to +privatize KVM's headers and exports to enforce this). + +Comments +~~~~~~~~ +Write comments using imperative mood and avoid pronouns. Use comments to +provide a high level overview of the code, and/or to explain why the code does +what it does. Do not reiterate what the code literally does; let the code +speak for itself. If the code itself is inscrutable, comments will not help. + +SDM and APM References +~~~~~~~~~~~~~~~~~~~~~~ +Much of KVM's code base is directly tied to architectural behavior defined in +Intel's Software Development Manual (SDM) and AMD's Architecture Programmer’s +Manual (APM). Use of "Intel's SDM" and "AMD's APM", or even just "SDM" or +"APM", without additional context is a-ok. + +Do not reference specific sections, tables, figures, etc. by number, especially +not in comments. Instead, if necessary (see below), copy-paste the relevant +snippet and reference sections/tables/figures by name. The layouts of the SDM +and APM are constantly changing, and so the numbers/labels aren't stable. + +Generally speaking, do not explicitly reference or copy-paste from the SDM or +APM in comments. With few exceptions, KVM *must* honor architectural behavior, +therefore it's implied that KVM behavior is emulating SDM and/or APM behavior. +Note, referencing the SDM/APM in changelogs to justify the change and provide +context is perfectly ok and encouraged. + +Shortlog +~~~~~~~~ +The preferred prefix format is ``KVM: <topic>:``, where ``<topic>`` is one of:: + + - x86 + - x86/mmu + - x86/pmu + - x86/xen + - selftests + - SVM + - nSVM + - VMX + - nVMX + +**DO NOT use x86/kvm!** ``x86/kvm`` is used exclusively for Linux-as-a-KVM-guest +changes, i.e. for arch/x86/kernel/kvm.c. Do not use file names or complete file +paths as the subject/shortlog prefix. + +Note, these don't align with the topics branches (the topic branches care much +more about code conflicts). + +All names are case sensitive! ``KVM: x86:`` is good, ``kvm: vmx:`` is not. + +Capitalize the first word of the condensed patch description, but omit ending +punctionation. E.g.:: + + KVM: x86: Fix a null pointer dereference in function_xyz() + +not:: + + kvm: x86: fix a null pointer dereference in function_xyz. + +If a patch touches multiple topics, traverse up the conceptual tree to find the +first common parent (which is often simply ``x86``). When in doubt, +``git log path/to/file`` should provide a reasonable hint. + +New topics do occasionally pop up, but please start an on-list discussion if +you want to propose introducing a new topic, i.e. don't go rogue. + +See :ref:`the_canonical_patch_format` for more information, with one amendment: +do not treat the 70-75 character limit as an absolute, hard limit. Instead, +use 75 characters as a firm-but-not-hard limit, and use 80 characters as a hard +limit. I.e. let the shortlog run a few characters over the standard limit if +you have good reason to do so. + +Changelog +~~~~~~~~~ +Most importantly, write changelogs using imperative mood and avoid pronouns. + +See :ref:`describe_changes` for more information, with one amendment: lead with +a short blurb on the actual changes, and then follow up with the context and +background. Note! This order directly conflicts with the tip tree's preferred +approach! Please follow the tip tree's preferred style when sending patches +that primarily target arch/x86 code that is _NOT_ KVM code. + +Stating what a patch does before diving into details is preferred by KVM x86 +for several reasons. First and foremost, what code is actually being changed +is arguably the most important information, and so that info should be easy to +find. Changelogs that bury the "what's actually changing" in a one-liner after +3+ paragraphs of background make it very hard to find that information. + +For initial review, one could argue the "what's broken" is more important, but +for skimming logs and git archaeology, the gory details matter less and less. +E.g. when doing a series of "git blame", the details of each change along the +way are useless, the details only matter for the culprit. Providing the "what +changed" makes it easy to quickly determine whether or not a commit might be of +interest. + +Another benefit of stating "what's changing" first is that it's almost always +possible to state "what's changing" in a single sentence. Conversely, all but +the most simple bugs require multiple sentences or paragraphs to fully describe +the problem. If both the "what's changing" and "what's the bug" are super +short then the order doesn't matter. But if one is shorter (almost always the +"what's changing), then covering the shorter one first is advantageous because +it's less of an inconvenience for readers/reviewers that have a strict ordering +preference. E.g. having to skip one sentence to get to the context is less +painful than having to skip three paragraphs to get to "what's changing". + +Fixes +~~~~~ +If a change fixes a KVM/kernel bug, add a Fixes: tag even if the change doesn't +need to be backported to stable kernels, and even if the change fixes a bug in +an older release. + +Conversely, if a fix does need to be backported, explicitly tag the patch with +"Cc: stable@vger.kernel" (though the email itself doesn't need to Cc: stable); +KVM x86 opts out of backporting Fixes: by default. Some auto-selected patches +do get backported, but require explicit maintainer approval (search MANUALSEL). + +Function References +~~~~~~~~~~~~~~~~~~~ +When a function is mentioned in a comment, changelog, or shortlog (or anywhere +for that matter), use the format ``function_name()``. The parentheses provide +context and disambiguate the reference. + +Testing +------- +At a bare minimum, *all* patches in a series must build cleanly for KVM_INTEL=m +KVM_AMD=m, and KVM_WERROR=y. Building every possible combination of Kconfigs +isn't feasible, but the more the merrier. KVM_SMM, KVM_XEN, PROVE_LOCKING, and +X86_64 are particularly interesting knobs to turn. + +Running KVM selftests and KVM-unit-tests is also mandatory (and stating the +obvious, the tests need to pass). The only exception is for changes that have +negligible probability of affecting runtime behavior, e.g. patches that only +modify comments. When possible and relevant, testing on both Intel and AMD is +strongly preferred. Booting an actual VM is encouraged, but not mandatory. + +For changes that touch KVM's shadow paging code, running with TDP (EPT/NPT) +disabled is mandatory. For changes that affect common KVM MMU code, running +with TDP disabled is strongly encouraged. For all other changes, if the code +being modified depends on and/or interacts with a module param, testing with +the relevant settings is mandatory. + +Note, KVM selftests and KVM-unit-tests do have known failures. If you suspect +a failure is not due to your changes, verify that the *exact same* failure +occurs with and without your changes. + +Changes that touch reStructured Text documentation, i.e. .rst files, must build +htmldocs cleanly, i.e. with no new warnings or errors. + +If you can't fully test a change, e.g. due to lack of hardware, clearly state +what level of testing you were able to do, e.g. in the cover letter. + +New Features +~~~~~~~~~~~~ +With one exception, new features *must* come with test coverage. KVM specific +tests aren't strictly required, e.g. if coverage is provided by running a +sufficiently enabled guest VM, or by running a related kernel selftest in a VM, +but dedicated KVM tests are preferred in all cases. Negative testcases in +particular are mandatory for enabling of new hardware features as error and +exception flows are rarely exercised simply by running a VM. + +The only exception to this rule is if KVM is simply advertising support for a +feature via KVM_GET_SUPPORTED_CPUID, i.e. for instructions/features that KVM +can't prevent a guest from using and for which there is no true enabling. + +Note, "new features" does not just mean "new hardware features"! New features +that can't be well validated using existing KVM selftests and/or KVM-unit-tests +must come with tests. + +Posting new feature development without tests to get early feedback is more +than welcome, but such submissions should be tagged RFC, and the cover letter +should clearly state what type of feedback is requested/expected. Do not abuse +the RFC process; RFCs will typically not receive in-depth review. + +Bug Fixes +~~~~~~~~~ +Except for "obvious" found-by-inspection bugs, fixes must be accompanied by a +reproducer for the bug being fixed. In many cases the reproducer is implicit, +e.g. for build errors and test failures, but it should still be clear to +readers what is broken and how to verify the fix. Some leeway is given for +bugs that are found via non-public workloads/tests, but providing regression +tests for such bugs is strongly preferred. + +In general, regression tests are preferred for any bug that is not trivial to +hit. E.g. even if the bug was originally found by a fuzzer such as syzkaller, +a targeted regression test may be warranted if the bug requires hitting a +one-in-a-million type race condition. + +Note, KVM bugs are rarely urgent *and* non-trivial to reproduce. Ask yourself +if a bug is really truly the end of the world before posting a fix without a +reproducer. + +Posting +------- + +Links +~~~~~ +Do not explicitly reference bug reports, prior versions of a patch/series, etc. +via ``In-Reply-To:`` headers. Using ``In-Reply-To:`` becomes an unholy mess +for large series and/or when the version count gets high, and ``In-Reply-To:`` +is useless for anyone that doesn't have the original message, e.g. if someone +wasn't Cc'd on the bug report or if the list of recipients changes between +versions. + +To link to a bug report, previous version, or anything of interest, use lore +links. For referencing previous version(s), generally speaking do not include +a Link: in the changelog as there is no need to record the history in git, i.e. +put the link in the cover letter or in the section git ignores. Do provide a +formal Link: for bug reports and/or discussions that led to the patch. The +context of why a change was made is highly valuable for future readers. + +Git Base +~~~~~~~~ +If you are using git version 2.9.0 or later (Googlers, this is all of you!), +use ``git format-patch`` with the ``--base`` flag to automatically include the +base tree information in the generated patches. + +Note, ``--base=auto`` works as expected if and only if a branch's upstream is +set to the base topic branch, e.g. it will do the wrong thing if your upstream +is set to your personal repository for backup purposes. An alternative "auto" +solution is to derive the names of your development branches based on their +KVM x86 topic, and feed that into ``--base``. E.g. ``x86/pmu/my_branch_name``, +and then write a small wrapper to extract ``pmu`` from the current branch name +to yield ``--base=x/pmu``, where ``x`` is whatever name your repository uses to +track the KVM x86 remote. + +Co-Posting Tests +~~~~~~~~~~~~~~~~ +KVM selftests that are associated with KVM changes, e.g. regression tests for +bug fixes, should be posted along with the KVM changes as a single series. The +standard kernel rules for bisection apply, i.e. KVM changes that result in test +failures should be ordered after the selftests updates, and vice versa, new +tests that fail due to KVM bugs should be ordered after the KVM fixes. + +KVM-unit-tests should *always* be posted separately. Tools, e.g. b4 am, don't +know that KVM-unit-tests is a separate repository and get confused when patches +in a series apply on different trees. To tie KVM-unit-tests patches back to +KVM patches, first post the KVM changes and then provide a lore Link: to the +KVM patch/series in the KVM-unit-tests patch(es). + +Notifications +------------- +When a patch/series is officially accepted, a notification email will be sent +in reply to the original posting (cover letter for multi-patch series). The +notification will include the tree and topic branch, along with the SHA1s of +the commits of applied patches. + +If a subset of patches is applied, this will be clearly stated in the +notification. Unless stated otherwise, it's implied that any patches in the +series that were not accepted need more work and should be submitted in a new +version. + +If for some reason a patch is dropped after officially being accepted, a reply +will be sent to the notification email explaining why the patch was dropped, as +well as the next steps. + +SHA1 Stability +~~~~~~~~~~~~~~ +SHA1s are not 100% guaranteed to be stable until they land in Linus' tree! A +SHA1 is *usually* stable once a notification has been sent, but things happen. +In most cases, an update to the notification email be provided if an applied +patch's SHA1 changes. However, in some scenarios, e.g. if all KVM x86 branches +need to be rebased, individual notifications will not be given. + +Vulnerabilities +--------------- +Bugs that can be exploited by the guest to attack the host (kernel or +userspace), or that can be exploited by a nested VM to *its* host (L2 attacking +L1), are of particular interest to KVM. Please follow the protocol for +:ref:`securitybugs` if you suspect a bug can lead to an escape, data leak, etc. + diff --git a/Documentation/process/maintainer-netdev.rst b/Documentation/process/maintainer-netdev.rst index 1fa5ab8754d3..989192421cc9 100644 --- a/Documentation/process/maintainer-netdev.rst +++ b/Documentation/process/maintainer-netdev.rst @@ -2,9 +2,9 @@ .. _netdev-FAQ: -========== -netdev FAQ -========== +============================= +Networking subsystem (netdev) +============================= tl;dr ----- @@ -15,49 +15,27 @@ tl;dr - don't repost your patches within one 24h period - reverse xmas tree -What is netdev? ---------------- -It is a mailing list for all network-related Linux stuff. This +netdev +------ + +netdev is a mailing list for all network-related Linux stuff. This includes anything found under net/ (i.e. core code like IPv6) and drivers/net (i.e. hardware specific drivers) in the Linux source tree. Note that some subsystems (e.g. wireless drivers) which have a high -volume of traffic have their own specific mailing lists. +volume of traffic have their own specific mailing lists and trees. -The netdev list is managed (like many other Linux mailing lists) through -VGER (http://vger.kernel.org/) with archives available at -https://lore.kernel.org/netdev/ +Like many other Linux mailing lists, the netdev list is hosted at +kernel.org with archives available at https://lore.kernel.org/netdev/. Aside from subsystems like those mentioned above, all network-related Linux development (i.e. RFC, review, comments, etc.) takes place on netdev. -How do the changes posted to netdev make their way into Linux? --------------------------------------------------------------- -There are always two trees (git repositories) in play. Both are -driven by David Miller, the main network maintainer. There is the -``net`` tree, and the ``net-next`` tree. As you can probably guess from -the names, the ``net`` tree is for fixes to existing code already in the -mainline tree from Linus, and ``net-next`` is where the new code goes -for the future release. You can find the trees here: - -- https://git.kernel.org/pub/scm/linux/kernel/git/netdev/net.git -- https://git.kernel.org/pub/scm/linux/kernel/git/netdev/net-next.git - -How do I indicate which tree (net vs. net-next) my patch should be in? ----------------------------------------------------------------------- -To help maintainers and CI bots you should explicitly mark which tree -your patch is targeting. Assuming that you use git, use the prefix -flag:: - - git format-patch --subject-prefix='PATCH net-next' start..finish - -Use ``net`` instead of ``net-next`` (always lower case) in the above for -bug-fix ``net`` content. +Development cycle +----------------- -How often do changes from these trees make it to the mainline Linus tree? -------------------------------------------------------------------------- -To understand this, you need to know a bit of background information on +Here is a bit of background information on the cadence of Linux development. Each new release starts off with a two week "merge window" where the main maintainers feed their new stuff to Linus for merging into the mainline tree. After the two weeks, the @@ -69,9 +47,33 @@ rc2 is released. This repeats on a roughly weekly basis until rc7 state of churn), and a week after the last vX.Y-rcN was done, the official vX.Y is released. -Relating that to netdev: At the beginning of the 2-week merge window, -the ``net-next`` tree will be closed - no new changes/features. The -accumulated new content of the past ~10 weeks will be passed onto +To find out where we are now in the cycle - load the mainline (Linus) +page here: + + https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git + +and note the top of the "tags" section. If it is rc1, it is early in +the dev cycle. If it was tagged rc7 a week ago, then a release is +probably imminent. If the most recent tag is a final release tag +(without an ``-rcN`` suffix) - we are most likely in a merge window +and ``net-next`` is closed. + +git trees and patch flow +------------------------ + +There are two networking trees (git repositories) in play. Both are +driven by David Miller, the main network maintainer. There is the +``net`` tree, and the ``net-next`` tree. As you can probably guess from +the names, the ``net`` tree is for fixes to existing code already in the +mainline tree from Linus, and ``net-next`` is where the new code goes +for the future release. You can find the trees here: + +- https://git.kernel.org/pub/scm/linux/kernel/git/netdev/net.git +- https://git.kernel.org/pub/scm/linux/kernel/git/netdev/net-next.git + +Relating that to kernel development: At the beginning of the 2-week +merge window, the ``net-next`` tree will be closed - no new changes/features. +The accumulated new content of the past ~10 weeks will be passed onto mainline/Linus via a pull request for vX.Y -- at the same time, the ``net`` tree will start accumulating fixes for this pulled content relating to vX.Y @@ -95,7 +97,7 @@ If you aren't subscribed to netdev and/or are simply unsure if repository link above for any new networking-related commits. You may also check the following website for the current status: - http://vger.kernel.org/~davem/net-next.html + https://netdev.bots.linux.dev/net-next.html The ``net`` tree continues to collect fixes for the vX.Y content, and is fed back to Linus at regular (~weekly) intervals. Meaning that the @@ -103,149 +105,188 @@ focus for ``net`` is on stabilization and bug fixes. Finally, the vX.Y gets released, and the whole cycle starts over. -So where are we now in this cycle? ----------------------------------- +netdev patch review +------------------- -Load the mainline (Linus) page here: - - https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git +.. _patch_status: -and note the top of the "tags" section. If it is rc1, it is early in -the dev cycle. If it was tagged rc7 a week ago, then a release is -probably imminent. If the most recent tag is a final release tag -(without an ``-rcN`` suffix) - we are most likely in a merge window -and ``net-next`` is closed. +Patch status +~~~~~~~~~~~~ -How can I tell the status of a patch I've sent? ------------------------------------------------ -Start by looking at the main patchworks queue for netdev: +Status of a patch can be checked by looking at the main patchwork +queue for netdev: https://patchwork.kernel.org/project/netdevbpf/list/ The "State" field will tell you exactly where things are at with your -patch. Patches are indexed by the ``Message-ID`` header of the emails +patch: + +================== ============================================================= +Patch state Description +================== ============================================================= +New, Under review pending review, patch is in the maintainer’s queue for + review; the two states are used interchangeably (depending on + the exact co-maintainer handling patchwork at the time) +Accepted patch was applied to the appropriate networking tree, this is + usually set automatically by the pw-bot +Needs ACK waiting for an ack from an area expert or testing +Changes requested patch has not passed the review, new revision is expected + with appropriate code and commit message changes +Rejected patch has been rejected and new revision is not expected +Not applicable patch is expected to be applied outside of the networking + subsystem +Awaiting upstream patch should be reviewed and handled by appropriate + sub-maintainer, who will send it on to the networking trees; + patches set to ``Awaiting upstream`` in netdev's patchwork + will usually remain in this state, whether the sub-maintainer + requested changes, accepted or rejected the patch +Deferred patch needs to be reposted later, usually due to dependency + or because it was posted for a closed tree +Superseded new version of the patch was posted, usually set by the + pw-bot +RFC not to be applied, usually not in maintainer’s review queue, + pw-bot can automatically set patches to this state based + on subject tags +================== ============================================================= + +Patches are indexed by the ``Message-ID`` header of the emails which carried them so if you have trouble finding your patch append the value of ``Message-ID`` to the URL above. -How long before my patch is accepted? -------------------------------------- +Updating patch status +~~~~~~~~~~~~~~~~~~~~~ + +Contributors and reviewers do not have the permissions to update patch +state directly in patchwork. Patchwork doesn't expose much information +about the history of the state of patches, therefore having multiple +people update the state leads to confusion. + +Instead of delegating patchwork permissions netdev uses a simple mail +bot which looks for special commands/lines within the emails sent to +the mailing list. For example to mark a series as Changes Requested +one needs to send the following line anywhere in the email thread:: + + pw-bot: changes-requested + +As a result the bot will set the entire series to Changes Requested. +This may be useful when author discovers a bug in their own series +and wants to prevent it from getting applied. + +The use of the bot is entirely optional, if in doubt ignore its existence +completely. Maintainers will classify and update the state of the patches +themselves. No email should ever be sent to the list with the main purpose +of communicating with the bot, the bot commands should be seen as metadata. + +The use of the bot is restricted to authors of the patches (the ``From:`` +header on patch submission and command must match!), maintainers of +the modified code according to the MAINTAINERS file (again, ``From:`` +must match the MAINTAINERS entry) and a handful of senior reviewers. + +Bot records its activity here: + + https://netdev.bots.linux.dev/pw-bot.html + +Review timelines +~~~~~~~~~~~~~~~~ + Generally speaking, the patches get triaged quickly (in less than 48h). But be patient, if your patch is active in patchwork (i.e. it's listed on the project's patch list) the chances it was missed are close to zero. -Asking the maintainer for status updates on your -patch is a good way to ensure your patch is ignored or pushed to the -bottom of the priority list. - -Should I directly update patchwork state of my own patches? ------------------------------------------------------------ -It may be tempting to help the maintainers and update the state of your -own patches when you post a new version or spot a bug. Please do not do that. -Interfering with the patch status on patchwork will only cause confusion. Leave -it to the maintainer to figure out what is the most recent and current -version that should be applied. If there is any doubt, the maintainer -will reply and ask what should be done. - -How do I divide my work into patches? -------------------------------------- -Put yourself in the shoes of the reviewer. Each patch is read separately -and therefore should constitute a comprehensible step towards your stated -goal. +The high volume of development on netdev makes reviewers move on +from discussions relatively quickly. New comments and replies +are very unlikely to arrive after a week of silence. If a patch +is no longer active in patchwork and the thread went idle for more +than a week - clarify the next steps and/or post the next version. -Avoid sending series longer than 15 patches. Larger series takes longer -to review as reviewers will defer looking at it until they find a large -chunk of time. A small series can be reviewed in a short time, so Maintainers -just do it. As a result, a sequence of smaller series gets merged quicker and -with better review coverage. Re-posting large series also increases the mailing -list traffic. +For RFC postings specifically, if nobody responded in a week - reviewers +either missed the posting or have no strong opinions. If the code is ready, +repost as a PATCH. + +Emails saying just "ping" or "bump" are considered rude. If you can't figure +out the status of the patch from patchwork or where the discussion has +landed - describe your best guess and ask if it's correct. For example:: + + I don't understand what the next steps are. Person X seems to be unhappy + with A, should I do B and repost the patches? + +.. _Changes requested: + +Changes requested +~~~~~~~~~~~~~~~~~ + +Patches :ref:`marked<patch_status>` as ``Changes Requested`` need +to be revised. The new version should come with a change log, +preferably including links to previous postings, for example:: + + [PATCH net-next v3] net: make cows go moo + + Even users who don't drink milk appreciate hearing the cows go "moo". -I made changes to only a few patches in a patch series should I resend only those changed? ------------------------------------------------------------------------------------------- -No, please resend the entire patch series and make sure you do number your + The amount of mooing will depend on packet rate so should match + the diurnal cycle quite well. + + Signed-off-by: Joe Defarmer <joe@barn.org> + --- + v3: + - add a note about time-of-day mooing fluctuation to the commit message + v2: https://lore.kernel.org/netdev/123themessageid@barn.org/ + - fix missing argument in kernel doc for netif_is_bovine() + - fix memory leak in netdev_register_cow() + v1: https://lore.kernel.org/netdev/456getstheclicks@barn.org/ + +The commit message should be revised to answer any questions reviewers +had to ask in previous discussions. Occasionally the update of +the commit message will be the only change in the new version. + +Partial resends +~~~~~~~~~~~~~~~ + +Please always resend the entire patch series and make sure you do number your patches such that it is clear this is the latest and greatest set of patches -that can be applied. +that can be applied. Do not try to resend just the patches which changed. -I have received review feedback, when should I post a revised version of the patches? -------------------------------------------------------------------------------------- -Allow at least 24 hours to pass between postings. This will ensure reviewers -from all geographical locations have a chance to chime in. Do not wait -too long (weeks) between postings either as it will make it harder for reviewers -to recall all the context. +Handling misapplied patches +~~~~~~~~~~~~~~~~~~~~~~~~~~~ -Make sure you address all the feedback in your new posting. Do not post a new -version of the code if the discussion about the previous version is still -ongoing, unless directly instructed by a reviewer. +Occasionally a patch series gets applied before receiving critical feedback, +or the wrong version of a series gets applied. -I submitted multiple versions of a patch series and it looks like a version other than the last one has been accepted, what should I do? ----------------------------------------------------------------------------------------------------------------------------------------- -There is no revert possible, once it is pushed out, it stays like that. +Making the patch disappear once it is pushed out is not possible, the commit +history in netdev trees is immutable. Please send incremental versions on top of what has been merged in order to fix the patches the way they would look like if your latest patch series was to be merged. -Are there special rules regarding stable submissions on netdev? ---------------------------------------------------------------- +In cases where full revert is needed the revert has to be submitted +as a patch to the list with a commit message explaining the technical +problems with the reverted commit. Reverts should be used as a last resort, +when original change is completely wrong; incremental fixes are preferred. + +Stable tree +~~~~~~~~~~~ + While it used to be the case that netdev submissions were not supposed to carry explicit ``CC: stable@vger.kernel.org`` tags that is no longer the case today. Please follow the standard stable rules in :ref:`Documentation/process/stable-kernel-rules.rst <stable_kernel_rules>`, and make sure you include appropriate Fixes tags! -Is the comment style convention different for the networking content? ---------------------------------------------------------------------- -Yes, in a largely trivial way. Instead of this:: - - /* - * foobar blah blah blah - * another line of text - */ +Security fixes +~~~~~~~~~~~~~~ -it is requested that you make it look like this:: - - /* foobar blah blah blah - * another line of text - */ - -What is "reverse xmas tree"? ----------------------------- - -Netdev has a convention for ordering local variables in functions. -Order the variable declaration lines longest to shortest, e.g.:: - - struct scatterlist *sg; - struct sk_buff *skb; - int err, i; - -If there are dependencies between the variables preventing the ordering -move the initialization out of line. - -I am working in existing code which uses non-standard formatting. Which formatting should I use? ------------------------------------------------------------------------------------------------- -Make your code follow the most recent guidelines, so that eventually all code -in the domain of netdev is in the preferred format. - -I found a bug that might have possible security implications or similar. Should I mail the main netdev maintainer off-list? ---------------------------------------------------------------------------------------------------------------------------- -No. The current netdev maintainer has consistently requested that +Do not email netdev maintainers directly if you think you discovered +a bug that might have possible security implications. +The current netdev maintainer has consistently requested that people use the mailing lists and not reach out directly. If you aren't OK with that, then perhaps consider mailing security@kernel.org or reading about http://oss-security.openwall.org/wiki/mailing-lists/distros as possible alternative mechanisms. -What level of testing is expected before I submit my change? ------------------------------------------------------------- -At the very minimum your changes must survive an ``allyesconfig`` and an -``allmodconfig`` build with ``W=1`` set without new warnings or failures. -Ideally you will have done run-time testing specific to your change, -and the patch series contains a set of kernel selftest for -``tools/testing/selftests/net`` or using the KUnit framework. +Co-posting changes to user space components +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -You are expected to test your changes on top of the relevant networking -tree (``net`` or ``net-next``) and not e.g. a stable tree or ``linux-next``. - -How do I post corresponding changes to user space components? -------------------------------------------------------------- User space code exercising kernel features should be posted alongside kernel patches. This gives reviewers a chance to see how any new interface is used and how well it works. @@ -270,42 +311,18 @@ to the mailing list, e.g.:: Posting as one thread is discouraged because it confuses patchwork (as of patchwork 2.2.2). -Can I reproduce the checks from patchwork on my local machine? --------------------------------------------------------------- - -Checks in patchwork are mostly simple wrappers around existing kernel -scripts, the sources are available at: +Co-posting selftests +~~~~~~~~~~~~~~~~~~~~ -https://github.com/kuba-moo/nipa/tree/master/tests +Selftests should be part of the same series as the code changes. +Specifically for fixes both code change and related test should go into +the same tree (the tests may lack a Fixes tag, which is expected). +Mixing code changes and test changes in a single commit is discouraged. -Running all the builds and checks locally is a pain, can I post my patches and have the patchwork bot validate them? --------------------------------------------------------------------------------------------------------------------- +Preparing changes +----------------- -No, you must ensure that your patches are ready by testing them locally -before posting to the mailing list. The patchwork build bot instance -gets overloaded very easily and netdev@vger really doesn't need more -traffic if we can help it. - -netdevsim is great, can I extend it for my out-of-tree tests? -------------------------------------------------------------- - -No, ``netdevsim`` is a test vehicle solely for upstream tests. -(Please add your tests under ``tools/testing/selftests/``.) - -We also give no guarantees that ``netdevsim`` won't change in the future -in a way which would break what would normally be considered uAPI. - -Is netdevsim considered a "user" of an API? -------------------------------------------- - -Linux kernel has a long standing rule that no API should be added unless -it has a real, in-tree user. Mock-ups and tests based on ``netdevsim`` are -strongly encouraged when adding new APIs, but ``netdevsim`` in itself -is **not** considered a use case/user. - -Any other tips to help ensure my net/net-next patch gets OK'd? --------------------------------------------------------------- -Attention to detail. Re-read your own work as if you were the +Attention to detail is important. Re-read your own work as if you were the reviewer. You can start with using ``checkpatch.pl``, perhaps even with the ``--strict`` flag. But do not be mindlessly robotic in doing so. If your change is a bug fix, make sure your commit log indicates the @@ -320,10 +337,214 @@ Finally, go back and read :ref:`Documentation/process/submitting-patches.rst <submittingpatches>` to be sure you are not repeating some common mistake documented there. -My company uses peer feedback in employee performance reviews. Can I ask netdev maintainers for feedback? ---------------------------------------------------------------------------------------------------------- +Indicating target tree +~~~~~~~~~~~~~~~~~~~~~~ + +To help maintainers and CI bots you should explicitly mark which tree +your patch is targeting. Assuming that you use git, use the prefix +flag:: + + git format-patch --subject-prefix='PATCH net-next' start..finish + +Use ``net`` instead of ``net-next`` (always lower case) in the above for +bug-fix ``net`` content. + +Dividing work into patches +~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Put yourself in the shoes of the reviewer. Each patch is read separately +and therefore should constitute a comprehensible step towards your stated +goal. + +Avoid sending series longer than 15 patches. Larger series takes longer +to review as reviewers will defer looking at it until they find a large +chunk of time. A small series can be reviewed in a short time, so Maintainers +just do it. As a result, a sequence of smaller series gets merged quicker and +with better review coverage. Re-posting large series also increases the mailing +list traffic. + +.. _rcs: + +Local variable ordering ("reverse xmas tree", "RCS") +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Netdev has a convention for ordering local variables in functions. +Order the variable declaration lines longest to shortest, e.g.:: + + struct scatterlist *sg; + struct sk_buff *skb; + int err, i; + +If there are dependencies between the variables preventing the ordering +move the initialization out of line. + +Format precedence +~~~~~~~~~~~~~~~~~ + +When working in existing code which uses nonstandard formatting make +your code follow the most recent guidelines, so that eventually all code +in the domain of netdev is in the preferred format. + +Using device-managed and cleanup.h constructs +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Netdev remains skeptical about promises of all "auto-cleanup" APIs, +including even ``devm_`` helpers, historically. They are not the preferred +style of implementation, merely an acceptable one. + +Use of ``guard()`` is discouraged within any function longer than 20 lines, +``scoped_guard()`` is considered more readable. Using normal lock/unlock is +still (weakly) preferred. + +Low level cleanup constructs (such as ``__free()``) can be used when building +APIs and helpers, especially scoped iterators. However, direct use of +``__free()`` within networking core and drivers is discouraged. +Similar guidance applies to declaring variables mid-function. + +Clean-up patches +~~~~~~~~~~~~~~~~ + +Netdev discourages patches which perform simple clean-ups, which are not in +the context of other work. For example: + +* Addressing ``checkpatch.pl``, and other trivial coding style warnings +* Addressing :ref:`Local variable ordering<rcs>` issues +* Conversions to device-managed APIs (``devm_`` helpers) + +This is because it is felt that the churn that such changes produce comes +at a greater cost than the value of such clean-ups. + +Conversely, spelling and grammar fixes are not discouraged. + +Resending after review +~~~~~~~~~~~~~~~~~~~~~~ + +Allow at least 24 hours to pass between postings. This will ensure reviewers +from all geographical locations have a chance to chime in. Do not wait +too long (weeks) between postings either as it will make it harder for reviewers +to recall all the context. + +Make sure you address all the feedback in your new posting. Do not post a new +version of the code if the discussion about the previous version is still +ongoing, unless directly instructed by a reviewer. + +The new version of patches should be posted as a separate thread, +not as a reply to the previous posting. Change log should include a link +to the previous posting (see :ref:`Changes requested`). + +Testing +------- + +Expected level of testing +~~~~~~~~~~~~~~~~~~~~~~~~~ + +At the very minimum your changes must survive an ``allyesconfig`` and an +``allmodconfig`` build with ``W=1`` set without new warnings or failures. + +Ideally you will have done run-time testing specific to your change, +and the patch series contains a set of kernel selftest for +``tools/testing/selftests/net`` or using the KUnit framework. + +You are expected to test your changes on top of the relevant networking +tree (``net`` or ``net-next``) and not e.g. a stable tree or ``linux-next``. + +patchwork checks +~~~~~~~~~~~~~~~~ + +Checks in patchwork are mostly simple wrappers around existing kernel +scripts, the sources are available at: + +https://github.com/linux-netdev/nipa/tree/master/tests + +**Do not** post your patches just to run them through the checks. +You must ensure that your patches are ready by testing them locally +before posting to the mailing list. The patchwork build bot instance +gets overloaded very easily and netdev@vger really doesn't need more +traffic if we can help it. + +netdevsim +~~~~~~~~~ + +``netdevsim`` is a test driver which can be used to exercise driver +configuration APIs without requiring capable hardware. +Mock-ups and tests based on ``netdevsim`` are strongly encouraged when +adding new APIs, but ``netdevsim`` in itself is **not** considered +a use case/user. You must also implement the new APIs in a real driver. + +We give no guarantees that ``netdevsim`` won't change in the future +in a way which would break what would normally be considered uAPI. + +``netdevsim`` is reserved for use by upstream tests only, so any +new ``netdevsim`` features must be accompanied by selftests under +``tools/testing/selftests/``. + +Supported status for drivers +---------------------------- + +.. note: The following requirements apply only to Ethernet NIC drivers. + +Netdev defines additional requirements for drivers which want to acquire +the ``Supported`` status in the MAINTAINERS file. ``Supported`` drivers must +be running all upstream driver tests and reporting the results twice a day. +Drivers which do not comply with this requirement should use the ``Maintained`` +status. There is currently no difference in how ``Supported`` and ``Maintained`` +drivers are treated upstream. + +The exact rules a driver must follow to acquire the ``Supported`` status: + +1. Must run all tests under ``drivers/net`` and ``drivers/net/hw`` targets + of Linux selftests. Running and reporting private / internal tests is + also welcome, but upstream tests are a must. + +2. The minimum run frequency is once every 12 hours. Must test the + designated branch from the selected branch feed. Note that branches + are auto-constructed and exposed to intentional malicious patch posting, + so the test systems must be isolated. + +3. Drivers supporting multiple generations of devices must test at + least one device from each generation. A testbed manifest (exact + format TBD) should describe the device models tested. + +4. The tests must run reliably, if multiple branches are skipped or tests + are failing due to execution environment problems the ``Supported`` + status will be withdrawn. + +5. Test failures due to bugs either in the driver or the test itself, + or lack of support for the feature the test is targgeting are + *not* a basis for losing the ``Supported`` status. + +netdev CI will maintain an official page of supported devices, listing their +recent test results. + +The driver maintainer may arrange for someone else to run the test, +there is no requirement for the person listed as maintainer (or their +employer) to be responsible for running the tests. Collaboration between +vendors, hosting GH CI, other repos under linux-netdev, etc. is most welcome. + +See https://github.com/linux-netdev/nipa/wiki for more information about +netdev CI. Feel free to reach out to maintainers or the list with any questions. + +Reviewer guidance +----------------- + +Reviewing other people's patches on the list is highly encouraged, +regardless of the level of expertise. For general guidance and +helpful tips please see :ref:`development_advancedtopics_reviews`. + +It's safe to assume that netdev maintainers know the community and the level +of expertise of the reviewers. The reviewers should not be concerned about +their comments impeding or derailing the patch flow. + +Less experienced reviewers are highly encouraged to do more in-depth +review of submissions and not focus exclusively on trivial or subjective +matters like code formatting, tags etc. + +Testimonials / feedback +----------------------- -Yes, especially if you spend significant amount of time reviewing code +Some companies use peer feedback in employee performance reviews. +Please feel free to request feedback from netdev maintainers, +especially if you spend significant amount of time reviewing code and go out of your way to improve shared infrastructure. The feedback must be requested by you, the contributor, and will always diff --git a/Documentation/process/maintainer-pgp-guide.rst b/Documentation/process/maintainer-pgp-guide.rst index 40bfbd3b7648..b6919bf606c3 100644 --- a/Documentation/process/maintainer-pgp-guide.rst +++ b/Documentation/process/maintainer-pgp-guide.rst @@ -49,7 +49,7 @@ hosting infrastructure, regardless of how good the security practices for the latter may be. The above guiding principle is the reason why this guide is needed. We -want to make sure that by placing trust into developers we do not simply +want to make sure that by placing trust into developers we do not merely shift the blame for potential future security incidents to someone else. The goal is to provide a set of guidelines developers can use to create a secure working environment and safeguard the PGP keys used to @@ -60,36 +60,18 @@ establish the integrity of the Linux kernel itself. PGP tools ========= -Use GnuPG v2 ------------- +Use GnuPG 2.4 or later +---------------------- Your distro should already have GnuPG installed by default, you just -need to verify that you are using version 2.x and not the legacy 1.4 -release -- many distributions still package both, with the default -``gpg`` command invoking GnuPG v.1. To check, run:: +need to verify that you are using a reasonably recent version of it. +To check, run:: $ gpg --version | head -n1 -If you see ``gpg (GnuPG) 1.4.x``, then you are using GnuPG v.1. Try the -``gpg2`` command (if you don't have it, you may need to install the -gnupg2 package):: - - $ gpg2 --version | head -n1 - -If you see ``gpg (GnuPG) 2.x.x``, then you are good to go. This guide -will assume you have the version 2.2 of GnuPG (or later). If you are -using version 2.0 of GnuPG, then some of the commands in this guide will -not work, and you should consider installing the latest 2.2 version of -GnuPG. Versions of gnupg-2.1.11 and later should be compatible for the -purposes of this guide as well. - -If you have both ``gpg`` and ``gpg2`` commands, you should make sure you -are always using GnuPG v2, not the legacy version. You can enforce this -by setting the appropriate alias:: - - $ alias gpg=gpg2 - -You can put that in your ``.bashrc`` to make sure it's always the case. +If you have version 2.4 or above, then you are good to go. If you have +an earlier version, then you are using a release of GnuPG that is no +longer maintained and some commands from this guide may not work. Configure gpg-agent options ~~~~~~~~~~~~~~~~~~~~~~~~~~~ @@ -150,9 +132,9 @@ PGP defines four capabilities that a key can have: The key with the **[C]** capability is often called the "master" key, but this terminology is misleading because it implies that the Certify key can be used in place of any of other subkey on the same chain (like -a physical "master key" can be used to open the locks made for other -keys). Since this is not the case, this guide will refer to it as "the -Certify key" to avoid any ambiguity. +a physical "master key" can be used to open locks made for other keys). +Since this is not the case, this guide will refer to it as "the Certify +key" to avoid any ambiguity. It is critical to fully understand the following: @@ -186,10 +168,10 @@ If you used the default parameters when generating your key, then that is what you will have. You can verify by running ``gpg --list-secret-keys``, for example:: - sec rsa2048 2018-01-23 [SC] [expires: 2020-01-23] + sec ed25519 2022-12-20 [SC] [expires: 2024-12-19] 000000000000000000000000AAAABBBBCCCCDDDD uid [ultimate] Alice Dev <adev@kernel.org> - ssb rsa2048 2018-01-23 [E] [expires: 2020-01-23] + ssb cv25519 2022-12-20 [E] [expires: 2024-12-19] The long line under the ``sec`` entry is your key fingerprint -- whenever you see ``[fpr]`` in the examples below, that 40-character @@ -217,22 +199,6 @@ separate signing subkey:: $ gpg --quick-addkey [fpr] ed25519 sign -.. note:: ECC support in GnuPG - - GnuPG 2.1 and later has full support for Elliptic Curve - Cryptography, with ability to combine ECC subkeys with traditional - RSA keys. The main upside of ECC cryptography is that it is much - faster computationally and creates much smaller signatures when - compared byte for byte with 2048+ bit RSA keys. Unless you plan on - using a smartcard device that does not support ECC operations, we - recommend that you create an ECC signing subkey for your kernel - work. - - Note, that if you plan to use a hardware device that does not - support ED25519 ECC keys, you should choose "nistp256" instead or - "ed25519." - - Back up your Certify key for disaster recovery ---------------------------------------------- @@ -240,7 +206,7 @@ The more signatures you have on your PGP key from other developers, the more reasons you have to create a backup version that lives on something other than digital media, for disaster recovery reasons. -The best way to create a printable hardcopy of your private key is by +A good way to create a printable hardcopy of your private key is by using the ``paperkey`` software written for this very purpose. See ``man paperkey`` for more details on the output format and its benefits over other solutions. Paperkey should already be packaged for most @@ -251,11 +217,11 @@ key:: $ gpg --export-secret-key [fpr] | paperkey -o /tmp/key-backup.txt -Print out that file (or pipe the output straight to lpr), then take a -pen and write your passphrase on the margin of the paper. **This is -strongly recommended** because the key printout is still encrypted with -that passphrase, and if you ever change it you will not remember what it -used to be when you had created the backup -- *guaranteed*. +Print out that file, then take a pen and write your passphrase on the +margin of the paper. **This is strongly recommended** because the key +printout is still encrypted with that passphrase, and if you ever change +it you will not remember what it used to be when you had created the +backup -- *guaranteed*. Put the resulting printout and the hand-written passphrase into an envelope and store in a secure and well-protected place, preferably away from your @@ -263,10 +229,9 @@ home, such as your bank vault. .. note:: - Your printer is probably no longer a simple dumb device connected to - your parallel port, but since the output is still encrypted with - your passphrase, printing out even to "cloud-integrated" modern - printers should remain a relatively safe operation. + The key is still encrypted with your passphrase, so printing out + even to "cloud-integrated" modern printers should remain a + relatively safe operation. Back up your whole GnuPG directory ---------------------------------- @@ -282,16 +247,17 @@ on these external copies whenever you need to use your Certify key -- such as when making changes to your own key or signing other people's keys after conferences and summits. -Start by getting a small USB "thumb" drive (preferably two!) that you -will use for backup purposes. You will need to encrypt them using LUKS --- refer to your distro's documentation on how to accomplish this. +Start by getting an external media card (preferably two!) that you will +use for backup purposes. You will need to create an encrypted partition +on this device using LUKS -- refer to your distro's documentation on how +to accomplish this. For the encryption passphrase, you can use the same one as on your PGP key. -Once the encryption process is over, re-insert the USB drive and make -sure it gets properly mounted. Copy your entire ``.gnupg`` directory -over to the encrypted storage:: +Once the encryption process is over, re-insert your device and make sure +it gets properly mounted. Copy your entire ``.gnupg`` directory over to +the encrypted storage:: $ cp -a ~/.gnupg /media/disk/foo/gnupg-backup @@ -300,11 +266,10 @@ You should now test to make sure everything still works:: $ gpg --homedir=/media/disk/foo/gnupg-backup --list-key [fpr] If you don't get any errors, then you should be good to go. Unmount the -USB drive, distinctly label it so you don't blow it away next time you -need to use a random USB drive, and put in a safe place -- but not too -far away, because you'll need to use it every now and again for things -like editing identities, adding or revoking subkeys, or signing other -people's keys. +device, distinctly label it so you don't overwrite it by accident, and +put in a safe place -- but not too far away, because you'll need to use +it every now and again for things like editing identities, adding or +revoking subkeys, or signing other people's keys. Remove the Certify key from your homedir ---------------------------------------- @@ -330,19 +295,19 @@ and store it on offline storage. your GnuPG directory in its entirety. What we are about to do will render your key useless if you do not have a usable backup! -First, identify the keygrip of your Certify key:: +First, identify the "keygrip" of your Certify key:: $ gpg --with-keygrip --list-key [fpr] The output will be something like this:: - pub rsa2048 2018-01-24 [SC] [expires: 2020-01-24] + pub ed25519 2022-12-20 [SC] [expires: 2022-12-19] 000000000000000000000000AAAABBBBCCCCDDDD Keygrip = 1111000000000000000000000000000000000000 uid [ultimate] Alice Dev <adev@kernel.org> - sub rsa2048 2018-01-24 [E] [expires: 2020-01-24] + sub cv25519 2022-12-20 [E] [expires: 2022-12-19] Keygrip = 2222000000000000000000000000000000000000 - sub ed25519 2018-01-24 [S] + sub ed25519 2022-12-20 [S] Keygrip = 3333000000000000000000000000000000000000 Find the keygrip entry that is beneath the ``pub`` line (right under the @@ -355,8 +320,8 @@ Certify key fingerprint). This will correspond directly to a file in your 2222000000000000000000000000000000000000.key 3333000000000000000000000000000000000000.key -All you have to do is simply remove the .key file that corresponds to -the Certify key keygrip:: +It is sufficient to remove the .key file that corresponds to the Certify +key keygrip:: $ cd ~/.gnupg/private-keys-v1.d $ rm 1111000000000000000000000000000000000000.key @@ -365,14 +330,14 @@ Now, if you issue the ``--list-secret-keys`` command, it will show that the Certify key is missing (the ``#`` indicates it is not available):: $ gpg --list-secret-keys - sec# rsa2048 2018-01-24 [SC] [expires: 2020-01-24] + sec# ed25519 2022-12-20 [SC] [expires: 2024-12-19] 000000000000000000000000AAAABBBBCCCCDDDD uid [ultimate] Alice Dev <adev@kernel.org> - ssb rsa2048 2018-01-24 [E] [expires: 2020-01-24] - ssb ed25519 2018-01-24 [S] + ssb cv25519 2022-12-20 [E] [expires: 2024-12-19] + ssb ed25519 2022-12-20 [S] You should also remove any ``secring.gpg`` files in the ``~/.gnupg`` -directory, which are left over from earlier versions of GnuPG. +directory, which may be left over from previous versions of GnuPG. If you don't have the "private-keys-v1.d" directory ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ @@ -399,7 +364,7 @@ GnuPG operation is performed, the keys are loaded into system memory and can be stolen from there by sufficiently advanced malware (think Meltdown and Spectre). -The best way to completely protect your keys is to move them to a +A good way to completely protect your keys is to move them to a specialized hardware device that is capable of smartcard operations. The benefits of smartcards @@ -410,11 +375,11 @@ private keys and performing crypto operations directly on the card itself. Because the key contents never leave the smartcard, the operating system of the computer into which you plug in the hardware device is not able to retrieve the private keys themselves. This is very -different from the encrypted USB storage device we used earlier for -backup purposes -- while that USB device is plugged in and mounted, the +different from the encrypted media storage device we used earlier for +backup purposes -- while that device is plugged in and mounted, the operating system is able to access the private key contents. -Using external encrypted USB media is not a substitute to having a +Using external encrypted media is not a substitute to having a smartcard-capable device. Available smartcard devices @@ -425,20 +390,17 @@ easiest is to get a specialized USB device that implements smartcard functionality. There are several options available: - `Nitrokey Start`_: Open hardware and Free Software, based on FSI - Japan's `Gnuk`_. One of the few available commercial devices that - support ED25519 ECC keys, but offer fewest security features (such as - resistance to tampering or some side-channel attacks). -- `Nitrokey Pro 2`_: Similar to the Nitrokey Start, but more - tamper-resistant and offers more security features. Pro 2 supports ECC - cryptography (NISTP). + Japan's `Gnuk`_. One of the cheapest options, but offers fewest + security features (such as resistance to tampering or some + side-channel attacks). +- `Nitrokey 3`_: Similar to the Nitrokey Start, but more + tamper-resistant and offers more security features and USB + form-factors. Supports ECC cryptography (ED25519 and NISTP). - `Yubikey 5`_: proprietary hardware and software, but cheaper than - Nitrokey Pro and comes available in the USB-C form that is more useful - with newer laptops. Offers additional security features such as FIDO - U2F, among others, and now finally supports NISTP and ED25519 ECC - keys. + Nitrokey with a similar set of features. Supports ECC cryptography + (ED25519 and NISTP). -`LWN has a good review`_ of some of the above models, as well as several -others. Your choice will depend on cost, shipping availability in your +Your choice will depend on cost, shipping availability in your geographical region, and open/proprietary hardware considerations. .. note:: @@ -447,11 +409,10 @@ geographical region, and open/proprietary hardware considerations. you `qualify for a free Nitrokey Start`_ courtesy of The Linux Foundation. -.. _`Nitrokey Start`: https://shop.nitrokey.com/shop/product/nitrokey-start-6 -.. _`Nitrokey Pro 2`: https://shop.nitrokey.com/shop/product/nkpr2-nitrokey-pro-2-3 +.. _`Nitrokey Start`: https://www.nitrokey.com/products/nitrokeys +.. _`Nitrokey 3`: https://www.nitrokey.com/products/nitrokeys .. _`Yubikey 5`: https://www.yubico.com/products/yubikey-5-overview/ .. _Gnuk: https://www.fsij.org/doc-gnuk/ -.. _`LWN has a good review`: https://lwn.net/Articles/736231/ .. _`qualify for a free Nitrokey Start`: https://www.kernel.org/nitrokey-digital-tokens-for-kernel-developers.html Configure your smartcard device @@ -484,7 +445,7 @@ the smartcard). You so rarely need to use the Admin PIN, that you will inevitably forget what it is if you do not record it. Getting back to the main card menu, you can also set other values (such -as name, sex, login data, etc), but it's not necessary and will +as name, gender, login data, etc), but it's not necessary and will additionally leak information about your smartcard should you lose it. .. note:: @@ -509,11 +470,11 @@ passphrase and the admin PIN of the card for most operations:: Secret subkeys are available. - pub rsa2048/AAAABBBBCCCCDDDD - created: 2018-01-23 expires: 2020-01-23 usage: SC + pub ed25519/AAAABBBBCCCCDDDD + created: 2022-12-20 expires: 2024-12-19 usage: SC trust: ultimate validity: ultimate - ssb rsa2048/1111222233334444 - created: 2018-01-23 expires: never usage: E + ssb cv25519/1111222233334444 + created: 2022-12-20 expires: never usage: E ssb ed25519/5555666677778888 created: 2017-12-07 expires: never usage: S [ultimate] (1). Alice Dev <adev@kernel.org> @@ -577,11 +538,11 @@ If you perform ``--list-secret-keys`` now, you will see a subtle difference in the output:: $ gpg --list-secret-keys - sec# rsa2048 2018-01-24 [SC] [expires: 2020-01-24] + sec# ed25519 2022-12-20 [SC] [expires: 2024-12-19] 000000000000000000000000AAAABBBBCCCCDDDD uid [ultimate] Alice Dev <adev@kernel.org> - ssb> rsa2048 2018-01-24 [E] [expires: 2020-01-24] - ssb> ed25519 2018-01-24 [S] + ssb> cv25519 2022-12-20 [E] [expires: 2024-12-19] + ssb> ed25519 2022-12-20 [S] The ``>`` in the ``ssb>`` output indicates that the subkey is only available on the smartcard. If you go back into your secret keys @@ -644,7 +605,7 @@ run:: You can also use a specific date if that is easier to remember (e.g. your birthday, January 1st, or Canada Day):: - $ gpg --quick-set-expire [fpr] 2020-07-01 + $ gpg --quick-set-expire [fpr] 2038-07-01 Remember to send the updated key back to keyservers:: @@ -685,9 +646,9 @@ hundreds of cloned repositories floating around, how does anyone verify that their copy of linux.git has not been tampered with by a malicious third party? -Or what happens if a backdoor is discovered in the code and the "Author" -line in the commit says it was done by you, while you're pretty sure you -had `nothing to do with it`_? +Or what happens if malicious code is discovered in the kernel and the +"Author" line in the commit says it was done by you, while you're pretty +sure you had `nothing to do with it`_? To address both of these issues, Git introduced PGP integration. Signed tags prove the repository integrity by assuring that its contents are @@ -707,17 +668,10 @@ should be used (``[fpr]`` is the fingerprint of your key):: $ git config --global user.signingKey [fpr] -**IMPORTANT**: If you have a distinct ``gpg2`` command, then you should -tell git to always use it instead of the legacy ``gpg`` from version 1:: - - $ git config --global gpg.program gpg2 - $ git config --global gpgv.program gpgv2 - How to work with signed tags ---------------------------- -To create a signed tag, simply pass the ``-s`` switch to the tag -command:: +To create a signed tag, pass the ``-s`` switch to the tag command:: $ git tag -s [tagname] @@ -728,7 +682,7 @@ not been maliciously altered. How to verify signed tags ~~~~~~~~~~~~~~~~~~~~~~~~~ -To verify a signed tag, simply use the ``verify-tag`` command:: +To verify a signed tag, use the ``verify-tag`` command:: $ git verify-tag [tagname] @@ -747,16 +701,9 @@ The merge message will contain something like this:: # gpg: Signature made [...] # gpg: Good signature from [...] -If you are verifying someone else's git tag, then you will need to -import their PGP key. Please refer to the -":ref:`verify_identities`" section below. - -.. note:: - - If you get "``gpg: Can't check signature: unknown pubkey - algorithm``" error, you need to tell git to use gpgv2 for - verification, so it properly processes signatures made by ECC keys. - See instructions at the start of this section. +If you are verifying someone else's git tag, you will first need to +import their PGP key. Please refer to the ":ref:`verify_identities`" +section below. Configure git to always sign annotated tags ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ @@ -770,16 +717,16 @@ configuration option:: How to work with signed commits ------------------------------- -It is easy to create signed commits, but it is much more difficult to -use them in Linux kernel development, since it relies on patches sent to -the mailing list, and this workflow does not preserve PGP commit -signatures. Furthermore, when rebasing your repository to match -upstream, even your own PGP commit signatures will end up discarded. For -this reason, most kernel developers don't bother signing their commits -and will ignore signed commits in any external repositories that they -rely upon in their work. +It is also possible to create signed commits, but they have limited +usefulness in Linux kernel development. The kernel contribution workflow +relies on sending in patches, and converting commits to patches does not +preserve git commit signatures. Furthermore, when rebasing your own +repository on a newer upstream, PGP commit signatures will end up +discarded. For this reason, most kernel developers don't bother signing +their commits and will ignore signed commits in any external +repositories that they rely upon in their work. -However, if you have your working git tree publicly available at some +That said, if you have your working git tree publicly available at some git hosting service (kernel.org, infradead.org, ozlabs.org, or others), then the recommendation is that you sign all your git commits even if upstream developers do not directly benefit from this practice. @@ -790,7 +737,7 @@ We recommend this for the following reasons: provenance, even externally maintained trees carrying PGP commit signatures will be valuable for such purposes. 2. If you ever need to re-clone your local repository (for example, - after a disk failure), this lets you easily verify the repository + after reinstalling your system), this lets you verify the repository integrity before resuming your work. 3. If someone needs to cherry-pick your commits, this allows them to quickly verify their integrity before applying them. @@ -798,9 +745,8 @@ We recommend this for the following reasons: Creating signed commits ~~~~~~~~~~~~~~~~~~~~~~~ -To create a signed commit, you just need to pass the ``-S`` flag to the -``git commit`` command (it's capital ``-S`` due to collision with -another flag):: +To create a signed commit, pass the ``-S`` flag to the ``git commit`` +command (it's capital ``-S`` due to collision with another flag):: $ git commit -S @@ -817,7 +763,6 @@ You can tell git to always sign commits:: .. _verify_identities: - How to work with signed patches ------------------------------- @@ -835,6 +780,11 @@ headers (a-la DKIM): Installing and configuring patatt ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +.. note:: + + If you use B4 to send in your patches, patatt is already installed + and integrated into your workflow. + Patatt is packaged for many distributions already, so please check there first. You can also install it from pypi using "``pip install patatt``". @@ -877,9 +827,9 @@ encounters, for example:: How to verify kernel developer identities ========================================= -Signing tags and commits is easy, but how does one go about verifying -that the key used to sign something belongs to the actual kernel -developer and not to a malicious imposter? +Signing tags and commits is straightforward, but how does one go about +verifying that the key used to sign something belongs to the actual +kernel developer and not to a malicious imposter? Configure auto-key-retrieval using WKD and DANE ----------------------------------------------- @@ -926,7 +876,7 @@ various software makers dictating who should be your trusted certifying entity, PGP leaves this responsibility to each user. Unfortunately, very few people understand how the Web of Trust works. -While it remains an important aspect of the OpenPGP specification, +While it is still an important part of the OpenPGP specification, recent versions of GnuPG (2.2 and above) have implemented an alternative mechanism called "Trust on First Use" (TOFU). You can think of TOFU as "the SSH-like approach to trust." With SSH, the first time you connect @@ -936,8 +886,8 @@ to connect, forcing you to make a decision on whether you choose to trust the changed key or not. Similarly, the first time you import someone's PGP key, it is assumed to be valid. If at any point in the future GnuPG comes across another key with the same identity, both the -previously imported key and the new key will be marked as invalid and -you will need to manually figure out which one to keep. +previously imported key and the new key will be marked for verification +and you will need to manually figure out which one to keep. We recommend that you use the combined TOFU+PGP trust model (which is the new default in GnuPG v2). To set it, add (or modify) the diff --git a/Documentation/process/maintainer-soc-clean-dts.rst b/Documentation/process/maintainer-soc-clean-dts.rst new file mode 100644 index 000000000000..5423fb7d6047 --- /dev/null +++ b/Documentation/process/maintainer-soc-clean-dts.rst @@ -0,0 +1,26 @@ +.. SPDX-License-Identifier: GPL-2.0 + +============================================== +SoC Platforms with DTS Compliance Requirements +============================================== + +Overview +-------- + +SoC platforms or subarchitectures should follow all the rules from +Documentation/process/maintainer-soc.rst. This document referenced in +MAINTAINERS impose additional requirements listed below. + +Strict DTS DT Schema and dtc Compliance +--------------------------------------- + +No changes to the SoC platform Devicetree sources (DTS files) should introduce +new ``make dtbs_check W=1`` warnings. Warnings in a new board DTS, which are +results of issues in an included DTSI file, are considered existing, not new +warnings. For series split between different trees (DT bindings go via driver +subsystem tree), warnings on linux-next are decisive. The platform maintainers +have automation in place which should point out any new warnings. + +If a commit introducing new warnings gets accepted somehow, the resulting +issues shall be fixed in reasonable time (e.g. within one release) or the +commit reverted. diff --git a/Documentation/process/maintainer-soc.rst b/Documentation/process/maintainer-soc.rst new file mode 100644 index 000000000000..3ba886f52a51 --- /dev/null +++ b/Documentation/process/maintainer-soc.rst @@ -0,0 +1,209 @@ +.. SPDX-License-Identifier: GPL-2.0 + +============= +SoC Subsystem +============= + +Overview +-------- + +The SoC subsystem is a place of aggregation for SoC-specific code. +The main components of the subsystem are: + +* devicetrees (DTS) for 32- & 64-bit ARM and RISC-V +* 32-bit ARM board files (arch/arm/mach*) +* 32- & 64-bit ARM defconfigs +* SoC-specific drivers across architectures, in particular for 32- & 64-bit + ARM, RISC-V and Loongarch + +These "SoC-specific drivers" do not include clock, GPIO etc drivers that have +other top-level maintainers. The drivers/soc/ directory is generally meant +for kernel-internal drivers that are used by other drivers to provide SoC- +specific functionality like identifying an SoC revision or interfacing with +power domains. + +The SoC subsystem also serves as an intermediate location for changes to +drivers/bus, drivers/firmware, drivers/reset and drivers/memory. The addition +of new platforms, or the removal of existing ones, often go through the SoC +tree as a dedicated branch covering multiple subsystems. + +The main SoC tree is housed on git.kernel.org: + https://git.kernel.org/pub/scm/linux/kernel/git/soc/soc.git/ + +Maintainers +----------- + +Clearly this is quite a wide range of topics, which no one person, or even +small group of people are capable of maintaining. Instead, the SoC subsystem +is comprised of many submaintainers (platform maintainers), each taking care of +individual platforms and driver subdirectories. +In this regard, "platform" usually refers to a series of SoCs from a given +vendor, for example, Nvidia's series of Tegra SoCs. Many submaintainers operate +on a vendor level, responsible for multiple product lines. For several reasons, +including acquisitions/different business units in a company, things vary +significantly here. The various submaintainers are documented in the +MAINTAINERS file. + +Most of these submaintainers have their own trees where they stage patches, +sending pull requests to the main SoC tree. These trees are usually, but not +always, listed in MAINTAINERS. + +What the SoC tree is not, however, is a location for architecture-specific code +changes. Each architecture has its own maintainers that are responsible for +architectural details, CPU errata and the like. + +Submitting Patches for Given SoC +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +All typical platform related patches should be sent via SoC submaintainers +(platform-specific maintainers). This includes also changes to per-platform or +shared defconfigs (scripts/get_maintainer.pl might not provide correct +addresses in such case). + +Submitting Patches to the Main SoC Maintainers +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +The main SoC maintainers can be reached via the alias soc@kernel.org only in +following cases: + +1. There are no platform-specific maintainers. + +2. Platform-specific maintainers are unresponsive. + +3. Introducing a completely new SoC platform. Such new SoC work should be sent + first to common mailing lists, pointed out by scripts/get_maintainer.pl, for + community review. After positive community review, work should be sent to + soc@kernel.org in one patchset containing new arch/foo/Kconfig entry, DTS + files, MAINTAINERS file entry and optionally initial drivers with their + Devicetree bindings. The MAINTAINERS file entry should list new + platform-specific maintainers, who are going to be responsible for handling + patches for the platform from now on. + +Note that the soc@kernel.org is usually not the place to discuss the patches, +thus work sent to this address should be already considered as acceptable by +the community. + +Information for (new) Submaintainers +------------------------------------ + +As new platforms spring up, they often bring with them new submaintainers, +many of whom work for the silicon vendor, and may not be familiar with the +process. + +Devicetree ABI Stability +~~~~~~~~~~~~~~~~~~~~~~~~ + +Perhaps one of the most important things to highlight is that dt-bindings +document the ABI between the devicetree and the kernel. +Please read Documentation/devicetree/bindings/ABI.rst. + +If changes are being made to a DTS that are incompatible with old +kernels, the DTS patch should not be applied until the driver is, or an +appropriate time later. Most importantly, any incompatible changes should be +clearly pointed out in the patch description and pull request, along with the +expected impact on existing users, such as bootloaders or other operating +systems. + +Driver Branch Dependencies +~~~~~~~~~~~~~~~~~~~~~~~~~~ + +A common problem is synchronizing changes between device drivers and devicetree +files. Even if a change is compatible in both directions, this may require +coordinating how the changes get merged through different maintainer trees. + +Usually the branch that includes a driver change will also include the +corresponding change to the devicetree binding description, to ensure they are +in fact compatible. This means that the devicetree branch can end up causing +warnings in the "make dtbs_check" step. If a devicetree change depends on +missing additions to a header file in include/dt-bindings/, it will fail the +"make dtbs" step and not get merged. + +There are multiple ways to deal with this: + +* Avoid defining custom macros in include/dt-bindings/ for hardware constants + that can be derived from a datasheet -- binding macros in header files should + only be used as a last resort if there is no natural way to define a binding + +* Use literal values in the devicetree file in place of macros even when a + header is required, and change them to the named representation in a + following release + +* Defer the devicetree changes to a release after the binding and driver have + already been merged + +* Change the bindings in a shared immutable branch that is used as the base for + both the driver change and the devicetree changes + +* Add duplicate defines in the devicetree file guarded by an #ifndef section, + removing them in a later release + +Devicetree Naming Convention +~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +The general naming scheme for devicetree files is as follows. The aspects of a +platform that are set at the SoC level, like CPU cores, are contained in a file +named $soc.dtsi, for example, jh7100.dtsi. Integration details, that will vary +from board to board, are described in $soc-$board.dts. An example of this is +jh7100-beaglev-starlight.dts. Often many boards are variations on a theme, and +frequently there are intermediate files, such as jh7100-common.dtsi, which sit +between the $soc.dtsi and $soc-$board.dts files, containing the descriptions of +common hardware. + +Some platforms also have System on Modules, containing an SoC, which are then +integrated into several different boards. For these platforms, $soc-$som.dtsi +and $soc-$som-$board.dts are typical. + +Directories are usually named after the vendor of the SoC at the time of its +inclusion, leading to some historical directory names in the tree. + +Validating Devicetree Files +~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +``make dtbs_check`` can be used to validate that devicetree files are compliant +with the dt-bindings that describe the ABI. Please read the section +"Running checks" of Documentation/devicetree/bindings/writing-schema.rst for +more information on the validation of devicetrees. + +For new platforms, or additions to existing ones, ``make dtbs_check`` should not +add any new warnings. For RISC-V and Samsung SoC, ``make dtbs_check W=1`` is +required to not add any new warnings. +If in any doubt about a devicetree change, reach out to the devicetree +maintainers. + +Branches and Pull Requests +~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Just as the main SoC tree has several branches, it is expected that +submaintainers will do the same. Driver, defconfig and devicetree changes should +all be split into separate branches and appear in separate pull requests to the +SoC maintainers. Each branch should be usable by itself and avoid +regressions that originate from dependencies on other branches. + +Small sets of patches can also be sent as separate emails to soc@kernel.org, +grouped into the same categories. + +If changes do not fit into the normal patterns, there can be additional +top-level branches, e.g. for a treewide rework, or the addition of new SoC +platforms including dts files and drivers. + +Branches with a lot of changes can benefit from getting split up into separate +topics branches, even if they end up getting merged into the same branch of the +SoC tree. An example here would be one branch for devicetree warning fixes, one +for a rework and one for newly added boards. + +Another common way to split up changes is to send an early pull request with the +majority of the changes at some point between rc1 and rc4, following up with one +or more smaller pull requests towards the end of the cycle that can add late +changes or address problems identified while testing the first set. + +While there is no cut-off time for late pull requests, it helps to only send +small branches as time gets closer to the merge window. + +Pull requests for bugfixes for the current release can be sent at any time, but +again having multiple smaller branches is better than trying to combine too many +patches into one pull request. + +The subject line of a pull request should begin with "[GIT PULL]" and made using +a signed tag, rather than a branch. This tag should contain a short description +summarising the changes in the pull request. For more detail on sending pull +requests, please see Documentation/maintainer/pull-requests.rst. diff --git a/Documentation/process/maintainer-tip.rst b/Documentation/process/maintainer-tip.rst index 572a3289c9cb..41d5855700cd 100644 --- a/Documentation/process/maintainer-tip.rst +++ b/Documentation/process/maintainer-tip.rst @@ -7,7 +7,7 @@ What is the tip tree? --------------------- The tip tree is a collection of several subsystems and areas of -development. The tip tree is both a direct development tree and a +development. The tip tree is both a direct development tree and an aggregation tree for several sub-maintainer trees. The tip tree gitweb URL is: https://git.kernel.org/pub/scm/linux/kernel/git/tip/tip.git @@ -121,15 +121,15 @@ The tip tree preferred format for patch subject prefixes is prefix. 'git log path/to/file' should give you a reasonable hint in most cases. -The condensed patch description in the subject line should start with a +The condensed patch description in the subject line should start with an uppercase letter and should be written in imperative tone. Changelog ^^^^^^^^^ -The general rules about changelogs in the process documentation, see -:ref:`Documentation/process/ <submittingpatches>`, apply. +The general rules about changelogs in the :ref:`Submitting patches guide +<describe_changes>`, apply. The tip tree maintainers set value on following these rules, especially on the request to write changelogs in imperative mood and not impersonating @@ -154,7 +154,7 @@ Examples for illustration: We modify the hot cpu handling to cancel the delayed work on the dying cpu and run the worker immediately on a different cpu in same domain. We - donot flush the worker because the MBM overflow worker reschedules the + do not flush the worker because the MBM overflow worker reschedules the worker on same CPU and scans the domain->cpu_mask to get the domain pointer. @@ -270,7 +270,7 @@ Ordering of commit tags To have a uniform view of the commit tags, the tip maintainers use the following tag ordering scheme: - - Fixes: 12char-SHA1 ("sub/sys: Original subject line") + - Fixes: 12+char-SHA1 ("sub/sys: Original subject line") A Fixes tag should be added even for changes which do not need to be backported to stable kernels, i.e. when addressing a recently introduced @@ -304,13 +304,15 @@ following tag ordering scheme: - Reported-by: ``Reporter <reporter@mail>`` + - Closes: ``URL or Message-ID of the bug report this is fixing`` + - Originally-by: ``Original author <original-author@mail>`` - Suggested-by: ``Suggester <suggester@mail>`` - Co-developed-by: ``Co-author <co-author@mail>`` - Signed-off: ``Co-author <co-author@mail>`` + Signed-off-by: ``Co-author <co-author@mail>`` Note, that Co-developed-by and Signed-off-by of the co-author(s) must come in pairs. @@ -370,17 +372,31 @@ following tag ordering scheme: - Link: ``https://link/to/information`` - For referring to an email on LKML or other kernel mailing lists, - please use the lore.kernel.org redirector URL:: + For referring to an email posted to the kernel mailing lists, please + use the lore.kernel.org redirector URL:: + + Link: https://lore.kernel.org/email-message-id@here + + This URL should be used when referring to relevant mailing list + topics, related patch sets, or other notable discussion threads. + A convenient way to associate ``Link:`` trailers with the commit + message is to use markdown-like bracketed notation, for example:: + + A similar approach was attempted before as part of a different + effort [1], but the initial implementation caused too many + regressions [2], so it was backed out and reimplemented. - https://lore.kernel.org/r/email-message@id + Link: https://lore.kernel.org/some-msgid@here # [1] + Link: https://bugzilla.example.org/bug/12345 # [2] - The kernel.org redirector is considered a stable URL, unlike other email - archives. + You can also use ``Link:`` trailers to indicate the origin of the + patch when applying it to your git tree. In that case, please use the + dedicated ``patch.msgid.link`` domain instead of ``lore.kernel.org``. + This practice makes it possible for automated tooling to identify + which link to use to retrieve the original patch submission. For + example:: - Maintainers will add a Link tag referencing the email of the patch - submission when they apply a patch to the tip tree. This tag is useful - for later reference and is also used for commit notifications. + Link: https://patch.msgid.link/patch-source-message-id@here Please do not use combined tags, e.g. ``Reported-and-tested-by``, as they just complicate automated extraction of tags. @@ -407,19 +423,22 @@ See :ref:`resend_reminders`. Merge window ^^^^^^^^^^^^ -Please do not expect large patch series to be handled during the merge -window or even during the week before. Such patches should be submitted in -mergeable state *at* *least* a week before the merge window opens. -Exceptions are made for bug fixes and *sometimes* for small standalone -drivers for new hardware or minimally invasive patches for hardware -enablement. +Please do not expect patches to be reviewed or merged by tip +maintainers around or during the merge window. The trees are closed +to all but urgent fixes during this time. They reopen once the merge +window closes and a new -rc1 kernel has been released. + +Large series should be submitted in mergeable state *at* *least* a week +before the merge window opens. Exceptions are made for bug fixes and +*sometimes* for small standalone drivers for new hardware or minimally +invasive patches for hardware enablement. During the merge window, the maintainers instead focus on following the upstream changes, fixing merge window fallout, collecting bug fixes, and allowing themselves a breath. Please respect that. -The release candidate -rc1 is the starting point for new patches to be -applied which are targeted for the next merge window. +So called _urgent_ branches will be merged into mainline during the +stabilization phase of each release. Git @@ -452,6 +471,8 @@ and can be added to an existing kernel config by running: Some of these options are x86-specific and can be left out when testing on other architectures. +.. _maintainer-tip-coding-style: + Coding style notes ------------------ @@ -473,7 +494,7 @@ Multi-line comments:: * Larger multi-line comments should be split into paragraphs. */ -No tail comments: +No tail comments (see below): Please refrain from using tail comments. Tail comments disturb the reading flow in almost all contexts, but especially in code:: @@ -494,6 +515,34 @@ No tail comments: /* This magic initialization needs a comment. Maybe not? */ seed = MAGIC_CONSTANT; + Use C++ style, tail comments when documenting structs in headers to + achieve a more compact layout and better readability:: + + // eax + u32 x2apic_shift : 5, // Number of bits to shift APIC ID right + // for the topology ID at the next level + : 27; // Reserved + // ebx + u32 num_processors : 16, // Number of processors at current level + : 16; // Reserved + + versus:: + + /* eax */ + /* + * Number of bits to shift APIC ID right for the topology ID + * at the next level + */ + u32 x2apic_shift : 5, + /* Reserved */ + : 27; + + /* ebx */ + /* Number of processors at current level */ + u32 num_processors : 16, + /* Reserved */ + : 16; + Comment the important things: Comments should be added where the operation is not obvious. Documenting diff --git a/Documentation/process/programming-language.rst b/Documentation/process/programming-language.rst index 5fc9160ca1fa..bc56dee6d0bc 100644 --- a/Documentation/process/programming-language.rst +++ b/Documentation/process/programming-language.rst @@ -12,10 +12,6 @@ under ``-std=gnu11`` [gcc-c-dialect-options]_: the GNU dialect of ISO C11. This dialect contains many extensions to the language [gnu-extensions]_, and many of them are used within the kernel as a matter of course. -There is some support for compiling the kernel with ``icc`` [icc]_ for several -of the architectures, although at the time of writing it is not completed, -requiring third-party patches. - Attributes ---------- @@ -35,12 +31,28 @@ in order to feature detect which ones can be used and/or to shorten the code. Please refer to ``include/linux/compiler_attributes.h`` for more information. +Rust +---- + +The kernel has experimental support for the Rust programming language +[rust-language]_ under ``CONFIG_RUST``. It is compiled with ``rustc`` [rustc]_ +under ``--edition=2021`` [rust-editions]_. Editions are a way to introduce +small changes to the language that are not backwards compatible. + +On top of that, some unstable features [rust-unstable-features]_ are used in +the kernel. Unstable features may change in the future, thus it is an important +goal to reach a point where only stable features are used. + +Please refer to Documentation/rust/index.rst for more information. + .. [c-language] http://www.open-std.org/jtc1/sc22/wg14/www/standards .. [gcc] https://gcc.gnu.org .. [clang] https://clang.llvm.org -.. [icc] https://software.intel.com/en-us/c-compilers .. [gcc-c-dialect-options] https://gcc.gnu.org/onlinedocs/gcc/C-Dialect-Options.html .. [gnu-extensions] https://gcc.gnu.org/onlinedocs/gcc/C-Extensions.html .. [gcc-attribute-syntax] https://gcc.gnu.org/onlinedocs/gcc/Attribute-Syntax.html .. [n2049] http://www.open-std.org/jtc1/sc22/wg14/www/docs/n2049.pdf - +.. [rust-language] https://www.rust-lang.org +.. [rustc] https://doc.rust-lang.org/rustc/ +.. [rust-editions] https://doc.rust-lang.org/edition-guide/editions/ +.. [rust-unstable-features] https://github.com/Rust-for-Linux/linux/issues/2 diff --git a/Documentation/process/researcher-guidelines.rst b/Documentation/process/researcher-guidelines.rst index afc944e0e898..beb484c5965d 100644 --- a/Documentation/process/researcher-guidelines.rst +++ b/Documentation/process/researcher-guidelines.rst @@ -44,6 +44,33 @@ explicit agreement of, and full disclosure to, the individual developers involved. Developers cannot be interacted with/experimented on without consent; this, too, is standard research ethics. +Surveys +======= + +Research often takes the form of surveys sent to maintainers or +contributors. As a general rule, though, the kernel community derives +little value from these surveys. The kernel development process works +because every developer benefits from their participation, even working +with others who have different goals. Responding to a survey, though, is a +one-way demand placed on busy developers with no corresponding benefit to +themselves or to the kernel community as a whole. For this reason, this +method of research is discouraged. + +Kernel community members already receive far too much email and are likely +to perceive survey requests as just another demand on their time. Sending +such requests deprives the community of valuable contributor time and is +unlikely to yield a statistically useful response. + +As an alternative, researchers should consider attending developer events, +hosting sessions where the research project and its benefits to the +participants can be explained, and interacting directly with the community +there. The information received will be far richer than that obtained from +an email survey, and the community will gain from the ability to learn from +your insights as well. + +Patches +======= + To help clarify: sending patches to developers *is* interacting with them, but they have already consented to receiving *good faith contributions*. Sending intentionally flawed/vulnerable patches or @@ -68,7 +95,7 @@ Before contributing, carefully read the appropriate documentation: * Documentation/process/development-process.rst * Documentation/process/submitting-patches.rst * Documentation/admin-guide/reporting-issues.rst -* Documentation/admin-guide/security-bugs.rst +* Documentation/process/security-bugs.rst Then send a patch (including a commit log with all the details listed below) and follow up on any feedback from other developers. @@ -140,4 +167,4 @@ If no one can be found to internally review patches and you need help finding such a person, or if you have any other questions related to this document and the developer community's expectations, please reach out to the private Technical Advisory Board mailing list: -<tech-board@lists.linux-foundation.org>. +<tech-board@groups.linuxfoundation.org>. diff --git a/Documentation/process/security-bugs.rst b/Documentation/process/security-bugs.rst new file mode 100644 index 000000000000..84657e7d2e5b --- /dev/null +++ b/Documentation/process/security-bugs.rst @@ -0,0 +1,116 @@ +.. _securitybugs: + +Security bugs +============= + +Linux kernel developers take security very seriously. As such, we'd +like to know when a security bug is found so that it can be fixed and +disclosed as quickly as possible. Please report security bugs to the +Linux kernel security team. + +The security team and maintainers almost always require additional +information beyond what was initially provided in a report and rely on +active and efficient collaboration with the reporter to perform further +testing (e.g., verifying versions, configuration options, mitigations, or +patches). Before contacting the security team, the reporter must ensure +they are available to explain their findings, engage in discussions, and +run additional tests. Reports where the reporter does not respond promptly +or cannot effectively discuss their findings may be abandoned if the +communication does not quickly improve. + +As it is with any bug, the more information provided the easier it +will be to diagnose and fix. Please review the procedure outlined in +'Documentation/admin-guide/reporting-issues.rst' if you are unclear about what +information is helpful. Any exploit code is very helpful and will not +be released without consent from the reporter unless it has already been +made public. + +The Linux kernel security team can be contacted by email at +<security@kernel.org>. This is a private list of security officers +who will help verify the bug report and develop and release a fix. +If you already have a fix, please include it with your report, as +that can speed up the process considerably. It is possible that the +security team will bring in extra help from area maintainers to +understand and fix the security vulnerability. + +Please send plain text emails without attachments where possible. +It is much harder to have a context-quoted discussion about a complex +issue if all the details are hidden away in attachments. Think of it like a +:doc:`regular patch submission <../process/submitting-patches>` +(even if you don't have a patch yet): describe the problem and impact, list +reproduction steps, and follow it with a proposed fix, all in plain text. + +Disclosure and embargoed information +------------------------------------ + +The security list is not a disclosure channel. For that, see Coordination +below. + +Once a robust fix has been developed, the release process starts. Fixes +for publicly known bugs are released immediately. + +Although our preference is to release fixes for publicly undisclosed bugs +as soon as they become available, this may be postponed at the request of +the reporter or an affected party for up to 7 calendar days from the start +of the release process, with an exceptional extension to 14 calendar days +if it is agreed that the criticality of the bug requires more time. The +only valid reason for deferring the publication of a fix is to accommodate +the logistics of QA and large scale rollouts which require release +coordination. + +While embargoed information may be shared with trusted individuals in +order to develop a fix, such information will not be published alongside +the fix or on any other disclosure channel without the permission of the +reporter. This includes but is not limited to the original bug report +and followup discussions (if any), exploits, CVE information or the +identity of the reporter. + +In other words our only interest is in getting bugs fixed. All other +information submitted to the security list and any followup discussions +of the report are treated confidentially even after the embargo has been +lifted, in perpetuity. + +Coordination with other groups +------------------------------ + +While the kernel security team solely focuses on getting bugs fixed, +other groups focus on fixing issues in distros and coordinating +disclosure between operating system vendors. Coordination is usually +handled by the "linux-distros" mailing list and disclosure by the +public "oss-security" mailing list, both of which are closely related +and presented in the linux-distros wiki: +<https://oss-security.openwall.org/wiki/mailing-lists/distros> + +Please note that the respective policies and rules are different since +the 3 lists pursue different goals. Coordinating between the kernel +security team and other teams is difficult since for the kernel security +team occasional embargoes (as subject to a maximum allowed number of +days) start from the availability of a fix, while for "linux-distros" +they start from the initial post to the list regardless of the +availability of a fix. + +As such, the kernel security team strongly recommends that as a reporter +of a potential security issue you DO NOT contact the "linux-distros" +mailing list UNTIL a fix is accepted by the affected code's maintainers +and you have read the distros wiki page above and you fully understand +the requirements that contacting "linux-distros" will impose on you and +the kernel community. This also means that in general it doesn't make +sense to Cc: both lists at once, except maybe for coordination if and +while an accepted fix has not yet been merged. In other words, until a +fix is accepted do not Cc: "linux-distros", and after it's merged do not +Cc: the kernel security team. + +CVE assignment +-------------- + +The security team does not assign CVEs, nor do we require them for +reports or fixes, as this can needlessly complicate the process and may +delay the bug handling. If a reporter wishes to have a CVE identifier +assigned for a confirmed issue, they can contact the :doc:`kernel CVE +assignment team<../process/cve>` to obtain one. + +Non-disclosure agreements +------------------------- + +The Linux kernel security team is not a formal body and therefore unable +to enter any non-disclosure agreements. diff --git a/Documentation/process/stable-kernel-rules.rst b/Documentation/process/stable-kernel-rules.rst index 2fd8aa593a28..edf90bbe30f4 100644 --- a/Documentation/process/stable-kernel-rules.rst +++ b/Documentation/process/stable-kernel-rules.rst @@ -6,30 +6,29 @@ Everything you ever wanted to know about Linux -stable releases Rules on what kind of patches are accepted, and which ones are not, into the "-stable" tree: - - It must be obviously correct and tested. - - It cannot be bigger than 100 lines, with context. - - It must fix only one thing. - - It must fix a real bug that bothers people (not a, "This could be a - problem..." type thing). - - It must fix a problem that causes a build error (but not for things - marked CONFIG_BROKEN), an oops, a hang, data corruption, a real - security issue, or some "oh, that's not good" issue. In short, something - critical. - - Serious issues as reported by a user of a distribution kernel may also - be considered if they fix a notable performance or interactivity issue. - As these fixes are not as obvious and have a higher risk of a subtle - regression they should only be submitted by a distribution kernel - maintainer and include an addendum linking to a bugzilla entry if it - exists and additional information on the user-visible impact. - - New device IDs and quirks are also accepted. - - No "theoretical race condition" issues, unless an explanation of how the - race can be exploited is also provided. - - It cannot contain any "trivial" fixes in it (spelling changes, - whitespace cleanups, etc). - - It must follow the - :ref:`Documentation/process/submitting-patches.rst <submittingpatches>` - rules. - - It or an equivalent fix must already exist in Linus' tree (upstream). +- It or an equivalent fix must already exist in Linux mainline (upstream). +- It must be obviously correct and tested. +- It cannot be bigger than 100 lines, with context. +- It must follow the + :ref:`Documentation/process/submitting-patches.rst <submittingpatches>` + rules. +- It must either fix a real bug that bothers people or just add a device ID. + To elaborate on the former: + + - It fixes a problem like an oops, a hang, data corruption, a real security + issue, a hardware quirk, a build error (but not for things marked + CONFIG_BROKEN), or some "oh, that's not good" issue. + - Serious issues as reported by a user of a distribution kernel may also + be considered if they fix a notable performance or interactivity issue. + As these fixes are not as obvious and have a higher risk of a subtle + regression they should only be submitted by a distribution kernel + maintainer and include an addendum linking to a bugzilla entry if it + exists and additional information on the user-visible impact. + - No "This could be a problem..." type of things like a "theoretical race + condition", unless an explanation of how the bug can be exploited is also + provided. + - No "trivial" fixes without benefit for users (spelling changes, whitespace + cleanups, etc). Procedure for submitting patches to the -stable tree @@ -39,166 +38,197 @@ Procedure for submitting patches to the -stable tree Security patches should not be handled (solely) by the -stable review process but should follow the procedures in - :ref:`Documentation/admin-guide/security-bugs.rst <securitybugs>`. + :ref:`Documentation/process/security-bugs.rst <securitybugs>`. -For all other submissions, choose one of the following procedures ------------------------------------------------------------------ +There are three options to submit a change to -stable trees: + +1. Add a 'stable tag' to the description of a patch you then submit for + mainline inclusion. +2. Ask the stable team to pick up a patch already mainlined. +3. Submit a patch to the stable team that is equivalent to a change already + mainlined. + +The sections below describe each of the options in more detail. + +:ref:`option_1` is **strongly** preferred, it is the easiest and most common. +:ref:`option_2` is mainly meant for changes where backporting was not considered +at the time of submission. :ref:`option_3` is an alternative to the two earlier +options for cases where a mainlined patch needs adjustments to apply in older +series (for example due to API changes). + +When using option 2 or 3 you can ask for your change to be included in specific +stable series. When doing so, ensure the fix or an equivalent is applicable, +submitted, or already present in all newer stable trees still supported. This is +meant to prevent regressions that users might later encounter on updating, if +e.g. a fix merged for 5.19-rc1 would be backported to 5.10.y, but not to 5.15.y. .. _option_1: Option 1 ******** -To have the patch automatically included in the stable tree, add the tag +To have a patch you submit for mainline inclusion later automatically picked up +for stable trees, add this tag in the sign-off area:: -.. code-block:: none + Cc: stable@vger.kernel.org - Cc: stable@vger.kernel.org +Use ``Cc: stable@kernel.org`` instead when fixing unpublished vulnerabilities: +it reduces the chance of accidentally exposing the fix to the public by way of +'git send-email', as mails sent to that address are not delivered anywhere. -in the sign-off area. Once the patch is merged it will be applied to -the stable tree without anything else needing to be done by the author -or subsystem maintainer. +Once the patch is mainlined it will be applied to the stable tree without +anything else needing to be done by the author or subsystem maintainer. -.. _option_2: +To send additional instructions to the stable team, use a shell-style inline +comment to pass arbitrary or predefined notes: -Option 2 -******** +* Specify any additional patch prerequisites for cherry picking:: -After the patch has been merged to Linus' tree, send an email to -stable@vger.kernel.org containing the subject of the patch, the commit ID, -why you think it should be applied, and what kernel version you wish it to -be applied to. + Cc: <stable@vger.kernel.org> # 3.3.x: a1f84a3: sched: Check for idle + Cc: <stable@vger.kernel.org> # 3.3.x: 1b9508f: sched: Rate-limit newidle + Cc: <stable@vger.kernel.org> # 3.3.x: fd21073: sched: Fix affinity logic + Cc: <stable@vger.kernel.org> # 3.3.x + Signed-off-by: Ingo Molnar <mingo@elte.hu> -.. _option_3: + The tag sequence has the meaning of:: -Option 3 -******** + git cherry-pick a1f84a3 + git cherry-pick 1b9508f + git cherry-pick fd21073 + git cherry-pick <this commit> -Send the patch, after verifying that it follows the above rules, to -stable@vger.kernel.org. You must note the upstream commit ID in the -changelog of your submission, as well as the kernel version you wish -it to be applied to. + Note that for a patch series, you do not have to list as prerequisites the + patches present in the series itself. For example, if you have the following + patch series:: + + patch1 + patch2 -:ref:`option_1` is **strongly** preferred, is the easiest and most common. -:ref:`option_2` and :ref:`option_3` are more useful if the patch isn't deemed -worthy at the time it is applied to a public git tree (for instance, because -it deserves more regression testing first). :ref:`option_3` is especially -useful if the original upstream patch needs to be backported (for example -the backport needs some special handling due to e.g. API changes). + where patch2 depends on patch1, you do not have to list patch1 as + prerequisite of patch2 if you have already marked patch1 for stable + inclusion. -Note that for :ref:`option_3`, if the patch deviates from the original -upstream patch (for example because it had to be backported) this must be very -clearly documented and justified in the patch description. +* Point out kernel version prerequisites:: -The upstream commit ID must be specified with a separate line above the commit -text, like this: + Cc: <stable@vger.kernel.org> # 3.3.x -.. code-block:: none + The tag has the meaning of:: - commit <sha1> upstream. + git cherry-pick <this commit> -or alternatively: + For each "-stable" tree starting with the specified version. -.. code-block:: none + Note, such tagging is unnecessary if the stable team can derive the + appropriate versions from Fixes: tags. - [ Upstream commit <sha1> ] +* Delay pick up of patches:: -Additionally, some patches submitted via :ref:`option_1` may have additional -patch prerequisites which can be cherry-picked. This can be specified in the -following format in the sign-off area: + Cc: <stable@vger.kernel.org> # after -rc3 -.. code-block:: none +* Point out known problems:: - Cc: <stable@vger.kernel.org> # 3.3.x: a1f84a3: sched: Check for idle - Cc: <stable@vger.kernel.org> # 3.3.x: 1b9508f: sched: Rate-limit newidle - Cc: <stable@vger.kernel.org> # 3.3.x: fd21073: sched: Fix affinity logic - Cc: <stable@vger.kernel.org> # 3.3.x - Signed-off-by: Ingo Molnar <mingo@elte.hu> + Cc: <stable@vger.kernel.org> # see patch description, needs adjustments for <= 6.3 -The tag sequence has the meaning of: +There furthermore is a variant of the stable tag you can use to make the stable +team's backporting tools (e.g AUTOSEL or scripts that look for commits +containing a 'Fixes:' tag) ignore a change:: -.. code-block:: none + Cc: <stable+noautosel@kernel.org> # reason goes here, and must be present - git cherry-pick a1f84a3 - git cherry-pick 1b9508f - git cherry-pick fd21073 - git cherry-pick <this commit> +.. _option_2: + +Option 2 +******** + +If the patch already has been merged to mainline, send an email to +stable@vger.kernel.org containing the subject of the patch, the commit ID, +why you think it should be applied, and what kernel versions you wish it to +be applied to. -Also, some patches may have kernel version prerequisites. This can be -specified in the following format in the sign-off area: +.. _option_3: -.. code-block:: none +Option 3 +******** - Cc: <stable@vger.kernel.org> # 3.3.x +Send the patch, after verifying that it follows the above rules, to +stable@vger.kernel.org and mention the kernel versions you wish it to be applied +to. When doing so, you must note the upstream commit ID in the changelog of your +submission with a separate line above the commit text, like this:: -The tag has the meaning of: + commit <sha1> upstream. -.. code-block:: none +Or alternatively:: - git cherry-pick <this commit> + [ Upstream commit <sha1> ] -For each "-stable" tree starting with the specified version. +If the submitted patch deviates from the original upstream patch (for example +because it had to be adjusted for the older API), this must be very clearly +documented and justified in the patch description. -Following the submission: - - The sender will receive an ACK when the patch has been accepted into the - queue, or a NAK if the patch is rejected. This response might take a few - days, according to the developer's schedules. - - If accepted, the patch will be added to the -stable queue, for review by - other developers and by the relevant subsystem maintainer. +Following the submission +------------------------ + +The sender will receive an ACK when the patch has been accepted into the +queue, or a NAK if the patch is rejected. This response might take a few +days, according to the schedules of the stable team members. + +If accepted, the patch will be added to the -stable queue, for review by other +developers and by the relevant subsystem maintainer. Review cycle ------------ - - When the -stable maintainers decide for a review cycle, the patches will be - sent to the review committee, and the maintainer of the affected area of - the patch (unless the submitter is the maintainer of the area) and CC: to - the linux-kernel mailing list. - - The review committee has 48 hours in which to ACK or NAK the patch. - - If the patch is rejected by a member of the committee, or linux-kernel - members object to the patch, bringing up issues that the maintainers and - members did not realize, the patch will be dropped from the queue. - - The ACKed patches will be posted again as part of release candidate (-rc) - to be tested by developers and testers. - - Usually only one -rc release is made, however if there are any outstanding - issues, some patches may be modified or dropped or additional patches may - be queued. Additional -rc releases are then released and tested until no - issues are found. - - Responding to the -rc releases can be done on the mailing list by sending - a "Tested-by:" email with any testing information desired. The "Tested-by:" - tags will be collected and added to the release commit. - - At the end of the review cycle, the new -stable release will be released - containing all the queued and tested patches. - - Security patches will be accepted into the -stable tree directly from the - security kernel team, and not go through the normal review cycle. - Contact the kernel security team for more details on this procedure. +- When the -stable maintainers decide for a review cycle, the patches will be + sent to the review committee, and the maintainer of the affected area of + the patch (unless the submitter is the maintainer of the area) and CC: to + the linux-kernel mailing list. +- The review committee has 48 hours in which to ACK or NAK the patch. +- If the patch is rejected by a member of the committee, or linux-kernel + members object to the patch, bringing up issues that the maintainers and + members did not realize, the patch will be dropped from the queue. +- The ACKed patches will be posted again as part of release candidate (-rc) + to be tested by developers and testers. +- Usually only one -rc release is made, however if there are any outstanding + issues, some patches may be modified or dropped or additional patches may + be queued. Additional -rc releases are then released and tested until no + issues are found. +- Responding to the -rc releases can be done on the mailing list by sending + a "Tested-by:" email with any testing information desired. The "Tested-by:" + tags will be collected and added to the release commit. +- At the end of the review cycle, the new -stable release will be released + containing all the queued and tested patches. +- Security patches will be accepted into the -stable tree directly from the + security kernel team, and not go through the normal review cycle. + Contact the kernel security team for more details on this procedure. + Trees ----- - - The queues of patches, for both completed versions and in progress - versions can be found at: +- The queues of patches, for both completed versions and in progress + versions can be found at: - https://git.kernel.org/pub/scm/linux/kernel/git/stable/stable-queue.git + https://git.kernel.org/pub/scm/linux/kernel/git/stable/stable-queue.git - - The finalized and tagged releases of all stable kernels can be found - in separate branches per version at: +- The finalized and tagged releases of all stable kernels can be found + in separate branches per version at: - https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git + https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git - - The release candidate of all stable kernel versions can be found at: +- The release candidate of all stable kernel versions can be found at: - https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable-rc.git/ + https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable-rc.git/ - .. warning:: - The -stable-rc tree is a snapshot in time of the stable-queue tree and - will change frequently, hence will be rebased often. It should only be - used for testing purposes (e.g. to be consumed by CI systems). + .. warning:: + The -stable-rc tree is a snapshot in time of the stable-queue tree and + will change frequently, hence will be rebased often. It should only be + used for testing purposes (e.g. to be consumed by CI systems). Review committee ---------------- - - This is made up of a number of kernel developers who have volunteered for - this task, and a few that haven't. +- This is made up of a number of kernel developers who have volunteered for + this task, and a few that haven't. diff --git a/Documentation/process/submit-checklist.rst b/Documentation/process/submit-checklist.rst index b1bc2d37bd0a..beb7f94279fd 100644 --- a/Documentation/process/submit-checklist.rst +++ b/Documentation/process/submit-checklist.rst @@ -1,7 +1,8 @@ .. _submitchecklist: +======================================= Linux Kernel patch submission checklist -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +======================================= Here are some basic things that developers should do if they want to see their kernel patch submissions accepted more quickly. @@ -10,111 +11,127 @@ These are all above and beyond the documentation that is provided in :ref:`Documentation/process/submitting-patches.rst <submittingpatches>` and elsewhere regarding submitting Linux kernel patches. +Review your code +================ 1) If you use a facility then #include the file that defines/declares that facility. Don't depend on other header files pulling in ones that you use. -2) Builds cleanly: - - a) with applicable or modified ``CONFIG`` options ``=y``, ``=m``, and - ``=n``. No ``gcc`` warnings/errors, no linker warnings/errors. - - b) Passes ``allnoconfig``, ``allmodconfig`` - - c) Builds successfully when using ``O=builddir`` - - d) Any Documentation/ changes build successfully without new warnings/errors. - Use ``make htmldocs`` or ``make pdfdocs`` to check the build and - fix any issues. +2) Check your patch for general style as detailed in + :ref:`Documentation/process/coding-style.rst <codingstyle>`. -3) Builds on multiple CPU architectures by using local cross-compile tools - or some other build farm. +3) All memory barriers {e.g., ``barrier()``, ``rmb()``, ``wmb()``} need a + comment in the source code that explains the logic of what they are doing + and why. -4) ppc64 is a good architecture for cross-compilation checking because it - tends to use ``unsigned long`` for 64-bit quantities. +Review Kconfig changes +====================== -5) Check your patch for general style as detailed in - :ref:`Documentation/process/coding-style.rst <codingstyle>`. - Check for trivial violations with the patch style checker prior to - submission (``scripts/checkpatch.pl``). - You should be able to justify all violations that remain in - your patch. - -6) Any new or modified ``CONFIG`` options do not muck up the config menu and +1) Any new or modified ``CONFIG`` options do not muck up the config menu and default to off unless they meet the exception criteria documented in ``Documentation/kbuild/kconfig-language.rst`` Menu attributes: default value. -7) All new ``Kconfig`` options have help text. +2) All new ``Kconfig`` options have help text. -8) Has been carefully reviewed with respect to relevant ``Kconfig`` - combinations. This is very hard to get right with testing -- brainpower +3) Has been carefully reviewed with respect to relevant ``Kconfig`` + combinations. This is very hard to get right with testing---brainpower pays off here. -9) Check cleanly with sparse. - -10) Use ``make checkstack`` and fix any problems that it finds. - - .. note:: +Provide documentation +===================== - ``checkstack`` does not point out problems explicitly, - but any one function that uses more than 512 bytes on the stack is a - candidate for change. +1) Include :ref:`kernel-doc <kernel_doc>` to document global kernel APIs. + (Not required for static functions, but OK there also.) -11) Include :ref:`kernel-doc <kernel_doc>` to document global kernel APIs. - (Not required for static functions, but OK there also.) Use - ``make htmldocs`` or ``make pdfdocs`` to check the - :ref:`kernel-doc <kernel_doc>` and fix any issues. +2) All new ``/proc`` entries are documented under ``Documentation/`` -12) Has been tested with ``CONFIG_PREEMPT``, ``CONFIG_DEBUG_PREEMPT``, - ``CONFIG_DEBUG_SLAB``, ``CONFIG_DEBUG_PAGEALLOC``, ``CONFIG_DEBUG_MUTEXES``, - ``CONFIG_DEBUG_SPINLOCK``, ``CONFIG_DEBUG_ATOMIC_SLEEP``, - ``CONFIG_PROVE_RCU`` and ``CONFIG_DEBUG_OBJECTS_RCU_HEAD`` all - simultaneously enabled. +3) All new kernel boot parameters are documented in + ``Documentation/admin-guide/kernel-parameters.rst``. -13) Has been build- and runtime tested with and without ``CONFIG_SMP`` and - ``CONFIG_PREEMPT.`` +4) All new module parameters are documented with ``MODULE_PARM_DESC()`` -14) All codepaths have been exercised with all lockdep features enabled. +5) All new userspace interfaces are documented in ``Documentation/ABI/``. + See Documentation/admin-guide/abi.rst (or ``Documentation/ABI/README``) + for more information. + Patches that change userspace interfaces should be CCed to + linux-api@vger.kernel.org. -15) All new ``/proc`` entries are documented under ``Documentation/`` +6) If any ioctl's are added by the patch, then also update + ``Documentation/userspace-api/ioctl/ioctl-number.rst``. -16) All new kernel boot parameters are documented in - ``Documentation/admin-guide/kernel-parameters.rst``. +Check your code with tools +========================== -17) All new module parameters are documented with ``MODULE_PARM_DESC()`` +1) Check for trivial violations with the patch style checker prior to + submission (``scripts/checkpatch.pl``). + You should be able to justify all violations that remain in + your patch. -18) All new userspace interfaces are documented in ``Documentation/ABI/``. - See ``Documentation/ABI/README`` for more information. - Patches that change userspace interfaces should be CCed to - linux-api@vger.kernel.org. +2) Check cleanly with sparse. -19) Has been checked with injection of at least slab and page-allocation - failures. See ``Documentation/fault-injection/``. +3) Use ``make checkstack`` and fix any problems that it finds. + Note that ``checkstack`` does not point out problems explicitly, + but any one function that uses more than 512 bytes on the stack is a + candidate for change. - If the new code is substantial, addition of subsystem-specific fault - injection might be appropriate. +Build your code +=============== -20) Newly-added code has been compiled with ``gcc -W`` (use - ``make KCFLAGS=-W``). This will generate lots of noise, but is good - for finding bugs like "warning: comparison between signed and unsigned". +1) Builds cleanly: -21) Tested after it has been merged into the -mm patchset to make sure - that it still works with all of the other queued patches and various - changes in the VM, VFS, and other subsystems. + a) with applicable or modified ``CONFIG`` options ``=y``, ``=m``, and + ``=n``. No ``gcc`` warnings/errors, no linker warnings/errors. -22) All memory barriers {e.g., ``barrier()``, ``rmb()``, ``wmb()``} need a - comment in the source code that explains the logic of what they are doing - and why. + b) Passes ``allnoconfig``, ``allmodconfig`` -23) If any ioctl's are added by the patch, then also update - ``Documentation/userspace-api/ioctl/ioctl-number.rst``. + c) Builds successfully when using ``O=builddir`` -24) If your modified source code depends on or uses any of the kernel - APIs or features that are related to the following ``Kconfig`` symbols, - then test multiple builds with the related ``Kconfig`` symbols disabled - and/or ``=m`` (if that option is available) [not all of these at the - same time, just various/random combinations of them]: + d) Any Documentation/ changes build successfully without new warnings/errors. + Use ``make htmldocs`` or ``make pdfdocs`` to check the build and + fix any issues. - ``CONFIG_SMP``, ``CONFIG_SYSFS``, ``CONFIG_PROC_FS``, ``CONFIG_INPUT``, ``CONFIG_PCI``, ``CONFIG_BLOCK``, ``CONFIG_PM``, ``CONFIG_MAGIC_SYSRQ``, - ``CONFIG_NET``, ``CONFIG_INET=n`` (but latter with ``CONFIG_NET=y``). +2) Builds on multiple CPU architectures by using local cross-compile tools + or some other build farm. + Note that testing against architectures of different word sizes + (32- and 64-bit) and different endianness (big- and little-) is effective + in catching various portability issues due to false assumptions on + representable quantity range, data alignment, or endianness, among + others. + +3) Newly-added code has been compiled with ``gcc -W`` (use + ``make KCFLAGS=-W``). This will generate lots of noise, but is good + for finding bugs like "warning: comparison between signed and unsigned". + +4) If your modified source code depends on or uses any of the kernel + APIs or features that are related to the following ``Kconfig`` symbols, + then test multiple builds with the related ``Kconfig`` symbols disabled + and/or ``=m`` (if that option is available) [not all of these at the + same time, just various/random combinations of them]: + + ``CONFIG_SMP``, ``CONFIG_SYSFS``, ``CONFIG_PROC_FS``, ``CONFIG_INPUT``, + ``CONFIG_PCI``, ``CONFIG_BLOCK``, ``CONFIG_PM``, ``CONFIG_MAGIC_SYSRQ``, + ``CONFIG_NET``, ``CONFIG_INET=n`` (but latter with ``CONFIG_NET=y``). + +Test your code +============== + +1) Has been tested with ``CONFIG_PREEMPT``, ``CONFIG_DEBUG_PREEMPT``, + ``CONFIG_SLUB_DEBUG``, ``CONFIG_DEBUG_PAGEALLOC``, ``CONFIG_DEBUG_MUTEXES``, + ``CONFIG_DEBUG_SPINLOCK``, ``CONFIG_DEBUG_ATOMIC_SLEEP``, + ``CONFIG_PROVE_RCU`` and ``CONFIG_DEBUG_OBJECTS_RCU_HEAD`` all + simultaneously enabled. + +2) Has been build- and runtime tested with and without ``CONFIG_SMP`` and + ``CONFIG_PREEMPT.`` + +3) All codepaths have been exercised with all lockdep features enabled. + +4) Has been checked with injection of at least slab and page-allocation + failures. See ``Documentation/fault-injection/``. + If the new code is substantial, addition of subsystem-specific fault + injection might be appropriate. + +5) Tested with the most recent tag of linux-next to make sure that it still + works with all of the other queued patches and various changes in the VM, + VFS, and other subsystems. diff --git a/Documentation/process/submitting-patches.rst b/Documentation/process/submitting-patches.rst index 7dc94555417d..9a509f1a6873 100644 --- a/Documentation/process/submitting-patches.rst +++ b/Documentation/process/submitting-patches.rst @@ -113,18 +113,16 @@ there is no collision with your six-character ID now, that condition may change five years from now. If related discussions or any other background information behind the change -can be found on the web, add 'Link:' tags pointing to it. In case your patch -fixes a bug, for example, add a tag with a URL referencing the report in the -mailing list archives or a bug tracker; if the patch is a result of some -earlier mailing list discussion or something documented on the web, point to -it. +can be found on the web, add 'Link:' tags pointing to it. If the patch is a +result of some earlier mailing list discussions or something documented on the +web, point to it. When linking to mailing list archives, preferably use the lore.kernel.org message archiver service. To create the link URL, use the contents of the -``Message-Id`` header of the message without the surrounding angle brackets. +``Message-ID`` header of the message without the surrounding angle brackets. For example:: - Link: https://lore.kernel.org/r/30th.anniversary.repost@klaava.Helsinki.FI/ + Link: https://lore.kernel.org/30th.anniversary.repost@klaava.Helsinki.FI Please check the link to make sure that it is actually working and points to the relevant message. @@ -134,11 +132,21 @@ resources. In addition to giving a URL to a mailing list archive or bug, summarize the relevant points of the discussion that led to the patch as submitted. +In case your patch fixes a bug, use the 'Closes:' tag with a URL referencing +the report in the mailing list archives or a public bug tracker. For example:: + + Closes: https://example.com/issues/1234 + +Some bug trackers have the ability to close issues automatically when a +commit with such a tag is applied. Some bots monitoring mailing lists can +also track such tags and take certain actions. Private bug trackers and +invalid URLs are forbidden. + If your patch fixes a bug in a specific commit, e.g. you found an issue using -``git bisect``, please use the 'Fixes:' tag with the first 12 characters of -the SHA-1 ID, and the one line summary. Do not split the tag across multiple -lines, tags are exempt from the "wrap at 75 columns" rule in order to simplify -parsing scripts. For example:: +``git bisect``, please use the 'Fixes:' tag with at least the first 12 +characters of the SHA-1 ID, and the one line summary. Do not split the tag +across multiple lines, tags are exempt from the "wrap at 75 columns" rule in +order to simplify parsing scripts. For example:: Fixes: 54a4f0239f2e ("KVM: MMU: make kvm_mmu_zap_page() return the number of pages it actually freed") @@ -223,26 +231,21 @@ patch. Select the recipients for your patch ------------------------------------ -You should always copy the appropriate subsystem maintainer(s) on any patch -to code that they maintain; look through the MAINTAINERS file and the -source code revision history to see who those maintainers are. The -script scripts/get_maintainer.pl can be very useful at this step (pass paths to -your patches as arguments to scripts/get_maintainer.pl). If you cannot find a +You should always copy the appropriate subsystem maintainer(s) and list(s) on +any patch to code that they maintain; look through the MAINTAINERS file and the +source code revision history to see who those maintainers are. The script +scripts/get_maintainer.pl can be very useful at this step (pass paths to your +patches as arguments to scripts/get_maintainer.pl). If you cannot find a maintainer for the subsystem you are working on, Andrew Morton (akpm@linux-foundation.org) serves as a maintainer of last resort. -You should also normally choose at least one mailing list to receive a copy -of your patch set. linux-kernel@vger.kernel.org should be used by default -for all patches, but the volume on that list has caused a number of -developers to tune it out. Look in the MAINTAINERS file for a -subsystem-specific list; your patch will probably get more attention there. -Please do not spam unrelated lists, though. +linux-kernel@vger.kernel.org should be used by default for all patches, but the +volume on that list has caused a number of developers to tune it out. Please +do not spam unrelated lists and unrelated people, though. -Many kernel-related lists are hosted on vger.kernel.org; you can find a -list of them at http://vger.kernel.org/vger-lists.html. There are -kernel-related lists hosted elsewhere as well, though. - -Do not send more than 15 patches at once to the vger mailing lists!!! +Many kernel-related lists are hosted at kernel.org; you can find a list +of them at https://subspace.kernel.org. There are kernel-related lists +hosted elsewhere as well, though. Linus Torvalds is the final arbiter of all changes accepted into the Linux kernel. His e-mail address is <torvalds@linux-foundation.org>. @@ -254,7 +257,7 @@ If you have a patch that fixes an exploitable security bug, send that patch to security@kernel.org. For severe bugs, a short embargo may be considered to allow distributors to get the patch out to users; in such cases, obviously, the patch should not be sent to any public lists. See also -Documentation/admin-guide/security-bugs.rst. +Documentation/process/security-bugs.rst. Patches that fix a severe bug in a released kernel should be directed toward the stable maintainers by putting a line like this:: @@ -320,12 +323,39 @@ for their time. Code review is a tiring and time-consuming process, and reviewers sometimes get grumpy. Even in that case, though, respond politely and address the problems they have pointed out. When sending a next version, add a ``patch changelog`` to the cover letter or to individual patches -explaining difference aganst previous submission (see +explaining difference against previous submission (see :ref:`the_canonical_patch_format`). +Notify people that commented on your patch about new versions by adding them to +the patches CC list. See Documentation/process/email-clients.rst for recommendations on email clients and mailing list etiquette. +.. _interleaved_replies: + +Use trimmed interleaved replies in email discussions +---------------------------------------------------- +Top-posting is strongly discouraged in Linux kernel development +discussions. Interleaved (or "inline") replies make conversations much +easier to follow. For more details see: +https://en.wikipedia.org/wiki/Posting_style#Interleaved_style + +As is frequently quoted on the mailing list:: + + A: http://en.wikipedia.org/wiki/Top_post + Q: Where do I find info about this thing called top-posting? + A: Because it messes up the order in which people normally read text. + Q: Why is top-posting such a bad thing? + A: Top-posting. + Q: What is the most annoying thing in e-mail? + +Similarly, please trim all unneeded quotations that aren't relevant +to your reply. This makes responses easier to find, and saves time and +space. For more details see: http://daringfireball.net/2007/07/on_top :: + + A: No. + Q: Should I include quotations after my reply? + .. _resend_reminders: Don't get discouraged - or impatient @@ -336,10 +366,10 @@ busy people and may not get to your patch right away. Once upon a time, patches used to disappear into the void without comment, but the development process works more smoothly than that now. You should -receive comments within a week or so; if that does not happen, make sure -that you have sent your patches to the right place. Wait for a minimum of -one week before resubmitting or pinging reviewers - possibly longer during -busy times like merge windows. +receive comments within a few weeks (typically 2-3); if that does not +happen, make sure that you have sent your patches to the right place. +Wait for a minimum of one week before resubmitting or pinging reviewers +- possibly longer during busy times like merge windows. It's also ok to resend the patch or the patch series after a couple of weeks with the word "RESEND" added to the subject line:: @@ -407,7 +437,7 @@ then you just add a line saying:: Signed-off-by: Random J Developer <random@developer.example.org> -using your real name (sorry, no pseudonyms or anonymous contributions.) +using a known identity (sorry, no anonymous contributions.) This will be done for you automatically if you use ``git commit -s``. Reverts should also include "Signed-off-by". ``git revert -s`` does that for you. @@ -433,28 +463,42 @@ If a person was not directly involved in the preparation or handling of a patch but wishes to signify and record their approval of it then they can ask to have an Acked-by: line added to the patch's changelog. -Acked-by: is often used by the maintainer of the affected code when that +Acked-by: is meant to be used by those responsible for or involved with the +affected code in one way or another. Most commonly, the maintainer when that maintainer neither contributed to nor forwarded the patch. +Acked-by: may also be used by other stakeholders, such as people with domain +knowledge (e.g. the original author of the code being modified), userspace-side +reviewers for a kernel uAPI patch or key users of a feature. Optionally, in +these cases, it can be useful to add a "# Suffix" to clarify its meaning:: + + Acked-by: The Stakeholder <stakeholder@example.org> # As primary user + Acked-by: is not as formal as Signed-off-by:. It is a record that the acker has at least reviewed the patch and has indicated acceptance. Hence patch mergers will sometimes manually convert an acker's "yep, looks good to me" into an Acked-by: (but note that it is usually better to ask for an explicit ack). +Acked-by: is also less formal than Reviewed-by:. For instance, maintainers may +use it to signify that they are OK with a patch landing, but they may not have +reviewed it as thoroughly as if a Reviewed-by: was provided. Similarly, a key +user may not have carried out a technical review of the patch, yet they may be +satisfied with the general approach, the feature or the user-facing interface. + Acked-by: does not necessarily indicate acknowledgement of the entire patch. For example, if a patch affects multiple subsystems and has an Acked-by: from one subsystem maintainer then this usually indicates acknowledgement of just the part which affects that maintainer's code. Judgement should be used here. When in doubt people should refer to the original discussion in the mailing -list archives. +list archives. A "# Suffix" may also be used in this case to clarify. If a person has had the opportunity to comment on a patch, but has not provided such comments, you may optionally add a ``Cc:`` tag to the patch. -This is the only tag which might be added without an explicit action by the -person it names - but it should indicate that this person was copied on the -patch. This tag documents that potentially interested parties -have been included in the discussion. +This tag documents that potentially interested parties have been included in +the discussion. Note, this is one of only three tags you might be able to use +without explicit permission of the person named (see 'Tagging people requires +permission' below for details). Co-developed-by: states that the patch was co-created by multiple developers; it is used to give attribution to co-authors (in addition to the author @@ -496,10 +540,13 @@ Using Reported-by:, Tested-by:, Reviewed-by:, Suggested-by: and Fixes: ---------------------------------------------------------------------- The Reported-by tag gives credit to people who find bugs and report them and it -hopefully inspires them to help us again in the future. Please note that if -the bug was reported in private, then ask for permission first before using the -Reported-by tag. The tag is intended for bugs; please do not use it to credit -feature requests. +hopefully inspires them to help us again in the future. The tag is intended for +bugs; please do not use it to credit feature requests. The tag should be +followed by a Closes: tag pointing to the report, unless the report is not +available on the web. The Link: tag can be used instead of Closes: if the patch +fixes a part of the issue(s) being reported. Note, the Reported-by tag is one +of only three tags you might be able to use without explicit permission of the +person named (see 'Tagging people requires permission' below for details). A Tested-by: tag indicates that the patch has been successfully tested (in some environment) by the person named. This tag informs maintainers that @@ -545,18 +592,19 @@ Both Tested-by and Reviewed-by tags, once received on mailing list from tester or reviewer, should be added by author to the applicable patches when sending next versions. However if the patch has changed substantially in following version, these tags might not be applicable anymore and thus should be removed. -Usually removal of someone's Tested-by or Reviewed-by tags should be mentioned -in the patch changelog (after the '---' separator). +Usually removal of someone's Acked-by, Tested-by or Reviewed-by tags should be +mentioned in the patch changelog with an explanation (after the '---' +separator). A Suggested-by: tag indicates that the patch idea is suggested by the person -named and ensures credit to the person for the idea. Please note that this -tag should not be added without the reporter's permission, especially if the -idea was not posted in a public forum. That said, if we diligently credit our -idea reporters, they will, hopefully, be inspired to help us again in the -future. - -A Fixes: tag indicates that the patch fixes an issue in a previous commit. It -is used to make it easy to determine where a bug originated, which can help +named and ensures credit to the person for the idea: if we diligently credit +our idea reporters, they will, hopefully, be inspired to help us again in the +future. Note, this is one of only three tags you might be able to use without +explicit permission of the person named (see 'Tagging people requires +permission' below for details). + +A Fixes: tag indicates that the patch fixes a bug in a previous commit. It +is used to make it easy to determine where an issue originated, which can help review a bug fix. This tag also assists the stable kernel team in determining which stable kernel versions should receive your fix. This is the preferred method for indicating a bug fixed by the patch. See :ref:`describe_changes` @@ -567,6 +615,25 @@ process nor the requirement to Cc: stable@vger.kernel.org on all stable patch candidates. For more information, please read Documentation/process/stable-kernel-rules.rst. +Finally, while providing tags is welcome and typically very appreciated, please +note that signers (i.e. submitters and maintainers) may use their discretion in +applying offered tags. + +.. _tagging_people: + +Tagging people requires permission +---------------------------------- + +Be careful in the addition of the aforementioned tags to your patches, as all +except for Cc:, Reported-by:, and Suggested-by: need explicit permission of the +person named. For those three implicit permission is sufficient if the person +contributed to the Linux kernel using that name and email address according +to the lore archives or the commit history -- and in case of Reported-by: +and Suggested-by: did the reporting or suggestion in public. Note, +bugzilla.kernel.org is a public place in this sense, but email addresses +used there are private; so do not expose them in tags, unless the person +used them in earlier contributions. + .. _the_canonical_patch_format: The canonical patch format @@ -577,6 +644,9 @@ that, if you have your patches stored in a ``git`` repository, proper patch formatting can be had with ``git format-patch``. The tools cannot create the necessary text, though, so read the instructions below anyway. +Subject Line +^^^^^^^^^^^^ + The canonical patch subject line is:: Subject: [PATCH 001/123] subsystem: summary phrase @@ -650,6 +720,9 @@ Here are some good example Subjects:: Subject: [PATCH v2] sub/sys: Condensed patch summary Subject: [PATCH v2 M/N] sub/sys: Condensed patch summary +From Line +^^^^^^^^^ + The ``from`` line must be the very first line in the message body, and has the form: @@ -660,6 +733,15 @@ patch in the permanent changelog. If the ``from`` line is missing, then the ``From:`` line from the email header will be used to determine the patch author in the changelog. +The author may indicate their affiliation or the sponsor of the work +by adding the name of an organization to the ``from`` and ``SoB`` lines, +e.g.: + + From: Patch Author (Company) <author@example.com> + +Explanation Body +^^^^^^^^^^^^^^^^ + The explanation body will be committed to the permanent source changelog, so should make sense to a competent reader who has long since forgotten the immediate details of the discussion that might have led to @@ -675,6 +757,31 @@ _all_ of the compile failures; just enough that it is likely that someone searching for the patch can find it. As in the ``summary phrase``, it is important to be both succinct as well as descriptive. +.. _backtraces: + +Backtraces in commit messages +""""""""""""""""""""""""""""" + +Backtraces help document the call chain leading to a problem. However, +not all backtraces are helpful. For example, early boot call chains are +unique and obvious. Copying the full dmesg output verbatim, however, +adds distracting information like timestamps, module lists, register and +stack dumps. + +Therefore, the most useful backtraces should distill the relevant +information from the dump, which makes it easier to focus on the real +issue. Here is an example of a well-trimmed backtrace:: + + unchecked MSR access error: WRMSR to 0xd51 (tried to write 0x0000000000000064) + at rIP: 0xffffffffae059994 (native_write_msr+0x4/0x20) + Call Trace: + mba_wrmsr + update_domains + rdtgroup_mkdir + +Commentary +^^^^^^^^^^ + The ``---`` marker line serves the essential purpose of marking for patch handling tools where the changelog message ends. @@ -713,28 +820,6 @@ patch:: See more details on the proper patch format in the following references. -.. _backtraces: - -Backtraces in commit messages -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -Backtraces help document the call chain leading to a problem. However, -not all backtraces are helpful. For example, early boot call chains are -unique and obvious. Copying the full dmesg output verbatim, however, -adds distracting information like timestamps, module lists, register and -stack dumps. - -Therefore, the most useful backtraces should distill the relevant -information from the dump, which makes it easier to focus on the real -issue. Here is an example of a well-trimmed backtrace:: - - unchecked MSR access error: WRMSR to 0xd51 (tried to write 0x0000000000000064) - at rIP: 0xffffffffae059994 (native_write_msr+0x4/0x20) - Call Trace: - mba_wrmsr - update_domains - rdtgroup_mkdir - .. _explicit_in_reply_to: Explicit In-Reply-To headers @@ -755,10 +840,14 @@ Providing base tree information ------------------------------- When other developers receive your patches and start the review process, -it is often useful for them to know where in the tree history they -should place your work. This is particularly useful for automated CI -processes that attempt to run a series of tests in order to establish -the quality of your submission before the maintainer starts the review. +it is absolutely necessary for them to know what is the base +commit/branch your work applies on, considering the sheer amount of +maintainer trees present nowadays. Note again the **T:** entry in the +MAINTAINERS file explained above. + +This is even more important for automated CI processes that attempt to +run a series of tests in order to establish the quality of your +submission before the maintainer starts the review. If you are using ``git format-patch`` to generate your patches, you can automatically include the base tree information in your submission by @@ -801,6 +890,17 @@ letter or in the first patch of the series and it should be placed either below the ``---`` line or at the very bottom of all other content, right before your email signature. +Make sure that base commit is in an official maintainer/mainline tree +and not in some internal, accessible only to you tree - otherwise it +would be worthless. + +Tooling +------- + +Many of the technical aspects of this process can be automated using +b4, documented at <https://b4.docs.kernel.org/en/latest/>. This can +help with things like tracking dependencies, running checkpatch and +with formatting and sending mails. References ---------- @@ -824,9 +924,6 @@ Greg Kroah-Hartman, "How to piss off a kernel subsystem maintainer". <http://www.kroah.com/log/linux/maintainer-06.html> -NO!!!! No more huge patch bombs to linux-kernel@vger.kernel.org people! - <https://lore.kernel.org/r/20050711.125305.08322243.davem@davemloft.net> - Kernel Documentation/process/coding-style.rst Linus Torvalds's mail on the canonical patch format: |