diff options
Diffstat (limited to 'Documentation/process')
42 files changed, 5400 insertions, 1745 deletions
diff --git a/Documentation/process/2.Process.rst b/Documentation/process/2.Process.rst index ae020d84d7c4..613a01da4717 100644 --- a/Documentation/process/2.Process.rst +++ b/Documentation/process/2.Process.rst @@ -18,18 +18,18 @@ major kernel release happening every two or three months. The recent release history looks like this: ====== ================= - 4.11 April 30, 2017 - 4.12 July 2, 2017 - 4.13 September 3, 2017 - 4.14 November 12, 2017 - 4.15 January 28, 2018 - 4.16 April 1, 2018 + 5.0 March 3, 2019 + 5.1 May 5, 2019 + 5.2 July 7, 2019 + 5.3 September 15, 2019 + 5.4 November 24, 2019 + 5.5 January 6, 2020 ====== ================= -Every 4.x release is a major kernel release with new features, internal -API changes, and more. A typical 4.x release contain about 13,000 -changesets with changes to several hundred thousand lines of code. 4.x is -thus the leading edge of Linux kernel development; the kernel uses a +Every 5.x release is a major kernel release with new features, internal +API changes, and more. A typical release can contain about 13,000 +changesets with changes to several hundred thousand lines of code. 5.x is +the leading edge of Linux kernel development; the kernel uses a rolling development model which is continually integrating major changes. A relatively straightforward discipline is followed with regard to the @@ -48,9 +48,9 @@ detail later on). The merge window lasts for approximately two weeks. At the end of this time, Linus Torvalds will declare that the window is closed and release the -first of the "rc" kernels. For the kernel which is destined to be 2.6.40, +first of the "rc" kernels. For the kernel which is destined to be 5.6, for example, the release which happens at the end of the merge window will -be called 2.6.40-rc1. The -rc1 release is the signal that the time to +be called 5.6-rc1. The -rc1 release is the signal that the time to merge new features has passed, and that the time to stabilize the next kernel has begun. @@ -67,22 +67,23 @@ add at any time). As fixes make their way into the mainline, the patch rate will slow over time. Linus releases new -rc kernels about once a week; a normal series will get up to somewhere between -rc6 and -rc9 before the kernel is -considered to be sufficiently stable and the final 2.6.x release is made. +considered to be sufficiently stable and the final release is made. At that point the whole process starts over again. -As an example, here is how the 4.16 development cycle went (all dates in -2018): +As an example, here is how the 5.4 development cycle went (all dates in +2019): ============== =============================== - January 28 4.15 stable release - February 11 4.16-rc1, merge window closes - February 18 4.16-rc2 - February 25 4.16-rc3 - March 4 4.16-rc4 - March 11 4.16-rc5 - March 18 4.16-rc6 - March 25 4.16-rc7 - April 1 4.16 stable release + September 15 5.3 stable release + September 30 5.4-rc1, merge window closes + October 6 5.4-rc2 + October 13 5.4-rc3 + October 20 5.4-rc4 + October 27 5.4-rc5 + November 3 5.4-rc6 + November 10 5.4-rc7 + November 17 5.4-rc8 + November 24 5.4 stable release ============== =============================== How do the developers decide when to close the development cycle and create @@ -98,43 +99,37 @@ release is made. In the real world, this kind of perfection is hard to achieve; there are just too many variables in a project of this size. There comes a point where delaying the final release just makes the problem worse; the pile of changes waiting for the next merge window will grow -larger, creating even more regressions the next time around. So most 4.x +larger, creating even more regressions the next time around. So most 5.x kernels go out with a handful of known regressions though, hopefully, none of them are serious. Once a stable release is made, its ongoing maintenance is passed off to the -"stable team," currently consisting of Greg Kroah-Hartman. The stable team -will release occasional updates to the stable release using the 4.x.y -numbering scheme. To be considered for an update release, a patch must (1) -fix a significant bug, and (2) already be merged into the mainline for the -next development kernel. Kernels will typically receive stable updates for -a little more than one development cycle past their initial release. So, -for example, the 4.13 kernel's history looked like: +"stable team," currently Greg Kroah-Hartman. The stable team will release +occasional updates to the stable release using the 5.x.y numbering scheme. +To be considered for an update release, a patch must (1) fix a significant +bug, and (2) already be merged into the mainline for the next development +kernel. Kernels will typically receive stable updates for a little more +than one development cycle past their initial release. So, for example, the +5.2 kernel's history looked like this (all dates in 2019): ============== =============================== - September 3 4.13 stable release - September 13 4.13.1 - September 20 4.13.2 - September 27 4.13.3 - October 5 4.13.4 - October 12 4.13.5 + July 7 5.2 stable release + July 14 5.2.1 + July 21 5.2.2 + July 26 5.2.3 + July 28 5.2.4 + July 31 5.2.5 ... ... - November 24 4.13.16 + October 11 5.2.21 ============== =============================== -4.13.16 was the final stable update of the 4.13 release. +5.2.21 was the final stable update of the 5.2 release. Some kernels are designated "long term" kernels; they will receive support -for a longer period. As of this writing, the current long term kernels -and their maintainers are: +for a longer period. Please refer to the following link for the list of active +long term kernel versions and their maintainers: - ====== ====================== ============================== - 3.16 Ben Hutchings (very long-term stable kernel) - 4.1 Sasha Levin - 4.4 Greg Kroah-Hartman (very long-term stable kernel) - 4.9 Greg Kroah-Hartman - 4.14 Greg Kroah-Hartman - ====== ====================== ============================== + https://www.kernel.org/category/releases.html The selection of a kernel for long-term support is purely a matter of a maintainer having the need and the time to maintain that release. There @@ -215,12 +210,12 @@ How patches get into the Kernel ------------------------------- There is exactly one person who can merge patches into the mainline kernel -repository: Linus Torvalds. But, of the over 9,500 patches which went -into the 2.6.38 kernel, only 112 (around 1.3%) were directly chosen by Linus -himself. The kernel project has long since grown to a size where no single -developer could possibly inspect and select every patch unassisted. The -way the kernel developers have addressed this growth is through the use of -a lieutenant system built around a chain of trust. +repository: Linus Torvalds. But, for example, of the over 9,500 patches +which went into the 2.6.38 kernel, only 112 (around 1.3%) were directly +chosen by Linus himself. The kernel project has long since grown to a size +where no single developer could possibly inspect and select every patch +unassisted. The way the kernel developers have addressed this growth is +through the use of a lieutenant system built around a chain of trust. The kernel code base is logically broken down into a set of subsystems: networking, specific architecture support, memory management, video @@ -293,7 +288,7 @@ mainline get there via -mm. The current -mm patch is available in the "mmotm" (-mm of the moment) directory at: - http://www.ozlabs.org/~akpm/mmotm/ + https://www.ozlabs.org/~akpm/mmotm/ Use of the MMOTM tree is likely to be a frustrating experience, though; there is a definite chance that it will not even compile. @@ -304,7 +299,7 @@ the mainline is expected to look like after the next merge window closes. Linux-next trees are announced on the linux-kernel and linux-next mailing lists when they are assembled; they can be downloaded from: - http://www.kernel.org/pub/linux/kernel/next/ + https://www.kernel.org/pub/linux/kernel/next/ Linux-next has become an integral part of the kernel development process; all patches merged during a given merge window should really have found @@ -363,21 +358,21 @@ to keep up with what other developers (and the mainline) are doing. Git is now packaged by almost all Linux distributions. There is a home page at: - http://git-scm.com/ + https://git-scm.com/ That page has pointers to documentation and tutorials. Among the kernel developers who do not use git, the most popular choice is almost certainly Mercurial: - http://www.selenic.com/mercurial/ + https://www.selenic.com/mercurial/ Mercurial shares many features with git, but it provides an interface which many find easier to use. The other tool worth knowing about is Quilt: - http://savannah.nongnu.org/projects/quilt/ + https://savannah.nongnu.org/projects/quilt/ Quilt is a patch management system, rather than a source code management system. It does not track history over time; it is, instead, oriented @@ -403,7 +398,7 @@ be found at: http://vger.kernel.org/vger-lists.html There are lists hosted elsewhere, though; a number of them are at -lists.redhat.com. +redhat.com/mailman/listinfo. The core mailing list for kernel development is, of course, linux-kernel. This list is an intimidating place to be; volume can reach 500 messages per @@ -439,9 +434,10 @@ There are a few hints which can help with linux-kernel survival: questions. Some developers can get impatient with people who clearly have not done their homework. -- Avoid top-posting (the practice of putting your answer above the quoted - text you are responding to). It makes your response harder to read and - makes a poor impression. +- Use interleaved ("inline") replies, which makes your response easier to + read. (i.e. avoid top-posting -- the practice of putting your answer above + the quoted text you are responding to.) For more details, see + :ref:`Documentation/process/submitting-patches.rst <interleaved_replies>`. - Ask on the correct mailing list. Linux-kernel may be the general meeting point, but it is not the best place to find developers from all @@ -492,7 +488,7 @@ Andrew Morton gives this advice for aspiring kernel developers with others on getting things fixed up (this can require persistence!) but that's fine - it's a part of kernel development. -(http://lwn.net/Articles/283982/). +(https://lwn.net/Articles/283982/). In the absence of obvious problems to fix, developers are advised to look at the current lists of regressions and open bugs in general. There is diff --git a/Documentation/process/3.Early-stage.rst b/Documentation/process/3.Early-stage.rst index be00716071d4..894a920041c6 100644 --- a/Documentation/process/3.Early-stage.rst +++ b/Documentation/process/3.Early-stage.rst @@ -46,7 +46,7 @@ and posted this: to communicate user requirements to these people is a waste of time. They are much too "intelligent" to listen to lesser mortals. -(http://lwn.net/Articles/131776/). +(https://lwn.net/Articles/131776/). The reality of the situation was different; the kernel developers were far more concerned about system stability, long-term maintenance, and finding @@ -154,10 +154,11 @@ that the kernel developers have added a script to ease the process: This script will return the current maintainer(s) for a given file or directory when given the "-f" option. If passed a patch on the command line, it will list the maintainers who should probably receive -copies of the patch. There are a number of options regulating how hard -get_maintainer.pl will search for maintainers; please be careful about -using the more aggressive options as you may end up including developers -who have no real interest in the code you are modifying. +copies of the patch. This is the preferred way (unlike "-f" option) to get the +list of people to Cc for your patches. There are a number of options +regulating how hard get_maintainer.pl will search for maintainers; please be +careful about using the more aggressive options as you may end up including +developers who have no real interest in the code you are modifying. If all else fails, talking to Andrew Morton can be an effective way to track down a maintainer for a specific piece of code. @@ -216,7 +217,7 @@ a non-disclosure agreement. The Linux Foundation operates an NDA program designed to help with this sort of situation; more information can be found at: - http://www.linuxfoundation.org/en/NDA_program + https://www.linuxfoundation.org/nda/ This kind of review is often enough to avoid serious problems later on without requiring public disclosure of the project. diff --git a/Documentation/process/4.Coding.rst b/Documentation/process/4.Coding.rst index 13dd893c9f88..c2046dec0c2f 100644 --- a/Documentation/process/4.Coding.rst +++ b/Documentation/process/4.Coding.rst @@ -66,6 +66,10 @@ for aligning variables/macros, for reflowing text and other similar tasks. See the file :ref:`Documentation/process/clang-format.rst <clangformat>` for more details. +Some basic editor settings, such as indentation and line endings, will be +set automatically if you are using an editor that is compatible with +EditorConfig. See the official EditorConfig website for more information: +https://editorconfig.org/ Abstraction layers ****************** @@ -210,7 +214,7 @@ breaks? The best answer to this question was expressed by Linus in July, progress at all. Is it two steps forwards, one step back, or one step forward and two steps back? -(http://lwn.net/Articles/243460/). +(https://lwn.net/Articles/243460/). An especially unwelcome type of regression is any sort of change to the user-space ABI. Once an interface has been exported to user space, it must @@ -242,17 +246,15 @@ and try to avoid "fixes" which make the warning go away without addressing its cause. Note that not all compiler warnings are enabled by default. Build the -kernel with "make EXTRA_CFLAGS=-W" to get the full set. +kernel with "make KCFLAGS=-W" to get the full set. The kernel provides several configuration options which turn on debugging features; most of these are found in the "kernel hacking" submenu. Several of these options should be turned on for any kernel used for development or testing purposes. In particular, you should turn on: - - ENABLE_MUST_CHECK and FRAME_WARN to get an - extra set of warnings for problems like the use of deprecated interfaces - or ignoring an important return value from a function. The output - generated by these warnings can be verbose, but one need not worry about + - FRAME_WARN to get warnings for stack frames larger than a given amount. + The output generated can be verbose, but one need not worry about warnings from other parts of the kernel. - DEBUG_OBJECTS will add code to track the lifetime of various objects @@ -323,7 +325,7 @@ other architectures. If you do not happen to have an S/390 system or a Blackfin development board handy, you can still perform the compilation step. A large set of cross compilers for x86 systems can be found at - http://www.kernel.org/pub/tools/crosstool/ + https://www.kernel.org/pub/tools/crosstool/ Some time spent installing and using these compilers will help avoid embarrassment later. diff --git a/Documentation/process/5.Posting.rst b/Documentation/process/5.Posting.rst index 855a70b80269..de4edd42d5c0 100644 --- a/Documentation/process/5.Posting.rst +++ b/Documentation/process/5.Posting.rst @@ -10,8 +10,7 @@ of conventions and procedures which are used in the posting of patches; following them will make life much easier for everybody involved. This document will attempt to cover these expectations in reasonable detail; more information can also be found in the files -:ref:`Documentation/process/submitting-patches.rst <submittingpatches>`, -:ref:`Documentation/process/submitting-drivers.rst <submittingdrivers>` +:ref:`Documentation/process/submitting-patches.rst <submittingpatches>` and :ref:`Documentation/process/submit-checklist.rst <submitchecklist>`. @@ -197,14 +196,39 @@ the build process, for example, or editor backup files) in the patch. The file "dontdiff" in the Documentation directory can help in this regard; pass it to diff with the "-X" option. -The tags mentioned above are used to describe how various developers have -been associated with the development of this patch. They are described in -detail in -the :ref:`Documentation/process/submitting-patches.rst <submittingpatches>` -document; what follows here is a brief summary. Each of these lines has -the format: +The tags already briefly mentioned above are used to provide insights how +the patch came into being. They are described in detail in the +:ref:`Documentation/process/submitting-patches.rst <submittingpatches>` +document; what follows here is a brief summary. -:: +One tag is used to refer to earlier commits which introduced problems fixed by +the patch:: + + Fixes: 1f2e3d4c5b6a ("The first line of the commit specified by the first 12 characters of its SHA-1 ID") + +Another tag is used for linking web pages with additional backgrounds or +details, for example an earlier discussion which leads to the patch or a +document with a specification implemented by the patch:: + + Link: https://example.com/somewhere.html optional-other-stuff + +Many maintainers when applying a patch also add this tag to link to the +latest public review posting of the patch; often this is automatically done +by tools like b4 or a git hook like the one described in +'Documentation/maintainer/configure-git.rst'. + +If the URL points to a public bug report being fixed by the patch, use the +"Closes:" tag instead:: + + Closes: https://example.com/issues/1234 optional-other-stuff + +Some bug trackers have the ability to close issues automatically when a +commit with such a tag is applied. Some bots monitoring mailing lists can +also track such tags and take certain actions. Private bug trackers and +invalid URLs are forbidden. + +Another kind of tag is used to document who was involved in the development of +the patch. Each of these uses this format:: tag: Full Name <email address> optional-other-stuff @@ -237,13 +261,18 @@ The tags in common use are: - Reported-by: names a user who reported a problem which is fixed by this patch; this tag is used to give credit to the (often underappreciated) people who test our code and let us know when things do not work - correctly. + correctly. Note, this tag should be followed by a Closes: tag pointing to + the report, unless the report is not available on the web. The Link: tag + can be used instead of Closes: if the patch fixes a part of the issue(s) + being reported. - Cc: the named person received a copy of the patch and had the opportunity to comment on it. -Be careful in the addition of tags to your patches: only Cc: is appropriate -for addition without the explicit permission of the person named. +Be careful in the addition of tags to your patches, as only Cc: is appropriate +for addition without the explicit permission of the person named; using +Reported-by: is fine most of the time as well, but ask for permission if +the bug was reported in private. Sending the patch diff --git a/Documentation/process/6.Followthrough.rst b/Documentation/process/6.Followthrough.rst index a173cd5f93d2..66fa400c6d94 100644 --- a/Documentation/process/6.Followthrough.rst +++ b/Documentation/process/6.Followthrough.rst @@ -51,6 +51,13 @@ mind: working toward the creation of the best kernel they can; they are not trying to create discomfort for their employers' competitors. + - Be prepared for seemingly silly requests for coding style changes + and requests to factor out some of your code to shared parts of + the kernel. One job the maintainers do is to keep things looking + the same. Sometimes this means that the clever hack in your driver + to get around a problem actually needs to become a generalized + kernel feature ready for next time. + What all of this comes down to is that, when reviewers send you comments, you need to pay attention to the technical observations that they are making. Do not let their form of expression or your own pride keep that diff --git a/Documentation/process/7.AdvancedTopics.rst b/Documentation/process/7.AdvancedTopics.rst index 172733cff097..43291704338e 100644 --- a/Documentation/process/7.AdvancedTopics.rst +++ b/Documentation/process/7.AdvancedTopics.rst @@ -29,9 +29,9 @@ long document in its own right. Instead, the focus here will be on how git fits into the kernel development process in particular. Developers who wish to come up to speed with git will find more information at: - http://git-scm.com/ + https://git-scm.com/ - http://www.kernel.org/pub/software/scm/git/docs/user-manual.html + https://www.kernel.org/pub/software/scm/git/docs/user-manual.html and on various tutorials found on the web. @@ -55,7 +55,7 @@ server with git-daemon is relatively straightforward if you have a system which is accessible to the Internet. Otherwise, free, public hosting sites (Github, for example) are starting to appear on the net. Established developers can get an account on kernel.org, but those are not easy to come -by; see http://kernel.org/faq/ for more information. +by; see https://kernel.org/faq/ for more information. The normal git workflow involves the use of a lot of branches. Each line of development can be separated into a separate "topic branch" and @@ -125,7 +125,7 @@ can affect your ability to get trees pulled in the future. Quoting Linus: to trust things *without* then having to go and check every individual change by hand. -(http://lwn.net/Articles/224135/). +(https://lwn.net/Articles/224135/). To avoid this kind of situation, ensure that all patches within a given branch stick closely to the associated topic; a "driver fixes" branch @@ -146,6 +146,7 @@ pull. The git request-pull command can be helpful in this regard; it will format the request as other developers expect, and will also check to be sure that you have remembered to push those changes to the public server. +.. _development_advancedtopics_reviews: Reviewing patches ----------------- @@ -167,6 +168,12 @@ comments as questions rather than criticisms. Asking "how does the lock get released in this path?" will always work better than stating "the locking here is wrong." +Another technique that is useful in case of a disagreement is to ask for others +to chime in. If a discussion reaches a stalemate after a few exchanges, +then call for opinions of other reviewers or maintainers. Often those in +agreement with a reviewer remain silent unless called upon. +The opinion of multiple people carries exponentially more weight. + Different developers will review code from different points of view. Some are mostly concerned with coding style and whether code lines have trailing white space. Others will focus primarily on whether the change implemented @@ -176,3 +183,14 @@ security issues, duplication of code found elsewhere, adequate documentation, adverse effects on performance, user-space ABI changes, etc. All types of review, if they lead to better code going into the kernel, are welcome and worthwhile. + +There is no strict requirement to use specific tags like ``Reviewed-by``. +In fact reviews in plain English are more informative and encouraged +even when a tag is provided, e.g. "I looked at aspects A, B and C of this +submission and it looks good to me." +Some form of a review message or reply is obviously necessary otherwise +maintainers will not know that the reviewer has looked at the patch at all! + +Last but not least patch review may become a negative process, focused +on pointing out problems. Please throw in a compliment once in a while, +particularly for newbies! diff --git a/Documentation/process/8.Conclusion.rst b/Documentation/process/8.Conclusion.rst index 8395aa2c1f3a..8c847dffe76b 100644 --- a/Documentation/process/8.Conclusion.rst +++ b/Documentation/process/8.Conclusion.rst @@ -5,35 +5,33 @@ For more information There are numerous sources of information on Linux kernel development and related topics. First among those will always be the Documentation -directory found in the kernel source distribution. The top-level :ref:`process/howto.rst <process_howto>` -file is an important starting point; :ref:`process/submitting-patches.rst <submittingpatches>` -and :ref:`process/submitting-drivers.rst <submittingdrivers>` -are also something which all kernel developers should -read. Many internal kernel APIs are documented using the kerneldoc -mechanism; "make htmldocs" or "make pdfdocs" can be used to generate those -documents in HTML or PDF format (though the version of TeX shipped by some -distributions runs into internal limits and fails to process the documents -properly). +directory found in the kernel source distribution. Start with the +top-level :ref:`process/howto.rst <process_howto>`; also read +:ref:`process/submitting-patches.rst <submittingpatches>`. Many internal +kernel APIs are documented using the kerneldoc mechanism; "make htmldocs" +or "make pdfdocs" can be used to generate those documents in HTML or PDF +format (though the version of TeX shipped by some distributions runs into +internal limits and fails to process the documents properly). Various web sites discuss kernel development at all levels of detail. Your -author would like to humbly suggest http://lwn.net/ as a source; +author would like to humbly suggest https://lwn.net/ as a source; information on many specific kernel topics can be found via the LWN kernel index at: - http://lwn.net/Kernel/Index/ + https://lwn.net/Kernel/Index/ Beyond that, a valuable resource for kernel developers is: - http://kernelnewbies.org/ + https://kernelnewbies.org/ -And, of course, one should not forget http://kernel.org/, the definitive +And, of course, one should not forget https://kernel.org/, the definitive location for kernel release information. There are a number of books on kernel development: Linux Device Drivers, 3rd Edition (Jonathan Corbet, Alessandro Rubini, and Greg Kroah-Hartman). Online at - http://lwn.net/Kernel/LDD3/. + https://lwn.net/Kernel/LDD3/. Linux Kernel Development (Robert Love). @@ -46,9 +44,9 @@ information to be found there. Documentation for git can be found at: - http://www.kernel.org/pub/software/scm/git/docs/ + https://www.kernel.org/pub/software/scm/git/docs/ - http://www.kernel.org/pub/software/scm/git/docs/user-manual.html + https://www.kernel.org/pub/software/scm/git/docs/user-manual.html Conclusion diff --git a/Documentation/process/adding-syscalls.rst b/Documentation/process/adding-syscalls.rst index 1c3a840d06b9..906c47f1a9e5 100644 --- a/Documentation/process/adding-syscalls.rst +++ b/Documentation/process/adding-syscalls.rst @@ -33,7 +33,7 @@ interface. to a somewhat opaque API. - If you're just exposing runtime system information, a new node in sysfs - (see ``Documentation/filesystems/sysfs.txt``) or the ``/proc`` filesystem may + (see ``Documentation/filesystems/sysfs.rst``) or the ``/proc`` filesystem may be more appropriate. However, access to these mechanisms requires that the relevant filesystem is mounted, which might not always be the case (e.g. in a namespaced/sandboxed/chrooted environment). Avoid adding any API to @@ -501,7 +501,7 @@ table, but not from elsewhere in the kernel. If the syscall functionality is useful to be used within the kernel, needs to be shared between an old and a new syscall, or needs to be shared between a syscall and its compatibility variant, it should be implemented by means of a "helper" function (such as -``kern_xyzzy()``). This kernel function may then be called within the +``ksys_xyzzy()``). This kernel function may then be called within the syscall stub (``sys_xyzzy()``), the compatibility syscall stub (``compat_sys_xyzzy()``), and/or other kernel code. @@ -541,25 +541,25 @@ References and Sources :manpage:`syscall(2)` man-page: http://man7.org/linux/man-pages/man2/syscall.2.html#NOTES - Collated emails from Linus Torvalds discussing the problems with ``ioctl()``: - http://yarchive.net/comp/linux/ioctl.html + https://yarchive.net/comp/linux/ioctl.html - "How to not invent kernel interfaces", Arnd Bergmann, - http://www.ukuug.org/events/linux2007/2007/papers/Bergmann.pdf + https://www.ukuug.org/events/linux2007/2007/papers/Bergmann.pdf - LWN article from Michael Kerrisk on avoiding new uses of CAP_SYS_ADMIN: https://lwn.net/Articles/486306/ - Recommendation from Andrew Morton that all related information for a new system call should come in the same email thread: - https://lkml.org/lkml/2014/7/24/641 + https://lore.kernel.org/r/20140724144747.3041b208832bbdf9fbce5d96@linux-foundation.org - Recommendation from Michael Kerrisk that a new system call should come with - a man page: https://lkml.org/lkml/2014/6/13/309 + a man page: https://lore.kernel.org/r/CAKgNAkgMA39AfoSoA5Pe1r9N+ZzfYQNvNPvcRN7tOvRb8+v06Q@mail.gmail.com - Suggestion from Thomas Gleixner that x86 wire-up should be in a separate - commit: https://lkml.org/lkml/2014/11/19/254 + commit: https://lore.kernel.org/r/alpine.DEB.2.11.1411191249560.3909@nanos - Suggestion from Greg Kroah-Hartman that it's good for new system calls to - come with a man-page & selftest: https://lkml.org/lkml/2014/3/19/710 + come with a man-page & selftest: https://lore.kernel.org/r/20140320025530.GA25469@kroah.com - Discussion from Michael Kerrisk of new system call vs. :manpage:`prctl(2)` extension: - https://lkml.org/lkml/2014/6/3/411 + https://lore.kernel.org/r/CAHO5Pa3F2MjfTtfNxa8LbnkeeU8=YJ+9tDqxZpw7Gz59E-4AUg@mail.gmail.com - Suggestion from Ingo Molnar that system calls that involve multiple arguments should encapsulate those arguments in a struct, which includes a - size field for future extensibility: https://lkml.org/lkml/2015/7/30/117 + size field for future extensibility: https://lore.kernel.org/r/20150730083831.GA22182@gmail.com - Numbering oddities arising from (re-)use of O_* numbering space flags: - commit 75069f2b5bfb ("vfs: renumber FMODE_NONOTIFY and add to uniqueness @@ -569,9 +569,9 @@ References and Sources - commit bb458c644a59 ("Safer ABI for O_TMPFILE") - Discussion from Matthew Wilcox about restrictions on 64-bit arguments: - https://lkml.org/lkml/2008/12/12/187 + https://lore.kernel.org/r/20081212152929.GM26095@parisc-linux.org - Recommendation from Greg Kroah-Hartman that unknown flags should be - policed: https://lkml.org/lkml/2014/7/17/577 + policed: https://lore.kernel.org/r/20140717193330.GB4703@kroah.com - Recommendation from Linus Torvalds that x32 system calls should prefer compatibility with 64-bit versions rather than 32-bit versions: - https://lkml.org/lkml/2011/8/31/244 + https://lore.kernel.org/r/CA+55aFxfmwfB7jbbrXxa=K7VBYPfAvmu3XOkGrLbB1UFjX1+Ew@mail.gmail.com diff --git a/Documentation/process/applying-patches.rst b/Documentation/process/applying-patches.rst index fbb9297e6360..c269f5e1a0a3 100644 --- a/Documentation/process/applying-patches.rst +++ b/Documentation/process/applying-patches.rst @@ -229,7 +229,7 @@ Although interdiff may save you a step or two you are generally advised to do the additional steps since interdiff can get things wrong in some cases. Another alternative is ``ketchup``, which is a python script for automatic -downloading and applying of patches (http://www.selenic.com/ketchup/). +downloading and applying of patches (https://www.selenic.com/ketchup/). Other nice tools are diffstat, which shows a summary of changes made by a patch; lsdiff, which displays a short listing of affected files in a patch @@ -241,7 +241,7 @@ the patch contains a given regular expression. Where can I download the patches? ================================= -The patches are available at http://kernel.org/ +The patches are available at https://kernel.org/ Most recent patches are linked from the front page, but they also have specific homes. @@ -249,6 +249,10 @@ The 5.x.y (-stable) and 5.x patches live at https://www.kernel.org/pub/linux/kernel/v5.x/ +The 5.x.y incremental patches live at + + https://www.kernel.org/pub/linux/kernel/v5.x/incr/ + The -rc patches are not stored on the webserver but are generated on demand from git tags such as @@ -308,12 +312,11 @@ versions. If no 5.x.y kernel is available, then the highest numbered 5.x kernel is the current stable kernel. -.. note:: +The -stable team provides normal as well as incremental patches. Below is +how to apply these patches. - The -stable team usually do make incremental patches available as well - as patches against the latest mainline release, but I only cover the - non-incremental ones below. The incremental ones can be found at - https://www.kernel.org/pub/linux/kernel/v5.x/incr/ +Normal patches +~~~~~~~~~~~~~~ These patches are not incremental, meaning that for example the 5.7.3 patch does not apply on top of the 5.7.2 kernel source, but rather on top @@ -331,6 +334,21 @@ Here's a small example:: $ cd .. $ mv linux-5.7.2 linux-5.7.3 # rename the kernel source dir +Incremental patches +~~~~~~~~~~~~~~~~~~~ + +Incremental patches are different: instead of being applied on top +of base 5.x kernel, they are applied on top of previous stable kernel +(5.x.y-1). + +Here's the example to apply these:: + + $ cd ~/linux-5.7.2 # change to the kernel source dir + $ patch -p1 < ../patch-5.7.2-3 # apply the new 5.7.3 patch + $ cd .. + $ mv linux-5.7.2 linux-5.7.3 # rename the kernel source dir + + The -rc kernels =============== @@ -389,7 +407,7 @@ The -mm patches are experimental patches released by Andrew Morton. In the past, -mm tree were used to also test subsystem patches, but this function is now done via the -`linux-next <https://www.kernel.org/doc/man-pages/linux-next.html>` +`linux-next` (https://www.kernel.org/doc/man-pages/linux-next.html) tree. The Subsystem maintainers push their patches first to linux-next, and, during the merge window, sends them directly to Linus. diff --git a/Documentation/process/backporting.rst b/Documentation/process/backporting.rst new file mode 100644 index 000000000000..e1a6ea0a1e8a --- /dev/null +++ b/Documentation/process/backporting.rst @@ -0,0 +1,604 @@ +.. SPDX-License-Identifier: GPL-2.0 + +=================================== +Backporting and conflict resolution +=================================== + +:Author: Vegard Nossum <vegard.nossum@oracle.com> + +.. contents:: + :local: + :depth: 3 + :backlinks: none + +Introduction +============ + +Some developers may never really have to deal with backporting patches, +merging branches, or resolving conflicts in their day-to-day work, so +when a merge conflict does pop up, it can be daunting. Luckily, +resolving conflicts is a skill like any other, and there are many useful +techniques you can use to make the process smoother and increase your +confidence in the result. + +This document aims to be a comprehensive, step-by-step guide to +backporting and conflict resolution. + +Applying the patch to a tree +============================ + +Sometimes the patch you are backporting already exists as a git commit, +in which case you just cherry-pick it directly using +``git cherry-pick``. However, if the patch comes from an email, as it +often does for the Linux kernel, you will need to apply it to a tree +using ``git am``. + +If you've ever used ``git am``, you probably already know that it is +quite picky about the patch applying perfectly to your source tree. In +fact, you've probably had nightmares about ``.rej`` files and trying to +edit the patch to make it apply. + +It is strongly recommended to instead find an appropriate base version +where the patch applies cleanly and *then* cherry-pick it over to your +destination tree, as this will make git output conflict markers and let +you resolve conflicts with the help of git and any other conflict +resolution tools you might prefer to use. For example, if you want to +apply a patch that just arrived on LKML to an older stable kernel, you +can apply it to the most recent mainline kernel and then cherry-pick it +to your older stable branch. + +It's generally better to use the exact same base as the one the patch +was generated from, but it doesn't really matter that much as long as it +applies cleanly and isn't too far from the original base. The only +problem with applying the patch to the "wrong" base is that it may pull +in more unrelated changes in the context of the diff when cherry-picking +it to the older branch. + +A good reason to prefer ``git cherry-pick`` over ``git am`` is that git +knows the precise history of an existing commit, so it will know when +code has moved around and changed the line numbers; this in turn makes +it less likely to apply the patch to the wrong place (which can result +in silent mistakes or messy conflicts). + +If you are using `b4`_. and you are applying the patch directly from an +email, you can use ``b4 am`` with the options ``-g``/``--guess-base`` +and ``-3``/``--prep-3way`` to do some of this automatically (see the +`b4 presentation`_ for more information). However, the rest of this +article will assume that you are doing a plain ``git cherry-pick``. + +.. _b4: https://people.kernel.org/monsieuricon/introducing-b4-and-patch-attestation +.. _b4 presentation: https://youtu.be/mF10hgVIx9o?t=2996 + +Once you have the patch in git, you can go ahead and cherry-pick it into +your source tree. Don't forget to cherry-pick with ``-x`` if you want a +written record of where the patch came from! + +Note that if you are submiting a patch for stable, the format is +slightly different; the first line after the subject line needs tobe +either:: + + commit <upstream commit> upstream + +or:: + + [ Upstream commit <upstream commit> ] + +Resolving conflicts +=================== + +Uh-oh; the cherry-pick failed with a vaguely threatening message:: + + CONFLICT (content): Merge conflict + +What to do now? + +In general, conflicts appear when the context of the patch (i.e., the +lines being changed and/or the lines surrounding the changes) doesn't +match what's in the tree you are trying to apply the patch *to*. + +For backports, what likely happened was that the branch you are +backporting from contains patches not in the branch you are backporting +to. However, the reverse is also possible. In any case, the result is a +conflict that needs to be resolved. + +If your attempted cherry-pick fails with a conflict, git automatically +edits the files to include so-called conflict markers showing you where +the conflict is and how the two branches have diverged. Resolving the +conflict typically means editing the end result in such a way that it +takes into account these other commits. + +Resolving the conflict can be done either by hand in a regular text +editor or using a dedicated conflict resolution tool. + +Many people prefer to use their regular text editor and edit the +conflict directly, as it may be easier to understand what you're doing +and to control the final result. There are definitely pros and cons to +each method, and sometimes there's value in using both. + +We will not cover using dedicated merge tools here beyond providing some +pointers to various tools that you could use: + +- `Emacs Ediff mode <https://www.emacswiki.org/emacs/EdiffMode>`__ +- `vimdiff/gvimdiff <https://linux.die.net/man/1/vimdiff>`__ +- `KDiff3 <http://kdiff3.sourceforge.net/>`__ +- `TortoiseMerge <https://tortoisesvn.net/TortoiseMerge.html>`__ +- `Meld <https://meldmerge.org/help/>`__ +- `P4Merge <https://www.perforce.com/products/helix-core-apps/merge-diff-tool-p4merge>`__ +- `Beyond Compare <https://www.scootersoftware.com/>`__ +- `IntelliJ <https://www.jetbrains.com/help/idea/resolve-conflicts.html>`__ +- `VSCode <https://code.visualstudio.com/docs/editor/versioncontrol>`__ + +To configure git to work with these, see ``git mergetool --help`` or +the official `git-mergetool documentation`_. + +.. _git-mergetool documentation: https://git-scm.com/docs/git-mergetool + +Prerequisite patches +-------------------- + +Most conflicts happen because the branch you are backporting to is +missing some patches compared to the branch you are backporting *from*. +In the more general case (such as merging two independent branches), +development could have happened on either branch, or the branches have +simply diverged -- perhaps your older branch had some other backports +applied to it that themselves needed conflict resolutions, causing a +divergence. + +It's important to always identify the commit or commits that caused the +conflict, as otherwise you cannot be confident in the correctness of +your resolution. As an added bonus, especially if the patch is in an +area you're not that famliar with, the changelogs of these commits will +often give you the context to understand the code and potential problems +or pitfalls with your conflict resolution. + +git log +~~~~~~~ + +A good first step is to look at ``git log`` for the file that has the +conflict -- this is usually sufficient when there aren't a lot of +patches to the file, but may get confusing if the file is big and +frequently patched. You should run ``git log`` on the range of commits +between your currently checked-out branch (``HEAD``) and the parent of +the patch you are picking (``<commit>``), i.e.:: + + git log HEAD..<commit>^ -- <path> + +Even better, if you want to restrict this output to a single function +(because that's where the conflict appears), you can use the following +syntax:: + + git log -L:'\<function\>':<path> HEAD..<commit>^ + +.. note:: + The ``\<`` and ``\>`` around the function name ensure that the + matches are anchored on a word boundary. This is important, as this + part is actually a regex and git only follows the first match, so + if you use ``-L:thread_stack:kernel/fork.c`` it may only give you + results for the function ``try_release_thread_stack_to_cache`` even + though there are many other functions in that file containing the + string ``thread_stack`` in their names. + +Another useful option for ``git log`` is ``-G``, which allows you to +filter on certain strings appearing in the diffs of the commits you are +listing:: + + git log -G'regex' HEAD..<commit>^ -- <path> + +This can also be a handy way to quickly find when something (e.g. a +function call or a variable) was changed, added, or removed. The search +string is a regular expression, which means you can potentially search +for more specific things like assignments to a specific struct member:: + + git log -G'\->index\>.*=' + +git blame +~~~~~~~~~ + +Another way to find prerequisite commits (albeit only the most recent +one for a given conflict) is to run ``git blame``. In this case, you +need to run it against the parent commit of the patch you are +cherry-picking and the file where the conflict appared, i.e.:: + + git blame <commit>^ -- <path> + +This command also accepts the ``-L`` argument (for restricting the +output to a single function), but in this case you specify the filename +at the end of the command as usual:: + + git blame -L:'\<function\>' <commit>^ -- <path> + +Navigate to the place where the conflict occurred. The first column of +the blame output is the commit ID of the patch that added a given line +of code. + +It might be a good idea to ``git show`` these commits and see if they +look like they might be the source of the conflict. Sometimes there will +be more than one of these commits, either because multiple commits +changed different lines of the same conflict area *or* because multiple +subsequent patches changed the same line (or lines) multiple times. In +the latter case, you may have to run ``git blame`` again and specify the +older version of the file to look at in order to dig further back in +the history of the file. + +Prerequisite vs. incidental patches +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Having found the patch that caused the conflict, you need to determine +whether it is a prerequisite for the patch you are backporting or +whether it is just incidental and can be skipped. An incidental patch +would be one that touches the same code as the patch you are +backporting, but does not change the semantics of the code in any +material way. For example, a whitespace cleanup patch is completely +incidental -- likewise, a patch that simply renames a function or a +variable would be incidental as well. On the other hand, if the function +being changed does not even exist in your current branch then this would +not be incidental at all and you need to carefully consider whether the +patch adding the function should be cherry-picked first. + +If you find that there is a necessary prerequisite patch, then you need +to stop and cherry-pick that instead. If you've already resolved some +conflicts in a different file and don't want to do it again, you can +create a temporary copy of that file. + +To abort the current cherry-pick, go ahead and run +``git cherry-pick --abort``, then restart the cherry-picking process +with the commit ID of the prerequisite patch instead. + +Understanding conflict markers +------------------------------ + +Combined diffs +~~~~~~~~~~~~~~ + +Let's say you've decided against picking (or reverting) additional +patches and you just want to resolve the conflict. Git will have +inserted conflict markers into your file. Out of the box, this will look +something like:: + + <<<<<<< HEAD + this is what's in your current tree before cherry-picking + ======= + this is what the patch wants it to be after cherry-picking + >>>>>>> <commit>... title + +This is what you would see if you opened the file in your editor. +However, if you were to run ``git diff`` without any arguments, the +output would look something like this:: + + $ git diff + [...] + ++<<<<<<<< HEAD + +this is what's in your current tree before cherry-picking + ++======== + + this is what the patch wants it to be after cherry-picking + ++>>>>>>>> <commit>... title + +When you are resolving a conflict, the behavior of ``git diff`` differs +from its normal behavior. Notice the two columns of diff markers +instead of the usual one; this is a so-called "`combined diff`_", here +showing the 3-way diff (or diff-of-diffs) between + +#. the current branch (before cherry-picking) and the current working + directory, and +#. the current branch (before cherry-picking) and the file as it looks + after the original patch has been applied. + +.. _combined diff: https://git-scm.com/docs/diff-format#_combined_diff_format + + +Better diffs +~~~~~~~~~~~~ + +3-way combined diffs include all the other changes that happened to the +file between your current branch and the branch you are cherry-picking +from. While this is useful for spotting other changes that you need to +take into account, this also makes the output of ``git diff`` somewhat +intimidating and difficult to read. You may instead prefer to run +``git diff HEAD`` (or ``git diff --ours``) which shows only the diff +between the current branch before cherry-picking and the current working +directory. It looks like this:: + + $ git diff HEAD + [...] + +<<<<<<<< HEAD + this is what's in your current tree before cherry-picking + +======== + +this is what the patch wants it to be after cherry-picking + +>>>>>>>> <commit>... title + +As you can see, this reads just like any other diff and makes it clear +which lines are in the current branch and which lines are being added +because they are part of the merge conflict or the patch being +cherry-picked. + +Merge styles and diff3 +~~~~~~~~~~~~~~~~~~~~~~ + +The default conflict marker style shown above is known as the ``merge`` +style. There is also another style available, known as the ``diff3`` +style, which looks like this:: + + <<<<<<< HEAD + this is what is in your current tree before cherry-picking + ||||||| parent of <commit> (title) + this is what the patch expected to find there + ======= + this is what the patch wants it to be after being applied + >>>>>>> <commit> (title) + +As you can see, this has 3 parts instead of 2, and includes what git +expected to find there but didn't. It is *highly recommended* to use +this conflict style as it makes it much clearer what the patch actually +changed; i.e., it allows you to compare the before-and-after versions +of the file for the commit you are cherry-picking. This allows you to +make better decisions about how to resolve the conflict. + +To change conflict marker styles, you can use the following command:: + + git config merge.conflictStyle diff3 + +There is a third option, ``zdiff3``, introduced in `Git 2.35`_, +which has the same 3 sections as ``diff3``, but where common lines have +been trimmed off, making the conflict area smaller in some cases. + +.. _Git 2.35: https://github.blog/2022-01-24-highlights-from-git-2-35/ + +Iterating on conflict resolutions +--------------------------------- + +The first step in any conflict resolution process is to understand the +patch you are backporting. For the Linux kernel this is especially +important, since an incorrect change can lead to the whole system +crashing -- or worse, an undetected security vulnerability. + +Understanding the patch can be easy or difficult depending on the patch +itself, the changelog, and your familiarity with the code being changed. +However, a good question for every change (or every hunk of the patch) +might be: "Why is this hunk in the patch?" The answers to these +questions will inform your conflict resolution. + +Resolution process +~~~~~~~~~~~~~~~~~~ + +Sometimes the easiest thing to do is to just remove all but the first +part of the conflict, leaving the file essentially unchanged, and apply +the changes by hand. Perhaps the patch is changing a function call +argument from ``0`` to ``1`` while a conflicting change added an +entirely new (and insignificant) parameter to the end of the parameter +list; in that case, it's easy enough to change the argument from ``0`` +to ``1`` by hand and leave the rest of the arguments alone. This +technique of manually applying changes is mostly useful if the conflict +pulled in a lot of unrelated context that you don't really need to care +about. + +For particularly nasty conflicts with many conflict markers, you can use +``git add`` or ``git add -i`` to selectively stage your resolutions to +get them out of the way; this also lets you use ``git diff HEAD`` to +always see what remains to be resolved or ``git diff --cached`` to see +what your patch looks like so far. + +Dealing with file renames +~~~~~~~~~~~~~~~~~~~~~~~~~ + +One of the most annoying things that can happen while backporting a +patch is discovering that one of the files being patched has been +renamed, as that typically means git won't even put in conflict markers, +but will just throw up its hands and say (paraphrased): "Unmerged path! +You do the work..." + +There are generally a few ways to deal with this. If the patch to the +renamed file is small, like a one-line change, the easiest thing is to +just go ahead and apply the change by hand and be done with it. On the +other hand, if the change is big or complicated, you definitely don't +want to do it by hand. + +As a first pass, you can try something like this, which will lower the +rename detection threshold to 30% (by default, git uses 50%, meaning +that two files need to have at least 50% in common for it to consider +an add-delete pair to be a potential rename):: + + git cherry-pick -strategy=recursive -Xrename-threshold=30 + +Sometimes the right thing to do will be to also backport the patch that +did the rename, but that's definitely not the most common case. Instead, +what you can do is to temporarily rename the file in the branch you're +backporting to (using ``git mv`` and committing the result), restart the +attempt to cherry-pick the patch, rename the file back (``git mv`` and +committing again), and finally squash the result using ``git rebase -i`` +(see the `rebase tutorial`_) so it appears as a single commit when you +are done. + +.. _rebase tutorial: https://medium.com/@slamflipstrom/a-beginners-guide-to-squashing-commits-with-git-rebase-8185cf6e62ec + +Gotchas +------- + +Function arguments +~~~~~~~~~~~~~~~~~~ + +Pay attention to changing function arguments! It's easy to gloss over +details and think that two lines are the same but actually they differ +in some small detail like which variable was passed as an argument +(especially if the two variables are both a single character that look +the same, like i and j). + +Error handling +~~~~~~~~~~~~~~ + +If you cherry-pick a patch that includes a ``goto`` statement (typically +for error handling), it is absolutely imperative to double check that +the target label is still correct in the branch you are backporting to. +The same goes for added ``return``, ``break``, and ``continue`` +statements. + +Error handling is typically located at the bottom of the function, so it +may not be part of the conflict even though could have been changed by +other patches. + +A good way to ensure that you review the error paths is to always use +``git diff -W`` and ``git show -W`` (AKA ``--function-context``) when +inspecting your changes. For C code, this will show you the whole +function that's being changed in a patch. One of the things that often +go wrong during backports is that something else in the function changed +on either of the branches that you're backporting from or to. By +including the whole function in the diff you get more context and can +more easily spot problems that might otherwise go unnoticed. + +Refactored code +~~~~~~~~~~~~~~~ + +Something that happens quite often is that code gets refactored by +"factoring out" a common code sequence or pattern into a helper +function. When backporting patches to an area where such a refactoring +has taken place, you effectively need to do the reverse when +backporting: a patch to a single location may need to be applied to +multiple locations in the backported version. (One giveaway for this +scenario is that a function was renamed -- but that's not always the +case.) + +To avoid incomplete backports, it's worth trying to figure out if the +patch fixes a bug that appears in more than one place. One way to do +this would be to use ``git grep``. (This is actually a good idea to do +in general, not just for backports.) If you do find that the same kind +of fix would apply to other places, it's also worth seeing if those +places exist upstream -- if they don't, it's likely the patch may need +to be adjusted. ``git log`` is your friend to figure out what happened +to these areas as ``git blame`` won't show you code that has been +removed. + +If you do find other instances of the same pattern in the upstream tree +and you're not sure whether it's also a bug, it may be worth asking the +patch author. It's not uncommon to find new bugs during backporting! + +Verifying the result +==================== + +colordiff +--------- + +Having committed a conflict-free new patch, you can now compare your +patch to the original patch. It is highly recommended that you use a +tool such as `colordiff`_ that can show two files side by side and color +them according to the changes between them:: + + colordiff -yw -W 200 <(git diff -W <upstream commit>^-) <(git diff -W HEAD^-) | less -SR + +.. _colordiff: https://www.colordiff.org/ + +Here, ``-y`` means to do a side-by-side comparison; ``-w`` ignores +whitespace, and ``-W 200`` sets the width of the output (as otherwise it +will use 130 by default, which is often a bit too little). + +The ``rev^-`` syntax is a handy shorthand for ``rev^..rev``, essentially +giving you just the diff for that single commit; also see +the official `git rev-parse documentation`_. + +.. _git rev-parse documentation: https://git-scm.com/docs/git-rev-parse#_other_rev_parent_shorthand_notations + +Again, note the inclusion of ``-W`` for ``git diff``; this ensures that +you will see the full function for any function that has changed. + +One incredibly important thing that colordiff does is to highlight lines +that are different. For example, if an error-handling ``goto`` has +changed labels between the original and backported patch, colordiff will +show these side-by-side but highlighted in a different color. Thus, it +is easy to see that the two ``goto`` statements are jumping to different +labels. Likewise, lines that were not modified by either patch but +differ in the context will also be highlighted and thus stand out during +a manual inspection. + +Of course, this is just a visual inspection; the real test is building +and running the patched kernel (or program). + +Build testing +------------- + +We won't cover runtime testing here, but it can be a good idea to build +just the files touched by the patch as a quick sanity check. For the +Linux kernel you can build single files like this, assuming you have the +``.config`` and build environment set up correctly:: + + make path/to/file.o + +Note that this won't discover linker errors, so you should still do a +full build after verifying that the single file compiles. By compiling +the single file first you can avoid having to wait for a full build *in +case* there are compiler errors in any of the files you've changed. + +Runtime testing +--------------- + +Even a successful build or boot test is not necessarily enough to rule +out a missing dependency somewhere. Even though the chances are small, +there could be code changes where two independent changes to the same +file result in no conflicts, no compile-time errors, and runtime errors +only in exceptional cases. + +One concrete example of this was a pair of patches to the system call +entry code where the first patch saved/restored a register and a later +patch made use of the same register somewhere in the middle of this +sequence. Since there was no overlap between the changes, one could +cherry-pick the second patch, have no conflicts, and believe that +everything was fine, when in fact the code was now scribbling over an +unsaved register. + +Although the vast majority of errors will be caught during compilation +or by superficially exercising the code, the only way to *really* verify +a backport is to review the final patch with the same level of scrutiny +as you would (or should) give to any other patch. Having unit tests and +regression tests or other types of automatic testing can help increase +the confidence in the correctness of a backport. + +Submitting backports to stable +============================== + +As the stable maintainers try to cherry-pick mainline fixes onto their +stable kernels, they may send out emails asking for backports when when +encountering conflicts, see e.g. +<https://lore.kernel.org/stable/2023101528-jawed-shelving-071a@gregkh/>. +These emails typically include the exact steps you need to cherry-pick +the patch to the correct tree and submit the patch. + +One thing to make sure is that your changelog conforms to the expected +format:: + + <original patch title> + + [ Upstream commit <mainline rev> ] + + <rest of the original changelog> + [ <summary of the conflicts and their resolutions> ] + Signed-off-by: <your name and email> + +The "Upstream commit" line is sometimes slightly different depending on +the stable version. Older version used this format:: + + commit <mainline rev> upstream. + +It is most common to indicate the kernel version the patch applies to +in the email subject line (using e.g. +``git send-email --subject-prefix='PATCH 6.1.y'``), but you can also put +it in the Signed-off-by:-area or below the ``---`` line. + +The stable maintainers expect separate submissions for each active +stable version, and each submission should also be tested separately. + +A few final words of advice +=========================== + +1) Approach the backporting process with humility. +2) Understand the patch you are backporting; this means reading both + the changelog and the code. +3) Be honest about your confidence in the result when submitting the + patch. +4) Ask relevant maintainers for explicit acks. + +Examples +======== + +The above shows roughly the idealized process of backporting a patch. +For a more concrete example, see this video tutorial where two patches +are backported from mainline to stable: +`Backporting Linux Kernel Patches`_. + +.. _Backporting Linux Kernel Patches: https://youtu.be/sBR7R1V2FeA diff --git a/Documentation/process/botching-up-ioctls.rst b/Documentation/process/botching-up-ioctls.rst index 2d4829b2fb09..a05e8401de1c 100644 --- a/Documentation/process/botching-up-ioctls.rst +++ b/Documentation/process/botching-up-ioctls.rst @@ -2,7 +2,7 @@ (How to avoid) Botching up ioctls ================================= -From: http://blog.ffwll.ch/2013/11/botching-up-ioctls.html +From: https://blog.ffwll.ch/2013/11/botching-up-ioctls.html By: Daniel Vetter, Copyright © 2013 Intel Corporation @@ -41,7 +41,7 @@ will need to add a 32-bit compat layer: structures to the kernel, or if the kernel checks the structure size, which e.g. the drm core does. - * Pointers are __u64, cast from/to a uintprt_t on the userspace side and + * Pointers are __u64, cast from/to a uintptr_t on the userspace side and from/to a void __user * in the kernel. Try really hard not to delay this conversion or worse, fiddle the raw __u64 through your code since that diminishes the checking tools like sparse can provide. The macro @@ -208,7 +208,7 @@ Not every problem needs a new ioctl: it's much quicker to push a driver-private interface than engaging in lengthy discussions for a more generic solution. And occasionally doing a private interface to spearhead a new concept is what's required. But in the - end, once the generic interface comes around you'll end up maintainer two + end, once the generic interface comes around you'll end up maintaining two interfaces. Indefinitely. * Consider other interfaces than ioctls. A sysfs attribute is much better for diff --git a/Documentation/process/changes.rst b/Documentation/process/changes.rst index 2284f2221f02..50b3d1cb1115 100644 --- a/Documentation/process/changes.rst +++ b/Documentation/process/changes.rst @@ -7,7 +7,7 @@ Intro ===== This document is designed to provide a list of the minimum levels of -software necessary to run the 4.x kernels. +software necessary to run the current kernel version. This document is originally based on my "Changes" file for 2.0.x kernels and therefore owes credit to the same people as that file (Jared Mauch, @@ -29,12 +29,17 @@ you probably needn't concern yourself with pcmciautils. ====================== =============== ======================================== Program Minimal version Command to check the version ====================== =============== ======================================== -GNU C 4.6 gcc --version -GNU make 3.81 make --version -binutils 2.21 ld -v +GNU C 5.1 gcc --version +Clang/LLVM (optional) 11.0.0 clang --version +Rust (optional) 1.74.1 rustc --version +bindgen (optional) 0.65.1 bindgen --version +GNU make 3.82 make --version +bash 4.2 bash --version +binutils 2.25 ld -v flex 2.5.35 flex --version bison 2.0 bison --version -util-linux 2.10o fdformat --version +pahole 1.16 pahole --version +util-linux 2.10o mount --version kmod 13 depmod -V e2fsprogs 1.41.4 e2fsck -V jfsutils 1.1.3 fsck.jfs -V @@ -47,14 +52,16 @@ quota-tools 3.09 quota -V PPP 2.4.0 pppd --version nfs-utils 1.0.5 showmount --version procps 3.2.0 ps --version -oprofile 0.9 oprofiled --version udev 081 udevd --version grub 0.93 grub --version || grub-install --version mcelog 0.6 mcelog --version iptables 1.4.2 iptables -V openssl & libcrypto 1.0.0 openssl version bc 1.06.95 bc --version -Sphinx\ [#f1]_ 1.3 sphinx-build --version +Sphinx\ [#f1]_ 2.4.4 sphinx-build --version +cpio any cpio --version +GNU tar 1.28 tar --version +gtags (optional) 6.6.5 gtags --version ====================== =============== ======================================== .. [#f1] Sphinx is needed only to build the Kernel documentation @@ -68,15 +75,53 @@ GCC The gcc version requirements may vary depending on the type of CPU in your computer. +Clang/LLVM (optional) +--------------------- + +The latest formal release of clang and LLVM utils (according to +`releases.llvm.org <https://releases.llvm.org>`_) are supported for building +kernels. Older releases aren't guaranteed to work, and we may drop workarounds +from the kernel that were used to support older versions. Please see additional +docs on :ref:`Building Linux with Clang/LLVM <kbuild_llvm>`. + +Rust (optional) +--------------- + +A particular version of the Rust toolchain is required. Newer versions may or +may not work because the kernel depends on some unstable Rust features, for +the moment. + +Each Rust toolchain comes with several "components", some of which are required +(like ``rustc``) and some that are optional. The ``rust-src`` component (which +is optional) needs to be installed to build the kernel. Other components are +useful for developing. + +Please see Documentation/rust/quick-start.rst for instructions on how to +satisfy the build requirements of Rust support. In particular, the ``Makefile`` +target ``rustavailable`` is useful to check why the Rust toolchain may not +be detected. + +bindgen (optional) +------------------ + +``bindgen`` is used to generate the Rust bindings to the C side of the kernel. +It depends on ``libclang``. + Make ---- -You will need GNU make 3.81 or later to build the kernel. +You will need GNU make 3.82 or later to build the kernel. + +Bash +---- + +Some bash scripts are used for the kernel build. +Bash 4.2 or newer is needed. Binutils -------- -Binutils 2.21 or newer is needed to build the kernel. +Binutils 2.25 or newer is needed to build the kernel. pkg-config ---------- @@ -99,6 +144,16 @@ Bison Since Linux 4.16, the build system generates parsers during build. This requires bison 2.0 or later. +pahole: +------- + +Since Linux 5.2, if CONFIG_DEBUG_INFO_BTF is selected, the build system +generates BTF (BPF Type Format) from DWARF in vmlinux, a bit later from kernel +modules as well. This requires pahole v1.16 or later. + +It is found in the 'dwarves' or 'pahole' distro packages or from +https://fedorapeople.org/~acme/dwarves/. + Perl ---- @@ -121,6 +176,18 @@ You will need openssl to build kernels 3.7 and higher if module signing is enabled. You will also need openssl development packages to build kernels 4.3 and higher. +Tar +--- + +GNU tar is needed if you want to enable access to the kernel headers via sysfs +(CONFIG_IKHEADERS). + +gtags / GNU GLOBAL (optional) +----------------------------- + +The kernel build requires GNU GLOBAL version 6.6.5 or later to generate +tag files through ``make gtags``. This is due to its use of the gtags +``-C (--directory)`` flag. System utilities **************** @@ -129,7 +196,7 @@ Architectural changes --------------------- DevFS has been obsoleted in favour of udev -(http://www.kernel.org/pub/linux/utils/kernel/hotplug/) +(https://www.kernel.org/pub/linux/utils/kernel/hotplug/) 32-bit UID support is now in place. Have fun! @@ -146,7 +213,7 @@ Util-linux New versions of util-linux provide ``fdisk`` support for larger disks, support new options to mount, recognize more supported partition -types, have a fdformat which works with 2.4 kernels, and similar goodies. +types, and similar goodies. You'll probably want to upgrade. Ksymoops @@ -320,6 +387,12 @@ Sphinx Please see :ref:`sphinx_install` in :ref:`Documentation/doc-guide/sphinx.rst <sphinxdoc>` for details about Sphinx requirements. +rustdoc +------- + +``rustdoc`` is used to generate the documentation for Rust code. Please see +Documentation/rust/general-information.rst for more information. + Getting updated software ======================== @@ -331,11 +404,31 @@ gcc - <ftp://ftp.gnu.org/gnu/gcc/> +Clang/LLVM +---------- + +- :ref:`Getting LLVM <getting_llvm>`. + +Rust +---- + +- Documentation/rust/quick-start.rst. + +bindgen +------- + +- Documentation/rust/quick-start.rst. + Make ---- - <ftp://ftp.gnu.org/gnu/make/> +Bash +---- + +- <ftp://ftp.gnu.org/gnu/bash/> + Binutils -------- @@ -383,22 +476,24 @@ Mkinitrd E2fsprogs --------- -- <http://prdownloads.sourceforge.net/e2fsprogs/e2fsprogs-1.29.tar.gz> +- <https://www.kernel.org/pub/linux/kernel/people/tytso/e2fsprogs/> +- <https://git.kernel.org/pub/scm/fs/ext2/e2fsprogs.git/> JFSutils -------- -- <http://jfs.sourceforge.net/> +- <https://jfs.sourceforge.net/> Reiserfsprogs ------------- -- <http://www.kernel.org/pub/linux/utils/fs/reiserfs/> +- <https://git.kernel.org/pub/scm/linux/kernel/git/jeffm/reiserfsprogs.git/> Xfsprogs -------- -- <ftp://oss.sgi.com/projects/xfs/> +- <https://git.kernel.org/pub/scm/fs/xfs/xfsprogs-dev.git> +- <https://www.kernel.org/pub/linux/utils/fs/xfs/xfsprogs/> Pcmciautils ----------- @@ -408,7 +503,7 @@ Pcmciautils Quota-tools ----------- -- <http://sourceforge.net/projects/linuxquota/> +- <https://sourceforge.net/projects/linuxquota/> Intel P6 microcode @@ -419,7 +514,7 @@ Intel P6 microcode udev ---- -- <http://www.freedesktop.org/software/systemd/man/udev.html> +- <https://www.freedesktop.org/software/systemd/man/udev.html> FUSE ---- @@ -429,7 +524,12 @@ FUSE mcelog ------ -- <http://www.mcelog.org/> +- <https://www.mcelog.org/> + +cpio +---- + +- <https://www.gnu.org/software/cpio/> Networking ********** @@ -437,17 +537,20 @@ Networking PPP --- -- <ftp://ftp.samba.org/pub/ppp/> +- <https://download.samba.org/pub/ppp/> +- <https://git.ozlabs.org/?p=ppp.git> +- <https://github.com/paulusmack/ppp/> NFS-utils --------- -- <http://sourceforge.net/project/showfiles.php?group_id=14> +- <https://sourceforge.net/project/showfiles.php?group_id=14> +- <https://nfs.sourceforge.net/> Iptables -------- -- <http://www.iptables.org/downloads.html> +- <https://netfilter.org/projects/iptables/index.html> Ip-route2 --------- @@ -457,12 +560,7 @@ Ip-route2 OProfile -------- -- <http://oprofile.sf.net/download/> - -NFS-Utils ---------- - -- <http://nfs.sourceforge.net/> +- <https://oprofile.sf.net/download/> Kernel documentation ******************** @@ -470,4 +568,4 @@ Kernel documentation Sphinx ------ -- <http://www.sphinx-doc.org/> +- <https://www.sphinx-doc.org/> diff --git a/Documentation/process/clang-format.rst b/Documentation/process/clang-format.rst index 6710c0707721..1d089a847c1b 100644 --- a/Documentation/process/clang-format.rst +++ b/Documentation/process/clang-format.rst @@ -32,7 +32,7 @@ Linux distributions for a long time. Search for ``clang-format`` in your repositories. Otherwise, you can either download pre-built LLVM/clang binaries or build the source code from: - http://releases.llvm.org/download.html + https://releases.llvm.org/download.html See more information about the tool at: @@ -97,7 +97,7 @@ it can be very useful. There are integrations for many popular text editors. For some of them, like vim, emacs, BBEdit and Visual Studio you can find support built-in. -For instructions, read the appropiate section at: +For instructions, read the appropriate section at: https://clang.llvm.org/docs/ClangFormat.html diff --git a/Documentation/process/code-of-conduct-interpretation.rst b/Documentation/process/code-of-conduct-interpretation.rst index e899f14a4ba2..66b07f14714c 100644 --- a/Documentation/process/code-of-conduct-interpretation.rst +++ b/Documentation/process/code-of-conduct-interpretation.rst @@ -51,7 +51,7 @@ the Technical Advisory Board (TAB) or other maintainers if you're uncertain how to handle situations that come up. It will not be considered a violation report unless you want it to be. If you are uncertain about approaching the TAB or any other maintainers, please -reach out to our conflict mediator, Mishi Choudhary <mishi@linux.com>. +reach out to our conflict mediator, Joanna Lee <jlee@linuxfoundation.org>. In the end, "be kind to each other" is really what the end goal is for everybody. We know everyone is human and we all fail at times, but the @@ -127,10 +127,12 @@ are listed at https://kernel.org/code-of-conduct.html. Members can not access reports made before they joined or after they have left the committee. -The initial Code of Conduct Committee consists of volunteer members of -the TAB, as well as a professional mediator acting as a neutral third -party. The first task of the committee is to establish documented -processes, which will be made public. +The Code of Conduct Committee consists of volunteer community members +appointed by the TAB, as well as a professional mediator acting as a +neutral third party. The processes the Code of Conduct committee will +use to address reports is varied and will depend on the individual +circumstance, however, this file serves as documentation for the +general process used. Any member of the committee, including the mediator, can be contacted directly if a reporter does not wish to include the full committee in a @@ -141,16 +143,16 @@ processes (see above) and consults with the TAB as needed and appropriate, for instance to request and receive information about the kernel community. -Any decisions by the committee will be brought to the TAB, for -implementation of enforcement with the relevant maintainers if needed. -A decision by the Code of Conduct Committee can be overturned by the TAB -by a two-thirds vote. +Any decisions regarding enforcement recommendations will be brought to +the TAB for implementation of enforcement with the relevant maintainers +if needed. A decision by the Code of Conduct Committee can be overturned +by the TAB by a two-thirds vote. At quarterly intervals, the Code of Conduct Committee and TAB will provide a report summarizing the anonymised reports that the Code of Conduct committee has received and their status, as well details of any overridden decisions including complete and identifiable voting details. -We expect to establish a different process for Code of Conduct Committee -staffing beyond the bootstrap period. This document will be updated -with that information when this occurs. +Because how we interpret and enforce the Code of Conduct will evolve over +time, this document will be updated when necessary to reflect any +changes. diff --git a/Documentation/process/coding-style.rst b/Documentation/process/coding-style.rst index ada573b7d703..c48382c6b477 100644 --- a/Documentation/process/coding-style.rst +++ b/Documentation/process/coding-style.rst @@ -69,9 +69,26 @@ something to hide: if (condition) do_this; do_something_everytime; +Don't use commas to avoid using braces: + +.. code-block:: c + + if (condition) + do_this(), do_that(); + +Always uses braces for multiple statements: + +.. code-block:: c + + if (condition) { + do_this(); + do_that(); + } + Don't put multiple assignments on a single line either. Kernel coding style is super simple. Avoid tricky expressions. + Outside of comments, documentation and except in Kconfig, spaces are never used for indentation, and the above example is deliberately broken. @@ -84,15 +101,20 @@ Get a decent editor and don't leave whitespace at the end of lines. Coding style is all about readability and maintainability using commonly available tools. -The limit on the length of lines is 80 columns and this is a strongly -preferred limit. +The preferred limit on the length of a single line is 80 columns. + +Statements longer than 80 columns should be broken into sensible chunks, +unless exceeding 80 columns significantly increases readability and does +not hide information. -Statements longer than 80 columns will be broken into sensible chunks, unless -exceeding 80 columns significantly increases readability and does not hide -information. Descendants are always substantially shorter than the parent and -are placed substantially to the right. The same applies to function headers -with a long argument list. However, never break user-visible strings such as -printk messages, because that breaks the ability to grep for them. +Descendants are always substantially shorter than the parent and +are placed substantially to the right. A very commonly used style +is to align descendants to a function open parenthesis. + +These same rules are applied to function headers with a long argument list. + +However, never break user-visible strings such as printk messages because +that breaks the ability to grep for them. 3) Placing Braces and Spaces @@ -284,9 +306,9 @@ context lines. 4) Naming --------- -C is a Spartan language, and so should your naming be. Unlike Modula-2 -and Pascal programmers, C programmers do not use cute names like -ThisVariableIsATemporaryCounter. A C programmer would call that +C is a Spartan language, and your naming conventions should follow suit. +Unlike Modula-2 and Pascal programmers, C programmers do not use cute +names like ThisVariableIsATemporaryCounter. A C programmer would call that variable ``tmp``, which is much easier to write, and not the least more difficult to understand. @@ -300,9 +322,8 @@ that counts the number of active users, you should call that ``count_active_users()`` or similar, you should **not** call it ``cntusr()``. Encoding the type of a function into the name (so-called Hungarian -notation) is brain damaged - the compiler knows the types anyway and can -check those, and it only confuses the programmer. No wonder MicroSoft -makes buggy programs. +notation) is asinine - the compiler knows the types anyway and can check +those, and it only confuses the programmer. LOCAL variable names should be short, and to the point. If you have some random integer loop counter, it should probably be called ``i``. @@ -314,6 +335,26 @@ If you are afraid to mix up your local variable names, you have another problem, which is called the function-growth-hormone-imbalance syndrome. See chapter 6 (Functions). +For symbol names and documentation, avoid introducing new usage of +'master / slave' (or 'slave' independent of 'master') and 'blacklist / +whitelist'. + +Recommended replacements for 'master / slave' are: + '{primary,main} / {secondary,replica,subordinate}' + '{initiator,requester} / {target,responder}' + '{controller,host} / {device,worker,proxy}' + 'leader / follower' + 'director / performer' + +Recommended replacements for 'blacklist/whitelist' are: + 'denylist / allowlist' + 'blocklist / passlist' + +Exceptions for introducing new usage is to maintain a userspace ABI/API, +or when updating code for an existing (as of 2020) hardware or protocol +specification that mandates those terms. For new specifications +translate specification usage of the terminology to the kernel coding +standard where possible. 5) Typedefs ----------- @@ -439,13 +480,48 @@ closing function brace line. E.g.: } EXPORT_SYMBOL(system_is_up); +6.1) Function prototypes +************************ + In function prototypes, include parameter names with their data types. Although this is not required by the C language, it is preferred in Linux because it is a simple way to add valuable information for the reader. -Do not use the ``extern`` keyword with function prototypes as this makes +Do not use the ``extern`` keyword with function declarations as this makes lines longer and isn't strictly necessary. +When writing function prototypes, please keep the `order of elements regular +<https://lore.kernel.org/mm-commits/CAHk-=wiOCLRny5aifWNhr621kYrJwhfURsa0vFPeUEm8mF0ufg@mail.gmail.com/>`_. +For example, using this function declaration example:: + + __init void * __must_check action(enum magic value, size_t size, u8 count, + char *fmt, ...) __printf(4, 5) __malloc; + +The preferred order of elements for a function prototype is: + +- storage class (below, ``static __always_inline``, noting that ``__always_inline`` + is technically an attribute but is treated like ``inline``) +- storage class attributes (here, ``__init`` -- i.e. section declarations, but also + things like ``__cold``) +- return type (here, ``void *``) +- return type attributes (here, ``__must_check``) +- function name (here, ``action``) +- function parameters (here, ``(enum magic value, size_t size, u8 count, char *fmt, ...)``, + noting that parameter names should always be included) +- function parameter attributes (here, ``__printf(4, 5)``) +- function behavior attributes (here, ``__malloc``) + +Note that for a function **definition** (i.e. the actual function body), +the compiler does not allow function parameter attributes after the +function parameters. In these cases, they should go after the storage +class attributes (e.g. note the changed position of ``__printf(4, 5)`` +below, compared to the **declaration** example above):: + + static __always_inline __init __printf(4, 5) void * __must_check action(enum magic value, + size_t size, u8 count, char *fmt, ...) __malloc + { + ... + } 7) Centralized exiting of functions ----------------------------------- @@ -659,6 +735,10 @@ for aligning variables/macros, for reflowing text and other similar tasks. See the file :ref:`Documentation/process/clang-format.rst <clangformat>` for more details. +Some basic editor settings, such as indentation and line endings, will be +set automatically if you are using an editor that is compatible with +EditorConfig. See the official EditorConfig website for more information: +https://editorconfig.org/ 10) Kconfig configuration files ------------------------------- @@ -806,15 +886,15 @@ covers RTL which is used frequently with assembly language in the kernel. ---------------------------- Kernel developers like to be seen as literate. Do mind the spelling -of kernel messages to make a good impression. Do not use crippled -words like ``dont``; use ``do not`` or ``don't`` instead. Make the messages -concise, clear, and unambiguous. +of kernel messages to make a good impression. Do not use incorrect +contractions like ``dont``; use ``do not`` or ``don't`` instead. Make the +messages concise, clear, and unambiguous. Kernel messages do not have to be terminated with a period. Printing numbers in parentheses (%d) adds no value and should be avoided. -There are a number of driver model diagnostic macros in <linux/device.h> +There are a number of driver model diagnostic macros in <linux/dev_printk.h> which you should use to make sure messages are matched to the right device and driver, and are tagged with the right level: dev_err(), dev_warn(), dev_info(), and so forth. For messages that aren't associated with a @@ -988,7 +1068,7 @@ Similarly, if you need to calculate the size of some structure member, use .. code-block:: c - #define FIELD_SIZEOF(t, f) (sizeof(((t*)0)->f)) + #define sizeof_field(t, f) (sizeof(((t*)0)->f)) There are also min() and max() macros that do strict type checking if you need them. Feel free to peruse that header file to see what else is already @@ -1110,6 +1190,68 @@ expression used. For instance: #endif /* CONFIG_SOMETHING */ +22) Do not crash the kernel +--------------------------- + +In general, the decision to crash the kernel belongs to the user, rather +than to the kernel developer. + +Avoid panic() +************* + +panic() should be used with care and primarily only during system boot. +panic() is, for example, acceptable when running out of memory during boot and +not being able to continue. + +Use WARN() rather than BUG() +**************************** + +Do not add new code that uses any of the BUG() variants, such as BUG(), +BUG_ON(), or VM_BUG_ON(). Instead, use a WARN*() variant, preferably +WARN_ON_ONCE(), and possibly with recovery code. Recovery code is not +required if there is no reasonable way to at least partially recover. + +"I'm too lazy to do error handling" is not an excuse for using BUG(). Major +internal corruptions with no way of continuing may still use BUG(), but need +good justification. + +Use WARN_ON_ONCE() rather than WARN() or WARN_ON() +************************************************** + +WARN_ON_ONCE() is generally preferred over WARN() or WARN_ON(), because it +is common for a given warning condition, if it occurs at all, to occur +multiple times. This can fill up and wrap the kernel log, and can even slow +the system enough that the excessive logging turns into its own, additional +problem. + +Do not WARN lightly +******************* + +WARN*() is intended for unexpected, this-should-never-happen situations. +WARN*() macros are not to be used for anything that is expected to happen +during normal operation. These are not pre- or post-condition asserts, for +example. Again: WARN*() must not be used for a condition that is expected +to trigger easily, for example, by user space actions. pr_warn_once() is a +possible alternative, if you need to notify the user of a problem. + +Do not worry about panic_on_warn users +************************************** + +A few more words about panic_on_warn: Remember that ``panic_on_warn`` is an +available kernel option, and that many users set this option. This is why +there is a "Do not WARN lightly" writeup, above. However, the existence of +panic_on_warn users is not a valid reason to avoid the judicious use +WARN*(). That is because, whoever enables panic_on_warn has explicitly +asked the kernel to crash if a WARN*() fires, and such users must be +prepared to deal with the consequences of a system that is somewhat more +likely to crash. + +Use BUILD_BUG_ON() for compile-time assertions +********************************************** + +The use of BUILD_BUG_ON() is acceptable and encouraged, because it is a +compile-time assertion that has no effect at runtime. + Appendix I) References ---------------------- @@ -1124,10 +1266,10 @@ Addison-Wesley, Inc., 1999. ISBN 0-201-61586-X. GNU manuals - where in compliance with K&R and this text - for cpp, gcc, -gcc internals and indent, all available from http://www.gnu.org/manual/ +gcc internals and indent, all available from https://www.gnu.org/manual/ WG14 is the international standardization working group for the programming language C, URL: http://www.open-std.org/JTC1/SC22/WG14/ -Kernel :ref:`process/coding-style.rst <codingstyle>`, by greg@kroah.com at OLS 2002: +Kernel CodingStyle, by greg@kroah.com at OLS 2002: http://www.kroah.com/linux/talks/ols_2002_kernel_codingstyle_talk/html/ diff --git a/Documentation/process/contribution-maturity-model.rst b/Documentation/process/contribution-maturity-model.rst new file mode 100644 index 000000000000..b87ab34de22c --- /dev/null +++ b/Documentation/process/contribution-maturity-model.rst @@ -0,0 +1,109 @@ +.. SPDX-License-Identifier: GPL-2.0 + +======================================== +Linux Kernel Contribution Maturity Model +======================================== + + +Background +========== + +As a part of the 2021 Linux Kernel Maintainers’ Summit, there was a +`discussion <https://lwn.net/Articles/870581/>`_ about the challenges in +recruiting kernel maintainers as well as maintainer succession. Some of +the conclusions from that discussion included that companies which are a +part of the Linux Kernel community need to allow engineers to be +maintainers as part of their job, so they can grow into becoming +respected leaders and eventually, kernel maintainers. To support a +strong talent pipeline, developers should be allowed and encouraged to +take on upstream contributions such as reviewing other people’s patches, +refactoring kernel infrastructure, and writing documentation. + +To that end, the Linux Foundation Technical Advisory Board (TAB) +proposes this Linux Kernel Contribution Maturity Model. These common +expectations for upstream community engagement aim to increase the +influence of individual developers, increase the collaboration of +organizations, and improve the overall health of the Linux Kernel +ecosystem. + +The TAB urges organizations to continuously evaluate their Open Source +maturity model and commit to improvements to align with this model. To +be effective, this evaluation should incorporate feedback from across +the organization, including management and developers at all seniority +levels. In the spirit of Open Source, we encourage organizations to +publish their evaluations and plans to improve their engagement with the +upstream community. + +Level 0 +======= + +* Software Engineers are not allowed to contribute patches to the Linux + kernel. + + +Level 1 +======= + +* Software Engineers are allowed to contribute patches to the Linux + kernel, either as part of their job responsibilities or on their own + time. + +Level 2 +======= + +* Software Engineers are expected to contribute to the Linux Kernel as + part of their job responsibilities. +* Software Engineers will be supported to attend Linux-related + conferences as a part of their job. +* A Software Engineer’s upstream code contributions will be considered + in promotion and performance reviews. + +Level 3 +======= + +* Software Engineers are expected to review patches (including patches + authored by engineers from other companies) as part of their job + responsibilities +* Contributing presentations or papers to Linux-related or academic + conferences (such those organized by the Linux Foundation, Usenix, + ACM, etc.), are considered part of an engineer’s work. +* A Software Engineer’s community contributions will be considered in + promotion and performance reviews. +* Organizations will regularly report metrics of their open source + contributions and track these metrics over time. These metrics may be + published only internally within the organization, or at the + organization’s discretion, some or all may be published externally. + Metrics that are strongly suggested include: + + * The number of upstream kernel contributions by team or organization + (e.g., all people reporting up to a manager, director, or VP). + * The percentage of kernel developers who have made upstream + contributions relative to the total kernel developers in the + organization. + * The time interval between kernels used in the organization’s servers + and/or products, and the publication date of the upstream kernel + upon which the internal kernel is based. + * The number of out-of-tree commits present in internal kernels. + +Level 4 +======= + +* Software Engineers are encouraged to spend a portion of their work + time focused on Upstream Work, which is defined as reviewing patches, + serving on program committees, improving core project infrastructure + such as writing or maintaining tests, upstream tech debt reduction, + writing documentation, etc. +* Software Engineers are supported in helping to organize Linux-related + conferences. +* Organizations will consider community member feedback in official + performance reviews. + +Level 5 +======= + +* Upstream kernel development is considered a formal job position, with + at least a third of the engineer’s time spent doing Upstream Work. +* Organizations will actively seek out community member feedback as a + factor in official performance reviews. +* Organizations will regularly report internally on the ratio of + Upstream Work to work focused on directly pursuing business goals. diff --git a/Documentation/process/deprecated.rst b/Documentation/process/deprecated.rst index 179f2a5625a0..1f7f3e6c9cda 100644 --- a/Documentation/process/deprecated.rst +++ b/Documentation/process/deprecated.rst @@ -29,6 +29,28 @@ a header file, it isn't the full solution. Such interfaces must either be fully removed from the kernel, or added to this file to discourage others from using them in the future. +BUG() and BUG_ON() +------------------ +Use WARN() and WARN_ON() instead, and handle the "impossible" +error condition as gracefully as possible. While the BUG()-family +of APIs were originally designed to act as an "impossible situation" +assert and to kill a kernel thread "safely", they turn out to just be +too risky. (e.g. "In what order do locks need to be released? Have +various states been restored?") Very commonly, using BUG() will +destabilize a system or entirely break it, which makes it impossible +to debug or even get viable crash reports. Linus has `very strong +<https://lore.kernel.org/lkml/CA+55aFy6jNLsywVYdGp83AMrXBo_P-pkjkphPGrO=82SPKCpLQ@mail.gmail.com/>`_ +feelings `about this +<https://lore.kernel.org/lkml/CAHk-=whDHsbK3HTOpTF=ue_o04onRwTEaK_ZoJp_fjbqq4+=Jw@mail.gmail.com/>`_. + +Note that the WARN()-family should only be used for "expected to +be unreachable" situations. If you want to warn about "reachable +but undesirable" situations, please use the pr_warn()-family of +functions. System owners may have set the *panic_on_warn* sysctl, +to make sure their systems do not continue running in the face of +"unreachable" conditions. (For example, see commits like `this one +<https://git.kernel.org/linus/d4689846881d160a4d12a514e991a740bcb5d65a>`_.) + open-coded arithmetic in allocator arguments -------------------------------------------- Dynamic size calculations (especially multiplication) should not be @@ -37,8 +59,9 @@ risk of them overflowing. This could lead to values wrapping around and a smaller allocation being made than the caller was expecting. Using those allocations could lead to linear overflows of heap memory and other misbehaviors. (One exception to this is literal values where the compiler -can warn if they might overflow. Though using literals for arguments as -suggested below is also harmless.) +can warn if they might overflow. However, the preferred way in these +cases is to refactor the code as suggested below to avoid the open-coded +arithmetic.) For example, do not use ``count * size`` as an argument, as in:: @@ -48,10 +71,13 @@ Instead, the 2-factor form of the allocator should be used:: foo = kmalloc_array(count, size, GFP_KERNEL); +Specifically, kmalloc() can be replaced with kmalloc_array(), and +kzalloc() can be replaced with kcalloc(). + If no 2-factor form is available, the saturate-on-overflow helpers should be used:: - bar = vmalloc(array_size(count, size)); + bar = dma_alloc_coherent(dev, array_size(count, size), &dma, GFP_KERNEL); Another common case to avoid is calculating the size of a structure with a trailing array of others structures, as in:: @@ -63,51 +89,102 @@ Instead, use the helper:: header = kzalloc(struct_size(header, item, count), GFP_KERNEL); -See :c:func:`array_size`, :c:func:`array3_size`, and :c:func:`struct_size`, -for more details as well as the related :c:func:`check_add_overflow` and -:c:func:`check_mul_overflow` family of functions. +.. note:: If you are using struct_size() on a structure containing a zero-length + or a one-element array as a trailing array member, please refactor such + array usage and switch to a `flexible array member + <#zero-length-and-one-element-arrays>`_ instead. + +For other calculations, please compose the use of the size_mul(), +size_add(), and size_sub() helpers. For example, in the case of:: + + foo = krealloc(current_size + chunk_size * (count - 3), GFP_KERNEL); + +Instead, use the helpers:: + + foo = krealloc(size_add(current_size, + size_mul(chunk_size, + size_sub(count, 3))), GFP_KERNEL); + +For more details, also see array3_size() and flex_array_size(), +as well as the related check_mul_overflow(), check_add_overflow(), +check_sub_overflow(), and check_shl_overflow() family of functions. simple_strtol(), simple_strtoll(), simple_strtoul(), simple_strtoull() ---------------------------------------------------------------------- -The :c:func:`simple_strtol`, :c:func:`simple_strtoll`, -:c:func:`simple_strtoul`, and :c:func:`simple_strtoull` functions +The simple_strtol(), simple_strtoll(), +simple_strtoul(), and simple_strtoull() functions explicitly ignore overflows, which may lead to unexpected results -in callers. The respective :c:func:`kstrtol`, :c:func:`kstrtoll`, -:c:func:`kstrtoul`, and :c:func:`kstrtoull` functions tend to be the +in callers. The respective kstrtol(), kstrtoll(), +kstrtoul(), and kstrtoull() functions tend to be the correct replacements, though note that those require the string to be NUL or newline terminated. strcpy() -------- -:c:func:`strcpy` performs no bounds checking on the destination -buffer. This could result in linear overflows beyond the -end of the buffer, leading to all kinds of misbehaviors. While -`CONFIG_FORTIFY_SOURCE=y` and various compiler flags help reduce the -risk of using this function, there is no good reason to add new uses of -this function. The safe replacement is :c:func:`strscpy`. +strcpy() performs no bounds checking on the destination buffer. This +could result in linear overflows beyond the end of the buffer, leading to +all kinds of misbehaviors. While `CONFIG_FORTIFY_SOURCE=y` and various +compiler flags help reduce the risk of using this function, there is +no good reason to add new uses of this function. The safe replacement +is strscpy(), though care must be given to any cases where the return +value of strcpy() was used, since strscpy() does not return a pointer to +the destination, but rather a count of non-NUL bytes copied (or negative +errno when it truncates). strncpy() on NUL-terminated strings ----------------------------------- -Use of :c:func:`strncpy` does not guarantee that the destination buffer -will be NUL terminated. This can lead to various linear read overflows -and other misbehavior due to the missing termination. It also NUL-pads the -destination buffer if the source contents are shorter than the destination -buffer size, which may be a needless performance penalty for callers using -only NUL-terminated strings. The safe replacement is :c:func:`strscpy`. -(Users of :c:func:`strscpy` still needing NUL-padding will need an -explicit :c:func:`memset` added.) - -If a caller is using non-NUL-terminated strings, :c:func:`strncpy()` can -still be used, but destinations should be marked with the `__nonstring +Use of strncpy() does not guarantee that the destination buffer will +be NUL terminated. This can lead to various linear read overflows and +other misbehavior due to the missing termination. It also NUL-pads +the destination buffer if the source contents are shorter than the +destination buffer size, which may be a needless performance penalty +for callers using only NUL-terminated strings. + +When the destination is required to be NUL-terminated, the replacement is +strscpy(), though care must be given to any cases where the return value +of strncpy() was used, since strscpy() does not return a pointer to the +destination, but rather a count of non-NUL bytes copied (or negative +errno when it truncates). Any cases still needing NUL-padding should +instead use strscpy_pad(). + +If a caller is using non-NUL-terminated strings, strtomem() should be +used, and the destinations should be marked with the `__nonstring <https://gcc.gnu.org/onlinedocs/gcc/Common-Variable-Attributes.html>`_ -attribute to avoid future compiler warnings. +attribute to avoid future compiler warnings. For cases still needing +NUL-padding, strtomem_pad() can be used. strlcpy() --------- -:c:func:`strlcpy` reads the entire source buffer first, possibly exceeding -the given limit of bytes to copy. This is inefficient and can lead to -linear read overflows if a source string is not NUL-terminated. The -safe replacement is :c:func:`strscpy`. +strlcpy() reads the entire source buffer first (since the return value +is meant to match that of strlen()). This read may exceed the destination +size limit. This is both inefficient and can lead to linear read overflows +if a source string is not NUL-terminated. The safe replacement is strscpy(), +though care must be given to any cases where the return value of strlcpy() +is used, since strscpy() will return negative errno values when it truncates. + +%p format specifier +------------------- +Traditionally, using "%p" in format strings would lead to regular address +exposure flaws in dmesg, proc, sysfs, etc. Instead of leaving these to +be exploitable, all "%p" uses in the kernel are being printed as a hashed +value, rendering them unusable for addressing. New uses of "%p" should not +be added to the kernel. For text addresses, using "%pS" is likely better, +as it produces the more useful symbol name instead. For nearly everything +else, just do not add "%p" at all. + +Paraphrasing Linus's current `guidance <https://lore.kernel.org/lkml/CA+55aFwQEd_d40g4mUCSsVRZzrFPUJt74vc6PPpb675hYNXcKw@mail.gmail.com/>`_: + +- If the hashed "%p" value is pointless, ask yourself whether the pointer + itself is important. Maybe it should be removed entirely? +- If you really think the true pointer value is important, why is some + system state or user privilege level considered "special"? If you think + you can justify it (in comments and commit log) well enough to stand + up to Linus's scrutiny, maybe you can use "%px", along with making sure + you have sensible permissions. + +If you are debugging something where "%p" hashing is causing problems, +you can temporarily boot with the debug flag "`no_hash_pointers +<https://git.kernel.org/linus/5ead723a20e0447bc7db33dc3070b420e5f80aa6>`_". Variable Length Arrays (VLAs) ----------------------------- @@ -122,27 +199,176 @@ memory adjacent to the stack (when built without `CONFIG_VMAP_STACK=y`) Implicit switch case fall-through --------------------------------- -The C language allows switch cases to "fall-through" when a "break" statement -is missing at the end of a case. This, however, introduces ambiguity in the -code, as it's not always clear if the missing break is intentional or a bug. +The C language allows switch cases to fall through to the next case +when a "break" statement is missing at the end of a case. This, however, +introduces ambiguity in the code, as it's not always clear if the missing +break is intentional or a bug. For example, it's not obvious just from +looking at the code if `STATE_ONE` is intentionally designed to fall +through into `STATE_TWO`:: + + switch (value) { + case STATE_ONE: + do_something(); + case STATE_TWO: + do_other(); + break; + default: + WARN("unknown state"); + } As there have been a long list of flaws `due to missing "break" statements <https://cwe.mitre.org/data/definitions/484.html>`_, we no longer allow -"implicit fall-through". - -In order to identify intentional fall-through cases, we have adopted a -pseudo-keyword macro 'fallthrough' which expands to gcc's extension -__attribute__((__fallthrough__)). `Statement Attributes -<https://gcc.gnu.org/onlinedocs/gcc/Statement-Attributes.html>`_ - -When the C17/C18 [[fallthrough]] syntax is more commonly supported by +implicit fall-through. In order to identify intentional fall-through +cases, we have adopted a pseudo-keyword macro "fallthrough" which +expands to gcc's extension `__attribute__((__fallthrough__)) +<https://gcc.gnu.org/onlinedocs/gcc/Statement-Attributes.html>`_. +(When the C17/C18 `[[fallthrough]]` syntax is more commonly supported by C compilers, static analyzers, and IDEs, we can switch to using that syntax -for the macro pseudo-keyword. +for the macro pseudo-keyword.) All switch/case blocks must end in one of: - break; - fallthrough; - continue; - goto <label>; - return [expression]; +* break; +* fallthrough; +* continue; +* goto <label>; +* return [expression]; + +Zero-length and one-element arrays +---------------------------------- +There is a regular need in the kernel to provide a way to declare having +a dynamically sized set of trailing elements in a structure. Kernel code +should always use `"flexible array members" <https://en.wikipedia.org/wiki/Flexible_array_member>`_ +for these cases. The older style of one-element or zero-length arrays should +no longer be used. + +In older C code, dynamically sized trailing elements were done by specifying +a one-element array at the end of a structure:: + + struct something { + size_t count; + struct foo items[1]; + }; + +This led to fragile size calculations via sizeof() (which would need to +remove the size of the single trailing element to get a correct size of +the "header"). A `GNU C extension <https://gcc.gnu.org/onlinedocs/gcc/Zero-Length.html>`_ +was introduced to allow for zero-length arrays, to avoid these kinds of +size problems:: + + struct something { + size_t count; + struct foo items[0]; + }; + +But this led to other problems, and didn't solve some problems shared by +both styles, like not being able to detect when such an array is accidentally +being used _not_ at the end of a structure (which could happen directly, or +when such a struct was in unions, structs of structs, etc). + +C99 introduced "flexible array members", which lacks a numeric size for +the array declaration entirely:: + + struct something { + size_t count; + struct foo items[]; + }; + +This is the way the kernel expects dynamically sized trailing elements +to be declared. It allows the compiler to generate errors when the +flexible array does not occur last in the structure, which helps to prevent +some kind of `undefined behavior +<https://git.kernel.org/linus/76497732932f15e7323dc805e8ea8dc11bb587cf>`_ +bugs from being inadvertently introduced to the codebase. It also allows +the compiler to correctly analyze array sizes (via sizeof(), +`CONFIG_FORTIFY_SOURCE`, and `CONFIG_UBSAN_BOUNDS`). For instance, +there is no mechanism that warns us that the following application of the +sizeof() operator to a zero-length array always results in zero:: + + struct something { + size_t count; + struct foo items[0]; + }; + + struct something *instance; + + instance = kmalloc(struct_size(instance, items, count), GFP_KERNEL); + instance->count = count; + + size = sizeof(instance->items) * instance->count; + memcpy(instance->items, source, size); + +At the last line of code above, ``size`` turns out to be ``zero``, when one might +have thought it represents the total size in bytes of the dynamic memory recently +allocated for the trailing array ``items``. Here are a couple examples of this +issue: `link 1 +<https://git.kernel.org/linus/f2cd32a443da694ac4e28fbf4ac6f9d5cc63a539>`_, +`link 2 +<https://git.kernel.org/linus/ab91c2a89f86be2898cee208d492816ec238b2cf>`_. +Instead, `flexible array members have incomplete type, and so the sizeof() +operator may not be applied <https://gcc.gnu.org/onlinedocs/gcc/Zero-Length.html>`_, +so any misuse of such operators will be immediately noticed at build time. + +With respect to one-element arrays, one has to be acutely aware that `such arrays +occupy at least as much space as a single object of the type +<https://gcc.gnu.org/onlinedocs/gcc/Zero-Length.html>`_, +hence they contribute to the size of the enclosing structure. This is prone +to error every time people want to calculate the total size of dynamic memory +to allocate for a structure containing an array of this kind as a member:: + + struct something { + size_t count; + struct foo items[1]; + }; + + struct something *instance; + + instance = kmalloc(struct_size(instance, items, count - 1), GFP_KERNEL); + instance->count = count; + + size = sizeof(instance->items) * instance->count; + memcpy(instance->items, source, size); + +In the example above, we had to remember to calculate ``count - 1`` when using +the struct_size() helper, otherwise we would have --unintentionally-- allocated +memory for one too many ``items`` objects. The cleanest and least error-prone way +to implement this is through the use of a `flexible array member`, together with +struct_size() and flex_array_size() helpers:: + + struct something { + size_t count; + struct foo items[]; + }; + + struct something *instance; + + instance = kmalloc(struct_size(instance, items, count), GFP_KERNEL); + instance->count = count; + + memcpy(instance->items, source, flex_array_size(instance, items, instance->count)); + +There are two special cases of replacement where the DECLARE_FLEX_ARRAY() +helper needs to be used. (Note that it is named __DECLARE_FLEX_ARRAY() for +use in UAPI headers.) Those cases are when the flexible array is either +alone in a struct or is part of a union. These are disallowed by the C99 +specification, but for no technical reason (as can be seen by both the +existing use of such arrays in those places and the work-around that +DECLARE_FLEX_ARRAY() uses). For example, to convert this:: + + struct something { + ... + union { + struct type1 one[0]; + struct type2 two[0]; + }; + }; + +The helper must be used:: + + struct something { + ... + union { + DECLARE_FLEX_ARRAY(struct type1, one); + DECLARE_FLEX_ARRAY(struct type2, two); + }; + }; diff --git a/Documentation/process/development-process.rst b/Documentation/process/development-process.rst index 61c627e41ba8..e34d7da58b7f 100644 --- a/Documentation/process/development-process.rst +++ b/Documentation/process/development-process.rst @@ -3,9 +3,17 @@ A guide to the Kernel Development Process ========================================= -Contents: +The purpose of this document is to help developers (and their managers) +work with the development community with a minimum of frustration. It is +an attempt to document how this community works in a way which is +accessible to those who are not intimately familiar with Linux kernel +development (or, indeed, free software development in general). While +there is some technical material here, this is very much a process-oriented +discussion which does not require a deep knowledge of kernel programming to +understand. .. toctree:: + :caption: Contents :numbered: :maxdepth: 2 @@ -17,12 +25,3 @@ Contents: 6.Followthrough 7.AdvancedTopics 8.Conclusion - -The purpose of this document is to help developers (and their managers) -work with the development community with a minimum of frustration. It is -an attempt to document how this community works in a way which is -accessible to those who are not intimately familiar with Linux kernel -development (or, indeed, free software development in general). While -there is some technical material here, this is very much a process-oriented -discussion which does not require a deep knowledge of kernel programming to -understand. diff --git a/Documentation/process/email-clients.rst b/Documentation/process/email-clients.rst index 5273d06c8ff6..471e1f93fa09 100644 --- a/Documentation/process/email-clients.rst +++ b/Documentation/process/email-clients.rst @@ -25,6 +25,11 @@ attachments, but then the attachments should have content-type it makes quoting portions of the patch more difficult in the patch review process. +It's also strongly recommended that you use plain text in your email body, +for patches and other emails alike. https://useplaintext.email may be useful +for information on how to configure your preferred email client, as well as +listing recommended email clients should you not already have a preference. + Email clients that are used for Linux kernel patches should send the patch text untouched. For example, they should not modify or delete tabs or spaces, even at the beginning or end of lines. @@ -237,9 +242,9 @@ using Mutt to send patches through Gmail:: The Mutt docs have lots more information: - http://dev.mutt.org/trac/wiki/UseCases/Gmail + https://gitlab.com/muttmua/mutt/-/wikis/UseCases/Gmail - http://dev.mutt.org/doc/manual.html + http://www.mutt.org/doc/manual/ Pine (TUI) ********** @@ -272,36 +277,61 @@ Thunderbird (GUI) Thunderbird is an Outlook clone that likes to mangle text, but there are ways to coerce it into behaving. +After doing the modifications, this includes installing the extensions, +you need to restart Thunderbird. + - Allow use of an external editor: - The easiest thing to do with Thunderbird and patches is to use an - "external editor" extension and then just use your favorite ``$EDITOR`` - for reading/merging patches into the body text. To do this, download - and install the extension, then add a button for it using - :menuselection:`View-->Toolbars-->Customize...` and finally just click on it - when in the :menuselection:`Compose` dialog. - - Please note that "external editor" requires that your editor must not - fork, or in other words, the editor must not return before closing. - You may have to pass additional flags or change the settings of your - editor. Most notably if you are using gvim then you must pass the -f - option to gvim by putting ``/usr/bin/gvim -f`` (if the binary is in - ``/usr/bin``) to the text editor field in :menuselection:`external editor` - settings. If you are using some other editor then please read its manual - to find out how to do this. + + The easiest thing to do with Thunderbird and patches is to use extensions + which open your favorite external editor. + + Here are some example extensions which are capable of doing this. + + - "External Editor Revived" + + https://github.com/Frederick888/external-editor-revived + + https://addons.thunderbird.net/en-GB/thunderbird/addon/external-editor-revived/ + + It requires installing a "native messaging host". + Please read the wiki which can be found here: + https://github.com/Frederick888/external-editor-revived/wiki + + - "External Editor" + + https://github.com/exteditor/exteditor + + To do this, download and install the extension, then open the + :menuselection:`compose` window, add a button for it using + :menuselection:`View-->Toolbars-->Customize...` + then just click on the new button when you wish to use the external editor. + + Please note that "External Editor" requires that your editor must not + fork, or in other words, the editor must not return before closing. + You may have to pass additional flags or change the settings of your + editor. Most notably if you are using gvim then you must pass the -f + option to gvim by putting ``/usr/bin/gvim --nofork"`` (if the binary is in + ``/usr/bin``) to the text editor field in :menuselection:`external editor` + settings. If you are using some other editor then please read its manual + to find out how to do this. To beat some sense out of the internal editor, do this: -- Edit your Thunderbird config settings so that it won't use ``format=flowed``. - Go to :menuselection:`edit-->preferences-->advanced-->config editor` to bring up - the thunderbird's registry editor. +- Edit your Thunderbird config settings so that it won't use ``format=flowed``! + Go to your main window and find the button for your main dropdown menu. + :menuselection:`Main Menu-->Preferences-->General-->Config Editor...` + to bring up the thunderbird's registry editor. -- Set ``mailnews.send_plaintext_flowed`` to ``false`` + - Set ``mailnews.send_plaintext_flowed`` to ``false`` -- Set ``mailnews.wraplength`` from ``72`` to ``0`` + - Set ``mailnews.wraplength`` from ``72`` to ``0`` -- :menuselection:`View-->Message Body As-->Plain Text` +- Don't write HTML messages! Go to the main window + :menuselection:`Main Menu-->Account Settings-->youracc@server.something-->Composition & Addressing`! + There you can disable the option "Compose messages in HTML format". -- :menuselection:`View-->Character Encoding-->Unicode (UTF-8)` +- Open messages only as plain text! Go to the main window + :menuselection:`Main Menu-->View-->Message Body As-->Plain Text`! TkRat (GUI) *********** @@ -320,3 +350,23 @@ although tab2space problem can be solved with external editor. Another problem is that Gmail will base64-encode any message that has a non-ASCII character. That includes things like European names. + +Proton Mail +*********** + +Proton Mail has a "feature" where it looks up keys using Web Key Directory +(WKD) and encrypts mail to any recipients for which it finds a key. +Kernel.org publishes the WKD for all developers who have kernel.org accounts. +As a result, emails sent using Proton Mail to kernel.org addresses will be +encrypted. +Unfortunately, Proton Mail does not provide a mechanism to disable the +automatic encryption, viewing it as a privacy feature. +The automatic encryption feature is also enabled for mail sent via the Proton +Mail Bridge, so this affects all outgoing messages, including patches sent with +``git send-email``. +Encrypted mail adds unnecessary friction, as other developers may not have mail +clients, or tooling, configured for use with encrypted mail and some mail +clients may encrypt responses to encrypted mail for all recipients, including +the mailing lists. +Unless a way to disable this "feature" is introduced, Proton Mail is unsuited +to kernel development. diff --git a/Documentation/process/embargoed-hardware-issues.rst b/Documentation/process/embargoed-hardware-issues.rst index 799580acc8de..31000f075707 100644 --- a/Documentation/process/embargoed-hardware-issues.rst +++ b/Documentation/process/embargoed-hardware-issues.rst @@ -1,3 +1,5 @@ +.. _embargoed_hardware_issues: + Embargoed hardware issues ========================= @@ -23,20 +25,23 @@ Contact The Linux kernel hardware security team is separate from the regular Linux kernel security team. -The team only handles the coordination of embargoed hardware security -issues. Reports of pure software security bugs in the Linux kernel are not +The team only handles developing fixes for embargoed hardware security +issues. Reports of pure software security bugs in the Linux kernel are not handled by this team and the reporter will be guided to contact the regular Linux kernel security team (:ref:`Documentation/admin-guide/ <securitybugs>`) instead. The team can be contacted by email at <hardware-security@kernel.org>. This -is a private list of security officers who will help you to coordinate an -issue according to our documented process. +is a private list of security officers who will help you to coordinate a +fix according to our documented process. The list is encrypted and email to the list can be sent by either PGP or S/MIME encrypted and must be signed with the reporter's PGP key or S/MIME certificate. The list's PGP key and S/MIME certificate are available from -https://www.kernel.org/.... +the following URLs: + + - PGP: https://www.kernel.org/static/files/hardware-security.asc + - S/MIME: https://www.kernel.org/static/files/hardware-security.crt While hardware security issues are often handled by the affected hardware vendor, we welcome contact from researchers or individuals who have @@ -55,14 +60,14 @@ Operation of mailing-lists ^^^^^^^^^^^^^^^^^^^^^^^^^^ The encrypted mailing-lists which are used in our process are hosted on -Linux Foundation's IT infrastructure. By providing this service Linux -Foundation's director of IT Infrastructure security technically has the -ability to access the embargoed information, but is obliged to -confidentiality by his employment contract. Linux Foundation's director of -IT Infrastructure security is also responsible for the kernel.org -infrastructure. - -The Linux Foundation's current director of IT Infrastructure security is +Linux Foundation's IT infrastructure. By providing this service, members +of Linux Foundation's IT operations personnel technically have the +ability to access the embargoed information, but are obliged to +confidentiality by their employment contract. Linux Foundation IT +personnel are also responsible for operating and managing the rest of +kernel.org infrastructure. + +The Linux Foundation's current director of IT Project infrastructure is Konstantin Ryabitsev. @@ -127,11 +132,11 @@ other hardware could be affected. The hardware security team will provide an incident-specific encrypted mailing-list which will be used for initial discussion with the reporter, -further disclosure and coordination. +further disclosure, and coordination of fixes. The hardware security team will provide the disclosing party a list of developers (domain experts) who should be informed initially about the -issue after confirming with the developers that they will adhere to this +issue after confirming with the developers that they will adhere to this Memorandum of Understanding and the documented process. These developers form the initial response team and will be responsible for handling the issue after initial contact. The hardware security team is supporting the @@ -147,7 +152,7 @@ The disclosing party should provide a list of contacts for all other entities who have already been, or should be, informed about the issue. This serves several purposes: - - The list of disclosed entities allows communication accross the + - The list of disclosed entities allows communication across the industry, e.g. other OS vendors, HW vendors, etc. - The disclosed entities can be contacted to name experts who should @@ -204,13 +209,18 @@ five work days this is taken as silent acknowledgement. After acknowledgement or resolution of an objection the expert is disclosed by the incident team and brought into the development process. +List participants may not communicate about the issue outside of the +private mailing list. List participants may not use any shared resources +(e.g. employer build farms, CI systems, etc) when working on patches. + Coordinated release """"""""""""""""""" The involved parties will negotiate the date and time where the embargo ends. At that point the prepared mitigations are integrated into the -relevant kernel trees and published. +relevant kernel trees and published. There is no pre-notification process: +fixes are published in public and available to everyone at the same time. While we understand that hardware security issues need coordinated embargo time, the embargo time should be constrained to the minimum time which is @@ -239,24 +249,28 @@ disclosure of a particular issue, unless requested by a response team or by an involved disclosed party. The current ambassadors list: ============= ======================================================== - ARM - AMD Tom Lendacky <tom.lendacky@amd.com> - IBM + AMD Tom Lendacky <thomas.lendacky@amd.com> + Ampere Darren Hart <darren@os.amperecomputing.com> + ARM Catalin Marinas <catalin.marinas@arm.com> + IBM Power Anton Blanchard <anton@linux.ibm.com> + IBM Z Christian Borntraeger <borntraeger@de.ibm.com> Intel Tony Luck <tony.luck@intel.com> Qualcomm Trilok Soni <tsoni@codeaurora.org> + RISC-V Palmer Dabbelt <palmer@dabbelt.com> + Samsung Javier González <javier.gonz@samsung.com> - Microsoft Sasha Levin <sashal@kernel.org> - VMware + Microsoft James Morris <jamorris@linux.microsoft.com> Xen Andrew Cooper <andrew.cooper3@citrix.com> - Canonical Tyler Hicks <tyhicks@canonical.com> + Canonical John Johansen <john.johansen@canonical.com> Debian Ben Hutchings <ben@decadent.org.uk> Oracle Konrad Rzeszutek Wilk <konrad.wilk@oracle.com> Red Hat Josh Poimboeuf <jpoimboe@redhat.com> SUSE Jiri Kosina <jkosina@suse.cz> - Amazon Google Kees Cook <keescook@chromium.org> + + LLVM Nick Desaulniers <ndesaulniers@google.com> ============= ======================================================== If you want your organization to be added to the ambassadors list, please @@ -274,7 +288,7 @@ software decrypts the email and re-encrypts it individually for each subscriber with the subscriber's PGP key or S/MIME certificate. Details about the mailing-list software and the setup which is used to ensure the security of the lists and protection of the data can be found here: -https://www.kernel.org/.... +https://korg.wiki.kernel.org/userdoc/remail. List keys ^^^^^^^^^ diff --git a/Documentation/process/handling-regressions.rst b/Documentation/process/handling-regressions.rst new file mode 100644 index 000000000000..5d3c3de3f4ec --- /dev/null +++ b/Documentation/process/handling-regressions.rst @@ -0,0 +1,790 @@ +.. SPDX-License-Identifier: (GPL-2.0+ OR CC-BY-4.0) +.. See the bottom of this file for additional redistribution information. + +Handling regressions +++++++++++++++++++++ + +*We don't cause regressions* -- this document describes what this "first rule of +Linux kernel development" means in practice for developers. It complements +Documentation/admin-guide/reporting-regressions.rst, which covers the topic from a +user's point of view; if you never read that text, go and at least skim over it +before continuing here. + +The important bits (aka "The TL;DR") +==================================== + +#. Ensure subscribers of the `regression mailing list <https://lore.kernel.org/regressions/>`_ + (regressions@lists.linux.dev) quickly become aware of any new regression + report: + + * When receiving a mailed report that did not CC the list, bring it into the + loop by immediately sending at least a brief "Reply-all" with the list + CCed. + + * Forward or bounce any reports submitted in bug trackers to the list. + +#. Make the Linux kernel regression tracking bot "regzbot" track the issue (this + is optional, but recommended): + + * For mailed reports, check if the reporter included a line like ``#regzbot + introduced v5.13..v5.14-rc1``. If not, send a reply (with the regressions + list in CC) containing a paragraph like the following, which tells regzbot + when the issue started to happen:: + + #regzbot ^introduced 1f2e3d4c5b6a + + * When forwarding reports from a bug tracker to the regressions list (see + above), include a paragraph like the following:: + + #regzbot introduced: v5.13..v5.14-rc1 + #regzbot from: Some N. Ice Human <some.human@example.com> + #regzbot monitor: http://some.bugtracker.example.com/ticket?id=123456789 + +#. When submitting fixes for regressions, add "Link:" tags to the patch + description pointing to all places where the issue was reported, as + mandated by Documentation/process/submitting-patches.rst and + :ref:`Documentation/process/5.Posting.rst <development_posting>`. + +#. Try to fix regressions quickly once the culprit has been identified; fixes + for most regressions should be merged within two weeks, but some need to be + resolved within two or three days. + + +All the details on Linux kernel regressions relevant for developers +=================================================================== + + +The important basics in more detail +----------------------------------- + + +What to do when receiving regression reports +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Ensure the Linux kernel's regression tracker and others subscribers of the +`regression mailing list <https://lore.kernel.org/regressions/>`_ +(regressions@lists.linux.dev) become aware of any newly reported regression: + + * When you receive a report by mail that did not CC the list, immediately bring + it into the loop by sending at least a brief "Reply-all" with the list CCed; + try to ensure it gets CCed again in case you reply to a reply that omitted + the list. + + * If a report submitted in a bug tracker hits your Inbox, forward or bounce it + to the list. Consider checking the list archives beforehand, if the reporter + already forwarded the report as instructed by + Documentation/admin-guide/reporting-issues.rst. + +When doing either, consider making the Linux kernel regression tracking bot +"regzbot" immediately start tracking the issue: + + * For mailed reports, check if the reporter included a "regzbot command" like + ``#regzbot introduced 1f2e3d4c5b6a``. If not, send a reply (with the + regressions list in CC) with a paragraph like the following::: + + #regzbot ^introduced: v5.13..v5.14-rc1 + + This tells regzbot the version range in which the issue started to happen; + you can specify a range using commit-ids as well or state a single commit-id + in case the reporter bisected the culprit. + + Note the caret (^) before the "introduced": it tells regzbot to treat the + parent mail (the one you reply to) as the initial report for the regression + you want to see tracked; that's important, as regzbot will later look out + for patches with "Link:" tags pointing to the report in the archives on + lore.kernel.org. + + * When forwarding a regressions reported to a bug tracker, include a paragraph + with these regzbot commands:: + + #regzbot introduced: 1f2e3d4c5b6a + #regzbot from: Some N. Ice Human <some.human@example.com> + #regzbot monitor: http://some.bugtracker.example.com/ticket?id=123456789 + + Regzbot will then automatically associate patches with the report that + contain "Link:" tags pointing to your mail or the mentioned ticket. + +What's important when fixing regressions +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +You don't need to do anything special when submitting fixes for regression, just +remember to do what Documentation/process/submitting-patches.rst, +:ref:`Documentation/process/5.Posting.rst <development_posting>`, and +Documentation/process/stable-kernel-rules.rst already explain in more detail: + + * Point to all places where the issue was reported using "Link:" tags:: + + Link: https://lore.kernel.org/r/30th.anniversary.repost@klaava.Helsinki.FI/ + Link: https://bugzilla.kernel.org/show_bug.cgi?id=1234567890 + + * Add a "Fixes:" tag to specify the commit causing the regression. + + * If the culprit was merged in an earlier development cycle, explicitly mark + the fix for backporting using the ``Cc: stable@vger.kernel.org`` tag. + +All this is expected from you and important when it comes to regression, as +these tags are of great value for everyone (you included) that might be looking +into the issue weeks, months, or years later. These tags are also crucial for +tools and scripts used by other kernel developers or Linux distributions; one of +these tools is regzbot, which heavily relies on the "Link:" tags to associate +reports for regression with changes resolving them. + +Expectations and best practices for fixing regressions +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +As a Linux kernel developer, you are expected to give your best to prevent +situations where a regression caused by a recent change of yours leaves users +only these options: + + * Run a kernel with a regression that impacts usage. + + * Switch to an older or newer kernel series. + + * Continue running an outdated and thus potentially insecure kernel for more + than three weeks after the regression's culprit was identified. Ideally it + should be less than two. And it ought to be just a few days, if the issue is + severe or affects many users -- either in general or in prevalent + environments. + +How to realize that in practice depends on various factors. Use the following +rules of thumb as a guide. + +In general: + + * Prioritize work on regressions over all other Linux kernel work, unless the + latter concerns a severe issue (e.g. acute security vulnerability, data loss, + bricked hardware, ...). + + * Expedite fixing mainline regressions that recently made it into a proper + mainline, stable, or longterm release (either directly or via backport). + + * Do not consider regressions from the current cycle as something that can wait + till the end of the cycle, as the issue might discourage or prevent users and + CI systems from testing mainline now or generally. + + * Work with the required care to avoid additional or bigger damage, even if + resolving an issue then might take longer than outlined below. + +On timing once the culprit of a regression is known: + + * Aim to mainline a fix within two or three days, if the issue is severe or + bothering many users -- either in general or in prevalent conditions like a + particular hardware environment, distribution, or stable/longterm series. + + * Aim to mainline a fix by Sunday after the next, if the culprit made it + into a recent mainline, stable, or longterm release (either directly or via + backport); if the culprit became known early during a week and is simple to + resolve, try to mainline the fix within the same week. + + * For other regressions, aim to mainline fixes before the hindmost Sunday + within the next three weeks. One or two Sundays later are acceptable, if the + regression is something people can live with easily for a while -- like a + mild performance regression. + + * It's strongly discouraged to delay mainlining regression fixes till the next + merge window, except when the fix is extraordinarily risky or when the + culprit was mainlined more than a year ago. + +On procedure: + + * Always consider reverting the culprit, as it's often the quickest and least + dangerous way to fix a regression. Don't worry about mainlining a fixed + variant later: that should be straight-forward, as most of the code went + through review once already. + + * Try to resolve any regressions introduced in mainline during the past + twelve months before the current development cycle ends: Linus wants such + regressions to be handled like those from the current cycle, unless fixing + bears unusual risks. + + * Consider CCing Linus on discussions or patch review, if a regression seems + tangly. Do the same in precarious or urgent cases -- especially if the + subsystem maintainer might be unavailable. Also CC the stable team, when you + know such a regression made it into a mainline, stable, or longterm release. + + * For urgent regressions, consider asking Linus to pick up the fix straight + from the mailing list: he is totally fine with that for uncontroversial + fixes. Ideally though such requests should happen in accordance with the + subsystem maintainers or come directly from them. + + * In case you are unsure if a fix is worth the risk applying just days before + a new mainline release, send Linus a mail with the usual lists and people in + CC; in it, summarize the situation while asking him to consider picking up + the fix straight from the list. He then himself can make the call and when + needed even postpone the release. Such requests again should ideally happen + in accordance with the subsystem maintainers or come directly from them. + +Regarding stable and longterm kernels: + + * You are free to leave regressions to the stable team, if they at no point in + time occurred with mainline or were fixed there already. + + * If a regression made it into a proper mainline release during the past + twelve months, ensure to tag the fix with "Cc: stable@vger.kernel.org", as a + "Fixes:" tag alone does not guarantee a backport. Please add the same tag, + in case you know the culprit was backported to stable or longterm kernels. + + * When receiving reports about regressions in recent stable or longterm kernel + series, please evaluate at least briefly if the issue might happen in current + mainline as well -- and if that seems likely, take hold of the report. If in + doubt, ask the reporter to check mainline. + + * Whenever you want to swiftly resolve a regression that recently also made it + into a proper mainline, stable, or longterm release, fix it quickly in + mainline; when appropriate thus involve Linus to fast-track the fix (see + above). That's because the stable team normally does neither revert nor fix + any changes that cause the same problems in mainline. + + * In case of urgent regression fixes you might want to ensure prompt + backporting by dropping the stable team a note once the fix was mainlined; + this is especially advisable during merge windows and shortly thereafter, as + the fix otherwise might land at the end of a huge patch queue. + +On patch flow: + + * Developers, when trying to reach the time periods mentioned above, remember + to account for the time it takes to get fixes tested, reviewed, and merged by + Linus, ideally with them being in linux-next at least briefly. Hence, if a + fix is urgent, make it obvious to ensure others handle it appropriately. + + * Reviewers, you are kindly asked to assist developers in reaching the time + periods mentioned above by reviewing regression fixes in a timely manner. + + * Subsystem maintainers, you likewise are encouraged to expedite the handling + of regression fixes. Thus evaluate if skipping linux-next is an option for + the particular fix. Also consider sending git pull requests more often than + usual when needed. And try to avoid holding onto regression fixes over + weekends -- especially when the fix is marked for backporting. + + +More aspects regarding regressions developers should be aware of +---------------------------------------------------------------- + + +How to deal with changes where a risk of regression is known +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Evaluate how big the risk of regressions is, for example by performing a code +search in Linux distributions and Git forges. Also consider asking other +developers or projects likely to be affected to evaluate or even test the +proposed change; if problems surface, maybe some solution acceptable for all +can be found. + +If the risk of regressions in the end seems to be relatively small, go ahead +with the change, but let all involved parties know about the risk. Hence, make +sure your patch description makes this aspect obvious. Once the change is +merged, tell the Linux kernel's regression tracker and the regressions mailing +list about the risk, so everyone has the change on the radar in case reports +trickle in. Depending on the risk, you also might want to ask the subsystem +maintainer to mention the issue in his mainline pull request. + +What else is there to known about regressions? +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Check out Documentation/admin-guide/reporting-regressions.rst, it covers a lot +of other aspects you want might want to be aware of: + + * the purpose of the "no regressions rule" + + * what issues actually qualify as regression + + * who's in charge for finding the root cause of a regression + + * how to handle tricky situations, e.g. when a regression is caused by a + security fix or when fixing a regression might cause another one + +Whom to ask for advice when it comes to regressions +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Send a mail to the regressions mailing list (regressions@lists.linux.dev) while +CCing the Linux kernel's regression tracker (regressions@leemhuis.info); if the +issue might better be dealt with in private, feel free to omit the list. + + +More about regression tracking and regzbot +------------------------------------------ + + +Why the Linux kernel has a regression tracker, and why is regzbot used? +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Rules like "no regressions" need someone to ensure they are followed, otherwise +they are broken either accidentally or on purpose. History has shown this to be +true for the Linux kernel as well. That's why Thorsten Leemhuis volunteered to +keep an eye on things as the Linux kernel's regression tracker, who's +occasionally helped by other people. Neither of them are paid to do this, +that's why regression tracking is done on a best effort basis. + +Earlier attempts to manually track regressions have shown it's an exhausting and +frustrating work, which is why they were abandoned after a while. To prevent +this from happening again, Thorsten developed regzbot to facilitate the work, +with the long term goal to automate regression tracking as much as possible for +everyone involved. + +How does regression tracking work with regzbot? +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +The bot watches for replies to reports of tracked regressions. Additionally, +it's looking out for posted or committed patches referencing such reports +with "Link:" tags; replies to such patch postings are tracked as well. +Combined this data provides good insights into the current state of the fixing +process. + +Regzbot tries to do its job with as little overhead as possible for both +reporters and developers. In fact, only reporters are burdened with an extra +duty: they need to tell regzbot about the regression report using the ``#regzbot +introduced`` command outlined above; if they don't do that, someone else can +take care of that using ``#regzbot ^introduced``. + +For developers there normally is no extra work involved, they just need to make +sure to do something that was expected long before regzbot came to light: add +"Link:" tags to the patch description pointing to all reports about the issue +fixed. + +Do I have to use regzbot? +~~~~~~~~~~~~~~~~~~~~~~~~~ + +It's in the interest of everyone if you do, as kernel maintainers like Linus +Torvalds partly rely on regzbot's tracking in their work -- for example when +deciding to release a new version or extend the development phase. For this they +need to be aware of all unfixed regression; to do that, Linus is known to look +into the weekly reports sent by regzbot. + +Do I have to tell regzbot about every regression I stumble upon? +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Ideally yes: we are all humans and easily forget problems when something more +important unexpectedly comes up -- for example a bigger problem in the Linux +kernel or something in real life that's keeping us away from keyboards for a +while. Hence, it's best to tell regzbot about every regression, except when you +immediately write a fix and commit it to a tree regularly merged to the affected +kernel series. + +How to see which regressions regzbot tracks currently? +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Check `regzbot's web-interface <https://linux-regtracking.leemhuis.info/regzbot/>`_ +for the latest info; alternatively, `search for the latest regression report +<https://lore.kernel.org/lkml/?q=%22Linux+regressions+report%22+f%3Aregzbot>`_, +which regzbot normally sends out once a week on Sunday evening (UTC), which is a +few hours before Linus usually publishes new (pre-)releases. + +What places is regzbot monitoring? +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Regzbot is watching the most important Linux mailing lists as well as the git +repositories of linux-next, mainline, and stable/longterm. + +What kind of issues are supposed to be tracked by regzbot? +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +The bot is meant to track regressions, hence please don't involve regzbot for +regular issues. But it's okay for the Linux kernel's regression tracker if you +use regzbot to track severe issues, like reports about hangs, corrupted data, +or internal errors (Panic, Oops, BUG(), warning, ...). + +Can I add regressions found by CI systems to regzbot's tracking? +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Feel free to do so, if the particular regression likely has impact on practical +use cases and thus might be noticed by users; hence, please don't involve +regzbot for theoretical regressions unlikely to show themselves in real world +usage. + +How to interact with regzbot? +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +By using a 'regzbot command' in a direct or indirect reply to the mail with the +regression report. These commands need to be in their own paragraph (IOW: they +need to be separated from the rest of the mail using blank lines). + +One such command is ``#regzbot introduced <version or commit>``, which makes +regzbot consider your mail as a regressions report added to the tracking, as +already described above; ``#regzbot ^introduced <version or commit>`` is another +such command, which makes regzbot consider the parent mail as a report for a +regression which it starts to track. + +Once one of those two commands has been utilized, other regzbot commands can be +used in direct or indirect replies to the report. You can write them below one +of the `introduced` commands or in replies to the mail that used one of them +or itself is a reply to that mail: + + * Set or update the title:: + + #regzbot title: foo + + * Monitor a discussion or bugzilla.kernel.org ticket where additions aspects of + the issue or a fix are discussed -- for example the posting of a patch fixing + the regression:: + + #regzbot monitor: https://lore.kernel.org/all/30th.anniversary.repost@klaava.Helsinki.FI/ + + Monitoring only works for lore.kernel.org and bugzilla.kernel.org; regzbot + will consider all messages in that thread or ticket as related to the fixing + process. + + * Point to a place with further details of interest, like a mailing list post + or a ticket in a bug tracker that are slightly related, but about a different + topic:: + + #regzbot link: https://bugzilla.kernel.org/show_bug.cgi?id=123456789 + + * Mark a regression as fixed by a commit that is heading upstream or already + landed:: + + #regzbot fixed-by: 1f2e3d4c5d + + * Mark a regression as a duplicate of another one already tracked by regzbot:: + + #regzbot dup-of: https://lore.kernel.org/all/30th.anniversary.repost@klaava.Helsinki.FI/ + + * Mark a regression as invalid:: + + #regzbot invalid: wasn't a regression, problem has always existed + +Is there more to tell about regzbot and its commands? +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +More detailed and up-to-date information about the Linux +kernel's regression tracking bot can be found on its +`project page <https://gitlab.com/knurd42/regzbot>`_, which among others +contains a `getting started guide <https://gitlab.com/knurd42/regzbot/-/blob/main/docs/getting_started.md>`_ +and `reference documentation <https://gitlab.com/knurd42/regzbot/-/blob/main/docs/reference.md>`_ +which both cover more details than the above section. + +Quotes from Linus about regression +---------------------------------- + +Find below a few real life examples of how Linus Torvalds expects regressions to +be handled: + + * From `2017-10-26 (1/2) + <https://lore.kernel.org/lkml/CA+55aFwiiQYJ+YoLKCXjN_beDVfu38mg=Ggg5LFOcqHE8Qi7Zw@mail.gmail.com/>`_:: + + If you break existing user space setups THAT IS A REGRESSION. + + It's not ok to say "but we'll fix the user space setup". + + Really. NOT OK. + + [...] + + The first rule is: + + - we don't cause regressions + + and the corollary is that when regressions *do* occur, we admit to + them and fix them, instead of blaming user space. + + The fact that you have apparently been denying the regression now for + three weeks means that I will revert, and I will stop pulling apparmor + requests until the people involved understand how kernel development + is done. + + * From `2017-10-26 (2/2) + <https://lore.kernel.org/lkml/CA+55aFxW7NMAMvYhkvz1UPbUTUJewRt6Yb51QAx5RtrWOwjebg@mail.gmail.com/>`_:: + + People should basically always feel like they can update their kernel + and simply not have to worry about it. + + I refuse to introduce "you can only update the kernel if you also + update that other program" kind of limitations. If the kernel used to + work for you, the rule is that it continues to work for you. + + There have been exceptions, but they are few and far between, and they + generally have some major and fundamental reasons for having happened, + that were basically entirely unavoidable, and people _tried_hard_ to + avoid them. Maybe we can't practically support the hardware any more + after it is decades old and nobody uses it with modern kernels any + more. Maybe there's a serious security issue with how we did things, + and people actually depended on that fundamentally broken model. Maybe + there was some fundamental other breakage that just _had_ to have a + flag day for very core and fundamental reasons. + + And notice that this is very much about *breaking* peoples environments. + + Behavioral changes happen, and maybe we don't even support some + feature any more. There's a number of fields in /proc/<pid>/stat that + are printed out as zeroes, simply because they don't even *exist* in + the kernel any more, or because showing them was a mistake (typically + an information leak). But the numbers got replaced by zeroes, so that + the code that used to parse the fields still works. The user might not + see everything they used to see, and so behavior is clearly different, + but things still _work_, even if they might no longer show sensitive + (or no longer relevant) information. + + But if something actually breaks, then the change must get fixed or + reverted. And it gets fixed in the *kernel*. Not by saying "well, fix + your user space then". It was a kernel change that exposed the + problem, it needs to be the kernel that corrects for it, because we + have a "upgrade in place" model. We don't have a "upgrade with new + user space". + + And I seriously will refuse to take code from people who do not + understand and honor this very simple rule. + + This rule is also not going to change. + + And yes, I realize that the kernel is "special" in this respect. I'm + proud of it. + + I have seen, and can point to, lots of projects that go "We need to + break that use case in order to make progress" or "you relied on + undocumented behavior, it sucks to be you" or "there's a better way to + do what you want to do, and you have to change to that new better + way", and I simply don't think that's acceptable outside of very early + alpha releases that have experimental users that know what they signed + up for. The kernel hasn't been in that situation for the last two + decades. + + We do API breakage _inside_ the kernel all the time. We will fix + internal problems by saying "you now need to do XYZ", but then it's + about internal kernel API's, and the people who do that then also + obviously have to fix up all the in-kernel users of that API. Nobody + can say "I now broke the API you used, and now _you_ need to fix it + up". Whoever broke something gets to fix it too. + + And we simply do not break user space. + + * From `2020-05-21 + <https://lore.kernel.org/all/CAHk-=wiVi7mSrsMP=fLXQrXK_UimybW=ziLOwSzFTtoXUacWVQ@mail.gmail.com/>`_:: + + The rules about regressions have never been about any kind of + documented behavior, or where the code lives. + + The rules about regressions are always about "breaks user workflow". + + Users are literally the _only_ thing that matters. + + No amount of "you shouldn't have used this" or "that behavior was + undefined, it's your own fault your app broke" or "that used to work + simply because of a kernel bug" is at all relevant. + + Now, reality is never entirely black-and-white. So we've had things + like "serious security issue" etc that just forces us to make changes + that may break user space. But even then the rule is that we don't + really have other options that would allow things to continue. + + And obviously, if users take years to even notice that something + broke, or if we have sane ways to work around the breakage that + doesn't make for too much trouble for users (ie "ok, there are a + handful of users, and they can use a kernel command line to work + around it" kind of things) we've also been a bit less strict. + + But no, "that was documented to be broken" (whether it's because the + code was in staging or because the man-page said something else) is + irrelevant. If staging code is so useful that people end up using it, + that means that it's basically regular kernel code with a flag saying + "please clean this up". + + The other side of the coin is that people who talk about "API + stability" are entirely wrong. API's don't matter either. You can make + any changes to an API you like - as long as nobody notices. + + Again, the regression rule is not about documentation, not about + API's, and not about the phase of the moon. + + It's entirely about "we caused problems for user space that used to work". + + * From `2017-11-05 + <https://lore.kernel.org/all/CA+55aFzUvbGjD8nQ-+3oiMBx14c_6zOj2n7KLN3UsJ-qsd4Dcw@mail.gmail.com/>`_:: + + And our regression rule has never been "behavior doesn't change". + That would mean that we could never make any changes at all. + + For example, we do things like add new error handling etc all the + time, which we then sometimes even add tests for in our kselftest + directory. + + So clearly behavior changes all the time and we don't consider that a + regression per se. + + The rule for a regression for the kernel is that some real user + workflow breaks. Not some test. Not a "look, I used to be able to do + X, now I can't". + + * From `2018-08-03 + <https://lore.kernel.org/all/CA+55aFwWZX=CXmWDTkDGb36kf12XmTehmQjbiMPCqCRG2hi9kw@mail.gmail.com/>`_:: + + YOU ARE MISSING THE #1 KERNEL RULE. + + We do not regress, and we do not regress exactly because your are 100% wrong. + + And the reason you state for your opinion is in fact exactly *WHY* you + are wrong. + + Your "good reasons" are pure and utter garbage. + + The whole point of "we do not regress" is so that people can upgrade + the kernel and never have to worry about it. + + > Kernel had a bug which has been fixed + + That is *ENTIRELY* immaterial. + + Guys, whether something was buggy or not DOES NOT MATTER. + + Why? + + Bugs happen. That's a fact of life. Arguing that "we had to break + something because we were fixing a bug" is completely insane. We fix + tens of bugs every single day, thinking that "fixing a bug" means that + we can break something is simply NOT TRUE. + + So bugs simply aren't even relevant to the discussion. They happen, + they get found, they get fixed, and it has nothing to do with "we + break users". + + Because the only thing that matters IS THE USER. + + How hard is that to understand? + + Anybody who uses "but it was buggy" as an argument is entirely missing + the point. As far as the USER was concerned, it wasn't buggy - it + worked for him/her. + + Maybe it worked *because* the user had taken the bug into account, + maybe it worked because the user didn't notice - again, it doesn't + matter. It worked for the user. + + Breaking a user workflow for a "bug" is absolutely the WORST reason + for breakage you can imagine. + + It's basically saying "I took something that worked, and I broke it, + but now it's better". Do you not see how f*cking insane that statement + is? + + And without users, your program is not a program, it's a pointless + piece of code that you might as well throw away. + + Seriously. This is *why* the #1 rule for kernel development is "we + don't break users". Because "I fixed a bug" is absolutely NOT AN + ARGUMENT if that bug fix broke a user setup. You actually introduced a + MUCH BIGGER bug by "fixing" something that the user clearly didn't + even care about. + + And dammit, we upgrade the kernel ALL THE TIME without upgrading any + other programs at all. It is absolutely required, because flag-days + and dependencies are horribly bad. + + And it is also required simply because I as a kernel developer do not + upgrade random other tools that I don't even care about as I develop + the kernel, and I want any of my users to feel safe doing the same + time. + + So no. Your rule is COMPLETELY wrong. If you cannot upgrade a kernel + without upgrading some other random binary, then we have a problem. + + * From `2021-06-05 + <https://lore.kernel.org/all/CAHk-=wiUVqHN76YUwhkjZzwTdjMMJf_zN4+u7vEJjmEGh3recw@mail.gmail.com/>`_:: + + THERE ARE NO VALID ARGUMENTS FOR REGRESSIONS. + + Honestly, security people need to understand that "not working" is not + a success case of security. It's a failure case. + + Yes, "not working" may be secure. But security in that case is *pointless*. + + * From `2011-05-06 (1/3) + <https://lore.kernel.org/all/BANLkTim9YvResB+PwRp7QTK-a5VNg2PvmQ@mail.gmail.com/>`_:: + + Binary compatibility is more important. + + And if binaries don't use the interface to parse the format (or just + parse it wrongly - see the fairly recent example of adding uuid's to + /proc/self/mountinfo), then it's a regression. + + And regressions get reverted, unless there are security issues or + similar that makes us go "Oh Gods, we really have to break things". + + I don't understand why this simple logic is so hard for some kernel + developers to understand. Reality matters. Your personal wishes matter + NOT AT ALL. + + If you made an interface that can be used without parsing the + interface description, then we're stuck with the interface. Theory + simply doesn't matter. + + You could help fix the tools, and try to avoid the compatibility + issues that way. There aren't that many of them. + + From `2011-05-06 (2/3) + <https://lore.kernel.org/all/BANLkTi=KVXjKR82sqsz4gwjr+E0vtqCmvA@mail.gmail.com/>`_:: + + it's clearly NOT an internal tracepoint. By definition. It's being + used by powertop. + + From `2011-05-06 (3/3) + <https://lore.kernel.org/all/BANLkTinazaXRdGovYL7rRVp+j6HbJ7pzhg@mail.gmail.com/>`_:: + + We have programs that use that ABI and thus it's a regression if they break. + + * From `2012-07-06 <https://lore.kernel.org/all/CA+55aFwnLJ+0sjx92EGREGTWOx84wwKaraSzpTNJwPVV8edw8g@mail.gmail.com/>`_:: + + > Now this got me wondering if Debian _unstable_ actually qualifies as a + > standard distro userspace. + + Oh, if the kernel breaks some standard user space, that counts. Tons + of people run Debian unstable + + * From `2019-09-15 + <https://lore.kernel.org/lkml/CAHk-=wiP4K8DRJWsCo=20hn_6054xBamGKF2kPgUzpB5aMaofA@mail.gmail.com/>`_:: + + One _particularly_ last-minute revert is the top-most commit (ignoring + the version change itself) done just before the release, and while + it's very annoying, it's perhaps also instructive. + + What's instructive about it is that I reverted a commit that wasn't + actually buggy. In fact, it was doing exactly what it set out to do, + and did it very well. In fact it did it _so_ well that the much + improved IO patterns it caused then ended up revealing a user-visible + regression due to a real bug in a completely unrelated area. + + The actual details of that regression are not the reason I point that + revert out as instructive, though. It's more that it's an instructive + example of what counts as a regression, and what the whole "no + regressions" kernel rule means. The reverted commit didn't change any + API's, and it didn't introduce any new bugs. But it ended up exposing + another problem, and as such caused a kernel upgrade to fail for a + user. So it got reverted. + + The point here being that we revert based on user-reported _behavior_, + not based on some "it changes the ABI" or "it caused a bug" concept. + The problem was really pre-existing, and it just didn't happen to + trigger before. The better IO patterns introduced by the change just + happened to expose an old bug, and people had grown to depend on the + previously benign behavior of that old issue. + + And never fear, we'll re-introduce the fix that improved on the IO + patterns once we've decided just how to handle the fact that we had a + bad interaction with an interface that people had then just happened + to rely on incidental behavior for before. It's just that we'll have + to hash through how to do that (there are no less than three different + patches by three different developers being discussed, and there might + be more coming...). In the meantime, I reverted the thing that exposed + the problem to users for this release, even if I hope it will be + re-introduced (perhaps even backported as a stable patch) once we have + consensus about the issue it exposed. + + Take-away from the whole thing: it's not about whether you change the + kernel-userspace ABI, or fix a bug, or about whether the old code + "should never have worked in the first place". It's about whether + something breaks existing users' workflow. + + Anyway, that was my little aside on the whole regression thing. Since + it's that "first rule of kernel programming", I felt it is perhaps + worth just bringing it up every once in a while + +.. + end-of-content +.. + This text is available under GPL-2.0+ or CC-BY-4.0, as stated at the top + of the file. If you want to distribute this text under CC-BY-4.0 only, + please use "The Linux kernel developers" for author attribution and link + this as source: + https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/plain/Documentation/process/handling-regressions.rst +.. + Note: Only the content of this RST file as found in the Linux kernel sources + is available under CC-BY-4.0, as versions of this text that were processed + (for example by the kernel's build system) might contain content taken from + files which use a more restrictive license. diff --git a/Documentation/process/howto.rst b/Documentation/process/howto.rst index b6f5a379ad6c..6c73889c98fc 100644 --- a/Documentation/process/howto.rst +++ b/Documentation/process/howto.rst @@ -36,7 +36,7 @@ experience, the following books are good for, if anything, reference: - "C: A Reference Manual" by Harbison and Steele [Prentice Hall] The kernel is written using GNU C and the GNU toolchain. While it -adheres to the ISO C89 standard, it uses a number of extensions that are +adheres to the ISO C11 standard, it uses a number of extensions that are not featured in the standard. The kernel is a freestanding C environment, with no reliance on the standard C library, so some portions of the C standard are not supported. Arbitrary long long @@ -82,8 +82,7 @@ documentation files are also added which explain how to use the feature. When a kernel change causes the interface that the kernel exposes to userspace to change, it is recommended that you send the information or a patch to the manual pages explaining the change to the manual pages -maintainer at mtk.manpages@gmail.com, and CC the list -linux-api@vger.kernel.org. +maintainer at alx@kernel.org, and CC the list linux-api@vger.kernel.org. Here is a list of files that are in the kernel source tree that are required reading: @@ -105,8 +104,8 @@ required reading: patches if these rules are followed, and many people will only review code if it is in the proper style. - :ref:`Documentation/process/submitting-patches.rst <submittingpatches>` and :ref:`Documentation/process/submitting-drivers.rst <submittingdrivers>` - These files describe in explicit detail how to successfully create + :ref:`Documentation/process/submitting-patches.rst <submittingpatches>` + This file describes in explicit detail how to successfully create and send a patch, including (but not limited to): - Email contents @@ -138,7 +137,7 @@ required reading: philosophy and is very important for people moving to Linux from development on other Operating Systems. - :ref:`Documentation/admin-guide/security-bugs.rst <securitybugs>` + :ref:`Documentation/process/security-bugs.rst <securitybugs>` If you feel you have found a security problem in the Linux kernel, please follow the steps in this document to help notify the kernel developers, and help solve the issue. @@ -243,10 +242,10 @@ branches. These different branches are: Mainline tree ~~~~~~~~~~~~~ -Mainline tree are maintained by Linus Torvalds, and can be found at +The mainline tree is maintained by Linus Torvalds, and can be found at https://kernel.org or in the repo. Its development process is as follows: - - As soon as a new kernel is released a two weeks window is open, + - As soon as a new kernel is released a two week window is open, during this period of time maintainers can submit big diffs to Linus, usually the patches that have already been included in the linux-next for a few weeks. The preferred way to submit big changes @@ -281,8 +280,9 @@ Various stable trees with multiple major numbers Kernels with 3-part versions are -stable kernels. They contain relatively small and critical fixes for security problems or significant -regressions discovered in a given major mainline release, with the first -2-part of version number are the same correspondingly. +regressions discovered in a given major mainline release. Each release +in a major stable series increments the third part of the version +number, keeping the first two parts the same. This is the recommended branch for users who want the most recent stable kernel and are not interested in helping test development/experimental @@ -341,17 +341,10 @@ Adventurous testers are very welcome to runtime-test the linux-next. Bug Reporting ------------- -https://bugzilla.kernel.org is where the Linux kernel developers track kernel -bugs. Users are encouraged to report all bugs that they find in this -tool. For details on how to use the kernel bugzilla, please see: - - https://bugzilla.kernel.org/page.cgi?id=faq.html - -The file :ref:`admin-guide/reporting-bugs.rst <reportingbugs>` -in the main kernel source directory has a good -template for how to report a possible kernel bug, and details what kind -of information is needed by the kernel developers to help track down the -problem. +The file 'Documentation/admin-guide/reporting-issues.rst' in the main kernel +source directory describes how to report a possible kernel bug, and details +what kind of information is needed by the kernel developers to help track +down the problem. Managing bug reports @@ -359,12 +352,18 @@ Managing bug reports One of the best ways to put into practice your hacking skills is by fixing bugs reported by other people. Not only you will help to make the kernel -more stable, you'll learn to fix real world problems and you will improve -your skills, and other developers will be aware of your presence. Fixing -bugs is one of the best ways to get merits among other developers, because -not many people like wasting time fixing other people's bugs. +more stable, but you'll also learn to fix real world problems and you will +improve your skills, and other developers will be aware of your presence. +Fixing bugs is one of the best ways to get merits among other developers, +because not many people like wasting time fixing other people's bugs. -To work in the already reported bug reports, go to https://bugzilla.kernel.org. +To work on already reported bug reports, find a subsystem you are interested in. +Check the MAINTAINERS file where bugs for that subsystem get reported to; often +it will be a mailing list, rarely a bugtracker. Search the archives of said +place for recent reports and help where you see fit. You may also want to check +https://bugzilla.kernel.org for bug reports; only a handful of kernel subsystems +use it actively for reporting or tracking, nevertheless bugs for the whole +kernel get filed there. Mailing lists @@ -379,7 +378,7 @@ to subscribe and unsubscribe from the list can be found at: There are archives of the mailing list on the web in many different places. Use a search engine to find these archives. For example: - http://dir.gmane.org/gmane.linux.kernel + https://lore.kernel.org/lkml/ It is highly recommended that you search the archives about the topic you want to bring up, before you post it to the list. A lot of things @@ -596,7 +595,7 @@ For more details on what this should all look like, please see the ChangeLog section of the document: "The Perfect Patch" - http://www.ozlabs.org/~akpm/stuff/tpp.txt + https://www.ozlabs.org/~akpm/stuff/tpp.txt All of these things are sometimes very hard to do. It can take years to diff --git a/Documentation/process/index.rst b/Documentation/process/index.rst index 21aa7d5358e6..6cb732dfcc72 100644 --- a/Documentation/process/index.rst +++ b/Documentation/process/index.rst @@ -5,6 +5,7 @@ .. _process_index: +============================================= Working with the kernel development community ============================================= @@ -14,52 +15,107 @@ to learn about how our community works. Reading these documents will make it much easier for you to get your changes merged with a minimum of trouble. -Below are the essential guides that every developer should read. +An introduction to how kernel development works +----------------------------------------------- + +Read these documents first: an understanding of the material here will ease +your entry into the kernel community. .. toctree:: :maxdepth: 1 - license-rules howto - code-of-conduct - code-of-conduct-interpretation development-process submitting-patches + submit-checklist + +Tools and technical guides for kernel developers +------------------------------------------------ + +This is a collection of material that kernel developers should be familiar +with. + +.. toctree:: + :maxdepth: 1 + + changes programming-language coding-style maintainer-pgp-guide email-clients - kernel-enforcement-statement - kernel-driver-statement + applying-patches + backporting + adding-syscalls + volatile-considered-harmful + botching-up-ioctls + +Policy guides and developer statements +-------------------------------------- -Other guides to the community that are of interest to most developers are: +These are the rules that we try to live by in the kernel community (and +beyond). .. toctree:: :maxdepth: 1 - changes - submitting-drivers + license-rules + code-of-conduct + code-of-conduct-interpretation + contribution-maturity-model + kernel-enforcement-statement + kernel-driver-statement stable-api-nonsense - management-style stable-kernel-rules - submit-checklist - kernel-docs - deprecated + management-style + researcher-guidelines + +Dealing with bugs +----------------- + +Bugs are a fact of life; it is important that we handle them properly. +The documents below describe our policies around the handling of a couple +of special classes of bugs: regressions and security problems. + +.. toctree:: + :maxdepth: 1 + + handling-regressions + security-bugs embargoed-hardware-issues + +Maintainer information +---------------------- + +How to find the people who will accept your patches. + +.. toctree:: + :maxdepth: 1 + + maintainer-handbooks maintainers +Other material +-------------- + +Here are some other guides to the community that are of interest to most +developers: + +.. toctree:: + :maxdepth: 1 + + kernel-docs + deprecated + These are some overall technical guides that have been put here for now for lack of a better place. .. toctree:: :maxdepth: 1 - applying-patches - adding-syscalls magic-number - volatile-considered-harmful - botching-up-ioctls clang-format + ../arch/riscv/patch-acceptance + ../core-api/unaligned-memory-access .. only:: subproject and html diff --git a/Documentation/process/kernel-docs.rst b/Documentation/process/kernel-docs.rst index 7a45a8e36ea7..8660493b91d0 100644 --- a/Documentation/process/kernel-docs.rst +++ b/Documentation/process/kernel-docs.rst @@ -1,9 +1,7 @@ .. _kernel_docs: -Index of Documentation for People Interested in Writing and/or Understanding the Linux Kernel -============================================================================================= - - Juan-Mariano de Goyeneche <jmseyas@dit.upm.es> +Index of Further Kernel Documentation +===================================== The need for a document like this one became apparent in the linux-kernel mailing list as the same questions, asking for pointers @@ -16,26 +14,23 @@ philosophy and design decisions behind this code. Unfortunately, not many documents are available for beginners to start. And, even if they exist, there was no "well-known" place which -kept track of them. These lines try to cover this lack. All documents -available on line known by the author are listed, while some reference -books are also mentioned. +kept track of them. These lines try to cover this lack. PLEASE, if you know any paper not listed here or write a new document, -send me an e-mail, and I'll include a reference to it here. Any -corrections, ideas or comments are also welcomed. - -The papers that follow are listed in no particular order. All are -cataloged with the following fields: the document's "Title", the -"Author"/s, the "URL" where they can be found, some "Keywords" helpful -when searching for specific topics, and a brief "Description" of the -Document. +include a reference to it here, following the kernel's patch submission +process. Any corrections, ideas or comments are also welcome. -Enjoy! +All documents are cataloged with the following fields: the document's +"Title", the "Author"/s, the "URL" where they can be found, some +"Keywords" helpful when searching for specific topics, and a brief +"Description" of the Document. .. note:: The documents on each section of this document are ordered by its - published date, from the newest to the oldest. + published date, from the newest to the oldest. The maintainer(s) should + periodically retire resources as they become obsolete or outdated; with + the exception of foundational books. Docs at the Linux Kernel tree ----------------------------- @@ -65,393 +60,54 @@ On-line docs a brief description of some of the acronyms and terms you may hear during discussion of the Linux kernel". - * Title: **Tracing the Way of Data in a TCP Connection through the Linux Kernel** - - :Author: Richard Sailer - :URL: https://archive.org/details/linux_kernel_data_flow_short_paper - :Date: 2016 - :Keywords: Linux Kernel Networking, TCP, tracing, ftrace - :Description: A seminar paper explaining ftrace and how to use it for - understanding linux kernel internals, - illustrated at tracing the way of a TCP packet through the kernel. - :Abstract: *This short paper outlines the usage of ftrace a tracing framework - as a tool to understand a running Linux system. - Having obtained a trace-log a kernel hacker can read and understand - source code more determined and with context. - In a detailed example this approach is demonstrated in tracing - and the way of data in a TCP Connection through the kernel. - Finally this trace-log is used as base for more a exact conceptual - exploration and description of the Linux TCP/IP implementation.* - - * Title: **On submitting kernel Patches** - - :Author: Andi Kleen - :URL: http://halobates.de/on-submitting-kernel-patches.pdf - :Date: 2008 - :Keywords: patches, review process, types of submissions, basic rules, case studies - :Description: This paper gives several experience values on what types of patches - there are and how likley they get merged. - :Abstract: - [...]. This paper examines some common problems for - submitting larger changes and some strategies to avoid problems. - - * Title: **Linux Device Drivers, Third Edition** - - :Author: Jonathan Corbet, Alessandro Rubini, Greg Kroah-Hartman - :URL: http://lwn.net/Kernel/LDD3/ - :Date: 2005 - :Description: A 600-page book covering the (2.6.10) driver - programming API and kernel hacking in general. Available under the - Creative Commons Attribution-ShareAlike 2.0 license. - :note: You can also :ref:`purchase a copy from O'Reilly or elsewhere <ldd3_published>`. - - * Title: **Writing an ALSA Driver** + * Title: **The Linux Kernel Module Programming Guide** - :Author: Takashi Iwai <tiwai@suse.de> - :URL: http://www.alsa-project.org/~iwai/writing-an-alsa-driver/index.html - :Date: 2005 - :Keywords: ALSA, sound, soundcard, driver, lowlevel, hardware. - :Description: Advanced Linux Sound Architecture for developers, - both at kernel and user-level sides. ALSA is the Linux kernel - sound architecture in the 2.6 kernel version. - - * Title: **Linux PCMCIA Programmer's Guide** - - :Author: David Hinds. - :URL: http://pcmcia-cs.sourceforge.net/ftp/doc/PCMCIA-PROG.html - :Date: 2003 - :Keywords: PCMCIA. - :Description: "This document describes how to write kernel device - drivers for the Linux PCMCIA Card Services interface. It also - describes how to write user-mode utilities for communicating with - Card Services. - - * Title: **Linux Kernel Module Programming Guide** - - :Author: Ori Pomerantz. - :URL: http://tldp.org/LDP/lkmpg/2.6/html/index.html - :Date: 2001 + :Author: Peter Jay Salzman, Michael Burian, Ori Pomerantz, Bob Mottram, + Jim Huang. + :URL: https://sysprog21.github.io/lkmpg/ + :Date: 2021 :Keywords: modules, GPL book, /proc, ioctls, system calls, interrupt handlers . - :Description: Very nice 92 pages GPL book on the topic of modules - programming. Lots of examples. - - * Title: **Global spinlock list and usage** - - :Author: Rick Lindsley. - :URL: http://lse.sourceforge.net/lockhier/global-spin-lock - :Date: 2001 - :Keywords: spinlock. - :Description: This is an attempt to document both the existence and - usage of the spinlocks in the Linux 2.4.5 kernel. Comprehensive - list of spinlocks showing when they are used, which functions - access them, how each lock is acquired, under what conditions it - is held, whether interrupts can occur or not while it is held... - - * Title: **A Linux vm README** - - :Author: Kanoj Sarcar. - :URL: http://kos.enix.org/pub/linux-vmm.html - :Date: 2001 - :Keywords: virtual memory, mm, pgd, vma, page, page flags, page - cache, swap cache, kswapd. - :Description: Telegraphic, short descriptions and definitions - relating the Linux virtual memory implementation. - - * Title: **Video4linux Drivers, Part 1: Video-Capture Device** - - :Author: Alan Cox. - :URL: http://www.linux-mag.com/id/406 - :Date: 2000 - :Keywords: video4linux, driver, video capture, capture devices, - camera driver. - :Description: The title says it all. - - * Title: **Video4linux Drivers, Part 2: Video-capture Devices** - - :Author: Alan Cox. - :URL: http://www.linux-mag.com/id/429 - :Date: 2000 - :Keywords: video4linux, driver, video capture, capture devices, - camera driver, control, query capabilities, capability, facility. - :Description: The title says it all. - - * Title: **Linux IP Networking. A Guide to the Implementation and Modification of the Linux Protocol Stack.** - - :Author: Glenn Herrin. - :URL: http://www.cs.unh.edu/cnrg/gherrin - :Date: 2000 - :Keywords: network, networking, protocol, IP, UDP, TCP, connection, - socket, receiving, transmitting, forwarding, routing, packets, - modules, /proc, sk_buff, FIB, tags. - :Description: Excellent paper devoted to the Linux IP Networking, - explaining anything from the kernel's to the user space - configuration tools' code. Very good to get a general overview of - the kernel networking implementation and understand all steps - packets follow from the time they are received at the network - device till they are delivered to applications. The studied kernel - code is from 2.2.14 version. Provides code for a working packet - dropper example. - - * Title: **How To Make Sure Your Driver Will Work On The Power Macintosh** - - :Author: Paul Mackerras. - :URL: http://www.linux-mag.com/id/261 - :Date: 1999 - :Keywords: Mac, Power Macintosh, porting, drivers, compatibility. - :Description: The title says it all. - - * Title: **An Introduction to SCSI Drivers** - - :Author: Alan Cox. - :URL: http://www.linux-mag.com/id/284 - :Date: 1999 - :Keywords: SCSI, device, driver. - :Description: The title says it all. - - * Title: **Advanced SCSI Drivers And Other Tales** - - :Author: Alan Cox. - :URL: http://www.linux-mag.com/id/307 - :Date: 1999 - :Keywords: SCSI, device, driver, advanced. - :Description: The title says it all. - - * Title: **Writing Linux Mouse Drivers** - - :Author: Alan Cox. - :URL: http://www.linux-mag.com/id/330 - :Date: 1999 - :Keywords: mouse, driver, gpm. - :Description: The title says it all. - - * Title: **More on Mouse Drivers** - - :Author: Alan Cox. - :URL: http://www.linux-mag.com/id/356 - :Date: 1999 - :Keywords: mouse, driver, gpm, races, asynchronous I/O. - :Description: The title still says it all. - - * Title: **Writing Video4linux Radio Driver** - - :Author: Alan Cox. - :URL: http://www.linux-mag.com/id/381 - :Date: 1999 - :Keywords: video4linux, driver, radio, radio devices. - :Description: The title says it all. - - * Title: **I/O Event Handling Under Linux** - - :Author: Richard Gooch. - :URL: http://web.mit.edu/~yandros/doc/io-events.html - :Date: 1999 - :Keywords: IO, I/O, select(2), poll(2), FDs, aio_read(2), readiness - event queues. - :Description: From the Introduction: "I/O Event handling is about - how your Operating System allows you to manage a large number of - open files (file descriptors in UNIX/POSIX, or FDs) in your - application. You want the OS to notify you when FDs become active - (have data ready to be read or are ready for writing). Ideally you - want a mechanism that is scalable. This means a large number of - inactive FDs cost very little in memory and CPU time to manage". - - * Title: **(nearly) Complete Linux Loadable Kernel Modules. The definitive guide for hackers, virus coders and system administrators.** - - :Author: pragmatic/THC. - :URL: http://packetstormsecurity.org/docs/hack/LKM_HACKING.html - :Date: 1999 - :Keywords: syscalls, intercept, hide, abuse, symbol table. - :Description: Interesting paper on how to abuse the Linux kernel in - order to intercept and modify syscalls, make - files/directories/processes invisible, become root, hijack ttys, - write kernel modules based virus... and solutions for admins to - avoid all those abuses. - :Notes: For 2.0.x kernels. Gives guidances to port it to 2.2.x - kernels. - - * Name: **Linux Virtual File System** - - :Author: Peter J. Braam. - :URL: http://www.coda.cs.cmu.edu/doc/talks/linuxvfs/ - :Date: 1998 - :Keywords: slides, VFS, inode, superblock, dentry, dcache. - :Description: Set of slides, presumably from a presentation on the - Linux VFS layer. Covers version 2.1.x, with dentries and the - dcache. - - * Title: **The Venus kernel interface** - - :Author: Peter J. Braam. - :URL: http://www.coda.cs.cmu.edu/doc/html/kernel-venus-protocol.html - :Date: 1998 - :Keywords: coda, filesystem, venus, cache manager. - :Description: "This document describes the communication between - Venus and kernel level file system code needed for the operation - of the Coda filesystem. This version document is meant to describe - the current interface (version 1.0) as well as improvements we - envisage". - - * Title: **Design and Implementation of the Second Extended Filesystem** - - :Author: Rémy Card, Theodore Ts'o, Stephen Tweedie. - :URL: http://web.mit.edu/tytso/www/linux/ext2intro.html - :Date: 1998 - :Keywords: ext2, linux fs history, inode, directory, link, devices, - VFS, physical structure, performance, benchmarks, ext2fs library, - ext2fs tools, e2fsck. - :Description: Paper written by three of the top ext2 hackers. - Covers Linux filesystems history, ext2 motivation, ext2 features, - design, physical structure on disk, performance, benchmarks, - e2fsck's passes description... A must read! - :Notes: This paper was first published in the Proceedings of the - First Dutch International Symposium on Linux, ISBN 90-367-0385-9. - - * Title: **The Linux RAID-1, 4, 5 Code** - - :Author: Ingo Molnar, Gadi Oxman and Miguel de Icaza. - :URL: http://www.linuxjournal.com/article.php?sid=2391 - :Date: 1997 - :Keywords: RAID, MD driver. - :Description: Linux Journal Kernel Korner article. Here is its - :Abstract: *A description of the implementation of the RAID-1, - RAID-4 and RAID-5 personalities of the MD device driver in the - Linux kernel, providing users with high performance and reliable, - secondary-storage capability using software*. - - * Title: **Linux Kernel Hackers' Guide** - - :Author: Michael K. Johnson. - :URL: http://www.tldp.org/LDP/khg/HyperNews/get/khg.html - :Date: 1997 - :Keywords: device drivers, files, VFS, kernel interface, character vs - block devices, hardware interrupts, scsi, DMA, access to user memory, - memory allocation, timers. - :Description: A guide designed to help you get up to speed on the - concepts that are not intuitevly obvious, and to document the internal - structures of Linux. - - * Title: **Dynamic Kernels: Modularized Device Drivers** - - :Author: Alessandro Rubini. - :URL: http://www.linuxjournal.com/article.php?sid=1219 - :Date: 1996 - :Keywords: device driver, module, loading/unloading modules, - allocating resources. - :Description: Linux Journal Kernel Korner article. Here is its - :Abstract: *This is the first of a series of four articles - co-authored by Alessandro Rubini and Georg Zezchwitz which present - a practical approach to writing Linux device drivers as kernel - loadable modules. This installment presents an introduction to the - topic, preparing the reader to understand next month's - installment*. - - * Title: **Dynamic Kernels: Discovery** - - :Author: Alessandro Rubini. - :URL: http://www.linuxjournal.com/article.php?sid=1220 - :Date: 1996 - :Keywords: character driver, init_module, clean_up module, - autodetection, mayor number, minor number, file operations, - open(), close(). - :Description: Linux Journal Kernel Korner article. Here is its - :Abstract: *This article, the second of four, introduces part of - the actual code to create custom module implementing a character - device driver. It describes the code for module initialization and - cleanup, as well as the open() and close() system calls*. - - * Title: **The Devil's in the Details** - - :Author: Georg v. Zezschwitz and Alessandro Rubini. - :URL: http://www.linuxjournal.com/article.php?sid=1221 - :Date: 1996 - :Keywords: read(), write(), select(), ioctl(), blocking/non - blocking mode, interrupt handler. - :Description: Linux Journal Kernel Korner article. Here is its - :Abstract: *This article, the third of four on writing character - device drivers, introduces concepts of reading, writing, and using - ioctl-calls*. - - * Title: **Dissecting Interrupts and Browsing DMA** - - :Author: Alessandro Rubini and Georg v. Zezschwitz. - :URL: http://www.linuxjournal.com/article.php?sid=1222 - :Date: 1996 - :Keywords: interrupts, irqs, DMA, bottom halves, task queues. - :Description: Linux Journal Kernel Korner article. Here is its - :Abstract: *This is the fourth in a series of articles about - writing character device drivers as loadable kernel modules. This - month, we further investigate the field of interrupt handling. - Though it is conceptually simple, practical limitations and - constraints make this an ''interesting'' part of device driver - writing, and several different facilities have been provided for - different situations. We also investigate the complex topic of - DMA*. - - * Title: **Device Drivers Concluded** - - :Author: Georg v. Zezschwitz. - :URL: http://www.linuxjournal.com/article.php?sid=1287 - :Date: 1996 - :Keywords: address spaces, pages, pagination, page management, - demand loading, swapping, memory protection, memory mapping, mmap, - virtual memory areas (VMAs), vremap, PCI. - :Description: Finally, the above turned out into a five articles - series. This latest one's introduction reads: "This is the last of - five articles about character device drivers. In this final - section, Georg deals with memory mapping devices, beginning with - an overall description of the Linux memory management concepts". - - * Title: **Network Buffers And Memory Management** - - :Author: Alan Cox. - :URL: http://www.linuxjournal.com/article.php?sid=1312 - :Date: 1996 - :Keywords: sk_buffs, network devices, protocol/link layer - variables, network devices flags, transmit, receive, - configuration, multicast. - :Description: Linux Journal Kernel Korner. - :Abstract: *Writing a network device driver for Linux is fundamentally - simple---most of the complexity (other than talking to the - hardware) involves managing network packets in memory*. - - * Title: **Analysis of the Ext2fs structure** - - :Author: Louis-Dominique Dubeau. - :URL: http://teaching.csse.uwa.edu.au/units/CITS2002/fs-ext2/ - :Date: 1994 - :Keywords: ext2, filesystem, ext2fs. - :Description: Description of ext2's blocks, directories, inodes, - bitmaps, invariants... + :Description: A very nice GPL book on the topic of modules + programming. Lots of examples. Currently the new version is being + actively maintained at https://github.com/sysprog21/lkmpg. Published books --------------- - * Title: **Linux Treiber entwickeln** + * Title: **Linux Kernel Debugging: Leverage proven tools and advanced techniques to effectively debug Linux kernels and kernel modules** + + :Author: Kaiwan N Billimoria + :Publisher: Packt Publishing Ltd + :Date: August, 2022 + :Pages: 638 + :ISBN: 978-1801075039 + :Notes: Debugging book + + * Title: **Linux Kernel Programming: A Comprehensive Guide to Kernel Internals, Writing Kernel Modules, and Kernel Synchronization** - :Author: Jürgen Quade, Eva-Katharina Kunst - :Publisher: dpunkt.verlag - :Date: Oct 2015 (4th edition) - :Pages: 688 - :ISBN: 978-3-86490-288-8 - :Note: German. The third edition from 2011 is - much cheaper and still quite up-to-date. + :Author: Kaiwan N Billimoria + :Publisher: Packt Publishing Ltd + :Date: March, 2021 + :Pages: 754 + :ISBN: 978-1789953435 - * Title: **Linux Kernel Networking: Implementation and Theory** + * Title: **Linux Kernel Programming Part 2 - Char Device Drivers and Kernel Synchronization: Create user-kernel interfaces, work with peripheral I/O, and handle hardware interrupts** - :Author: Rami Rosen - :Publisher: Apress - :Date: December 22, 2013 - :Pages: 648 - :ISBN: 978-1430261964 + :Author: Kaiwan N Billimoria + :Publisher: Packt Publishing Ltd + :Date: March, 2021 + :Pages: 452 + :ISBN: 978-1801079518 - * Title: **Embedded Linux Primer: A practical Real-World Approach, 2nd Edition** + * Title: **Linux System Programming: Talking Directly to the Kernel and C Library** - :Author: Christopher Hallinan - :Publisher: Pearson - :Date: November, 2010 - :Pages: 656 - :ISBN: 978-0137017836 + :Author: Robert Love + :Publisher: O'Reilly Media + :Date: June, 2013 + :Pages: 456 + :ISBN: 978-1449339531 + :Notes: Foundational book * Title: **Linux Kernel Development, 3rd Edition** @@ -460,14 +116,16 @@ Published books :Date: July, 2010 :Pages: 440 :ISBN: 978-0672329463 + :Notes: Foundational book - * Title: **Essential Linux Device Drivers** + * Title: **Practical Linux System Administration: A Guide to Installation, Configuration, and Management, 1st Edition** - :Author: Sreekrishnan Venkateswaran - :Published: Prentice Hall - :Date: April, 2008 - :Pages: 744 - :ISBN: 978-0132396554 + :Author: Kenneth Hess + :Publisher: O'Reilly Media + :Date: May, 2023 + :Pages: 246 + :ISBN: 978-1098109035 + :Notes: System administration .. _ldd3_published: @@ -478,67 +136,9 @@ Published books :Date: 2005 :Pages: 636 :ISBN: 0-596-00590-3 - :Notes: Further information in + :Notes: Foundational book. Further information in http://www.oreilly.com/catalog/linuxdrive3/ - PDF format, URL: http://lwn.net/Kernel/LDD3/ - - * Title: **Linux Kernel Internals** - - :Author: Michael Beck - :Publisher: Addison-Wesley - :Date: 1997 - :ISBN: 0-201-33143-8 (second edition) - - * Title: **Programmation Linux 2.0 API systeme et fonctionnement du noyau** - - :Author: Remy Card, Eric Dumas, Franck Mevel - :Publisher: Eyrolles - :Date: 1997 - :Pages: 520 - :ISBN: 2-212-08932-5 - :Notes: French - - * Title: **The Design and Implementation of the 4.4 BSD UNIX Operating System** - - :Author: Marshall Kirk McKusick, Keith Bostic, Michael J. Karels, - John S. Quarterman - :Publisher: Addison-Wesley - :Date: 1996 - :ISBN: 0-201-54979-4 - - * Title: **Unix internals -- the new frontiers** - - :Author: Uresh Vahalia - :Publisher: Prentice Hall - :Date: 1996 - :Pages: 600 - :ISBN: 0-13-101908-2 - - * Title: **Programming for the real world - POSIX.4** - - :Author: Bill O. Gallmeister - :Publisher: O'Reilly & Associates, Inc - :Date: 1995 - :Pages: 552 - :ISBN: I-56592-074-0 - :Notes: Though not being directly about Linux, Linux aims to be - POSIX. Good reference. - - * Title: **UNIX Systems for Modern Architectures: Symmetric Multiprocessing and Caching for Kernel Programmers** - - :Author: Curt Schimmel - :Publisher: Addison Wesley - :Date: June, 1994 - :Pages: 432 - :ISBN: 0-201-63338-8 - - * Title: **The Design and Implementation of the 4.3 BSD UNIX Operating System** - - :Author: Samuel J. Leffler, Marshall Kirk McKusick, Michael J - Karels, John S. Quarterman - :Publisher: Addison-Wesley - :Date: 1989 (reprinted with corrections on October, 1990) - :ISBN: 0-201-06196-1 + PDF format, URL: https://lwn.net/Kernel/LDD3/ * Title: **The Design of the UNIX Operating System** @@ -547,6 +147,7 @@ Published books :Date: 1986 :Pages: 471 :ISBN: 0-13-201757-1 + :Notes: Foundational book Miscellaneous ------------- @@ -561,16 +162,16 @@ Miscellaneous * Name: **Linux Weekly News** - :URL: http://lwn.net + :URL: https://lwn.net :Keywords: latest kernel news. :Description: The title says it all. There's a fixed kernel section summarizing developers' work, bug fixes, new features and versions - produced during the week. Published every Thursday. + produced during the week. * Name: **The home page of Linux-MM** :Author: The Linux-MM team. - :URL: http://linux-mm.org/ + :URL: https://linux-mm.org/ :Keywords: memory management, Linux-MM, mm patches, TODO, docs, mailing list. :Description: Site devoted to Linux Memory Management development. @@ -579,7 +180,7 @@ Miscellaneous * Name: **Kernel Newbies IRC Channel and Website** - :URL: http://www.kernelnewbies.org + :URL: https://www.kernelnewbies.org :Keywords: IRC, newbies, channel, asking doubts. :Description: #kernelnewbies on irc.oftc.net. #kernelnewbies is an IRC network dedicated to the 'newbie' @@ -602,7 +203,8 @@ Miscellaneous ------- -Document last updated on Tue 2016-Sep-20 +This document was originally based on: + + https://www.dit.upm.es/~jmseyas/linux/kernel/hackers-docs.html -This document is based on: - http://www.dit.upm.es/~jmseyas/linux/kernel/hackers-docs.html +and written by Juan-Mariano de Goyeneche diff --git a/Documentation/process/kernel-enforcement-statement.rst b/Documentation/process/kernel-enforcement-statement.rst index e5a1be476047..dc2d813b2e79 100644 --- a/Documentation/process/kernel-enforcement-statement.rst +++ b/Documentation/process/kernel-enforcement-statement.rst @@ -1,4 +1,4 @@ -.. _process_statement_kernel: +.. _process_statement_kernel: Linux Kernel Enforcement Statement ---------------------------------- diff --git a/Documentation/process/magic-number.rst b/Documentation/process/magic-number.rst index eee9b44553b3..7029c3c084ee 100644 --- a/Documentation/process/magic-number.rst +++ b/Documentation/process/magic-number.rst @@ -69,94 +69,16 @@ Changelog:: Magic Name Number Structure File ===================== ================ ======================== ========================================== PG_MAGIC 'P' pg_{read,write}_hdr ``include/linux/pg.h`` -CMAGIC 0x0111 user ``include/linux/a.out.h`` -MKISS_DRIVER_MAGIC 0x04bf mkiss_channel ``drivers/net/mkiss.h`` -HDLC_MAGIC 0x239e n_hdlc ``drivers/char/n_hdlc.c`` APM_BIOS_MAGIC 0x4101 apm_user ``arch/x86/kernel/apm_32.c`` -CYCLADES_MAGIC 0x4359 cyclades_port ``include/linux/cyclades.h`` -DB_MAGIC 0x4442 fc_info ``drivers/net/iph5526_novram.c`` -DL_MAGIC 0x444d fc_info ``drivers/net/iph5526_novram.c`` FASYNC_MAGIC 0x4601 fasync_struct ``include/linux/fs.h`` -FF_MAGIC 0x4646 fc_info ``drivers/net/iph5526_novram.c`` -ISICOM_MAGIC 0x4d54 isi_port ``include/linux/isicom.h`` -PTY_MAGIC 0x5001 ``drivers/char/pty.c`` -PPP_MAGIC 0x5002 ppp ``include/linux/if_pppvar.h`` -SSTATE_MAGIC 0x5302 serial_state ``include/linux/serial.h`` SLIP_MAGIC 0x5302 slip ``drivers/net/slip.h`` -STRIP_MAGIC 0x5303 strip ``drivers/net/strip.c`` -X25_ASY_MAGIC 0x5303 x25_asy ``drivers/net/x25_asy.h`` -SIXPACK_MAGIC 0x5304 sixpack ``drivers/net/hamradio/6pack.h`` -AX25_MAGIC 0x5316 ax_disp ``drivers/net/mkiss.h`` -TTY_MAGIC 0x5401 tty_struct ``include/linux/tty.h`` -MGSL_MAGIC 0x5401 mgsl_info ``drivers/char/synclink.c`` -TTY_DRIVER_MAGIC 0x5402 tty_driver ``include/linux/tty_driver.h`` -MGSLPC_MAGIC 0x5402 mgslpc_info ``drivers/char/pcmcia/synclink_cs.c`` -TTY_LDISC_MAGIC 0x5403 tty_ldisc ``include/linux/tty_ldisc.h`` -USB_SERIAL_MAGIC 0x6702 usb_serial ``drivers/usb/serial/usb-serial.h`` -FULL_DUPLEX_MAGIC 0x6969 ``drivers/net/ethernet/dec/tulip/de2104x.c`` -USB_BLUETOOTH_MAGIC 0x6d02 usb_bluetooth ``drivers/usb/class/bluetty.c`` -RFCOMM_TTY_MAGIC 0x6d02 ``net/bluetooth/rfcomm/tty.c`` -USB_SERIAL_PORT_MAGIC 0x7301 usb_serial_port ``drivers/usb/serial/usb-serial.h`` -CG_MAGIC 0x00090255 ufs_cylinder_group ``include/linux/ufs_fs.h`` -RPORT_MAGIC 0x00525001 r_port ``drivers/char/rocket_int.h`` -LSEMAGIC 0x05091998 lse ``drivers/fc4/fc.c`` -GDTIOCTL_MAGIC 0x06030f07 gdth_iowr_str ``drivers/scsi/gdth_ioctl.h`` -RIEBL_MAGIC 0x09051990 ``drivers/net/atarilance.c`` -NBD_REQUEST_MAGIC 0x12560953 nbd_request ``include/linux/nbd.h`` -RED_MAGIC2 0x170fc2a5 (any) ``mm/slab.c`` BAYCOM_MAGIC 0x19730510 baycom_state ``drivers/net/baycom_epp.c`` -ISDN_X25IFACE_MAGIC 0x1e75a2b9 isdn_x25iface_proto_data ``drivers/isdn/isdn_x25iface.h`` -ECP_MAGIC 0x21504345 cdkecpsig ``include/linux/cdk.h`` -LSOMAGIC 0x27091997 lso ``drivers/fc4/fc.c`` -LSMAGIC 0x2a3b4d2a ls ``drivers/fc4/fc.c`` -WANPIPE_MAGIC 0x414C4453 sdla_{dump,exec} ``include/linux/wanpipe.h`` -CS_CARD_MAGIC 0x43525553 cs_card ``sound/oss/cs46xx.c`` -LABELCL_MAGIC 0x4857434c labelcl_info_s ``include/asm/ia64/sn/labelcl.h`` -ISDN_ASYNC_MAGIC 0x49344C01 modem_info ``include/linux/isdn.h`` -CTC_ASYNC_MAGIC 0x49344C01 ctc_tty_info ``drivers/s390/net/ctctty.c`` -ISDN_NET_MAGIC 0x49344C02 isdn_net_local_s ``drivers/isdn/i4l/isdn_net_lib.h`` -SAVEKMSG_MAGIC2 0x4B4D5347 savekmsg ``arch/*/amiga/config.c`` -CS_STATE_MAGIC 0x4c4f4749 cs_state ``sound/oss/cs46xx.c`` -SLAB_C_MAGIC 0x4f17a36d kmem_cache ``mm/slab.c`` -COW_MAGIC 0x4f4f4f4d cow_header_v1 ``arch/um/drivers/ubd_user.c`` -I810_CARD_MAGIC 0x5072696E i810_card ``sound/oss/i810_audio.c`` -TRIDENT_CARD_MAGIC 0x5072696E trident_card ``sound/oss/trident.c`` -ROUTER_MAGIC 0x524d4157 wan_device [in ``wanrouter.h`` pre 3.9] -SAVEKMSG_MAGIC1 0x53415645 savekmsg ``arch/*/amiga/config.c`` -GDA_MAGIC 0x58464552 gda ``arch/mips/include/asm/sn/gda.h`` -RED_MAGIC1 0x5a2cf071 (any) ``mm/slab.c`` -EEPROM_MAGIC_VALUE 0x5ab478d2 lanai_dev ``drivers/atm/lanai.c`` HDLCDRV_MAGIC 0x5ac6e778 hdlcdrv_state ``include/linux/hdlcdrv.h`` -PCXX_MAGIC 0x5c6df104 channel ``drivers/char/pcxx.h`` KV_MAGIC 0x5f4b565f kernel_vars_s ``arch/mips/include/asm/sn/klkernvars.h`` -I810_STATE_MAGIC 0x63657373 i810_state ``sound/oss/i810_audio.c`` -TRIDENT_STATE_MAGIC 0x63657373 trient_state ``sound/oss/trident.c`` -M3_CARD_MAGIC 0x646e6f50 m3_card ``sound/oss/maestro3.c`` -FW_HEADER_MAGIC 0x65726F66 fw_header ``drivers/atm/fore200e.h`` -SLOT_MAGIC 0x67267321 slot ``drivers/hotplug/cpqphp.h`` -SLOT_MAGIC 0x67267322 slot ``drivers/hotplug/acpiphp.h`` -LO_MAGIC 0x68797548 nbd_device ``include/linux/nbd.h`` -OPROFILE_MAGIC 0x6f70726f super_block ``drivers/oprofile/oprofilefs.h`` -M3_STATE_MAGIC 0x734d724d m3_state ``sound/oss/maestro3.c`` -VMALLOC_MAGIC 0x87654320 snd_alloc_track ``sound/core/memory.c`` -KMALLOC_MAGIC 0x87654321 snd_alloc_track ``sound/core/memory.c`` -PWC_MAGIC 0x89DC10AB pwc_device ``drivers/usb/media/pwc.h`` -NBD_REPLY_MAGIC 0x96744668 nbd_reply ``include/linux/nbd.h`` -ENI155_MAGIC 0xa54b872d midway_eprom ``drivers/atm/eni.h`` CODA_MAGIC 0xC0DAC0DA coda_file_info ``fs/coda/coda_fs_i.h`` -DPMEM_MAGIC 0xc0ffee11 gdt_pci_sram ``drivers/scsi/gdth.h`` YAM_MAGIC 0xF10A7654 yam_port ``drivers/net/hamradio/yam.c`` CCB_MAGIC 0xf2691ad2 ccb ``drivers/scsi/ncr53c8xx.c`` QUEUE_MAGIC_FREE 0xf7e1c9a3 queue_entry ``drivers/scsi/arm/queue.c`` QUEUE_MAGIC_USED 0xf7e1cc33 queue_entry ``drivers/scsi/arm/queue.c`` -HTB_CMAGIC 0xFEFAFEF1 htb_class ``net/sched/sch_htb.c`` NMI_MAGIC 0x48414d4d455201 nmi_s ``arch/mips/include/asm/sn/nmi.h`` ===================== ================ ======================== ========================================== - -Note that there are also defined special per-driver magic numbers in sound -memory management. See ``include/sound/sndmagic.h`` for complete list of them. Many -OSS sound drivers have their magic numbers constructed from the soundcard PCI -ID - these are not listed here as well. - -HFS is another larger user of magic numbers - you can find them in -``fs/hfs/hfs.h``. diff --git a/Documentation/process/maintainer-handbooks.rst b/Documentation/process/maintainer-handbooks.rst new file mode 100644 index 000000000000..976391cec528 --- /dev/null +++ b/Documentation/process/maintainer-handbooks.rst @@ -0,0 +1,22 @@ +.. SPDX-License-Identifier: GPL-2.0 + +.. _maintainer_handbooks_main: + +Subsystem and maintainer tree specific development process notes +================================================================ + +The purpose of this document is to provide subsystem specific information +which is supplementary to the general development process handbook +:ref:`Documentation/process <development_process_main>`. + +Contents: + +.. toctree:: + :numbered: + :maxdepth: 2 + + maintainer-netdev + maintainer-soc + maintainer-soc-clean-dts + maintainer-tip + maintainer-kvm-x86 diff --git a/Documentation/process/maintainer-kvm-x86.rst b/Documentation/process/maintainer-kvm-x86.rst new file mode 100644 index 000000000000..9183bd449762 --- /dev/null +++ b/Documentation/process/maintainer-kvm-x86.rst @@ -0,0 +1,390 @@ +.. SPDX-License-Identifier: GPL-2.0 + +KVM x86 +======= + +Foreword +-------- +KVM strives to be a welcoming community; contributions from newcomers are +valued and encouraged. Please do not be discouraged or intimidated by the +length of this document and the many rules/guidelines it contains. Everyone +makes mistakes, and everyone was a newbie at some point. So long as you make +an honest effort to follow KVM x86's guidelines, are receptive to feedback, +and learn from any mistakes you make, you will be welcomed with open arms, not +torches and pitchforks. + +TL;DR +----- +Testing is mandatory. Be consistent with established styles and patterns. + +Trees +----- +KVM x86 is currently in a transition period from being part of the main KVM +tree, to being "just another KVM arch". As such, KVM x86 is split across the +main KVM tree, ``git.kernel.org/pub/scm/virt/kvm/kvm.git``, and a KVM x86 +specific tree, ``github.com/kvm-x86/linux.git``. + +Generally speaking, fixes for the current cycle are applied directly to the +main KVM tree, while all development for the next cycle is routed through the +KVM x86 tree. In the unlikely event that a fix for the current cycle is routed +through the KVM x86 tree, it will be applied to the ``fixes`` branch before +making its way to the main KVM tree. + +Note, this transition period is expected to last quite some time, i.e. will be +the status quo for the foreseeable future. + +Branches +~~~~~~~~ +The KVM x86 tree is organized into multiple topic branches. The purpose of +using finer-grained topic branches is to make it easier to keep tabs on an area +of development, and to limit the collateral damage of human errors and/or buggy +commits, e.g. dropping the HEAD commit of a topic branch has no impact on other +in-flight commits' SHA1 hashes, and having to reject a pull request due to bugs +delays only that topic branch. + +All topic branches, except for ``next`` and ``fixes``, are rolled into ``next`` +via a Cthulhu merge on an as-needed basis, i.e. when a topic branch is updated. +As a result, force pushes to ``next`` are common. + +Lifecycle +~~~~~~~~~ +Fixes that target the current release, a.k.a. mainline, are typically applied +directly to the main KVM tree, i.e. do not route through the KVM x86 tree. + +Changes that target the next release are routed through the KVM x86 tree. Pull +requests (from KVM x86 to main KVM) are sent for each KVM x86 topic branch, +typically the week before Linus' opening of the merge window, e.g. the week +following rc7 for "normal" releases. If all goes well, the topic branches are +rolled into the main KVM pull request sent during Linus' merge window. + +The KVM x86 tree doesn't have its own official merge window, but there's a soft +close around rc5 for new features, and a soft close around rc6 for fixes (for +the next release; see above for fixes that target the current release). + +Timeline +~~~~~~~~ +Submissions are typically reviewed and applied in FIFO order, with some wiggle +room for the size of a series, patches that are "cache hot", etc. Fixes, +especially for the current release and or stable trees, get to jump the queue. +Patches that will be taken through a non-KVM tree (most often through the tip +tree) and/or have other acks/reviews also jump the queue to some extent. + +Note, the vast majority of review is done between rc1 and rc6, give or take. +The period between rc6 and the next rc1 is used to catch up on other tasks, +i.e. radio silence during this period isn't unusual. + +Pings to get a status update are welcome, but keep in mind the timing of the +current release cycle and have realistic expectations. If you are pinging for +acceptance, i.e. not just for feedback or an update, please do everything you +can, within reason, to ensure that your patches are ready to be merged! Pings +on series that break the build or fail tests lead to unhappy maintainers! + +Development +----------- + +Base Tree/Branch +~~~~~~~~~~~~~~~~ +Fixes that target the current release, a.k.a. mainline, should be based on +``git://git.kernel.org/pub/scm/virt/kvm/kvm.git master``. Note, fixes do not +automatically warrant inclusion in the current release. There is no singular +rule, but typically only fixes for bugs that are urgent, critical, and/or were +introduced in the current release should target the current release. + +Everything else should be based on ``kvm-x86/next``, i.e. there is no need to +select a specific topic branch as the base. If there are conflicts and/or +dependencies across topic branches, it is the maintainer's job to sort them +out. + +The only exception to using ``kvm-x86/next`` as the base is if a patch/series +is a multi-arch series, i.e. has non-trivial modifications to common KVM code +and/or has more than superficial changes to other architectures' code. Multi- +arch patch/series should instead be based on a common, stable point in KVM's +history, e.g. the release candidate upon which ``kvm-x86 next`` is based. If +you're unsure whether a patch/series is truly multi-arch, err on the side of +caution and treat it as multi-arch, i.e. use a common base. + +Coding Style +~~~~~~~~~~~~ +When it comes to style, naming, patterns, etc., consistency is the number one +priority in KVM x86. If all else fails, match what already exists. + +With a few caveats listed below, follow the tip tree maintainers' preferred +:ref:`maintainer-tip-coding-style`, as patches/series often touch both KVM and +non-KVM x86 files, i.e. draw the attention of KVM *and* tip tree maintainers. + +Using reverse fir tree, a.k.a. reverse Christmas tree or reverse XMAS tree, for +variable declarations isn't strictly required, though it is still preferred. + +Except for a handful of special snowflakes, do not use kernel-doc comments for +functions. The vast majority of "public" KVM functions aren't truly public as +they are intended only for KVM-internal consumption (there are plans to +privatize KVM's headers and exports to enforce this). + +Comments +~~~~~~~~ +Write comments using imperative mood and avoid pronouns. Use comments to +provide a high level overview of the code, and/or to explain why the code does +what it does. Do not reiterate what the code literally does; let the code +speak for itself. If the code itself is inscrutable, comments will not help. + +SDM and APM References +~~~~~~~~~~~~~~~~~~~~~~ +Much of KVM's code base is directly tied to architectural behavior defined in +Intel's Software Development Manual (SDM) and AMD's Architecture Programmer’s +Manual (APM). Use of "Intel's SDM" and "AMD's APM", or even just "SDM" or +"APM", without additional context is a-ok. + +Do not reference specific sections, tables, figures, etc. by number, especially +not in comments. Instead, if necessary (see below), copy-paste the relevant +snippet and reference sections/tables/figures by name. The layouts of the SDM +and APM are constantly changing, and so the numbers/labels aren't stable. + +Generally speaking, do not explicitly reference or copy-paste from the SDM or +APM in comments. With few exceptions, KVM *must* honor architectural behavior, +therefore it's implied that KVM behavior is emulating SDM and/or APM behavior. +Note, referencing the SDM/APM in changelogs to justify the change and provide +context is perfectly ok and encouraged. + +Shortlog +~~~~~~~~ +The preferred prefix format is ``KVM: <topic>:``, where ``<topic>`` is one of:: + + - x86 + - x86/mmu + - x86/pmu + - x86/xen + - selftests + - SVM + - nSVM + - VMX + - nVMX + +**DO NOT use x86/kvm!** ``x86/kvm`` is used exclusively for Linux-as-a-KVM-guest +changes, i.e. for arch/x86/kernel/kvm.c. Do not use file names or complete file +paths as the subject/shortlog prefix. + +Note, these don't align with the topics branches (the topic branches care much +more about code conflicts). + +All names are case sensitive! ``KVM: x86:`` is good, ``kvm: vmx:`` is not. + +Capitalize the first word of the condensed patch description, but omit ending +punctionation. E.g.:: + + KVM: x86: Fix a null pointer dereference in function_xyz() + +not:: + + kvm: x86: fix a null pointer dereference in function_xyz. + +If a patch touches multiple topics, traverse up the conceptual tree to find the +first common parent (which is often simply ``x86``). When in doubt, +``git log path/to/file`` should provide a reasonable hint. + +New topics do occasionally pop up, but please start an on-list discussion if +you want to propose introducing a new topic, i.e. don't go rogue. + +See :ref:`the_canonical_patch_format` for more information, with one amendment: +do not treat the 70-75 character limit as an absolute, hard limit. Instead, +use 75 characters as a firm-but-not-hard limit, and use 80 characters as a hard +limit. I.e. let the shortlog run a few characters over the standard limit if +you have good reason to do so. + +Changelog +~~~~~~~~~ +Most importantly, write changelogs using imperative mood and avoid pronouns. + +See :ref:`describe_changes` for more information, with one amendment: lead with +a short blurb on the actual changes, and then follow up with the context and +background. Note! This order directly conflicts with the tip tree's preferred +approach! Please follow the tip tree's preferred style when sending patches +that primarily target arch/x86 code that is _NOT_ KVM code. + +Stating what a patch does before diving into details is preferred by KVM x86 +for several reasons. First and foremost, what code is actually being changed +is arguably the most important information, and so that info should be easy to +find. Changelogs that bury the "what's actually changing" in a one-liner after +3+ paragraphs of background make it very hard to find that information. + +For initial review, one could argue the "what's broken" is more important, but +for skimming logs and git archaeology, the gory details matter less and less. +E.g. when doing a series of "git blame", the details of each change along the +way are useless, the details only matter for the culprit. Providing the "what +changed" makes it easy to quickly determine whether or not a commit might be of +interest. + +Another benefit of stating "what's changing" first is that it's almost always +possible to state "what's changing" in a single sentence. Conversely, all but +the most simple bugs require multiple sentences or paragraphs to fully describe +the problem. If both the "what's changing" and "what's the bug" are super +short then the order doesn't matter. But if one is shorter (almost always the +"what's changing), then covering the shorter one first is advantageous because +it's less of an inconvenience for readers/reviewers that have a strict ordering +preference. E.g. having to skip one sentence to get to the context is less +painful than having to skip three paragraphs to get to "what's changing". + +Fixes +~~~~~ +If a change fixes a KVM/kernel bug, add a Fixes: tag even if the change doesn't +need to be backported to stable kernels, and even if the change fixes a bug in +an older release. + +Conversely, if a fix does need to be backported, explicitly tag the patch with +"Cc: stable@vger.kernel" (though the email itself doesn't need to Cc: stable); +KVM x86 opts out of backporting Fixes: by default. Some auto-selected patches +do get backported, but require explicit maintainer approval (search MANUALSEL). + +Function References +~~~~~~~~~~~~~~~~~~~ +When a function is mentioned in a comment, changelog, or shortlog (or anywhere +for that matter), use the format ``function_name()``. The parentheses provide +context and disambiguate the reference. + +Testing +------- +At a bare minimum, *all* patches in a series must build cleanly for KVM_INTEL=m +KVM_AMD=m, and KVM_WERROR=y. Building every possible combination of Kconfigs +isn't feasible, but the more the merrier. KVM_SMM, KVM_XEN, PROVE_LOCKING, and +X86_64 are particularly interesting knobs to turn. + +Running KVM selftests and KVM-unit-tests is also mandatory (and stating the +obvious, the tests need to pass). The only exception is for changes that have +negligible probability of affecting runtime behavior, e.g. patches that only +modify comments. When possible and relevant, testing on both Intel and AMD is +strongly preferred. Booting an actual VM is encouraged, but not mandatory. + +For changes that touch KVM's shadow paging code, running with TDP (EPT/NPT) +disabled is mandatory. For changes that affect common KVM MMU code, running +with TDP disabled is strongly encouraged. For all other changes, if the code +being modified depends on and/or interacts with a module param, testing with +the relevant settings is mandatory. + +Note, KVM selftests and KVM-unit-tests do have known failures. If you suspect +a failure is not due to your changes, verify that the *exact same* failure +occurs with and without your changes. + +Changes that touch reStructured Text documentation, i.e. .rst files, must build +htmldocs cleanly, i.e. with no new warnings or errors. + +If you can't fully test a change, e.g. due to lack of hardware, clearly state +what level of testing you were able to do, e.g. in the cover letter. + +New Features +~~~~~~~~~~~~ +With one exception, new features *must* come with test coverage. KVM specific +tests aren't strictly required, e.g. if coverage is provided by running a +sufficiently enabled guest VM, or by running a related kernel selftest in a VM, +but dedicated KVM tests are preferred in all cases. Negative testcases in +particular are mandatory for enabling of new hardware features as error and +exception flows are rarely exercised simply by running a VM. + +The only exception to this rule is if KVM is simply advertising support for a +feature via KVM_GET_SUPPORTED_CPUID, i.e. for instructions/features that KVM +can't prevent a guest from using and for which there is no true enabling. + +Note, "new features" does not just mean "new hardware features"! New features +that can't be well validated using existing KVM selftests and/or KVM-unit-tests +must come with tests. + +Posting new feature development without tests to get early feedback is more +than welcome, but such submissions should be tagged RFC, and the cover letter +should clearly state what type of feedback is requested/expected. Do not abuse +the RFC process; RFCs will typically not receive in-depth review. + +Bug Fixes +~~~~~~~~~ +Except for "obvious" found-by-inspection bugs, fixes must be accompanied by a +reproducer for the bug being fixed. In many cases the reproducer is implicit, +e.g. for build errors and test failures, but it should still be clear to +readers what is broken and how to verify the fix. Some leeway is given for +bugs that are found via non-public workloads/tests, but providing regression +tests for such bugs is strongly preferred. + +In general, regression tests are preferred for any bug that is not trivial to +hit. E.g. even if the bug was originally found by a fuzzer such as syzkaller, +a targeted regression test may be warranted if the bug requires hitting a +one-in-a-million type race condition. + +Note, KVM bugs are rarely urgent *and* non-trivial to reproduce. Ask yourself +if a bug is really truly the end of the world before posting a fix without a +reproducer. + +Posting +------- + +Links +~~~~~ +Do not explicitly reference bug reports, prior versions of a patch/series, etc. +via ``In-Reply-To:`` headers. Using ``In-Reply-To:`` becomes an unholy mess +for large series and/or when the version count gets high, and ``In-Reply-To:`` +is useless for anyone that doesn't have the original message, e.g. if someone +wasn't Cc'd on the bug report or if the list of recipients changes between +versions. + +To link to a bug report, previous version, or anything of interest, use lore +links. For referencing previous version(s), generally speaking do not include +a Link: in the changelog as there is no need to record the history in git, i.e. +put the link in the cover letter or in the section git ignores. Do provide a +formal Link: for bug reports and/or discussions that led to the patch. The +context of why a change was made is highly valuable for future readers. + +Git Base +~~~~~~~~ +If you are using git version 2.9.0 or later (Googlers, this is all of you!), +use ``git format-patch`` with the ``--base`` flag to automatically include the +base tree information in the generated patches. + +Note, ``--base=auto`` works as expected if and only if a branch's upstream is +set to the base topic branch, e.g. it will do the wrong thing if your upstream +is set to your personal repository for backup purposes. An alternative "auto" +solution is to derive the names of your development branches based on their +KVM x86 topic, and feed that into ``--base``. E.g. ``x86/pmu/my_branch_name``, +and then write a small wrapper to extract ``pmu`` from the current branch name +to yield ``--base=x/pmu``, where ``x`` is whatever name your repository uses to +track the KVM x86 remote. + +Co-Posting Tests +~~~~~~~~~~~~~~~~ +KVM selftests that are associated with KVM changes, e.g. regression tests for +bug fixes, should be posted along with the KVM changes as a single series. The +standard kernel rules for bisection apply, i.e. KVM changes that result in test +failures should be ordered after the selftests updates, and vice versa, new +tests that fail due to KVM bugs should be ordered after the KVM fixes. + +KVM-unit-tests should *always* be posted separately. Tools, e.g. b4 am, don't +know that KVM-unit-tests is a separate repository and get confused when patches +in a series apply on different trees. To tie KVM-unit-tests patches back to +KVM patches, first post the KVM changes and then provide a lore Link: to the +KVM patch/series in the KVM-unit-tests patch(es). + +Notifications +------------- +When a patch/series is officially accepted, a notification email will be sent +in reply to the original posting (cover letter for multi-patch series). The +notification will include the tree and topic branch, along with the SHA1s of +the commits of applied patches. + +If a subset of patches is applied, this will be clearly stated in the +notification. Unless stated otherwise, it's implied that any patches in the +series that were not accepted need more work and should be submitted in a new +version. + +If for some reason a patch is dropped after officially being accepted, a reply +will be sent to the notification email explaining why the patch was dropped, as +well as the next steps. + +SHA1 Stability +~~~~~~~~~~~~~~ +SHA1s are not 100% guaranteed to be stable until they land in Linus' tree! A +SHA1 is *usually* stable once a notification has been sent, but things happen. +In most cases, an update to the notification email be provided if an applied +patch's SHA1 changes. However, in some scenarios, e.g. if all KVM x86 branches +need to be rebased, individual notifications will not be given. + +Vulnerabilities +--------------- +Bugs that can be exploited by the guest to attack the host (kernel or +userspace), or that can be exploited by a nested VM to *its* host (L2 attacking +L1), are of particular interest to KVM. Please follow the protocol for +:ref:`securitybugs` if you suspect a bug can lead to an escape, data leak, etc. + diff --git a/Documentation/process/maintainer-netdev.rst b/Documentation/process/maintainer-netdev.rst new file mode 100644 index 000000000000..84ee60fceef2 --- /dev/null +++ b/Documentation/process/maintainer-netdev.rst @@ -0,0 +1,483 @@ +.. SPDX-License-Identifier: GPL-2.0 + +.. _netdev-FAQ: + +============================= +Networking subsystem (netdev) +============================= + +tl;dr +----- + + - designate your patch to a tree - ``[PATCH net]`` or ``[PATCH net-next]`` + - for fixes the ``Fixes:`` tag is required, regardless of the tree + - don't post large series (> 15 patches), break them up + - don't repost your patches within one 24h period + - reverse xmas tree + +netdev +------ + +netdev is a mailing list for all network-related Linux stuff. This +includes anything found under net/ (i.e. core code like IPv6) and +drivers/net (i.e. hardware specific drivers) in the Linux source tree. + +Note that some subsystems (e.g. wireless drivers) which have a high +volume of traffic have their own specific mailing lists and trees. + +The netdev list is managed (like many other Linux mailing lists) through +VGER (http://vger.kernel.org/) with archives available at +https://lore.kernel.org/netdev/ + +Aside from subsystems like those mentioned above, all network-related +Linux development (i.e. RFC, review, comments, etc.) takes place on +netdev. + +Development cycle +----------------- + +Here is a bit of background information on +the cadence of Linux development. Each new release starts off with a +two week "merge window" where the main maintainers feed their new stuff +to Linus for merging into the mainline tree. After the two weeks, the +merge window is closed, and it is called/tagged ``-rc1``. No new +features get mainlined after this -- only fixes to the rc1 content are +expected. After roughly a week of collecting fixes to the rc1 content, +rc2 is released. This repeats on a roughly weekly basis until rc7 +(typically; sometimes rc6 if things are quiet, or rc8 if things are in a +state of churn), and a week after the last vX.Y-rcN was done, the +official vX.Y is released. + +To find out where we are now in the cycle - load the mainline (Linus) +page here: + + https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git + +and note the top of the "tags" section. If it is rc1, it is early in +the dev cycle. If it was tagged rc7 a week ago, then a release is +probably imminent. If the most recent tag is a final release tag +(without an ``-rcN`` suffix) - we are most likely in a merge window +and ``net-next`` is closed. + +git trees and patch flow +------------------------ + +There are two networking trees (git repositories) in play. Both are +driven by David Miller, the main network maintainer. There is the +``net`` tree, and the ``net-next`` tree. As you can probably guess from +the names, the ``net`` tree is for fixes to existing code already in the +mainline tree from Linus, and ``net-next`` is where the new code goes +for the future release. You can find the trees here: + +- https://git.kernel.org/pub/scm/linux/kernel/git/netdev/net.git +- https://git.kernel.org/pub/scm/linux/kernel/git/netdev/net-next.git + +Relating that to kernel development: At the beginning of the 2-week +merge window, the ``net-next`` tree will be closed - no new changes/features. +The accumulated new content of the past ~10 weeks will be passed onto +mainline/Linus via a pull request for vX.Y -- at the same time, the +``net`` tree will start accumulating fixes for this pulled content +relating to vX.Y + +An announcement indicating when ``net-next`` has been closed is usually +sent to netdev, but knowing the above, you can predict that in advance. + +.. warning:: + Do not send new ``net-next`` content to netdev during the + period during which ``net-next`` tree is closed. + +RFC patches sent for review only are obviously welcome at any time +(use ``--subject-prefix='RFC net-next'`` with ``git format-patch``). + +Shortly after the two weeks have passed (and vX.Y-rc1 is released), the +tree for ``net-next`` reopens to collect content for the next (vX.Y+1) +release. + +If you aren't subscribed to netdev and/or are simply unsure if +``net-next`` has re-opened yet, simply check the ``net-next`` git +repository link above for any new networking-related commits. You may +also check the following website for the current status: + + https://netdev.bots.linux.dev/net-next.html + +The ``net`` tree continues to collect fixes for the vX.Y content, and is +fed back to Linus at regular (~weekly) intervals. Meaning that the +focus for ``net`` is on stabilization and bug fixes. + +Finally, the vX.Y gets released, and the whole cycle starts over. + +netdev patch review +------------------- + +.. _patch_status: + +Patch status +~~~~~~~~~~~~ + +Status of a patch can be checked by looking at the main patchwork +queue for netdev: + + https://patchwork.kernel.org/project/netdevbpf/list/ + +The "State" field will tell you exactly where things are at with your +patch: + +================== ============================================================= +Patch state Description +================== ============================================================= +New, Under review pending review, patch is in the maintainer’s queue for + review; the two states are used interchangeably (depending on + the exact co-maintainer handling patchwork at the time) +Accepted patch was applied to the appropriate networking tree, this is + usually set automatically by the pw-bot +Needs ACK waiting for an ack from an area expert or testing +Changes requested patch has not passed the review, new revision is expected + with appropriate code and commit message changes +Rejected patch has been rejected and new revision is not expected +Not applicable patch is expected to be applied outside of the networking + subsystem +Awaiting upstream patch should be reviewed and handled by appropriate + sub-maintainer, who will send it on to the networking trees; + patches set to ``Awaiting upstream`` in netdev's patchwork + will usually remain in this state, whether the sub-maintainer + requested changes, accepted or rejected the patch +Deferred patch needs to be reposted later, usually due to dependency + or because it was posted for a closed tree +Superseded new version of the patch was posted, usually set by the + pw-bot +RFC not to be applied, usually not in maintainer’s review queue, + pw-bot can automatically set patches to this state based + on subject tags +================== ============================================================= + +Patches are indexed by the ``Message-ID`` header of the emails +which carried them so if you have trouble finding your patch append +the value of ``Message-ID`` to the URL above. + +Updating patch status +~~~~~~~~~~~~~~~~~~~~~ + +Contributors and reviewers do not have the permissions to update patch +state directly in patchwork. Patchwork doesn't expose much information +about the history of the state of patches, therefore having multiple +people update the state leads to confusion. + +Instead of delegating patchwork permissions netdev uses a simple mail +bot which looks for special commands/lines within the emails sent to +the mailing list. For example to mark a series as Changes Requested +one needs to send the following line anywhere in the email thread:: + + pw-bot: changes-requested + +As a result the bot will set the entire series to Changes Requested. +This may be useful when author discovers a bug in their own series +and wants to prevent it from getting applied. + +The use of the bot is entirely optional, if in doubt ignore its existence +completely. Maintainers will classify and update the state of the patches +themselves. No email should ever be sent to the list with the main purpose +of communicating with the bot, the bot commands should be seen as metadata. + +The use of the bot is restricted to authors of the patches (the ``From:`` +header on patch submission and command must match!), maintainers of +the modified code according to the MAINTAINERS file (again, ``From:`` +must match the MAINTAINERS entry) and a handful of senior reviewers. + +Bot records its activity here: + + https://netdev.bots.linux.dev/pw-bot.html + +Review timelines +~~~~~~~~~~~~~~~~ + +Generally speaking, the patches get triaged quickly (in less than +48h). But be patient, if your patch is active in patchwork (i.e. it's +listed on the project's patch list) the chances it was missed are close to zero. + +The high volume of development on netdev makes reviewers move on +from discussions relatively quickly. New comments and replies +are very unlikely to arrive after a week of silence. If a patch +is no longer active in patchwork and the thread went idle for more +than a week - clarify the next steps and/or post the next version. + +For RFC postings specifically, if nobody responded in a week - reviewers +either missed the posting or have no strong opinions. If the code is ready, +repost as a PATCH. + +Emails saying just "ping" or "bump" are considered rude. If you can't figure +out the status of the patch from patchwork or where the discussion has +landed - describe your best guess and ask if it's correct. For example:: + + I don't understand what the next steps are. Person X seems to be unhappy + with A, should I do B and repost the patches? + +.. _Changes requested: + +Changes requested +~~~~~~~~~~~~~~~~~ + +Patches :ref:`marked<patch_status>` as ``Changes Requested`` need +to be revised. The new version should come with a change log, +preferably including links to previous postings, for example:: + + [PATCH net-next v3] net: make cows go moo + + Even users who don't drink milk appreciate hearing the cows go "moo". + + The amount of mooing will depend on packet rate so should match + the diurnal cycle quite well. + + Signed-of-by: Joe Defarmer <joe@barn.org> + --- + v3: + - add a note about time-of-day mooing fluctuation to the commit message + v2: https://lore.kernel.org/netdev/123themessageid@barn.org/ + - fix missing argument in kernel doc for netif_is_bovine() + - fix memory leak in netdev_register_cow() + v1: https://lore.kernel.org/netdev/456getstheclicks@barn.org/ + +The commit message should be revised to answer any questions reviewers +had to ask in previous discussions. Occasionally the update of +the commit message will be the only change in the new version. + +Partial resends +~~~~~~~~~~~~~~~ + +Please always resend the entire patch series and make sure you do number your +patches such that it is clear this is the latest and greatest set of patches +that can be applied. Do not try to resend just the patches which changed. + +Handling misapplied patches +~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Occasionally a patch series gets applied before receiving critical feedback, +or the wrong version of a series gets applied. + +Making the patch disappear once it is pushed out is not possible, the commit +history in netdev trees is immutable. +Please send incremental versions on top of what has been merged in order to fix +the patches the way they would look like if your latest patch series was to be +merged. + +In cases where full revert is needed the revert has to be submitted +as a patch to the list with a commit message explaining the technical +problems with the reverted commit. Reverts should be used as a last resort, +when original change is completely wrong; incremental fixes are preferred. + +Stable tree +~~~~~~~~~~~ + +While it used to be the case that netdev submissions were not supposed +to carry explicit ``CC: stable@vger.kernel.org`` tags that is no longer +the case today. Please follow the standard stable rules in +:ref:`Documentation/process/stable-kernel-rules.rst <stable_kernel_rules>`, +and make sure you include appropriate Fixes tags! + +Security fixes +~~~~~~~~~~~~~~ + +Do not email netdev maintainers directly if you think you discovered +a bug that might have possible security implications. +The current netdev maintainer has consistently requested that +people use the mailing lists and not reach out directly. If you aren't +OK with that, then perhaps consider mailing security@kernel.org or +reading about http://oss-security.openwall.org/wiki/mailing-lists/distros +as possible alternative mechanisms. + + +Co-posting changes to user space components +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +User space code exercising kernel features should be posted +alongside kernel patches. This gives reviewers a chance to see +how any new interface is used and how well it works. + +When user space tools reside in the kernel repo itself all changes +should generally come as one series. If series becomes too large +or the user space project is not reviewed on netdev include a link +to a public repo where user space patches can be seen. + +In case user space tooling lives in a separate repository but is +reviewed on netdev (e.g. patches to ``iproute2`` tools) kernel and +user space patches should form separate series (threads) when posted +to the mailing list, e.g.:: + + [PATCH net-next 0/3] net: some feature cover letter + └─ [PATCH net-next 1/3] net: some feature prep + └─ [PATCH net-next 2/3] net: some feature do it + └─ [PATCH net-next 3/3] selftest: net: some feature + + [PATCH iproute2-next] ip: add support for some feature + +Posting as one thread is discouraged because it confuses patchwork +(as of patchwork 2.2.2). + +Preparing changes +----------------- + +Attention to detail is important. Re-read your own work as if you were the +reviewer. You can start with using ``checkpatch.pl``, perhaps even with +the ``--strict`` flag. But do not be mindlessly robotic in doing so. +If your change is a bug fix, make sure your commit log indicates the +end-user visible symptom, the underlying reason as to why it happens, +and then if necessary, explain why the fix proposed is the best way to +get things done. Don't mangle whitespace, and as is common, don't +mis-indent function arguments that span multiple lines. If it is your +first patch, mail it to yourself so you can test apply it to an +unpatched tree to confirm infrastructure didn't mangle it. + +Finally, go back and read +:ref:`Documentation/process/submitting-patches.rst <submittingpatches>` +to be sure you are not repeating some common mistake documented there. + +Indicating target tree +~~~~~~~~~~~~~~~~~~~~~~ + +To help maintainers and CI bots you should explicitly mark which tree +your patch is targeting. Assuming that you use git, use the prefix +flag:: + + git format-patch --subject-prefix='PATCH net-next' start..finish + +Use ``net`` instead of ``net-next`` (always lower case) in the above for +bug-fix ``net`` content. + +Dividing work into patches +~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Put yourself in the shoes of the reviewer. Each patch is read separately +and therefore should constitute a comprehensible step towards your stated +goal. + +Avoid sending series longer than 15 patches. Larger series takes longer +to review as reviewers will defer looking at it until they find a large +chunk of time. A small series can be reviewed in a short time, so Maintainers +just do it. As a result, a sequence of smaller series gets merged quicker and +with better review coverage. Re-posting large series also increases the mailing +list traffic. + +Multi-line comments +~~~~~~~~~~~~~~~~~~~ + +Comment style convention is slightly different for networking and most of +the tree. Instead of this:: + + /* + * foobar blah blah blah + * another line of text + */ + +it is requested that you make it look like this:: + + /* foobar blah blah blah + * another line of text + */ + +Local variable ordering ("reverse xmas tree", "RCS") +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Netdev has a convention for ordering local variables in functions. +Order the variable declaration lines longest to shortest, e.g.:: + + struct scatterlist *sg; + struct sk_buff *skb; + int err, i; + +If there are dependencies between the variables preventing the ordering +move the initialization out of line. + +Format precedence +~~~~~~~~~~~~~~~~~ + +When working in existing code which uses nonstandard formatting make +your code follow the most recent guidelines, so that eventually all code +in the domain of netdev is in the preferred format. + +Resending after review +~~~~~~~~~~~~~~~~~~~~~~ + +Allow at least 24 hours to pass between postings. This will ensure reviewers +from all geographical locations have a chance to chime in. Do not wait +too long (weeks) between postings either as it will make it harder for reviewers +to recall all the context. + +Make sure you address all the feedback in your new posting. Do not post a new +version of the code if the discussion about the previous version is still +ongoing, unless directly instructed by a reviewer. + +The new version of patches should be posted as a separate thread, +not as a reply to the previous posting. Change log should include a link +to the previous posting (see :ref:`Changes requested`). + +Testing +------- + +Expected level of testing +~~~~~~~~~~~~~~~~~~~~~~~~~ + +At the very minimum your changes must survive an ``allyesconfig`` and an +``allmodconfig`` build with ``W=1`` set without new warnings or failures. + +Ideally you will have done run-time testing specific to your change, +and the patch series contains a set of kernel selftest for +``tools/testing/selftests/net`` or using the KUnit framework. + +You are expected to test your changes on top of the relevant networking +tree (``net`` or ``net-next``) and not e.g. a stable tree or ``linux-next``. + +patchwork checks +~~~~~~~~~~~~~~~~ + +Checks in patchwork are mostly simple wrappers around existing kernel +scripts, the sources are available at: + +https://github.com/kuba-moo/nipa/tree/master/tests + +**Do not** post your patches just to run them through the checks. +You must ensure that your patches are ready by testing them locally +before posting to the mailing list. The patchwork build bot instance +gets overloaded very easily and netdev@vger really doesn't need more +traffic if we can help it. + +netdevsim +~~~~~~~~~ + +``netdevsim`` is a test driver which can be used to exercise driver +configuration APIs without requiring capable hardware. +Mock-ups and tests based on ``netdevsim`` are strongly encouraged when +adding new APIs, but ``netdevsim`` in itself is **not** considered +a use case/user. You must also implement the new APIs in a real driver. + +We give no guarantees that ``netdevsim`` won't change in the future +in a way which would break what would normally be considered uAPI. + +``netdevsim`` is reserved for use by upstream tests only, so any +new ``netdevsim`` features must be accompanied by selftests under +``tools/testing/selftests/``. + +Reviewer guidance +----------------- + +Reviewing other people's patches on the list is highly encouraged, +regardless of the level of expertise. For general guidance and +helpful tips please see :ref:`development_advancedtopics_reviews`. + +It's safe to assume that netdev maintainers know the community and the level +of expertise of the reviewers. The reviewers should not be concerned about +their comments impeding or derailing the patch flow. + +Less experienced reviewers are highly encouraged to do more in-depth +review of submissions and not focus exclusively on trivial or subjective +matters like code formatting, tags etc. + +Testimonials / feedback +----------------------- + +Some companies use peer feedback in employee performance reviews. +Please feel free to request feedback from netdev maintainers, +especially if you spend significant amount of time reviewing code +and go out of your way to improve shared infrastructure. + +The feedback must be requested by you, the contributor, and will always +be shared with you (even if you request for it to be submitted to your +manager). diff --git a/Documentation/process/maintainer-pgp-guide.rst b/Documentation/process/maintainer-pgp-guide.rst index 17db11b7ed48..f5277993b195 100644 --- a/Documentation/process/maintainer-pgp-guide.rst +++ b/Documentation/process/maintainer-pgp-guide.rst @@ -60,36 +60,18 @@ establish the integrity of the Linux kernel itself. PGP tools ========= -Use GnuPG v2 ------------- +Use GnuPG 2.2 or later +---------------------- Your distro should already have GnuPG installed by default, you just -need to verify that you are using version 2.x and not the legacy 1.4 -release -- many distributions still package both, with the default -``gpg`` command invoking GnuPG v.1. To check, run:: +need to verify that you are using a reasonably recent version of it. +To check, run:: $ gpg --version | head -n1 -If you see ``gpg (GnuPG) 1.4.x``, then you are using GnuPG v.1. Try the -``gpg2`` command (if you don't have it, you may need to install the -gnupg2 package):: - - $ gpg2 --version | head -n1 - -If you see ``gpg (GnuPG) 2.x.x``, then you are good to go. This guide -will assume you have the version 2.2 of GnuPG (or later). If you are -using version 2.0 of GnuPG, then some of the commands in this guide will -not work, and you should consider installing the latest 2.2 version of -GnuPG. Versions of gnupg-2.1.11 and later should be compatible for the -purposes of this guide as well. - -If you have both ``gpg`` and ``gpg2`` commands, you should make sure you -are always using GnuPG v2, not the legacy version. You can enforce this -by setting the appropriate alias:: - - $ alias gpg=gpg2 - -You can put that in your ``.bashrc`` to make sure it's always the case. +If you have version 2.2 or above, then you are good to go. If you have a +version that is prior than 2.2, then some commands from this guide may +not work. Configure gpg-agent options ~~~~~~~~~~~~~~~~~~~~~~~~~~~ @@ -121,57 +103,56 @@ edit your ``~/.gnupg/gpg-agent.conf`` file to set your own values:: to remove anything you had in place for older versions of GnuPG, as it may not be doing the right thing any more. -Set up a refresh cronjob -~~~~~~~~~~~~~~~~~~~~~~~~ +.. _protect_your_key: -You will need to regularly refresh your keyring in order to get the -latest changes on other people's public keys, which is best done with a -daily cronjob:: - - @daily /usr/bin/gpg2 --refresh >/dev/null 2>&1 - -Check the full path to your ``gpg`` or ``gpg2`` command and use the -``gpg2`` command if regular ``gpg`` for you is the legacy GnuPG v.1. - -.. _master_key: - -Protect your master PGP key -=========================== +Protect your PGP key +==================== This guide assumes that you already have a PGP key that you use for Linux kernel development purposes. If you do not yet have one, please see the "`Protecting Code Integrity`_" document mentioned earlier for guidance on how to create a new one. -You should also make a new key if your current one is weaker than 2048 bits -(RSA). - -Master key vs. Subkeys ----------------------- - -Subkeys are fully independent PGP keypairs that are tied to the "master" -key using certifying key signatures (certificates). It is important to -understand the following: - -1. There are no technical differences between the "master key" and "subkeys." -2. At creation time, we assign functional limitations to each key by - giving it specific capabilities. -3. A PGP key can have 4 capabilities: - - - **[S]** key can be used for signing - - **[E]** key can be used for encryption - - **[A]** key can be used for authentication - - **[C]** key can be used for certifying other keys - -4. A single key may have multiple capabilities. -5. A subkey is fully independent from the master key. A message - encrypted to a subkey cannot be decrypted with the master key. If you - lose your private subkey, it cannot be recreated from the master key - in any way. - -The key carrying the **[C]** (certify) capability is considered the -"master" key because it is the only key that can be used to indicate -relationship with other keys. Only the **[C]** key can be used to: +You should also make a new key if your current one is weaker than 2048 +bits (RSA). + +Understanding PGP Subkeys +------------------------- + +A PGP key rarely consists of a single keypair -- usually it is a +collection of independent subkeys that can be used for different +purposes based on their capabilities, assigned at their creation time. +PGP defines four capabilities that a key can have: + +- **[S]** keys can be used for signing +- **[E]** keys can be used for encryption +- **[A]** keys can be used for authentication +- **[C]** keys can be used for certifying other keys + +The key with the **[C]** capability is often called the "master" key, +but this terminology is misleading because it implies that the Certify +key can be used in place of any of other subkey on the same chain (like +a physical "master key" can be used to open locks made for other keys). +Since this is not the case, this guide will refer to it as "the Certify +key" to avoid any ambiguity. + +It is critical to fully understand the following: + +1. All subkeys are fully independent from each other. If you lose a + private subkey, it cannot be restored or recreated from any other + private key on your chain. +2. With the exception of the Certify key, there can be multiple subkeys + with identical capabilities (e.g. you can have 2 valid encryption + subkeys, 3 valid signing subkeys, but only one valid certification + subkey). All subkeys are fully independent -- a message encrypted to + one **[E]** subkey cannot be decrypted with any other **[E]** subkey + you may also have. +3. A single subkey may have multiple capabilities (e.g. your **[C]** key + can also be your **[S]** key). + +The key carrying the **[C]** (certify) capability is the only key that +can be used to indicate relationship with other keys. Only the **[C]** +key can be used to: - add or revoke other keys (subkeys) with S/E/A capabilities - add, change or revoke identities (uids) associated with the key @@ -180,20 +161,17 @@ relationship with other keys. Only the **[C]** key can be used to: By default, GnuPG creates the following when generating new keys: -- A master key carrying both Certify and Sign capabilities (**[SC]**) +- One subkey carrying both Certify and Sign capabilities (**[SC]**) - A separate subkey with the Encryption capability (**[E]**) If you used the default parameters when generating your key, then that is what you will have. You can verify by running ``gpg --list-secret-keys``, for example:: - sec rsa2048 2018-01-23 [SC] [expires: 2020-01-23] + sec ed25519 2022-12-20 [SC] [expires: 2024-12-19] 000000000000000000000000AAAABBBBCCCCDDDD uid [ultimate] Alice Dev <adev@kernel.org> - ssb rsa2048 2018-01-23 [E] [expires: 2020-01-23] - -Any key carrying the **[C]** capability is your master key, regardless -of any other capabilities it may have assigned to it. + ssb cv25519 2022-12-20 [E] [expires: 2024-12-19] The long line under the ``sec`` entry is your key fingerprint -- whenever you see ``[fpr]`` in the examples below, that 40-character @@ -215,37 +193,21 @@ strong passphrase. To set it or change it, use:: Create a separate Signing subkey -------------------------------- -Our goal is to protect your master key by moving it to offline media, so -if you only have a combined **[SC]** key, then you should create a separate -signing subkey:: +Our goal is to protect your Certify key by moving it to offline media, +so if you only have a combined **[SC]** key, then you should create a +separate signing subkey:: $ gpg --quick-addkey [fpr] ed25519 sign -Remember to tell the keyservers about this change, so others can pull down -your new subkey:: - - $ gpg --send-key [fpr] - .. note:: ECC support in GnuPG - GnuPG 2.1 and later has full support for Elliptic Curve - Cryptography, with ability to combine ECC subkeys with traditional - RSA master keys. The main upside of ECC cryptography is that it is - much faster computationally and creates much smaller signatures when - compared byte for byte with 2048+ bit RSA keys. Unless you plan on - using a smartcard device that does not support ECC operations, we - recommend that you create an ECC signing subkey for your kernel - work. + Note, that if you intend to use a hardware token that does not + support ED25519 ECC keys, you should choose "nistp256" instead or + "ed25519." See the section below on recommended hardware devices. - If for some reason you prefer to stay with RSA subkeys, just replace - "ed25519" with "rsa2048" in the above command. Additionally, if you - plan to use a hardware device that does not support ED25519 ECC - keys, like Nitrokey Pro or a Yubikey, then you should use - "nistp256" instead or "ed25519." - -Back up your master key for disaster recovery ---------------------------------------------- +Back up your Certify key for disaster recovery +---------------------------------------------- The more signatures you have on your PGP key from other developers, the more reasons you have to create a backup version that lives on something @@ -277,9 +239,7 @@ home, such as your bank vault. Your printer is probably no longer a simple dumb device connected to your parallel port, but since the output is still encrypted with your passphrase, printing out even to "cloud-integrated" modern - printers should remain a relatively safe operation. One option is to - change the passphrase on your master key immediately after you are - done with paperkey. + printers should remain a relatively safe operation. Back up your whole GnuPG directory ---------------------------------- @@ -300,7 +260,7 @@ will use for backup purposes. You will need to encrypt them using LUKS -- refer to your distro's documentation on how to accomplish this. For the encryption passphrase, you can use the same one as on your -master key. +PGP key. Once the encryption process is over, re-insert the USB drive and make sure it gets properly mounted. Copy your entire ``.gnupg`` directory @@ -319,7 +279,7 @@ far away, because you'll need to use it every now and again for things like editing identities, adding or revoking subkeys, or signing other people's keys. -Remove the master key from your homedir +Remove the Certify key from your homedir ---------------------------------------- The files in our home directory are not as well protected as we like to @@ -334,7 +294,7 @@ think. They can be leaked or stolen via many different means: Protecting your key with a good passphrase greatly helps reduce the risk of any of the above, but passphrases can be discovered via keyloggers, shoulder-surfing, or any number of other means. For this reason, the -recommended setup is to remove your master key from your home directory +recommended setup is to remove your Certify key from your home directory and store it on offline storage. .. warning:: @@ -343,23 +303,23 @@ and store it on offline storage. your GnuPG directory in its entirety. What we are about to do will render your key useless if you do not have a usable backup! -First, identify the keygrip of your master key:: +First, identify the keygrip of your Certify key:: $ gpg --with-keygrip --list-key [fpr] The output will be something like this:: - pub rsa2048 2018-01-24 [SC] [expires: 2020-01-24] + pub ed25519 2022-12-20 [SC] [expires: 2022-12-19] 000000000000000000000000AAAABBBBCCCCDDDD Keygrip = 1111000000000000000000000000000000000000 uid [ultimate] Alice Dev <adev@kernel.org> - sub rsa2048 2018-01-24 [E] [expires: 2020-01-24] + sub cv25519 2022-12-20 [E] [expires: 2022-12-19] Keygrip = 2222000000000000000000000000000000000000 - sub ed25519 2018-01-24 [S] + sub ed25519 2022-12-20 [S] Keygrip = 3333000000000000000000000000000000000000 Find the keygrip entry that is beneath the ``pub`` line (right under the -master key fingerprint). This will correspond directly to a file in your +Certify key fingerprint). This will correspond directly to a file in your ``~/.gnupg`` directory:: $ cd ~/.gnupg/private-keys-v1.d @@ -369,23 +329,23 @@ master key fingerprint). This will correspond directly to a file in your 3333000000000000000000000000000000000000.key All you have to do is simply remove the .key file that corresponds to -the master keygrip:: +the Certify key keygrip:: $ cd ~/.gnupg/private-keys-v1.d $ rm 1111000000000000000000000000000000000000.key Now, if you issue the ``--list-secret-keys`` command, it will show that -the master key is missing (the ``#`` indicates it is not available):: +the Certify key is missing (the ``#`` indicates it is not available):: $ gpg --list-secret-keys - sec# rsa2048 2018-01-24 [SC] [expires: 2020-01-24] + sec# ed25519 2022-12-20 [SC] [expires: 2024-12-19] 000000000000000000000000AAAABBBBCCCCDDDD uid [ultimate] Alice Dev <adev@kernel.org> - ssb rsa2048 2018-01-24 [E] [expires: 2020-01-24] - ssb ed25519 2018-01-24 [S] + ssb cv25519 2022-12-20 [E] [expires: 2024-12-19] + ssb ed25519 2022-12-20 [S] You should also remove any ``secring.gpg`` files in the ``~/.gnupg`` -directory, which are left over from earlier versions of GnuPG. +directory, which may be left over from previous versions of GnuPG. If you don't have the "private-keys-v1.d" directory ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ @@ -404,7 +364,7 @@ file, which still contains your private keys. Move the subkeys to a dedicated crypto device ============================================= -Even though the master key is now safe from being leaked or stolen, the +Even though the Certify key is now safe from being leaked or stolen, the subkeys are still in your home directory. Anyone who manages to get their hands on those will be able to decrypt your communication or fake your signatures (if they know the passphrase). Furthermore, each time a @@ -447,10 +407,10 @@ functionality. There are several options available: - `Yubikey 5`_: proprietary hardware and software, but cheaper than Nitrokey Pro and comes available in the USB-C form that is more useful with newer laptops. Offers additional security features such as FIDO - U2F, among others, and now finally supports ECC keys (NISTP). + U2F, among others, and now finally supports NISTP and ED25519 ECC + keys. -`LWN has a good review`_ of some of the above models, as well as several -others. Your choice will depend on cost, shipping availability in your +Your choice will depend on cost, shipping availability in your geographical region, and open/proprietary hardware considerations. .. note:: @@ -460,10 +420,9 @@ geographical region, and open/proprietary hardware considerations. Foundation. .. _`Nitrokey Start`: https://shop.nitrokey.com/shop/product/nitrokey-start-6 -.. _`Nitrokey Pro 2`: https://shop.nitrokey.com/shop/product/nitrokey-pro-2-3 +.. _`Nitrokey Pro 2`: https://shop.nitrokey.com/shop/product/nkpr2-nitrokey-pro-2-3 .. _`Yubikey 5`: https://www.yubico.com/products/yubikey-5-overview/ -.. _Gnuk: http://www.fsij.org/doc-gnuk/ -.. _`LWN has a good review`: https://lwn.net/Articles/736231/ +.. _Gnuk: https://www.fsij.org/doc-gnuk/ .. _`qualify for a free Nitrokey Start`: https://www.kernel.org/nitrokey-digital-tokens-for-kernel-developers.html Configure your smartcard device @@ -521,11 +480,11 @@ passphrase and the admin PIN of the card for most operations:: Secret subkeys are available. - pub rsa2048/AAAABBBBCCCCDDDD - created: 2018-01-23 expires: 2020-01-23 usage: SC + pub ed25519/AAAABBBBCCCCDDDD + created: 2022-12-20 expires: 2024-12-19 usage: SC trust: ultimate validity: ultimate - ssb rsa2048/1111222233334444 - created: 2018-01-23 expires: never usage: E + ssb cv25519/1111222233334444 + created: 2022-12-20 expires: never usage: E ssb ed25519/5555666677778888 created: 2017-12-07 expires: never usage: S [ultimate] (1). Alice Dev <adev@kernel.org> @@ -589,11 +548,11 @@ If you perform ``--list-secret-keys`` now, you will see a subtle difference in the output:: $ gpg --list-secret-keys - sec# rsa2048 2018-01-24 [SC] [expires: 2020-01-24] + sec# ed25519 2022-12-20 [SC] [expires: 2024-12-19] 000000000000000000000000AAAABBBBCCCCDDDD uid [ultimate] Alice Dev <adev@kernel.org> - ssb> rsa2048 2018-01-24 [E] [expires: 2020-01-24] - ssb> ed25519 2018-01-24 [S] + ssb> cv25519 2022-12-20 [E] [expires: 2024-12-19] + ssb> ed25519 2022-12-20 [S] The ``>`` in the ``ssb>`` output indicates that the subkey is only available on the smartcard. If you go back into your secret keys @@ -627,10 +586,10 @@ Other common GnuPG operations Here is a quick reference for some common operations you'll need to do with your PGP key. -Mounting your master key offline storage -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +Mounting your safe offline storage +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -You will need your master key for any of the operations below, so you +You will need your Certify key for any of the operations below, so you will first need to mount your backup offline storage and tell GnuPG to use it:: @@ -644,7 +603,7 @@ your regular home directory location). Extending key expiration date ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -The master key has the default expiration date of 2 years from the date +The Certify key has the default expiration date of 2 years from the date of creation. This is done both for security reasons and to make obsolete keys eventually disappear from keyservers. @@ -656,7 +615,7 @@ run:: You can also use a specific date if that is easier to remember (e.g. your birthday, January 1st, or Canada Day):: - $ gpg --quick-set-expire [fpr] 2020-07-01 + $ gpg --quick-set-expire [fpr] 2025-07-01 Remember to send the updated key back to keyservers:: @@ -685,6 +644,7 @@ remote end. .. _`Agent Forwarding over SSH`: https://wiki.gnupg.org/AgentForwarding +.. _pgp_with_git: Using PGP with Git ================== @@ -718,12 +678,6 @@ should be used (``[fpr]`` is the fingerprint of your key):: $ git config --global user.signingKey [fpr] -**IMPORTANT**: If you have a distinct ``gpg2`` command, then you should -tell git to always use it instead of the legacy ``gpg`` from version 1:: - - $ git config --global gpg.program gpg2 - $ git config --global gpgv.program gpgv2 - How to work with signed tags ---------------------------- @@ -762,13 +716,6 @@ If you are verifying someone else's git tag, then you will need to import their PGP key. Please refer to the ":ref:`verify_identities`" section below. -.. note:: - - If you get "``gpg: Can't check signature: unknown pubkey - algorithm``" error, you need to tell git to use gpgv2 for - verification, so it properly processes signatures made by ECC keys. - See instructions at the start of this section. - Configure git to always sign annotated tags ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ @@ -828,6 +775,63 @@ You can tell git to always sign commits:: .. _verify_identities: + +How to work with signed patches +------------------------------- + +It is possible to use your PGP key to sign patches sent to kernel +developer mailing lists. Since existing email signature mechanisms +(PGP-Mime or PGP-inline) tend to cause problems with regular code +review tasks, you should use the tool kernel.org created for this +purpose that puts cryptographic attestation signatures into message +headers (a-la DKIM): + +- `Patatt Patch Attestation`_ + +.. _`Patatt Patch Attestation`: https://pypi.org/project/patatt/ + +Installing and configuring patatt +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Patatt is packaged for many distributions already, so please check there +first. You can also install it from pypi using "``pip install patatt``". + +If you already have your PGP key configured with git (via the +``user.signingKey`` configuration parameter), then patatt requires no +further configuration. You can start signing your patches by installing +the git-send-email hook in the repository you want:: + + patatt install-hook + +Now any patches you send with ``git send-email`` will be automatically +signed with your cryptographic signature. + +Checking patatt signatures +~~~~~~~~~~~~~~~~~~~~~~~~~~ + +If you are using ``b4`` to retrieve and apply patches, then it will +automatically attempt to verify all DKIM and patatt signatures it +encounters, for example:: + + $ b4 am 20220720205013.890942-1-broonie@kernel.org + [...] + Checking attestation on all messages, may take a moment... + --- + ✓ [PATCH v1 1/3] kselftest/arm64: Correct buffer allocation for SVE Z registers + ✓ [PATCH v1 2/3] arm64/sve: Document our actual ABI for clearing registers on syscall + ✓ [PATCH v1 3/3] kselftest/arm64: Enforce actual ABI for SVE syscalls + --- + ✓ Signed: openpgp/broonie@kernel.org + ✓ Signed: DKIM/kernel.org + +.. note:: + + Patatt and b4 are still in active development and you should check + the latest documentation for these projects for any new or updated + features. + +.. _kernel_identities: + How to verify kernel developer identities ========================================= @@ -899,69 +903,17 @@ the new default in GnuPG v2). To set it, add (or modify) the trust-model tofu+pgp -How to use keyservers (more) safely ------------------------------------ - -If you get a "No public key" error when trying to validate someone's -tag, then you should attempt to lookup that key using a keyserver. It is -important to keep in mind that there is absolutely no guarantee that the -key you retrieve from PGP keyservers belongs to the actual person -- -that much is by design. You are supposed to use the Web of Trust to -establish key validity. - -How to properly maintain the Web of Trust is beyond the scope of this -document, simply because doing it properly requires both effort and -dedication that tends to be beyond the caring threshold of most human -beings. Here are some shortcuts that will help you reduce the risk of -importing a malicious key. - -First, let's say you've tried to run ``git verify-tag`` but it returned -an error saying the key is not found:: - - $ git verify-tag sunxi-fixes-for-4.15-2 - gpg: Signature made Sun 07 Jan 2018 10:51:55 PM EST - gpg: using RSA key DA73759BF8619E484E5A3B47389A54219C0F2430 - gpg: issuer "wens@...org" - gpg: Can't check signature: No public key - -Let's query the keyserver for more info about that key fingerprint (the -fingerprint probably belongs to a subkey, so we can't use it directly -without finding out the ID of the master key it is associated with):: - - $ gpg --search DA73759BF8619E484E5A3B47389A54219C0F2430 - gpg: data source: hkp://keys.gnupg.net - (1) Chen-Yu Tsai <wens@...org> - 4096 bit RSA key C94035C21B4F2AEB, created: 2017-03-14, expires: 2019-03-15 - Keys 1-1 of 1 for "DA73759BF8619E484E5A3B47389A54219C0F2430". Enter number(s), N)ext, or Q)uit > q - -Locate the ID of the master key in the output, in our example -``C94035C21B4F2AEB``. Now display the key of Linus Torvalds that you -have on your keyring:: - - $ gpg --list-key torvalds@kernel.org - pub rsa2048 2011-09-20 [SC] - ABAF11C65A2970B130ABE3C479BE3E4300411886 - uid [ unknown] Linus Torvalds <torvalds@kernel.org> - sub rsa2048 2011-09-20 [E] - -Next, open the `PGP pathfinder`_. In the "From" field, paste the key -fingerprint of Linus Torvalds from the output above. In the "To" field, -paste the key-id you found via ``gpg --search`` of the unknown key, and -check the results: - -- `Finding paths to Linus`_ - -If you get a few decent trust paths, then it's a pretty good indication -that it is a valid key. You can add it to your keyring from the -keyserver now:: - - $ gpg --recv-key C94035C21B4F2AEB - -This process is not perfect, and you are obviously trusting the -administrators of the PGP Pathfinder service to not be malicious (in -fact, this goes against :ref:`devs_not_infra`). However, if you -do not carefully maintain your own web of trust, then it is a marked -improvement over blindly trusting keyservers. - -.. _`PGP pathfinder`: https://pgp.cs.uu.nl/ -.. _`Finding paths to Linus`: https://pgp.cs.uu.nl/paths/79BE3E4300411886/to/C94035C21B4F2AEB.html +Using the kernel.org web of trust repository +-------------------------------------------- + +Kernel.org maintains a git repository with developers' public keys as a +replacement for replicating keyserver networks that have gone mostly +dark in the past few years. The full documentation for how to set up +that repository as your source of public keys can be found here: + +- `Kernel developer PGP Keyring`_ + +If you are a kernel developer, please consider submitting your key for +inclusion into that keyring. + +.. _`Kernel developer PGP Keyring`: https://korg.docs.kernel.org/pgpkeys.html diff --git a/Documentation/process/maintainer-soc-clean-dts.rst b/Documentation/process/maintainer-soc-clean-dts.rst new file mode 100644 index 000000000000..1b32430d0cfc --- /dev/null +++ b/Documentation/process/maintainer-soc-clean-dts.rst @@ -0,0 +1,25 @@ +.. SPDX-License-Identifier: GPL-2.0 + +============================================== +SoC Platforms with DTS Compliance Requirements +============================================== + +Overview +-------- + +SoC platforms or subarchitectures should follow all the rules from +Documentation/process/maintainer-soc.rst. This document referenced in +MAINTAINERS impose additional requirements listed below. + +Strict DTS DT Schema and dtc Compliance +--------------------------------------- + +No changes to the SoC platform Devicetree sources (DTS files) should introduce +new ``make dtbs_check W=1`` warnings. Warnings in a new board DTS, which are +results of issues in an included DTSI file, are considered existing, not new +warnings. The platform maintainers have automation in place which should point +out any new warnings. + +If a commit introducing new warnings gets accepted somehow, the resulting +issues shall be fixed in reasonable time (e.g. within one release) or the +commit reverted. diff --git a/Documentation/process/maintainer-soc.rst b/Documentation/process/maintainer-soc.rst new file mode 100644 index 000000000000..12637530d68f --- /dev/null +++ b/Documentation/process/maintainer-soc.rst @@ -0,0 +1,177 @@ +.. SPDX-License-Identifier: GPL-2.0 + +============= +SoC Subsystem +============= + +Overview +-------- + +The SoC subsystem is a place of aggregation for SoC-specific code. +The main components of the subsystem are: + +* devicetrees for 32- & 64-bit ARM and RISC-V +* 32-bit ARM board files (arch/arm/mach*) +* 32- & 64-bit ARM defconfigs +* SoC-specific drivers across architectures, in particular for 32- & 64-bit + ARM, RISC-V and Loongarch + +These "SoC-specific drivers" do not include clock, GPIO etc drivers that have +other top-level maintainers. The drivers/soc/ directory is generally meant +for kernel-internal drivers that are used by other drivers to provide SoC- +specific functionality like identifying an SoC revision or interfacing with +power domains. + +The SoC subsystem also serves as an intermediate location for changes to +drivers/bus, drivers/firmware, drivers/reset and drivers/memory. The addition +of new platforms, or the removal of existing ones, often go through the SoC +tree as a dedicated branch covering multiple subsystems. + +The main SoC tree is housed on git.kernel.org: + https://git.kernel.org/pub/scm/linux/kernel/git/soc/soc.git/ + +Clearly this is quite a wide range of topics, which no one person, or even +small group of people are capable of maintaining. Instead, the SoC subsystem +is comprised of many submaintainers, each taking care of individual platforms +and driver subdirectories. +In this regard, "platform" usually refers to a series of SoCs from a given +vendor, for example, Nvidia's series of Tegra SoCs. Many submaintainers operate +on a vendor level, responsible for multiple product lines. For several reasons, +including acquisitions/different business units in a company, things vary +significantly here. The various submaintainers are documented in the +MAINTAINERS file. + +Most of these submaintainers have their own trees where they stage patches, +sending pull requests to the main SoC tree. These trees are usually, but not +always, listed in MAINTAINERS. The main SoC maintainers can be reached via the +alias soc@kernel.org if there is no platform-specific maintainer, or if they +are unresponsive. + +What the SoC tree is not, however, is a location for architecture-specific code +changes. Each architecture has its own maintainers that are responsible for +architectural details, CPU errata and the like. + +Information for (new) Submaintainers +------------------------------------ + +As new platforms spring up, they often bring with them new submaintainers, +many of whom work for the silicon vendor, and may not be familiar with the +process. + +Devicetree ABI Stability +~~~~~~~~~~~~~~~~~~~~~~~~ + +Perhaps one of the most important things to highlight is that dt-bindings +document the ABI between the devicetree and the kernel. +Please read Documentation/devicetree/bindings/ABI.rst. + +If changes are being made to a devicetree that are incompatible with old +kernels, the devicetree patch should not be applied until the driver is, or an +appropriate time later. Most importantly, any incompatible changes should be +clearly pointed out in the patch description and pull request, along with the +expected impact on existing users, such as bootloaders or other operating +systems. + +Driver Branch Dependencies +~~~~~~~~~~~~~~~~~~~~~~~~~~ + +A common problem is synchronizing changes between device drivers and devicetree +files. Even if a change is compatible in both directions, this may require +coordinating how the changes get merged through different maintainer trees. + +Usually the branch that includes a driver change will also include the +corresponding change to the devicetree binding description, to ensure they are +in fact compatible. This means that the devicetree branch can end up causing +warnings in the "make dtbs_check" step. If a devicetree change depends on +missing additions to a header file in include/dt-bindings/, it will fail the +"make dtbs" step and not get merged. + +There are multiple ways to deal with this: + +* Avoid defining custom macros in include/dt-bindings/ for hardware constants + that can be derived from a datasheet -- binding macros in header files should + only be used as a last resort if there is no natural way to define a binding + +* Use literal values in the devicetree file in place of macros even when a + header is required, and change them to the named representation in a + following release + +* Defer the devicetree changes to a release after the binding and driver have + already been merged + +* Change the bindings in a shared immutable branch that is used as the base for + both the driver change and the devicetree changes + +* Add duplicate defines in the devicetree file guarded by an #ifndef section, + removing them in a later release + +Devicetree Naming Convention +~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +The general naming scheme for devicetree files is as follows. The aspects of a +platform that are set at the SoC level, like CPU cores, are contained in a file +named $soc.dtsi, for example, jh7100.dtsi. Integration details, that will vary +from board to board, are described in $soc-$board.dts. An example of this is +jh7100-beaglev-starlight.dts. Often many boards are variations on a theme, and +frequently there are intermediate files, such as jh7100-common.dtsi, which sit +between the $soc.dtsi and $soc-$board.dts files, containing the descriptions of +common hardware. + +Some platforms also have System on Modules, containing an SoC, which are then +integrated into several different boards. For these platforms, $soc-$som.dtsi +and $soc-$som-$board.dts are typical. + +Directories are usually named after the vendor of the SoC at the time of its +inclusion, leading to some historical directory names in the tree. + +Validating Devicetree Files +~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +``make dtbs_check`` can be used to validate that devicetree files are compliant +with the dt-bindings that describe the ABI. Please read the section +"Running checks" of Documentation/devicetree/bindings/writing-schema.rst for +more information on the validation of devicetrees. + +For new platforms, or additions to existing ones, ``make dtbs_check`` should not +add any new warnings. For RISC-V and Samsung SoC, ``make dtbs_check W=1`` is +required to not add any new warnings. +If in any doubt about a devicetree change, reach out to the devicetree +maintainers. + +Branches and Pull Requests +~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Just as the main SoC tree has several branches, it is expected that +submaintainers will do the same. Driver, defconfig and devicetree changes should +all be split into separate branches and appear in separate pull requests to the +SoC maintainers. Each branch should be usable by itself and avoid +regressions that originate from dependencies on other branches. + +Small sets of patches can also be sent as separate emails to soc@kernel.org, +grouped into the same categories. + +If changes do not fit into the normal patterns, there can be additional +top-level branches, e.g. for a treewide rework, or the addition of new SoC +platforms including dts files and drivers. + +Branches with a lot of changes can benefit from getting split up into separate +topics branches, even if they end up getting merged into the same branch of the +SoC tree. An example here would be one branch for devicetree warning fixes, one +for a rework and one for newly added boards. + +Another common way to split up changes is to send an early pull request with the +majority of the changes at some point between rc1 and rc4, following up with one +or more smaller pull requests towards the end of the cycle that can add late +changes or address problems identified while testing the first set. + +While there is no cut-off time for late pull requests, it helps to only send +small branches as time gets closer to the merge window. + +Pull requests for bugfixes for the current release can be sent at any time, but +again having multiple smaller branches is better than trying to combine too many +patches into one pull request. + +The subject line of a pull request should begin with "[GIT PULL]" and made using +a signed tag, rather than a branch. This tag should contain a short description +summarising the changes in the pull request. For more detail on sending pull +requests, please see Documentation/maintainer/pull-requests.rst. diff --git a/Documentation/process/maintainer-tip.rst b/Documentation/process/maintainer-tip.rst new file mode 100644 index 000000000000..08dd0f804410 --- /dev/null +++ b/Documentation/process/maintainer-tip.rst @@ -0,0 +1,804 @@ +.. SPDX-License-Identifier: GPL-2.0 + +The tip tree handbook +===================== + +What is the tip tree? +--------------------- + +The tip tree is a collection of several subsystems and areas of +development. The tip tree is both a direct development tree and a +aggregation tree for several sub-maintainer trees. The tip tree gitweb URL +is: https://git.kernel.org/pub/scm/linux/kernel/git/tip/tip.git + +The tip tree contains the following subsystems: + + - **x86 architecture** + + The x86 architecture development takes place in the tip tree except + for the x86 KVM and XEN specific parts which are maintained in the + corresponding subsystems and routed directly to mainline from + there. It's still good practice to Cc the x86 maintainers on + x86-specific KVM and XEN patches. + + Some x86 subsystems have their own maintainers in addition to the + overall x86 maintainers. Please Cc the overall x86 maintainers on + patches touching files in arch/x86 even when they are not called out + by the MAINTAINER file. + + Note, that ``x86@kernel.org`` is not a mailing list. It is merely a + mail alias which distributes mails to the x86 top-level maintainer + team. Please always Cc the Linux Kernel mailing list (LKML) + ``linux-kernel@vger.kernel.org``, otherwise your mail ends up only in + the private inboxes of the maintainers. + + - **Scheduler** + + Scheduler development takes place in the -tip tree, in the + sched/core branch - with occasional sub-topic trees for + work-in-progress patch-sets. + + - **Locking and atomics** + + Locking development (including atomics and other synchronization + primitives that are connected to locking) takes place in the -tip + tree, in the locking/core branch - with occasional sub-topic trees + for work-in-progress patch-sets. + + - **Generic interrupt subsystem and interrupt chip drivers**: + + - interrupt core development happens in the irq/core branch + + - interrupt chip driver development also happens in the irq/core + branch, but the patches are usually applied in a separate maintainer + tree and then aggregated into irq/core + + - **Time, timers, timekeeping, NOHZ and related chip drivers**: + + - timekeeping, clocksource core, NTP and alarmtimer development + happens in the timers/core branch, but patches are usually applied in + a separate maintainer tree and then aggregated into timers/core + + - clocksource/event driver development happens in the timers/core + branch, but patches are mostly applied in a separate maintainer tree + and then aggregated into timers/core + + - **Performance counters core, architecture support and tooling**: + + - perf core and architecture support development happens in the + perf/core branch + + - perf tooling development happens in the perf tools maintainer + tree and is aggregated into the tip tree. + + - **CPU hotplug core** + + - **RAS core** + + Mostly x86-specific RAS patches are collected in the tip ras/core + branch. + + - **EFI core** + + EFI development in the efi git tree. The collected patches are + aggregated in the tip efi/core branch. + + - **RCU** + + RCU development happens in the linux-rcu tree. The resulting changes + are aggregated into the tip core/rcu branch. + + - **Various core code components**: + + - debugobjects + + - objtool + + - random bits and pieces + + +Patch submission notes +---------------------- + +Selecting the tree/branch +^^^^^^^^^^^^^^^^^^^^^^^^^ + +In general, development against the head of the tip tree master branch is +fine, but for the subsystems which are maintained separately, have their +own git tree and are only aggregated into the tip tree, development should +take place against the relevant subsystem tree or branch. + +Bug fixes which target mainline should always be applicable against the +mainline kernel tree. Potential conflicts against changes which are already +queued in the tip tree are handled by the maintainers. + +Patch subject +^^^^^^^^^^^^^ + +The tip tree preferred format for patch subject prefixes is +'subsys/component:', e.g. 'x86/apic:', 'x86/mm/fault:', 'sched/fair:', +'genirq/core:'. Please do not use file names or complete file paths as +prefix. 'git log path/to/file' should give you a reasonable hint in most +cases. + +The condensed patch description in the subject line should start with a +uppercase letter and should be written in imperative tone. + + +Changelog +^^^^^^^^^ + +The general rules about changelogs in the :ref:`Submitting patches guide +<describe_changes>`, apply. + +The tip tree maintainers set value on following these rules, especially on +the request to write changelogs in imperative mood and not impersonating +code or the execution of it. This is not just a whim of the +maintainers. Changelogs written in abstract words are more precise and +tend to be less confusing than those written in the form of novels. + +It's also useful to structure the changelog into several paragraphs and not +lump everything together into a single one. A good structure is to explain +the context, the problem and the solution in separate paragraphs and this +order. + +Examples for illustration: + + Example 1:: + + x86/intel_rdt/mbm: Fix MBM overflow handler during hot cpu + + When a CPU is dying, we cancel the worker and schedule a new worker on a + different CPU on the same domain. But if the timer is already about to + expire (say 0.99s) then we essentially double the interval. + + We modify the hot cpu handling to cancel the delayed work on the dying + cpu and run the worker immediately on a different cpu in same domain. We + donot flush the worker because the MBM overflow worker reschedules the + worker on same CPU and scans the domain->cpu_mask to get the domain + pointer. + + Improved version:: + + x86/intel_rdt/mbm: Fix MBM overflow handler during CPU hotplug + + When a CPU is dying, the overflow worker is canceled and rescheduled on a + different CPU in the same domain. But if the timer is already about to + expire this essentially doubles the interval which might result in a non + detected overflow. + + Cancel the overflow worker and reschedule it immediately on a different CPU + in the same domain. The work could be flushed as well, but that would + reschedule it on the same CPU. + + Example 2:: + + time: POSIX CPU timers: Ensure that variable is initialized + + If cpu_timer_sample_group returns -EINVAL, it will not have written into + *sample. Checking for cpu_timer_sample_group's return value precludes the + potential use of an uninitialized value of now in the following block. + Given an invalid clock_idx, the previous code could otherwise overwrite + *oldval in an undefined manner. This is now prevented. We also exploit + short-circuiting of && to sample the timer only if the result will + actually be used to update *oldval. + + Improved version:: + + posix-cpu-timers: Make set_process_cpu_timer() more robust + + Because the return value of cpu_timer_sample_group() is not checked, + compilers and static checkers can legitimately warn about a potential use + of the uninitialized variable 'now'. This is not a runtime issue as all + call sites hand in valid clock ids. + + Also cpu_timer_sample_group() is invoked unconditionally even when the + result is not used because *oldval is NULL. + + Make the invocation conditional and check the return value. + + Example 3:: + + The entity can also be used for other purposes. + + Let's rename it to be more generic. + + Improved version:: + + The entity can also be used for other purposes. + + Rename it to be more generic. + + +For complex scenarios, especially race conditions and memory ordering +issues, it is valuable to depict the scenario with a table which shows +the parallelism and the temporal order of events. Here is an example:: + + CPU0 CPU1 + free_irq(X) interrupt X + spin_lock(desc->lock) + wake irq thread() + spin_unlock(desc->lock) + spin_lock(desc->lock) + remove action() + shutdown_irq() + release_resources() thread_handler() + spin_unlock(desc->lock) access released resources. + ^^^^^^^^^^^^^^^^^^^^^^^^^ + synchronize_irq() + +Lockdep provides similar useful output to depict a possible deadlock +scenario:: + + CPU0 CPU1 + rtmutex_lock(&rcu->rt_mutex) + spin_lock(&rcu->rt_mutex.wait_lock) + local_irq_disable() + spin_lock(&timer->it_lock) + spin_lock(&rcu->mutex.wait_lock) + --> Interrupt + spin_lock(&timer->it_lock) + + +Function references in changelogs +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +When a function is mentioned in the changelog, either the text body or the +subject line, please use the format 'function_name()'. Omitting the +brackets after the function name can be ambiguous:: + + Subject: subsys/component: Make reservation_count static + + reservation_count is only used in reservation_stats. Make it static. + +The variant with brackets is more precise:: + + Subject: subsys/component: Make reservation_count() static + + reservation_count() is only called from reservation_stats(). Make it + static. + + +Backtraces in changelogs +^^^^^^^^^^^^^^^^^^^^^^^^ + +See :ref:`backtraces`. + +Ordering of commit tags +^^^^^^^^^^^^^^^^^^^^^^^ + +To have a uniform view of the commit tags, the tip maintainers use the +following tag ordering scheme: + + - Fixes: 12char-SHA1 ("sub/sys: Original subject line") + + A Fixes tag should be added even for changes which do not need to be + backported to stable kernels, i.e. when addressing a recently introduced + issue which only affects tip or the current head of mainline. These tags + are helpful to identify the original commit and are much more valuable + than prominently mentioning the commit which introduced a problem in the + text of the changelog itself because they can be automatically + extracted. + + The following example illustrates the difference:: + + Commit + + abcdef012345678 ("x86/xxx: Replace foo with bar") + + left an unused instance of variable foo around. Remove it. + + Signed-off-by: J.Dev <j.dev@mail> + + Please say instead:: + + The recent replacement of foo with bar left an unused instance of + variable foo around. Remove it. + + Fixes: abcdef012345678 ("x86/xxx: Replace foo with bar") + Signed-off-by: J.Dev <j.dev@mail> + + The latter puts the information about the patch into the focus and + amends it with the reference to the commit which introduced the issue + rather than putting the focus on the original commit in the first place. + + - Reported-by: ``Reporter <reporter@mail>`` + + - Originally-by: ``Original author <original-author@mail>`` + + - Suggested-by: ``Suggester <suggester@mail>`` + + - Co-developed-by: ``Co-author <co-author@mail>`` + + Signed-off: ``Co-author <co-author@mail>`` + + Note, that Co-developed-by and Signed-off-by of the co-author(s) must + come in pairs. + + - Signed-off-by: ``Author <author@mail>`` + + The first Signed-off-by (SOB) after the last Co-developed-by/SOB pair is the + author SOB, i.e. the person flagged as author by git. + + - Signed-off-by: ``Patch handler <handler@mail>`` + + SOBs after the author SOB are from people handling and transporting + the patch, but were not involved in development. SOB chains should + reflect the **real** route a patch took as it was propagated to us, + with the first SOB entry signalling primary authorship of a single + author. Acks should be given as Acked-by lines and review approvals + as Reviewed-by lines. + + If the handler made modifications to the patch or the changelog, then + this should be mentioned **after** the changelog text and **above** + all commit tags in the following format:: + + ... changelog text ends. + + [ handler: Replaced foo by bar and updated changelog ] + + First-tag: ..... + + Note the two empty new lines which separate the changelog text and the + commit tags from that notice. + + If a patch is sent to the mailing list by a handler then the author has + to be noted in the first line of the changelog with:: + + From: Author <author@mail> + + Changelog text starts here.... + + so the authorship is preserved. The 'From:' line has to be followed + by a empty newline. If that 'From:' line is missing, then the patch + would be attributed to the person who sent (transported, handled) it. + The 'From:' line is automatically removed when the patch is applied + and does not show up in the final git changelog. It merely affects + the authorship information of the resulting Git commit. + + - Tested-by: ``Tester <tester@mail>`` + + - Reviewed-by: ``Reviewer <reviewer@mail>`` + + - Acked-by: ``Acker <acker@mail>`` + + - Cc: ``cc-ed-person <person@mail>`` + + If the patch should be backported to stable, then please add a '``Cc: + stable@vger.kernel.org``' tag, but do not Cc stable when sending your + mail. + + - Link: ``https://link/to/information`` + + For referring to an email on LKML or other kernel mailing lists, + please use the lore.kernel.org redirector URL:: + + https://lore.kernel.org/r/email-message@id + + The kernel.org redirector is considered a stable URL, unlike other email + archives. + + Maintainers will add a Link tag referencing the email of the patch + submission when they apply a patch to the tip tree. This tag is useful + for later reference and is also used for commit notifications. + +Please do not use combined tags, e.g. ``Reported-and-tested-by``, as +they just complicate automated extraction of tags. + + +Links to documentation +^^^^^^^^^^^^^^^^^^^^^^ + +Providing links to documentation in the changelog is a great help to later +debugging and analysis. Unfortunately, URLs often break very quickly +because companies restructure their websites frequently. Non-'volatile' +exceptions include the Intel SDM and the AMD APM. + +Therefore, for 'volatile' documents, please create an entry in the kernel +bugzilla https://bugzilla.kernel.org and attach a copy of these documents +to the bugzilla entry. Finally, provide the URL of the bugzilla entry in +the changelog. + +Patch resend or reminders +^^^^^^^^^^^^^^^^^^^^^^^^^ + +See :ref:`resend_reminders`. + +Merge window +^^^^^^^^^^^^ + +Please do not expect large patch series to be handled during the merge +window or even during the week before. Such patches should be submitted in +mergeable state *at* *least* a week before the merge window opens. +Exceptions are made for bug fixes and *sometimes* for small standalone +drivers for new hardware or minimally invasive patches for hardware +enablement. + +During the merge window, the maintainers instead focus on following the +upstream changes, fixing merge window fallout, collecting bug fixes, and +allowing themselves a breath. Please respect that. + +The release candidate -rc1 is the starting point for new patches to be +applied which are targeted for the next merge window. + +So called _urgent_ branches will be merged into mainline during the +stabilization phase of each release. + + +Git +^^^ + +The tip maintainers accept git pull requests from maintainers who provide +subsystem changes for aggregation in the tip tree. + +Pull requests for new patch submissions are usually not accepted and do not +replace proper patch submission to the mailing list. The main reason for +this is that the review workflow is email based. + +If you submit a larger patch series it is helpful to provide a git branch +in a private repository which allows interested people to easily pull the +series for testing. The usual way to offer this is a git URL in the cover +letter of the patch series. + +Testing +^^^^^^^ + +Code should be tested before submitting to the tip maintainers. Anything +other than minor changes should be built, booted and tested with +comprehensive (and heavyweight) kernel debugging options enabled. + +These debugging options can be found in kernel/configs/x86_debug.config +and can be added to an existing kernel config by running: + + make x86_debug.config + +Some of these options are x86-specific and can be left out when testing +on other architectures. + +.. _maintainer-tip-coding-style: + +Coding style notes +------------------ + +Comment style +^^^^^^^^^^^^^ + +Sentences in comments start with an uppercase letter. + +Single line comments:: + + /* This is a single line comment */ + +Multi-line comments:: + + /* + * This is a properly formatted + * multi-line comment. + * + * Larger multi-line comments should be split into paragraphs. + */ + +No tail comments: + + Please refrain from using tail comments. Tail comments disturb the + reading flow in almost all contexts, but especially in code:: + + if (somecondition_is_true) /* Don't put a comment here */ + dostuff(); /* Neither here */ + + seed = MAGIC_CONSTANT; /* Nor here */ + + Use freestanding comments instead:: + + /* This condition is not obvious without a comment */ + if (somecondition_is_true) { + /* This really needs to be documented */ + dostuff(); + } + + /* This magic initialization needs a comment. Maybe not? */ + seed = MAGIC_CONSTANT; + +Comment the important things: + + Comments should be added where the operation is not obvious. Documenting + the obvious is just a distraction:: + + /* Decrement refcount and check for zero */ + if (refcount_dec_and_test(&p->refcnt)) { + do; + lots; + of; + magic; + things; + } + + Instead, comments should explain the non-obvious details and document + constraints:: + + if (refcount_dec_and_test(&p->refcnt)) { + /* + * Really good explanation why the magic things below + * need to be done, ordering and locking constraints, + * etc.. + */ + do; + lots; + of; + magic; + /* Needs to be the last operation because ... */ + things; + } + +Function documentation comments: + + To document functions and their arguments please use kernel-doc format + and not free form comments:: + + /** + * magic_function - Do lots of magic stuff + * @magic: Pointer to the magic data to operate on + * @offset: Offset in the data array of @magic + * + * Deep explanation of mysterious things done with @magic along + * with documentation of the return values. + * + * Note, that the argument descriptors above are arranged + * in a tabular fashion. + */ + + This applies especially to globally visible functions and inline + functions in public header files. It might be overkill to use kernel-doc + format for every (static) function which needs a tiny explanation. The + usage of descriptive function names often replaces these tiny comments. + Apply common sense as always. + + +Documenting locking requirements +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + Documenting locking requirements is a good thing, but comments are not + necessarily the best choice. Instead of writing:: + + /* Caller must hold foo->lock */ + void func(struct foo *foo) + { + ... + } + + Please use:: + + void func(struct foo *foo) + { + lockdep_assert_held(&foo->lock); + ... + } + + In PROVE_LOCKING kernels, lockdep_assert_held() emits a warning + if the caller doesn't hold the lock. Comments can't do that. + +Bracket rules +^^^^^^^^^^^^^ + +Brackets should be omitted only if the statement which follows 'if', 'for', +'while' etc. is truly a single line:: + + if (foo) + do_something(); + +The following is not considered to be a single line statement even +though C does not require brackets:: + + for (i = 0; i < end; i++) + if (foo[i]) + do_something(foo[i]); + +Adding brackets around the outer loop enhances the reading flow:: + + for (i = 0; i < end; i++) { + if (foo[i]) + do_something(foo[i]); + } + + +Variable declarations +^^^^^^^^^^^^^^^^^^^^^ + +The preferred ordering of variable declarations at the beginning of a +function is reverse fir tree order:: + + struct long_struct_name *descriptive_name; + unsigned long foo, bar; + unsigned int tmp; + int ret; + +The above is faster to parse than the reverse ordering:: + + int ret; + unsigned int tmp; + unsigned long foo, bar; + struct long_struct_name *descriptive_name; + +And even more so than random ordering:: + + unsigned long foo, bar; + int ret; + struct long_struct_name *descriptive_name; + unsigned int tmp; + +Also please try to aggregate variables of the same type into a single +line. There is no point in wasting screen space:: + + unsigned long a; + unsigned long b; + unsigned long c; + unsigned long d; + +It's really sufficient to do:: + + unsigned long a, b, c, d; + +Please also refrain from introducing line splits in variable declarations:: + + struct long_struct_name *descriptive_name = container_of(bar, + struct long_struct_name, + member); + struct foobar foo; + +It's way better to move the initialization to a separate line after the +declarations:: + + struct long_struct_name *descriptive_name; + struct foobar foo; + + descriptive_name = container_of(bar, struct long_struct_name, member); + + +Variable types +^^^^^^^^^^^^^^ + +Please use the proper u8, u16, u32, u64 types for variables which are meant +to describe hardware or are used as arguments for functions which access +hardware. These types are clearly defining the bit width and avoid +truncation, expansion and 32/64-bit confusion. + +u64 is also recommended in code which would become ambiguous for 32-bit +kernels when 'unsigned long' would be used instead. While in such +situations 'unsigned long long' could be used as well, u64 is shorter +and also clearly shows that the operation is required to be 64 bits wide +independent of the target CPU. + +Please use 'unsigned int' instead of 'unsigned'. + + +Constants +^^^^^^^^^ + +Please do not use literal (hexa)decimal numbers in code or initializers. +Either use proper defines which have descriptive names or consider using +an enum. + + +Struct declarations and initializers +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +Struct declarations should align the struct member names in a tabular +fashion:: + + struct bar_order { + unsigned int guest_id; + int ordered_item; + struct menu *menu; + }; + +Please avoid documenting struct members within the declaration, because +this often results in strangely formatted comments and the struct members +become obfuscated:: + + struct bar_order { + unsigned int guest_id; /* Unique guest id */ + int ordered_item; + /* Pointer to a menu instance which contains all the drinks */ + struct menu *menu; + }; + +Instead, please consider using the kernel-doc format in a comment preceding +the struct declaration, which is easier to read and has the added advantage +of including the information in the kernel documentation, for example, as +follows:: + + + /** + * struct bar_order - Description of a bar order + * @guest_id: Unique guest id + * @ordered_item: The item number from the menu + * @menu: Pointer to the menu from which the item + * was ordered + * + * Supplementary information for using the struct. + * + * Note, that the struct member descriptors above are arranged + * in a tabular fashion. + */ + struct bar_order { + unsigned int guest_id; + int ordered_item; + struct menu *menu; + }; + +Static struct initializers must use C99 initializers and should also be +aligned in a tabular fashion:: + + static struct foo statfoo = { + .a = 0, + .plain_integer = CONSTANT_DEFINE_OR_ENUM, + .bar = &statbar, + }; + +Note that while C99 syntax allows the omission of the final comma, +we recommend the use of a comma on the last line because it makes +reordering and addition of new lines easier, and makes such future +patches slightly easier to read as well. + +Line breaks +^^^^^^^^^^^ + +Restricting line length to 80 characters makes deeply indented code hard to +read. Consider breaking out code into helper functions to avoid excessive +line breaking. + +The 80 character rule is not a strict rule, so please use common sense when +breaking lines. Especially format strings should never be broken up. + +When splitting function declarations or function calls, then please align +the first argument in the second line with the first argument in the first +line:: + + static int long_function_name(struct foobar *barfoo, unsigned int id, + unsigned int offset) + { + + if (!id) { + ret = longer_function_name(barfoo, DEFAULT_BARFOO_ID, + offset); + ... + +Namespaces +^^^^^^^^^^ + +Function/variable namespaces improve readability and allow easy +grepping. These namespaces are string prefixes for globally visible +function and variable names, including inlines. These prefixes should +combine the subsystem and the component name such as 'x86_comp\_', +'sched\_', 'irq\_', and 'mutex\_'. + +This also includes static file scope functions that are immediately put +into globally visible driver templates - it's useful for those symbols +to carry a good prefix as well, for backtrace readability. + +Namespace prefixes may be omitted for local static functions and +variables. Truly local functions, only called by other local functions, +can have shorter descriptive names - our primary concern is greppability +and backtrace readability. + +Please note that 'xxx_vendor\_' and 'vendor_xxx_` prefixes are not +helpful for static functions in vendor-specific files. After all, it +is already clear that the code is vendor-specific. In addition, vendor +names should only be for truly vendor-specific functionality. + +As always apply common sense and aim for consistency and readability. + + +Commit notifications +-------------------- + +The tip tree is monitored by a bot for new commits. The bot sends an email +for each new commit to a dedicated mailing list +(``linux-tip-commits@vger.kernel.org``) and Cc's all people who are +mentioned in one of the commit tags. It uses the email message ID from the +Link tag at the end of the tag list to set the In-Reply-To email header so +the message is properly threaded with the patch submission email. + +The tip maintainers and submaintainers try to reply to the submitter +when merging a patch, but they sometimes forget or it does not fit the +workflow of the moment. While the bot message is purely mechanical, it +also implies a 'Thank you! Applied.'. diff --git a/Documentation/process/management-style.rst b/Documentation/process/management-style.rst index 186753ff3d2d..dfbc69bf49d4 100644 --- a/Documentation/process/management-style.rst +++ b/Documentation/process/management-style.rst @@ -227,7 +227,7 @@ incompetence will grudgingly admit that you at least didn't try to weasel out of it. Then make the developer who really screwed up (if you can find them) know -**in_private** that they screwed up. Not just so they can avoid it in the +**in private** that they screwed up. Not just so they can avoid it in the future, but so that they know they owe you one. And, perhaps even more importantly, they're also likely the person who can fix it. Because, let's face it, it sure ain't you. diff --git a/Documentation/process/programming-language.rst b/Documentation/process/programming-language.rst index e5f5f065dc24..bc56dee6d0bc 100644 --- a/Documentation/process/programming-language.rst +++ b/Documentation/process/programming-language.rst @@ -5,16 +5,13 @@ Programming Language The kernel is written in the C programming language [c-language]_. More precisely, the kernel is typically compiled with ``gcc`` [gcc]_ -under ``-std=gnu89`` [gcc-c-dialect-options]_: the GNU dialect of ISO C90 -(including some C99 features). +under ``-std=gnu11`` [gcc-c-dialect-options]_: the GNU dialect of ISO C11. +``clang`` [clang]_ is also supported, see docs on +:ref:`Building Linux with Clang/LLVM <kbuild_llvm>`. This dialect contains many extensions to the language [gnu-extensions]_, and many of them are used within the kernel as a matter of course. -There is some support for compiling the kernel with ``clang`` [clang]_ -and ``icc`` [icc]_ for several of the architectures, although at the time -of writing it is not completed, requiring third-party patches. - Attributes ---------- @@ -34,12 +31,28 @@ in order to feature detect which ones can be used and/or to shorten the code. Please refer to ``include/linux/compiler_attributes.h`` for more information. +Rust +---- + +The kernel has experimental support for the Rust programming language +[rust-language]_ under ``CONFIG_RUST``. It is compiled with ``rustc`` [rustc]_ +under ``--edition=2021`` [rust-editions]_. Editions are a way to introduce +small changes to the language that are not backwards compatible. + +On top of that, some unstable features [rust-unstable-features]_ are used in +the kernel. Unstable features may change in the future, thus it is an important +goal to reach a point where only stable features are used. + +Please refer to Documentation/rust/index.rst for more information. + .. [c-language] http://www.open-std.org/jtc1/sc22/wg14/www/standards .. [gcc] https://gcc.gnu.org .. [clang] https://clang.llvm.org -.. [icc] https://software.intel.com/en-us/c-compilers .. [gcc-c-dialect-options] https://gcc.gnu.org/onlinedocs/gcc/C-Dialect-Options.html .. [gnu-extensions] https://gcc.gnu.org/onlinedocs/gcc/C-Extensions.html .. [gcc-attribute-syntax] https://gcc.gnu.org/onlinedocs/gcc/Attribute-Syntax.html .. [n2049] http://www.open-std.org/jtc1/sc22/wg14/www/docs/n2049.pdf - +.. [rust-language] https://www.rust-lang.org +.. [rustc] https://doc.rust-lang.org/rustc/ +.. [rust-editions] https://doc.rust-lang.org/edition-guide/editions/ +.. [rust-unstable-features] https://github.com/Rust-for-Linux/linux/issues/2 diff --git a/Documentation/process/researcher-guidelines.rst b/Documentation/process/researcher-guidelines.rst new file mode 100644 index 000000000000..d159cd4f5e5b --- /dev/null +++ b/Documentation/process/researcher-guidelines.rst @@ -0,0 +1,170 @@ +.. SPDX-License-Identifier: GPL-2.0 + +.. _researcher_guidelines: + +Researcher Guidelines ++++++++++++++++++++++ + +The Linux kernel community welcomes transparent research on the Linux +kernel, the activities involved in producing it, and any other byproducts +of its development. Linux benefits greatly from this kind of research, and +most aspects of Linux are driven by research in one form or another. + +The community greatly appreciates if researchers can share preliminary +findings before making their results public, especially if such research +involves security. Getting involved early helps both improve the quality +of research and ability for Linux to improve from it. In any case, +sharing open access copies of the published research with the community +is recommended. + +This document seeks to clarify what the Linux kernel community considers +acceptable and non-acceptable practices when conducting such research. At +the very least, such research and related activities should follow +standard research ethics rules. For more background on research ethics +generally, ethics in technology, and research of developer communities +in particular, see: + +* `History of Research Ethics <https://www.unlv.edu/research/ORI-HSR/history-ethics>`_ +* `IEEE Ethics <https://www.ieee.org/about/ethics/index.html>`_ +* `Developer and Researcher Views on the Ethics of Experiments on Open-Source Projects <https://arxiv.org/pdf/2112.13217.pdf>`_ + +The Linux kernel community expects that everyone interacting with the +project is participating in good faith to make Linux better. Research on +any publicly-available artifact (including, but not limited to source +code) produced by the Linux kernel community is welcome, though research +on developers must be distinctly opt-in. + +Passive research that is based entirely on publicly available sources, +including posts to public mailing lists and commits to public +repositories, is clearly permissible. Though, as with any research, +standard ethics must still be followed. + +Active research on developer behavior, however, must be done with the +explicit agreement of, and full disclosure to, the individual developers +involved. Developers cannot be interacted with/experimented on without +consent; this, too, is standard research ethics. + +Surveys +======= + +Research often takes the form of surveys sent to maintainers or +contributors. As a general rule, though, the kernel community derives +little value from these surveys. The kernel development process works +because every developer benefits from their participation, even working +with others who have different goals. Responding to a survey, though, is a +one-way demand placed on busy developers with no corresponding benefit to +themselves or to the kernel community as a whole. For this reason, this +method of research is discouraged. + +Kernel community members already receive far too much email and are likely +to perceive survey requests as just another demand on their time. Sending +such requests deprives the community of valuable contributor time and is +unlikely to yield a statistically useful response. + +As an alternative, researchers should consider attending developer events, +hosting sessions where the research project and its benefits to the +participants can be explained, and interacting directly with the community +there. The information received will be far richer than that obtained from +an email survey, and the community will gain from the ability to learn from +your insights as well. + +Patches +======= + +To help clarify: sending patches to developers *is* interacting +with them, but they have already consented to receiving *good faith +contributions*. Sending intentionally flawed/vulnerable patches or +contributing misleading information to discussions is not consented +to. Such communication can be damaging to the developer (e.g. draining +time, effort, and morale) and damaging to the project by eroding +the entire developer community's trust in the contributor (and the +contributor's organization as a whole), undermining efforts to provide +constructive feedback to contributors, and putting end users at risk of +software flaws. + +Participation in the development of Linux itself by researchers, as +with anyone, is welcomed and encouraged. Research into Linux code is +a common practice, especially when it comes to developing or running +analysis tools that produce actionable results. + +When engaging with the developer community, sending a patch has +traditionally been the best way to make an impact. Linux already has +plenty of known bugs -- what's much more helpful is having vetted fixes. +Before contributing, carefully read the appropriate documentation: + +* Documentation/process/development-process.rst +* Documentation/process/submitting-patches.rst +* Documentation/admin-guide/reporting-issues.rst +* Documentation/process/security-bugs.rst + +Then send a patch (including a commit log with all the details listed +below) and follow up on any feedback from other developers. + +When sending patches produced from research, the commit logs should +contain at least the following details, so that developers have +appropriate context for understanding the contribution. Answer: + +* What is the specific problem that has been found? +* How could the problem be reached on a running system? +* What effect would encountering the problem have on the system? +* How was the problem found? Specifically include details about any + testing, static or dynamic analysis programs, and any other tools or + methods used to perform the work. +* Which version of Linux was the problem found on? Using the most recent + release or a recent linux-next branch is strongly preferred (see + Documentation/process/howto.rst). +* What was changed to fix the problem, and why it is believed to be correct? +* How was the change build tested and run-time tested? +* What prior commit does this change fix? This should go in a "Fixes:" + tag as the documentation describes. +* Who else has reviewed this patch? This should go in appropriate + "Reviewed-by:" tags; see below. + +For example:: + + From: Author <author@email> + Subject: [PATCH] drivers/foo_bar: Add missing kfree() + + The error path in foo_bar driver does not correctly free the allocated + struct foo_bar_info. This can happen if the attached foo_bar device + rejects the initialization packets sent during foo_bar_probe(). This + would result in a 64 byte slab memory leak once per device attach, + wasting memory resources over time. + + This flaw was found using an experimental static analysis tool we are + developing, LeakMagic[1], which reported the following warning when + analyzing the v5.15 kernel release: + + path/to/foo_bar.c:187: missing kfree() call? + + Add the missing kfree() to the error path. No other references to + this memory exist outside the probe function, so this is the only + place it can be freed. + + x86_64 and arm64 defconfig builds with CONFIG_FOO_BAR=y using GCC + 11.2 show no new warnings, and LeakMagic no longer warns about this + code path. As we don't have a FooBar device to test with, no runtime + testing was able to be performed. + + [1] https://url/to/leakmagic/details + + Reported-by: Researcher <researcher@email> + Fixes: aaaabbbbccccdddd ("Introduce support for FooBar") + Signed-off-by: Author <author@email> + Reviewed-by: Reviewer <reviewer@email> + +If you are a first time contributor it is recommended that the patch +itself be vetted by others privately before being posted to public lists. +(This is required if you have been explicitly told your patches need +more careful internal review.) These people are expected to have their +"Reviewed-by" tag included in the resulting patch. Finding another +developer familiar with Linux contribution, especially within your own +organization, and having them help with reviews before sending them to +the public mailing lists tends to significantly improve the quality of the +resulting patches, and there by reduces the burden on other developers. + +If no one can be found to internally review patches and you need +help finding such a person, or if you have any other questions +related to this document and the developer community's expectations, +please reach out to the private Technical Advisory Board mailing list: +<tech-board@lists.linux-foundation.org>. diff --git a/Documentation/process/security-bugs.rst b/Documentation/process/security-bugs.rst new file mode 100644 index 000000000000..692a3ba56cca --- /dev/null +++ b/Documentation/process/security-bugs.rst @@ -0,0 +1,110 @@ +.. _securitybugs: + +Security bugs +============= + +Linux kernel developers take security very seriously. As such, we'd +like to know when a security bug is found so that it can be fixed and +disclosed as quickly as possible. Please report security bugs to the +Linux kernel security team. + +Contact +------- + +The Linux kernel security team can be contacted by email at +<security@kernel.org>. This is a private list of security officers +who will help verify the bug report and develop and release a fix. +If you already have a fix, please include it with your report, as +that can speed up the process considerably. It is possible that the +security team will bring in extra help from area maintainers to +understand and fix the security vulnerability. + +As it is with any bug, the more information provided the easier it +will be to diagnose and fix. Please review the procedure outlined in +'Documentation/admin-guide/reporting-issues.rst' if you are unclear about what +information is helpful. Any exploit code is very helpful and will not +be released without consent from the reporter unless it has already been +made public. + +Please send plain text emails without attachments where possible. +It is much harder to have a context-quoted discussion about a complex +issue if all the details are hidden away in attachments. Think of it like a +:doc:`regular patch submission <../process/submitting-patches>` +(even if you don't have a patch yet): describe the problem and impact, list +reproduction steps, and follow it with a proposed fix, all in plain text. + +Disclosure and embargoed information +------------------------------------ + +The security list is not a disclosure channel. For that, see Coordination +below. + +Once a robust fix has been developed, the release process starts. Fixes +for publicly known bugs are released immediately. + +Although our preference is to release fixes for publicly undisclosed bugs +as soon as they become available, this may be postponed at the request of +the reporter or an affected party for up to 7 calendar days from the start +of the release process, with an exceptional extension to 14 calendar days +if it is agreed that the criticality of the bug requires more time. The +only valid reason for deferring the publication of a fix is to accommodate +the logistics of QA and large scale rollouts which require release +coordination. + +While embargoed information may be shared with trusted individuals in +order to develop a fix, such information will not be published alongside +the fix or on any other disclosure channel without the permission of the +reporter. This includes but is not limited to the original bug report +and followup discussions (if any), exploits, CVE information or the +identity of the reporter. + +In other words our only interest is in getting bugs fixed. All other +information submitted to the security list and any followup discussions +of the report are treated confidentially even after the embargo has been +lifted, in perpetuity. + +Coordination with other groups +------------------------------ + +While the kernel security team solely focuses on getting bugs fixed, +other groups focus on fixing issues in distros and coordinating +disclosure between operating system vendors. Coordination is usually +handled by the "linux-distros" mailing list and disclosure by the +public "oss-security" mailing list, both of which are closely related +and presented in the linux-distros wiki: +<https://oss-security.openwall.org/wiki/mailing-lists/distros> + +Please note that the respective policies and rules are different since +the 3 lists pursue different goals. Coordinating between the kernel +security team and other teams is difficult since for the kernel security +team occasional embargoes (as subject to a maximum allowed number of +days) start from the availability of a fix, while for "linux-distros" +they start from the initial post to the list regardless of the +availability of a fix. + +As such, the kernel security team strongly recommends that as a reporter +of a potential security issue you DO NOT contact the "linux-distros" +mailing list UNTIL a fix is accepted by the affected code's maintainers +and you have read the distros wiki page above and you fully understand +the requirements that contacting "linux-distros" will impose on you and +the kernel community. This also means that in general it doesn't make +sense to Cc: both lists at once, except maybe for coordination if and +while an accepted fix has not yet been merged. In other words, until a +fix is accepted do not Cc: "linux-distros", and after it's merged do not +Cc: the kernel security team. + +CVE assignment +-------------- + +The security team does not assign CVEs, nor do we require them for +reports or fixes, as this can needlessly complicate the process and may +delay the bug handling. If a reporter wishes to have a CVE identifier +assigned, they should find one by themselves, for example by contacting +MITRE directly. However under no circumstances will a patch inclusion +be delayed to wait for a CVE identifier to arrive. + +Non-disclosure agreements +------------------------- + +The Linux kernel security team is not a formal body and therefore unable +to enter any non-disclosure agreements. diff --git a/Documentation/process/stable-kernel-rules.rst b/Documentation/process/stable-kernel-rules.rst index 06f743b612c4..1704f1c686d0 100644 --- a/Documentation/process/stable-kernel-rules.rst +++ b/Documentation/process/stable-kernel-rules.rst @@ -6,144 +6,189 @@ Everything you ever wanted to know about Linux -stable releases Rules on what kind of patches are accepted, and which ones are not, into the "-stable" tree: + - It or an equivalent fix must already exist in Linus' tree (upstream). - It must be obviously correct and tested. - It cannot be bigger than 100 lines, with context. - - It must fix only one thing. - - It must fix a real bug that bothers people (not a, "This could be a - problem..." type thing). - - It must fix a problem that causes a build error (but not for things - marked CONFIG_BROKEN), an oops, a hang, data corruption, a real - security issue, or some "oh, that's not good" issue. In short, something - critical. - - Serious issues as reported by a user of a distribution kernel may also - be considered if they fix a notable performance or interactivity issue. - As these fixes are not as obvious and have a higher risk of a subtle - regression they should only be submitted by a distribution kernel - maintainer and include an addendum linking to a bugzilla entry if it - exists and additional information on the user-visible impact. - - New device IDs and quirks are also accepted. - - No "theoretical race condition" issues, unless an explanation of how the - race can be exploited is also provided. - - It cannot contain any "trivial" fixes in it (spelling changes, - whitespace cleanups, etc). - It must follow the :ref:`Documentation/process/submitting-patches.rst <submittingpatches>` rules. - - It or an equivalent fix must already exist in Linus' tree (upstream). + - It must either fix a real bug that bothers people or just add a device ID. + To elaborate on the former: + + - It fixes a problem like an oops, a hang, data corruption, a real security + issue, a hardware quirk, a build error (but not for things marked + CONFIG_BROKEN), or some "oh, that's not good" issue. + - Serious issues as reported by a user of a distribution kernel may also + be considered if they fix a notable performance or interactivity issue. + As these fixes are not as obvious and have a higher risk of a subtle + regression they should only be submitted by a distribution kernel + maintainer and include an addendum linking to a bugzilla entry if it + exists and additional information on the user-visible impact. + - No "This could be a problem..." type of things like a "theoretical race + condition", unless an explanation of how the bug can be exploited is also + provided. + - No "trivial" fixes without benefit for users (spelling changes, whitespace + cleanups, etc). Procedure for submitting patches to the -stable tree ---------------------------------------------------- - - If the patch covers files in net/ or drivers/net please follow netdev stable - submission guidelines as described in - :ref:`Documentation/networking/netdev-FAQ.rst <netdev-FAQ>` - after first checking the stable networking queue at - https://patchwork.ozlabs.org/bundle/davem/stable/?series=&submitter=&state=*&q=&archive= - to ensure the requested patch is not already queued up. - - Security patches should not be handled (solely) by the -stable review +.. note:: + + Security patches should not be handled (solely) by the -stable review process but should follow the procedures in - :ref:`Documentation/admin-guide/security-bugs.rst <securitybugs>`. + :ref:`Documentation/process/security-bugs.rst <securitybugs>`. + +There are three options to submit a change to -stable trees: + + 1. Add a 'stable tag' to the description of a patch you then submit for + mainline inclusion. + 2. Ask the stable team to pick up a patch already mainlined. + 3. Submit a patch to the stable team that is equivalent to a change already + mainlined. -For all other submissions, choose one of the following procedures ------------------------------------------------------------------ +The sections below describe each of the options in more detail. + +:ref:`option_1` is **strongly** preferred, it is the easiest and most common. +:ref:`option_2` is mainly meant for changes where backporting was not considered +at the time of submission. :ref:`option_3` is an alternative to the two earlier +options for cases where a mainlined patch needs adjustments to apply in older +series (for example due to API changes). + +When using option 2 or 3 you can ask for your change to be included in specific +stable series. When doing so, ensure the fix or an equivalent is applicable, +submitted, or already present in all newer stable trees still supported. This is +meant to prevent regressions that users might later encounter on updating, if +e.g. a fix merged for 5.19-rc1 would be backported to 5.10.y, but not to 5.15.y. .. _option_1: Option 1 ******** -To have the patch automatically included in the stable tree, add the tag +To have a patch you submit for mainline inclusion later automatically picked up +for stable trees, add the tag .. code-block:: none Cc: stable@vger.kernel.org -in the sign-off area. Once the patch is merged it will be applied to -the stable tree without anything else needing to be done by the author -or subsystem maintainer. +in the sign-off area. Once the patch is mainlined it will be applied to the +stable tree without anything else needing to be done by the author or +subsystem maintainer. -.. _option_2: +To sent additional instructions to the stable team, use a shell-style inline +comment: -Option 2 -******** + * To specify any additional patch prerequisites for cherry picking use the + following format in the sign-off area: -After the patch has been merged to Linus' tree, send an email to -stable@vger.kernel.org containing the subject of the patch, the commit ID, -why you think it should be applied, and what kernel version you wish it to -be applied to. + .. code-block:: none -.. _option_3: + Cc: <stable@vger.kernel.org> # 3.3.x: a1f84a3: sched: Check for idle + Cc: <stable@vger.kernel.org> # 3.3.x: 1b9508f: sched: Rate-limit newidle + Cc: <stable@vger.kernel.org> # 3.3.x: fd21073: sched: Fix affinity logic + Cc: <stable@vger.kernel.org> # 3.3.x + Signed-off-by: Ingo Molnar <mingo@elte.hu> -Option 3 -******** + The tag sequence has the meaning of: -Send the patch, after verifying that it follows the above rules, to -stable@vger.kernel.org. You must note the upstream commit ID in the -changelog of your submission, as well as the kernel version you wish -it to be applied to. + .. code-block:: none -:ref:`option_1` is **strongly** preferred, is the easiest and most common. -:ref:`option_2` and :ref:`option_3` are more useful if the patch isn't deemed -worthy at the time it is applied to a public git tree (for instance, because -it deserves more regression testing first). :ref:`option_3` is especially -useful if the patch needs some special handling to apply to an older kernel -(e.g., if API's have changed in the meantime). + git cherry-pick a1f84a3 + git cherry-pick 1b9508f + git cherry-pick fd21073 + git cherry-pick <this commit> -Note that for :ref:`option_3`, if the patch deviates from the original -upstream patch (for example because it had to be backported) this must be very -clearly documented and justified in the patch description. + Note that for a patch series, you do not have to list as prerequisites the + patches present in the series itself. For example, if you have the following + patch series: -The upstream commit ID must be specified with a separate line above the commit -text, like this: + .. code-block:: none -.. code-block:: none + patch1 + patch2 - commit <sha1> upstream. + where patch2 depends on patch1, you do not have to list patch1 as + prerequisite of patch2 if you have already marked patch1 for stable + inclusion. -Additionally, some patches submitted via :ref:`option_1` may have additional -patch prerequisites which can be cherry-picked. This can be specified in the -following format in the sign-off area: + * For patches that may have kernel version prerequisites specify them using + the following format in the sign-off area: -.. code-block:: none + .. code-block:: none - Cc: <stable@vger.kernel.org> # 3.3.x: a1f84a3: sched: Check for idle - Cc: <stable@vger.kernel.org> # 3.3.x: 1b9508f: sched: Rate-limit newidle - Cc: <stable@vger.kernel.org> # 3.3.x: fd21073: sched: Fix affinity logic Cc: <stable@vger.kernel.org> # 3.3.x - Signed-off-by: Ingo Molnar <mingo@elte.hu> -The tag sequence has the meaning of: + The tag has the meaning of: -.. code-block:: none + .. code-block:: none - git cherry-pick a1f84a3 - git cherry-pick 1b9508f - git cherry-pick fd21073 git cherry-pick <this commit> -Also, some patches may have kernel version prerequisites. This can be -specified in the following format in the sign-off area: + For each "-stable" tree starting with the specified version. + + Note, such tagging is unnecessary if the stable team can derive the + appropriate versions from Fixes: tags. + + * To delay pick up of patches, use the following format: + + .. code-block:: none + + Cc: <stable@vger.kernel.org> # after 4 weeks in mainline + + * For any other requests, just add a note to the stable tag. This for example + can be used to point out known problems: + + .. code-block:: none + + Cc: <stable@vger.kernel.org> # see patch description, needs adjustments for <= 6.3 + +.. _option_2: + +Option 2 +******** + +If the patch already has been merged to mainline, send an email to +stable@vger.kernel.org containing the subject of the patch, the commit ID, +why you think it should be applied, and what kernel versions you wish it to +be applied to. + +.. _option_3: + +Option 3 +******** + +Send the patch, after verifying that it follows the above rules, to +stable@vger.kernel.org and mention the kernel versions you wish it to be applied +to. When doing so, you must note the upstream commit ID in the changelog of your +submission with a separate line above the commit text, like this: .. code-block:: none - Cc: <stable@vger.kernel.org> # 3.3.x + commit <sha1> upstream. -The tag has the meaning of: +or alternatively: .. code-block:: none - git cherry-pick <this commit> + [ Upstream commit <sha1> ] + +If the submitted patch deviates from the original upstream patch (for example +because it had to be adjusted for the older API), this must be very clearly +documented and justified in the patch description. + -For each "-stable" tree starting with the specified version. +Following the submission +------------------------ -Following the submission: +The sender will receive an ACK when the patch has been accepted into the +queue, or a NAK if the patch is rejected. This response might take a few +days, according to the schedules of the stable team members. - - The sender will receive an ACK when the patch has been accepted into the - queue, or a NAK if the patch is rejected. This response might take a few - days, according to the developer's schedules. - - If accepted, the patch will be added to the -stable queue, for review by - other developers and by the relevant subsystem maintainer. +If accepted, the patch will be added to the -stable queue, for review by other +developers and by the relevant subsystem maintainer. Review cycle @@ -157,12 +202,22 @@ Review cycle - If the patch is rejected by a member of the committee, or linux-kernel members object to the patch, bringing up issues that the maintainers and members did not realize, the patch will be dropped from the queue. - - At the end of the review cycle, the ACKed patches will be added to the - latest -stable release, and a new -stable release will happen. + - The ACKed patches will be posted again as part of release candidate (-rc) + to be tested by developers and testers. + - Usually only one -rc release is made, however if there are any outstanding + issues, some patches may be modified or dropped or additional patches may + be queued. Additional -rc releases are then released and tested until no + issues are found. + - Responding to the -rc releases can be done on the mailing list by sending + a "Tested-by:" email with any testing information desired. The "Tested-by:" + tags will be collected and added to the release commit. + - At the end of the review cycle, the new -stable release will be released + containing all the queued and tested patches. - Security patches will be accepted into the -stable tree directly from the security kernel team, and not go through the normal review cycle. Contact the kernel security team for more details on this procedure. + Trees ----- @@ -174,7 +229,16 @@ Trees - The finalized and tagged releases of all stable kernels can be found in separate branches per version at: - https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable.git + https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git + + - The release candidate of all stable kernel versions can be found at: + + https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable-rc.git/ + + .. warning:: + The -stable-rc tree is a snapshot in time of the stable-queue tree and + will change frequently, hence will be rebased often. It should only be + used for testing purposes (e.g. to be consumed by CI systems). Review committee diff --git a/Documentation/process/submit-checklist.rst b/Documentation/process/submit-checklist.rst index 8e56337d422d..b1bc2d37bd0a 100644 --- a/Documentation/process/submit-checklist.rst +++ b/Documentation/process/submit-checklist.rst @@ -24,6 +24,10 @@ and elsewhere regarding submitting Linux kernel patches. c) Builds successfully when using ``O=builddir`` + d) Any Documentation/ changes build successfully without new warnings/errors. + Use ``make htmldocs`` or ``make pdfdocs`` to check the build and + fix any issues. + 3) Builds on multiple CPU architectures by using local cross-compile tools or some other build farm. @@ -49,8 +53,7 @@ and elsewhere regarding submitting Linux kernel patches. 9) Check cleanly with sparse. -10) Use ``make checkstack`` and ``make namespacecheck`` and fix any problems - that they find. +10) Use ``make checkstack`` and fix any problems that it finds. .. note:: @@ -72,44 +75,42 @@ and elsewhere regarding submitting Linux kernel patches. 13) Has been build- and runtime tested with and without ``CONFIG_SMP`` and ``CONFIG_PREEMPT.`` -16) All codepaths have been exercised with all lockdep features enabled. +14) All codepaths have been exercised with all lockdep features enabled. -17) All new ``/proc`` entries are documented under ``Documentation/`` +15) All new ``/proc`` entries are documented under ``Documentation/`` -18) All new kernel boot parameters are documented in +16) All new kernel boot parameters are documented in ``Documentation/admin-guide/kernel-parameters.rst``. -19) All new module parameters are documented with ``MODULE_PARM_DESC()`` +17) All new module parameters are documented with ``MODULE_PARM_DESC()`` -20) All new userspace interfaces are documented in ``Documentation/ABI/``. +18) All new userspace interfaces are documented in ``Documentation/ABI/``. See ``Documentation/ABI/README`` for more information. Patches that change userspace interfaces should be CCed to linux-api@vger.kernel.org. -21) Check that it all passes ``make headers_check``. - -22) Has been checked with injection of at least slab and page-allocation +19) Has been checked with injection of at least slab and page-allocation failures. See ``Documentation/fault-injection/``. If the new code is substantial, addition of subsystem-specific fault injection might be appropriate. -23) Newly-added code has been compiled with ``gcc -W`` (use - ``make EXTRA_CFLAGS=-W``). This will generate lots of noise, but is good +20) Newly-added code has been compiled with ``gcc -W`` (use + ``make KCFLAGS=-W``). This will generate lots of noise, but is good for finding bugs like "warning: comparison between signed and unsigned". -24) Tested after it has been merged into the -mm patchset to make sure +21) Tested after it has been merged into the -mm patchset to make sure that it still works with all of the other queued patches and various changes in the VM, VFS, and other subsystems. -25) All memory barriers {e.g., ``barrier()``, ``rmb()``, ``wmb()``} need a +22) All memory barriers {e.g., ``barrier()``, ``rmb()``, ``wmb()``} need a comment in the source code that explains the logic of what they are doing and why. -26) If any ioctl's are added by the patch, then also update - ``Documentation/ioctl/ioctl-number.rst``. +23) If any ioctl's are added by the patch, then also update + ``Documentation/userspace-api/ioctl/ioctl-number.rst``. -27) If your modified source code depends on or uses any of the kernel +24) If your modified source code depends on or uses any of the kernel APIs or features that are related to the following ``Kconfig`` symbols, then test multiple builds with the related ``Kconfig`` symbols disabled and/or ``=m`` (if that option is available) [not all of these at the diff --git a/Documentation/process/submitting-drivers.rst b/Documentation/process/submitting-drivers.rst deleted file mode 100644 index 1acaa14903d6..000000000000 --- a/Documentation/process/submitting-drivers.rst +++ /dev/null @@ -1,193 +0,0 @@ -.. _submittingdrivers: - -Submitting Drivers For The Linux Kernel -======================================= - -This document is intended to explain how to submit device drivers to the -various kernel trees. Note that if you are interested in video card drivers -you should probably talk to XFree86 (http://www.xfree86.org/) and/or X.Org -(http://x.org/) instead. - -.. note:: - - This document is old and has seen little maintenance in recent years; it - should probably be updated or, perhaps better, just deleted. Most of - what is here can be found in the other development documents anyway. - - Oh, and we don't really recommend submitting changes to XFree86 :) - -Also read the :ref:`Documentation/process/submitting-patches.rst <submittingpatches>` -document. - - -Allocating Device Numbers -------------------------- - -Major and minor numbers for block and character devices are allocated -by the Linux assigned name and number authority (currently this is -Torben Mathiasen). The site is http://www.lanana.org/. This -also deals with allocating numbers for devices that are not going to -be submitted to the mainstream kernel. -See :ref:`Documentation/admin-guide/devices.rst <admin_devices>` -for more information on this. - -If you don't use assigned numbers then when your device is submitted it will -be given an assigned number even if that is different from values you may -have shipped to customers before. - -Who To Submit Drivers To ------------------------- - -Linux 2.0: - No new drivers are accepted for this kernel tree. - -Linux 2.2: - No new drivers are accepted for this kernel tree. - -Linux 2.4: - If the code area has a general maintainer then please submit it to - the maintainer listed in MAINTAINERS in the kernel file. If the - maintainer does not respond or you cannot find the appropriate - maintainer then please contact Willy Tarreau <w@1wt.eu>. - -Linux 2.6 and upper: - The same rules apply as 2.4 except that you should follow linux-kernel - to track changes in API's. The final contact point for Linux 2.6+ - submissions is Andrew Morton. - -What Criteria Determine Acceptance ----------------------------------- - -Licensing: - The code must be released to us under the - GNU General Public License. We don't insist on any kind - of exclusive GPL licensing, and if you wish the driver - to be useful to other communities such as BSD you may well - wish to release under multiple licenses. - See accepted licenses at include/linux/module.h - -Copyright: - The copyright owner must agree to use of GPL. - It's best if the submitter and copyright owner - are the same person/entity. If not, the name of - the person/entity authorizing use of GPL should be - listed in case it's necessary to verify the will of - the copyright owner. - -Interfaces: - If your driver uses existing interfaces and behaves like - other drivers in the same class it will be much more likely - to be accepted than if it invents gratuitous new ones. - If you need to implement a common API over Linux and NT - drivers do it in userspace. - -Code: - Please use the Linux style of code formatting as documented - in :ref:`Documentation/process/coding-style.rst <codingStyle>`. - If you have sections of code - that need to be in other formats, for example because they - are shared with a windows driver kit and you want to - maintain them just once separate them out nicely and note - this fact. - -Portability: - Pointers are not always 32bits, not all computers are little - endian, people do not all have floating point and you - shouldn't use inline x86 assembler in your driver without - careful thought. Pure x86 drivers generally are not popular. - If you only have x86 hardware it is hard to test portability - but it is easy to make sure the code can easily be made - portable. - -Clarity: - It helps if anyone can see how to fix the driver. It helps - you because you get patches not bug reports. If you submit a - driver that intentionally obfuscates how the hardware works - it will go in the bitbucket. - -PM support: - Since Linux is used on many portable and desktop systems, your - driver is likely to be used on such a system and therefore it - should support basic power management by implementing, if - necessary, the .suspend and .resume methods used during the - system-wide suspend and resume transitions. You should verify - that your driver correctly handles the suspend and resume, but - if you are unable to ensure that, please at least define the - .suspend method returning the -ENOSYS ("Function not - implemented") error. You should also try to make sure that your - driver uses as little power as possible when it's not doing - anything. For the driver testing instructions see - Documentation/power/drivers-testing.rst and for a relatively - complete overview of the power management issues related to - drivers see :ref:`Documentation/driver-api/pm/devices.rst <driverapi_pm_devices>`. - -Control: - In general if there is active maintenance of a driver by - the author then patches will be redirected to them unless - they are totally obvious and without need of checking. - If you want to be the contact and update point for the - driver it is a good idea to state this in the comments, - and include an entry in MAINTAINERS for your driver. - -What Criteria Do Not Determine Acceptance ------------------------------------------ - -Vendor: - Being the hardware vendor and maintaining the driver is - often a good thing. If there is a stable working driver from - other people already in the tree don't expect 'we are the - vendor' to get your driver chosen. Ideally work with the - existing driver author to build a single perfect driver. - -Author: - It doesn't matter if a large Linux company wrote the driver, - or you did. Nobody has any special access to the kernel - tree. Anyone who tells you otherwise isn't telling the - whole story. - - -Resources ---------- - -Linux kernel master tree: - ftp.\ *country_code*\ .kernel.org:/pub/linux/kernel/... - - where *country_code* == your country code, such as - **us**, **uk**, **fr**, etc. - - http://git.kernel.org/?p=linux/kernel/git/torvalds/linux.git - -Linux kernel mailing list: - linux-kernel@vger.kernel.org - [mail majordomo@vger.kernel.org to subscribe] - -Linux Device Drivers, Third Edition (covers 2.6.10): - http://lwn.net/Kernel/LDD3/ (free version) - -LWN.net: - Weekly summary of kernel development activity - http://lwn.net/ - - 2.6 API changes: - - http://lwn.net/Articles/2.6-kernel-api/ - - Porting drivers from prior kernels to 2.6: - - http://lwn.net/Articles/driver-porting/ - -KernelNewbies: - Documentation and assistance for new kernel programmers - - http://kernelnewbies.org/ - -Linux USB project: - http://www.linux-usb.org/ - -How to NOT write kernel driver by Arjan van de Ven: - http://www.fenrus.org/how-to-not-write-a-device-driver-paper.pdf - -Kernel Janitor: - http://kernelnewbies.org/KernelJanitors - -GIT, Fast Version Control System: - http://git-scm.com/ diff --git a/Documentation/process/submitting-patches.rst b/Documentation/process/submitting-patches.rst index ba5e944c7a63..66029999b587 100644 --- a/Documentation/process/submitting-patches.rst +++ b/Documentation/process/submitting-patches.rst @@ -10,22 +10,23 @@ can greatly increase the chances of your change being accepted. This document contains a large number of suggestions in a relatively terse format. For detailed information on how the kernel development process -works, see :ref:`Documentation/process <development_process_main>`. -Also, read :ref:`Documentation/process/submit-checklist.rst <submitchecklist>` -for a list of items to check before -submitting code. If you are submitting a driver, also read -:ref:`Documentation/process/submitting-drivers.rst <submittingdrivers>`; -for device tree binding patches, read -Documentation/devicetree/bindings/submitting-patches.txt. - -Many of these steps describe the default behavior of the ``git`` version -control system; if you use ``git`` to prepare your patches, you'll find much -of the mechanical work done for you, though you'll still need to prepare -and document a sensible set of patches. In general, use of ``git`` will make -your life as a kernel developer easier. - -0) Obtain a current source tree -------------------------------- +works, see Documentation/process/development-process.rst. Also, read +Documentation/process/submit-checklist.rst +for a list of items to check before submitting code. +For device tree binding patches, read +Documentation/devicetree/bindings/submitting-patches.rst. + +This documentation assumes that you're using ``git`` to prepare your patches. +If you're unfamiliar with ``git``, you would be well-advised to learn how to +use it, it will make your life as a kernel developer and in general much +easier. + +Some subsystems and maintainer trees have additional information about +their workflow and expectations, see +:ref:`Documentation/process/maintainer-handbooks.rst <maintainer_handbooks_main>`. + +Obtain a current source tree +---------------------------- If you do not have a repository with the current kernel source handy, use ``git`` to obtain one. You'll want to start with the mainline repository, @@ -39,68 +40,10 @@ patches prepared against those trees. See the **T:** entry for the subsystem in the MAINTAINERS file to find that tree, or simply ask the maintainer if the tree is not listed there. -It is still possible to download kernel releases via tarballs (as described -in the next section), but that is the hard way to do kernel development. - -1) ``diff -up`` ---------------- - -If you must generate your patches by hand, use ``diff -up`` or ``diff -uprN`` -to create patches. Git generates patches in this form by default; if -you're using ``git``, you can skip this section entirely. - -All changes to the Linux kernel occur in the form of patches, as -generated by :manpage:`diff(1)`. When creating your patch, make sure to -create it in "unified diff" format, as supplied by the ``-u`` argument -to :manpage:`diff(1)`. -Also, please use the ``-p`` argument which shows which C function each -change is in - that makes the resultant ``diff`` a lot easier to read. -Patches should be based in the root kernel source directory, -not in any lower subdirectory. - -To create a patch for a single file, it is often sufficient to do:: - - SRCTREE=linux - MYFILE=drivers/net/mydriver.c - - cd $SRCTREE - cp $MYFILE $MYFILE.orig - vi $MYFILE # make your change - cd .. - diff -up $SRCTREE/$MYFILE{.orig,} > /tmp/patch - -To create a patch for multiple files, you should unpack a "vanilla", -or unmodified kernel source tree, and generate a ``diff`` against your -own source tree. For example:: - - MYSRC=/devel/linux - - tar xvfz linux-3.19.tar.gz - mv linux-3.19 linux-3.19-vanilla - diff -uprN -X linux-3.19-vanilla/Documentation/dontdiff \ - linux-3.19-vanilla $MYSRC > /tmp/patch - -``dontdiff`` is a list of files which are generated by the kernel during -the build process, and should be ignored in any :manpage:`diff(1)`-generated -patch. - -Make sure your patch does not include any extra files which do not -belong in a patch submission. Make sure to review your patch -after- -generating it with :manpage:`diff(1)`, to ensure accuracy. - -If your changes produce a lot of deltas, you need to split them into -individual patches which modify things in logical stages; see -:ref:`split_changes`. This will facilitate review by other kernel developers, -very important if you want your patch accepted. - -If you're using ``git``, ``git rebase -i`` can help you with this process. If -you're not using ``git``, ``quilt`` <http://savannah.nongnu.org/projects/quilt> -is another popular alternative. - .. _describe_changes: -2) Describe your changes ------------------------- +Describe your changes +--------------------- Describe your problem. Whether your patch is a one-line bug fix or 5000 lines of a new feature, there must be an underlying problem that @@ -133,7 +76,7 @@ as you intend it to. The maintainer will thank you if you write your patch description in a form which can be easily pulled into Linux's source code management -system, ``git``, as a "commit log". See :ref:`explicit_in_reply_to`. +system, ``git``, as a "commit log". See :ref:`the_canonical_patch_format`. Solve only one problem per patch. If your description starts to get long, that's a sign that you probably need to split up your patch. @@ -153,17 +96,6 @@ instead of "[This patch] makes xyzzy do frotz" or "[I] changed xyzzy to do frotz", as if you are giving orders to the codebase to change its behaviour. -If the patch fixes a logged bug entry, refer to that bug entry by -number and URL. If the patch follows from a mailing list discussion, -give a URL to the mailing list archive; use the https://lkml.kernel.org/ -redirector with a ``Message-Id``, to ensure that the links cannot become -stale. - -However, try to make your explanation understandable without external -resources. In addition to giving a URL to a mailing list archive or -bug, summarize the relevant points of the discussion that led to the -patch as submitted. - If you want to refer to a specific commit, don't just refer to the SHA-1 ID of the commit. Please also include the oneline summary of the commit, to make it easier for reviewers to know what it is about. @@ -180,6 +112,36 @@ collisions with shorter IDs a real possibility. Bear in mind that, even if there is no collision with your six-character ID now, that condition may change five years from now. +If related discussions or any other background information behind the change +can be found on the web, add 'Link:' tags pointing to it. If the patch is a +result of some earlier mailing list discussions or something documented on the +web, point to it. + +When linking to mailing list archives, preferably use the lore.kernel.org +message archiver service. To create the link URL, use the contents of the +``Message-Id`` header of the message without the surrounding angle brackets. +For example:: + + Link: https://lore.kernel.org/r/30th.anniversary.repost@klaava.Helsinki.FI/ + +Please check the link to make sure that it is actually working and points +to the relevant message. + +However, try to make your explanation understandable without external +resources. In addition to giving a URL to a mailing list archive or bug, +summarize the relevant points of the discussion that led to the +patch as submitted. + +In case your patch fixes a bug, use the 'Closes:' tag with a URL referencing +the report in the mailing list archives or a public bug tracker. For example:: + + Closes: https://example.com/issues/1234 + +Some bug trackers have the ability to close issues automatically when a +commit with such a tag is applied. Some bots monitoring mailing lists can +also track such tags and take certain actions. Private bug trackers and +invalid URLs are forbidden. + If your patch fixes a bug in a specific commit, e.g. you found an issue using ``git bisect``, please use the 'Fixes:' tag with the first 12 characters of the SHA-1 ID, and the one line summary. Do not split the tag across multiple @@ -196,10 +158,15 @@ outputting the above style in the ``git log`` or ``git show`` commands:: [pretty] fixes = Fixes: %h (\"%s\") +An example call:: + + $ git log -1 --pretty=fixes 54a4f0239f2e + Fixes: 54a4f0239f2e ("KVM: MMU: make kvm_mmu_zap_page() return the number of pages it actually freed") + .. _split_changes: -3) Separate your changes ------------------------- +Separate your changes +--------------------- Separate each **logical change** into a separate patch. @@ -231,12 +198,11 @@ then only post say 15 or so at a time and wait for review and integration. -4) Style-check your changes ---------------------------- +Style-check your changes +------------------------ Check your patch for basic style violations, details of which can be -found in -:ref:`Documentation/process/coding-style.rst <codingstyle>`. +found in Documentation/process/coding-style.rst. Failure to do so simply wastes the reviewers time and will get your patch rejected, probably without even being read. @@ -262,22 +228,20 @@ You should be able to justify all violations that remain in your patch. -5) Select the recipients for your patch ---------------------------------------- +Select the recipients for your patch +------------------------------------ -You should always copy the appropriate subsystem maintainer(s) on any patch -to code that they maintain; look through the MAINTAINERS file and the -source code revision history to see who those maintainers are. The -script scripts/get_maintainer.pl can be very useful at this step. If you -cannot find a maintainer for the subsystem you are working on, Andrew -Morton (akpm@linux-foundation.org) serves as a maintainer of last resort. +You should always copy the appropriate subsystem maintainer(s) and list(s) on +any patch to code that they maintain; look through the MAINTAINERS file and the +source code revision history to see who those maintainers are. The script +scripts/get_maintainer.pl can be very useful at this step (pass paths to your +patches as arguments to scripts/get_maintainer.pl). If you cannot find a +maintainer for the subsystem you are working on, Andrew Morton +(akpm@linux-foundation.org) serves as a maintainer of last resort. -You should also normally choose at least one mailing list to receive a copy -of your patch set. linux-kernel@vger.kernel.org functions as a list of -last resort, but the volume on that list has caused a number of developers -to tune it out. Look in the MAINTAINERS file for a subsystem-specific -list; your patch will probably get more attention there. Please do not -spam unrelated lists, though. +linux-kernel@vger.kernel.org should be used by default for all patches, but the +volume on that list has caused a number of developers to tune it out. Please +do not spam unrelated lists and unrelated people, though. Many kernel-related lists are hosted on vger.kernel.org; you can find a list of them at http://vger.kernel.org/vger-lists.html. There are @@ -294,7 +258,8 @@ sending him e-mail. If you have a patch that fixes an exploitable security bug, send that patch to security@kernel.org. For severe bugs, a short embargo may be considered to allow distributors to get the patch out to users; in such cases, -obviously, the patch should not be sent to any public lists. +obviously, the patch should not be sent to any public lists. See also +Documentation/process/security-bugs.rst. Patches that fix a severe bug in a released kernel should be directed toward the stable maintainers by putting a line like this:: @@ -302,14 +267,8 @@ toward the stable maintainers by putting a line like this:: Cc: stable@vger.kernel.org into the sign-off area of your patch (note, NOT an email recipient). You -should also read -:ref:`Documentation/process/stable-kernel-rules.rst <stable_kernel_rules>` -in addition to this file. - -Note, however, that some subsystem maintainers want to come to their own -conclusions on which patches should go to the stable trees. The networking -maintainer, in particular, would rather not see individual developers -adding lines like the above to their patches. +should also read Documentation/process/stable-kernel-rules.rst +in addition to this document. If changes affect userland-kernel interfaces, please send the MAN-PAGES maintainer (as listed in the MAINTAINERS file) a man-pages patch, or at @@ -317,35 +276,21 @@ least a notification of the change, so that some information makes its way into the manual pages. User-space API changes should also be copied to linux-api@vger.kernel.org. -For small patches you may want to CC the Trivial Patch Monkey -trivial@kernel.org which collects "trivial" patches. Have a look -into the MAINTAINERS file for its current manager. - -Trivial patches must qualify for one of the following rules: - -- Spelling fixes in documentation -- Spelling fixes for errors which could break :manpage:`grep(1)` -- Warning fixes (cluttering with useless warnings is bad) -- Compilation fixes (only if they are actually correct) -- Runtime fixes (only if they actually fix things) -- Removing use of deprecated functions/macros -- Contact detail and documentation fixes -- Non-portable code replaced by portable code (even in arch-specific, - since people copy, as long as it's trivial) -- Any fix by the author/maintainer of the file (ie. patch monkey - in re-transmission mode) - - -6) No MIME, no links, no compression, no attachments. Just plain text ----------------------------------------------------------------------- +No MIME, no links, no compression, no attachments. Just plain text +------------------------------------------------------------------- Linus and other kernel developers need to be able to read and comment on the changes you are submitting. It is important for a kernel developer to be able to "quote" your changes, using standard e-mail tools, so that they may comment on specific portions of your code. -For this reason, all patches should be submitted by e-mail "inline". +For this reason, all patches should be submitted by e-mail "inline". The +easiest way to do this is with ``git send-email``, which is strongly +recommended. An interactive tutorial for ``git send-email`` is available at +https://git-send-email.io. + +If you choose not to use ``git send-email``: .. warning:: @@ -361,62 +306,97 @@ decreasing the likelihood of your MIME-attached change being accepted. Exception: If your mailer is mangling patches then someone may ask you to re-send them using MIME. -See :ref:`Documentation/process/email-clients.rst <email_clients>` -for hints about configuring your e-mail client so that it sends your patches -untouched. - -7) E-mail size --------------- +See Documentation/process/email-clients.rst for hints about configuring +your e-mail client so that it sends your patches untouched. -Large changes are not appropriate for mailing lists, and some -maintainers. If your patch, uncompressed, exceeds 300 kB in size, -it is preferred that you store your patch on an Internet-accessible -server, and provide instead a URL (link) pointing to your patch. But note -that if your patch exceeds 300 kB, it almost certainly needs to be broken up -anyway. - -8) Respond to review comments ------------------------------ +Respond to review comments +-------------------------- Your patch will almost certainly get comments from reviewers on ways in -which the patch can be improved. You must respond to those comments; -ignoring reviewers is a good way to get ignored in return. Review comments -or questions that do not lead to a code change should almost certainly +which the patch can be improved, in the form of a reply to your email. You must +respond to those comments; ignoring reviewers is a good way to get ignored in +return. You can simply reply to their emails to answer their comments. Review +comments or questions that do not lead to a code change should almost certainly bring about a comment or changelog entry so that the next reviewer better understands what is going on. Be sure to tell the reviewers what changes you are making and to thank them for their time. Code review is a tiring and time-consuming process, and reviewers sometimes get grumpy. Even in that case, though, respond -politely and address the problems they have pointed out. +politely and address the problems they have pointed out. When sending a next +version, add a ``patch changelog`` to the cover letter or to individual patches +explaining difference against previous submission (see +:ref:`the_canonical_patch_format`). +Notify people that commented on your patch about new versions by adding them to +the patches CC list. + +See Documentation/process/email-clients.rst for recommendations on email +clients and mailing list etiquette. +.. _interleaved_replies: -9) Don't get discouraged - or impatient ---------------------------------------- +Use trimmed interleaved replies in email discussions +---------------------------------------------------- +Top-posting is strongly discouraged in Linux kernel development +discussions. Interleaved (or "inline") replies make conversations much +easier to follow. For more details see: +https://en.wikipedia.org/wiki/Posting_style#Interleaved_style + +As is frequently quoted on the mailing list:: + + A: http://en.wikipedia.org/wiki/Top_post + Q: Were do I find info about this thing called top-posting? + A: Because it messes up the order in which people normally read text. + Q: Why is top-posting such a bad thing? + A: Top-posting. + Q: What is the most annoying thing in e-mail? + +Similarly, please trim all unneeded quotations that aren't relevant +to your reply. This makes responses easier to find, and saves time and +space. For more details see: http://daringfireball.net/2007/07/on_top :: + + A: No. + Q: Should I include quotations after my reply? + +.. _resend_reminders: + +Don't get discouraged - or impatient +------------------------------------ After you have submitted your change, be patient and wait. Reviewers are busy people and may not get to your patch right away. Once upon a time, patches used to disappear into the void without comment, but the development process works more smoothly than that now. You should -receive comments within a week or so; if that does not happen, make sure -that you have sent your patches to the right place. Wait for a minimum of -one week before resubmitting or pinging reviewers - possibly longer during -busy times like merge windows. +receive comments within a few weeks (typically 2-3); if that does not +happen, make sure that you have sent your patches to the right place. +Wait for a minimum of one week before resubmitting or pinging reviewers +- possibly longer during busy times like merge windows. + +It's also ok to resend the patch or the patch series after a couple of +weeks with the word "RESEND" added to the subject line:: + [PATCH Vx RESEND] sub/sys: Condensed patch summary -10) Include PATCH in the subject --------------------------------- +Don't add "RESEND" when you are submitting a modified version of your +patch or patch series - "RESEND" only applies to resubmission of a +patch or patch series which have not been modified in any way from the +previous submission. + + +Include PATCH in the subject +----------------------------- Due to high e-mail traffic to Linus, and to linux-kernel, it is common convention to prefix your subject line with [PATCH]. This lets Linus and other kernel developers more easily distinguish patches from other e-mail discussions. +``git send-email`` will do this for you automatically. -11) Sign your work - the Developer's Certificate of Origin ----------------------------------------------------------- +Sign your work - the Developer's Certificate of Origin +------------------------------------------------------ To improve tracking of who did what, especially with patches that can percolate to their final resting place in the kernel through several @@ -459,61 +439,24 @@ then you just add a line saying:: Signed-off-by: Random J Developer <random@developer.example.org> -using your real name (sorry, no pseudonyms or anonymous contributions.) +using a known identity (sorry, no anonymous contributions.) +This will be done for you automatically if you use ``git commit -s``. +Reverts should also include "Signed-off-by". ``git revert -s`` does that +for you. Some people also put extra tags at the end. They'll just be ignored for now, but you can do this to mark internal company procedures or just point out some special detail about the sign-off. -If you are a subsystem or branch maintainer, sometimes you need to slightly -modify patches you receive in order to merge them, because the code is not -exactly the same in your tree and the submitters'. If you stick strictly to -rule (c), you should ask the submitter to rediff, but this is a totally -counter-productive waste of time and energy. Rule (b) allows you to adjust -the code, but then it is very impolite to change one submitter's code and -make him endorse your bugs. To solve this problem, it is recommended that -you add a line between the last Signed-off-by header and yours, indicating -the nature of your changes. While there is nothing mandatory about this, it -seems like prepending the description with your mail and/or name, all -enclosed in square brackets, is noticeable enough to make it obvious that -you are responsible for last-minute changes. Example:: - - Signed-off-by: Random J Developer <random@developer.example.org> - [lucky@maintainer.example.org: struct foo moved from foo.c to foo.h] - Signed-off-by: Lucky K Maintainer <lucky@maintainer.example.org> - -This practice is particularly helpful if you maintain a stable branch and -want at the same time to credit the author, track changes, merge the fix, -and protect the submitter from complaints. Note that under no circumstances -can you change the author's identity (the From header), as it is the one -which appears in the changelog. +Any further SoBs (Signed-off-by:'s) following the author's SoB are from +people handling and transporting the patch, but were not involved in its +development. SoB chains should reflect the **real** route a patch took +as it was propagated to the maintainers and ultimately to Linus, with +the first SoB entry signalling primary authorship of a single author. -Special note to back-porters: It seems to be a common and useful practice -to insert an indication of the origin of a patch at the top of the commit -message (just after the subject line) to facilitate tracking. For instance, -here's what we see in a 3.x-stable release:: - Date: Tue Oct 7 07:26:38 2014 -0400 - - libata: Un-break ATA blacklist - - commit 1c40279960bcd7d52dbdf1d466b20d24b99176c8 upstream. - -And here's what might appear in an older kernel once a patch is backported:: - - Date: Tue May 13 22:12:27 2008 +0200 - - wireless, airo: waitbusy() won't delay - - [backport of 2.6 commit b7acbdfbd1f277c1eb23f344f899cfa4cd0bf36a] - -Whatever the format, this information provides a valuable help to people -tracking your trees, and to people trying to troubleshoot bugs in your -tree. - - -12) When to use Acked-by:, Cc:, and Co-developed-by: -------------------------------------------------------- +When to use Acked-by:, Cc:, and Co-developed-by: +------------------------------------------------ The Signed-off-by: tag indicates that the signer was involved in the development of the patch, or that he/she was in the patch's delivery path. @@ -546,7 +489,7 @@ patch. This tag documents that potentially interested parties have been included in the discussion. Co-developed-by: states that the patch was co-created by multiple developers; -it is a used to give attribution to co-authors (in addition to the author +it is used to give attribution to co-authors (in addition to the author attributed by the From: tag) when several people work on a single patch. Since Co-developed-by: denotes authorship, every Co-developed-by: must be immediately followed by a Signed-off-by: of the associated co-author. Standard sign-off @@ -581,13 +524,17 @@ Example of a patch submitted by a Co-developed-by: author:: Signed-off-by: Submitting Co-Author <sub@coauthor.example.org> -13) Using Reported-by:, Tested-by:, Reviewed-by:, Suggested-by: and Fixes: --------------------------------------------------------------------------- +Using Reported-by:, Tested-by:, Reviewed-by:, Suggested-by: and Fixes: +---------------------------------------------------------------------- The Reported-by tag gives credit to people who find bugs and report them and it -hopefully inspires them to help us again in the future. Please note that if -the bug was reported in private, then ask for permission first before using the -Reported-by tag. +hopefully inspires them to help us again in the future. The tag is intended for +bugs; please do not use it to credit feature requests. The tag should be +followed by a Closes: tag pointing to the report, unless the report is not +available on the web. The Link: tag can be used instead of Closes: if the patch +fixes a part of the issue(s) being reported. Please note that if the bug was +reported in private, then ask for permission first before using the Reported-by +tag. A Tested-by: tag indicates that the patch has been successfully tested (in some environment) by the person named. This tag informs maintainers that @@ -629,6 +576,13 @@ done on the patch. Reviewed-by: tags, when supplied by reviewers known to understand the subject area and to perform thorough reviews, will normally increase the likelihood of your patch getting into the kernel. +Both Tested-by and Reviewed-by tags, once received on mailing list from tester +or reviewer, should be added by author to the applicable patches when sending +next versions. However if the patch has changed substantially in following +version, these tags might not be applicable anymore and thus should be removed. +Usually removal of someone's Tested-by or Reviewed-by tags should be mentioned +in the patch changelog (after the '---' separator). + A Suggested-by: tag indicates that the patch idea is suggested by the person named and ensures credit to the person for the idea. Please note that this tag should not be added without the reporter's permission, especially if the @@ -643,10 +597,15 @@ which stable kernel versions should receive your fix. This is the preferred method for indicating a bug fixed by the patch. See :ref:`describe_changes` for more details. +Note: Attaching a Fixes: tag does not subvert the stable kernel rules +process nor the requirement to Cc: stable@vger.kernel.org on all stable +patch candidates. For more information, please read +Documentation/process/stable-kernel-rules.rst. + .. _the_canonical_patch_format: -14) The canonical patch format ------------------------------- +The canonical patch format +-------------------------- This section describes how the patch itself should be formatted. Note that, if you have your patches stored in a ``git`` repository, proper patch @@ -712,16 +671,19 @@ not considered part of the summary phrase, but describe how the patch should be treated. Common tags might include a version descriptor if the multiple versions of the patch have been sent out in response to comments (i.e., "v1, v2, v3"), or "RFC" to indicate a request for -comments. If there are four patches in a patch series the individual -patches may be numbered like this: 1/4, 2/4, 3/4, 4/4. This assures -that developers understand the order in which the patches should be -applied and that they have reviewed or applied all of the patches in -the patch series. +comments. -A couple of example Subjects:: +If there are four patches in a patch series the individual patches may +be numbered like this: 1/4, 2/4, 3/4, 4/4. This assures that developers +understand the order in which the patches should be applied and that +they have reviewed or applied all of the patches in the patch series. + +Here are some good example Subjects:: Subject: [PATCH 2/5] ext2: improve scalability of bitmap searching Subject: [PATCH v2 01/27] x86: fix eflags tracking + Subject: [PATCH v2] sub/sys: Condensed patch summary + Subject: [PATCH v2 M/N] sub/sys: Condensed patch summary The ``from`` line must be the very first line in the message body, and has the form: @@ -734,42 +696,84 @@ then the ``From:`` line from the email header will be used to determine the patch author in the changelog. The explanation body will be committed to the permanent source -changelog, so should make sense to a competent reader who has long -since forgotten the immediate details of the discussion that might -have led to this patch. Including symptoms of the failure which the -patch addresses (kernel log messages, oops messages, etc.) is -especially useful for people who might be searching the commit logs -looking for the applicable patch. If a patch fixes a compile failure, -it may not be necessary to include _all_ of the compile failures; just -enough that it is likely that someone searching for the patch can find -it. As in the ``summary phrase``, it is important to be both succinct as -well as descriptive. - -The ``---`` marker line serves the essential purpose of marking for patch -handling tools where the changelog message ends. - -One good use for the additional comments after the ``---`` marker is for -a ``diffstat``, to show what files have changed, and the number of -inserted and deleted lines per file. A ``diffstat`` is especially useful -on bigger patches. Other comments relevant only to the moment or the -maintainer, not suitable for the permanent changelog, should also go -here. A good example of such comments might be ``patch changelogs`` -which describe what has changed between the v1 and v2 version of the -patch. - -If you are going to include a ``diffstat`` after the ``---`` marker, please -use ``diffstat`` options ``-p 1 -w 70`` so that filenames are listed from -the top of the kernel source tree and don't use too much horizontal -space (easily fit in 80 columns, maybe with some indentation). (``git`` -generates appropriate diffstats by default.) +changelog, so should make sense to a competent reader who has long since +forgotten the immediate details of the discussion that might have led to +this patch. Including symptoms of the failure which the patch addresses +(kernel log messages, oops messages, etc.) are especially useful for +people who might be searching the commit logs looking for the applicable +patch. The text should be written in such detail so that when read +weeks, months or even years later, it can give the reader the needed +details to grasp the reasoning for **why** the patch was created. + +If a patch fixes a compile failure, it may not be necessary to include +_all_ of the compile failures; just enough that it is likely that +someone searching for the patch can find it. As in the ``summary +phrase``, it is important to be both succinct as well as descriptive. + +The ``---`` marker line serves the essential purpose of marking for +patch handling tools where the changelog message ends. + +One good use for the additional comments after the ``---`` marker is +for a ``diffstat``, to show what files have changed, and the number of +inserted and deleted lines per file. A ``diffstat`` is especially useful +on bigger patches. If you are going to include a ``diffstat`` after the +``---`` marker, please use ``diffstat`` options ``-p 1 -w 70`` so that +filenames are listed from the top of the kernel source tree and don't +use too much horizontal space (easily fit in 80 columns, maybe with some +indentation). (``git`` generates appropriate diffstats by default.) + +Other comments relevant only to the moment or the maintainer, not +suitable for the permanent changelog, should also go here. A good +example of such comments might be ``patch changelogs`` which describe +what has changed between the v1 and v2 version of the patch. + +Please put this information **after** the ``---`` line which separates +the changelog from the rest of the patch. The version information is +not part of the changelog which gets committed to the git tree. It is +additional information for the reviewers. If it's placed above the +commit tags, it needs manual interaction to remove it. If it is below +the separator line, it gets automatically stripped off when applying the +patch:: + + <commit message> + ... + Signed-off-by: Author <author@mail> + --- + V2 -> V3: Removed redundant helper function + V1 -> V2: Cleaned up coding style and addressed review comments + + path/to/file | 5+++-- + ... See more details on the proper patch format in the following references. +.. _backtraces: + +Backtraces in commit messages +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +Backtraces help document the call chain leading to a problem. However, +not all backtraces are helpful. For example, early boot call chains are +unique and obvious. Copying the full dmesg output verbatim, however, +adds distracting information like timestamps, module lists, register and +stack dumps. + +Therefore, the most useful backtraces should distill the relevant +information from the dump, which makes it easier to focus on the real +issue. Here is an example of a well-trimmed backtrace:: + + unchecked MSR access error: WRMSR to 0xd51 (tried to write 0x0000000000000064) + at rIP: 0xffffffffae059994 (native_write_msr+0x4/0x20) + Call Trace: + mba_wrmsr + update_domains + rdtgroup_mkdir + .. _explicit_in_reply_to: -15) Explicit In-Reply-To headers --------------------------------- +Explicit In-Reply-To headers +---------------------------- It can be helpful to manually add In-Reply-To: headers to a patch (e.g., when using ``git send-email``) to associate the patch with @@ -778,18 +782,22 @@ the bug report. However, for a multi-patch series, it is generally best to avoid using In-Reply-To: to link to older versions of the series. This way multiple versions of the patch don't become an unmanageable forest of references in email clients. If a link is -helpful, you can use the https://lkml.kernel.org/ redirector (e.g., in +helpful, you can use the https://lore.kernel.org/ redirector (e.g., in the cover email text) to link to an earlier version of the patch series. -16) Providing base tree information ------------------------------------ +Providing base tree information +------------------------------- When other developers receive your patches and start the review process, -it is often useful for them to know where in the tree history they -should place your work. This is particularly useful for automated CI -processes that attempt to run a series of tests in order to establish -the quality of your submission before the maintainer starts the review. +it is absolutely necessary for them to know what is the base +commit/branch your work applies on, considering the sheer amount of +maintainer trees present nowadays. Note again the **T:** entry in the +MAINTAINERS file explained above. + +This is even more important for automated CI processes that attempt to +run a series of tests in order to establish the quality of your +submission before the maintainer starts the review. If you are using ``git format-patch`` to generate your patches, you can automatically include the base tree information in your submission by @@ -832,67 +840,15 @@ letter or in the first patch of the series and it should be placed either below the ``---`` line or at the very bottom of all other content, right before your email signature. - -17) Sending ``git pull`` requests ---------------------------------- - -If you have a series of patches, it may be most convenient to have the -maintainer pull them directly into the subsystem repository with a -``git pull`` operation. Note, however, that pulling patches from a developer -requires a higher degree of trust than taking patches from a mailing list. -As a result, many subsystem maintainers are reluctant to take pull -requests, especially from new, unknown developers. If in doubt you can use -the pull request as the cover letter for a normal posting of the patch -series, giving the maintainer the option of using either. - -A pull request should have [GIT PULL] in the subject line. The -request itself should include the repository name and the branch of -interest on a single line; it should look something like:: - - Please pull from - - git://jdelvare.pck.nerim.net/jdelvare-2.6 i2c-for-linus - - to get these changes: - -A pull request should also include an overall message saying what will be -included in the request, a ``git shortlog`` listing of the patches -themselves, and a ``diffstat`` showing the overall effect of the patch series. -The easiest way to get all this information together is, of course, to let -``git`` do it for you with the ``git request-pull`` command. - -Some maintainers (including Linus) want to see pull requests from signed -commits; that increases their confidence that the request actually came -from you. Linus, in particular, will not pull from public hosting sites -like GitHub in the absence of a signed tag. - -The first step toward creating such tags is to make a GNUPG key and get it -signed by one or more core kernel developers. This step can be hard for -new developers, but there is no way around it. Attending conferences can -be a good way to find developers who can sign your key. - -Once you have prepared a patch series in ``git`` that you wish to have somebody -pull, create a signed tag with ``git tag -s``. This will create a new tag -identifying the last commit in the series and containing a signature -created with your private key. You will also have the opportunity to add a -changelog-style message to the tag; this is an ideal place to describe the -effects of the pull request as a whole. - -If the tree the maintainer will be pulling from is not the repository you -are working from, don't forget to push the signed tag explicitly to the -public tree. - -When generating your pull request, use the signed tag as the target. A -command like this will do the trick:: - - git request-pull master git://my.public.tree/linux.git my-signed-tag - +Make sure that base commit is in an official maintainer/mainline tree +and not in some internal, accessible only to you tree - otherwise it +would be worthless. References ---------- Andrew Morton, "The perfect patch" (tpp). - <http://www.ozlabs.org/~akpm/stuff/tpp.txt> + <https://www.ozlabs.org/~akpm/stuff/tpp.txt> Jeff Garzik, "Linux kernel patch submission format". <https://web.archive.org/web/20180829112450/http://linux.yyz.us/patch-format.html> @@ -911,13 +867,12 @@ Greg Kroah-Hartman, "How to piss off a kernel subsystem maintainer". <http://www.kroah.com/log/linux/maintainer-06.html> NO!!!! No more huge patch bombs to linux-kernel@vger.kernel.org people! - <https://lkml.org/lkml/2005/7/11/336> + <https://lore.kernel.org/r/20050711.125305.08322243.davem@davemloft.net> -Kernel Documentation/process/coding-style.rst: - :ref:`Documentation/process/coding-style.rst <codingstyle>` +Kernel Documentation/process/coding-style.rst Linus Torvalds's mail on the canonical patch format: - <http://lkml.org/lkml/2005/4/7/183> + <https://lore.kernel.org/r/Pine.LNX.4.58.0504071023190.28951@ppc970.osdl.org> Andi Kleen, "On submitting kernel patches" Some strategies to get difficult or controversial changes in. diff --git a/Documentation/process/volatile-considered-harmful.rst b/Documentation/process/volatile-considered-harmful.rst index 4934e656a6f3..7eb6bd7c9214 100644 --- a/Documentation/process/volatile-considered-harmful.rst +++ b/Documentation/process/volatile-considered-harmful.rst @@ -109,9 +109,9 @@ been properly thought through. References ========== -[1] http://lwn.net/Articles/233481/ +[1] https://lwn.net/Articles/233481/ -[2] http://lwn.net/Articles/233482/ +[2] https://lwn.net/Articles/233482/ Credits ======= |