summaryrefslogtreecommitdiff
path: root/scripts/adjust_autoksyms.sh
AgeCommit message (Collapse)Author
2020-03-03kbuild: split adjust_autoksyms.sh in two partsQuentin Perret
In order to prepare the ground for a build-time optimization, split adjust_autoksyms.sh into two scripts: one that generates autoksyms.h based on all currently available information (whitelist, and .mod files), and the other to inspect the diff between two versions of autoksyms.h and trigger appropriate rebuilds. Acked-by: Nicolas Pitre <nico@fluxnic.net> Tested-by: Matthias Maennich <maennich@google.com> Reviewed-by: Matthias Maennich <maennich@google.com> Signed-off-by: Quentin Perret <qperret@google.com> Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
2020-03-03kbuild: allow symbol whitelisting with TRIM_UNUSED_KSYMSQuentin Perret
CONFIG_TRIM_UNUSED_KSYMS currently removes all unused exported symbols from ksymtab. This works really well when using in-tree drivers, but cannot be used in its current form if some of them are out-of-tree. Indeed, even if the list of symbols required by out-of-tree drivers is known at compile time, the only solution today to guarantee these don't get trimmed is to set CONFIG_TRIM_UNUSED_KSYMS=n. This not only wastes space, but also makes it difficult to control the ABI usable by vendor modules in distribution kernels such as Android. Being able to control the kernel ABI surface is particularly useful to ship a unique Generic Kernel Image (GKI) for all vendors, which is a first step in the direction of getting all vendors to contribute their code upstream. As such, attempt to improve the situation by enabling users to specify a symbol 'whitelist' at compile time. Any symbol specified in this whitelist will be kept exported when CONFIG_TRIM_UNUSED_KSYMS is set, even if it has no in-tree user. The whitelist is defined as a simple text file, listing symbols, one per line. Acked-by: Jessica Yu <jeyu@kernel.org> Acked-by: Nicolas Pitre <nico@fluxnic.net> Tested-by: Matthias Maennich <maennich@google.com> Reviewed-by: Matthias Maennich <maennich@google.com> Signed-off-by: Quentin Perret <qperret@google.com> Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
2019-07-18kbuild: remove the first line of *.mod filesMasahiro Yamada
The current format of *.mod is like this: line 1: directory path to the .ko file line 2: a list of objects linked into this module line 3: unresolved symbols (only when CONFIG_TRIM_UNUSED_KSYMS=y) Now that *.mod and *.ko are created in the same directory, the line 1 provides no valuable information. It can be derived by replacing the extension .mod with .ko. In fact, nobody uses the first line any more. Cut down the first line. Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
2019-07-18kbuild: create *.mod with full directory path and remove MODVERDIRMasahiro Yamada
While descending directories, Kbuild produces objects for modules, but do not link final *.ko files; it is done in the modpost. To keep track of modules, Kbuild creates a *.mod file in $(MODVERDIR) for every module it is building. Some post-processing steps read the necessary information from *.mod files. This avoids descending into directories again. This mechanism was introduced in 2003 or so. Later, commit 551559e13af1 ("kbuild: implement modules.order") added modules.order. So, we can simply read it out to know all the modules with directory paths. This is easier than parsing the first line of *.mod files. $(MODVERDIR) has a flat directory structure, that is, *.mod files are named only with base names. This is based on the assumption that the module name is unique across the tree. This assumption is really fragile. Stephen Rothwell reported a race condition caused by a module name conflict: https://lkml.org/lkml/2019/5/13/991 In parallel building, two different threads could write to the same $(MODVERDIR)/*.mod simultaneously. Non-unique module names are the source of all kind of troubles, hence commit 3a48a91901c5 ("kbuild: check uniqueness of module names") introduced a new checker script. However, it is still fragile in the build system point of view because this race happens before scripts/modules-check.sh is invoked. If it happens again, the modpost will emit unclear error messages. To fix this issue completely, create *.mod with full directory path so that two threads never attempt to write to the same file. $(MODVERDIR) is no longer needed. Since modules with directory paths are listed in modules.order, Kbuild is still able to find *.mod files without additional descending. I also killed cmd_secanalysis; scripts/mod/sumversion.c computes MD4 hash for modules with MODULE_VERSION(). When CONFIG_DEBUG_SECTION_MISMATCH=y, it occurs not only in the modpost stage, but also during directory descending, where sumversion.c may parse stale *.mod files. It would emit 'No such file or directory' warning when an object consisting a module is renamed, or when a single-obj module is turned into a multi-obj module or vice versa. Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com> Acked-by: Nicolas Pitre <nico@fluxnic.net>
2019-06-19treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 500Thomas Gleixner
Based on 2 normalized pattern(s): this program is free software you can redistribute it and or modify it under the terms of the gnu general public license version 2 as published by the free software foundation this program is free software you can redistribute it and or modify it under the terms of the gnu general public license version 2 as published by the free software foundation # extracted by the scancode license scanner the SPDX license identifier GPL-2.0-only has been chosen to replace the boilerplate/reference in 4122 file(s). Signed-off-by: Thomas Gleixner <tglx@linutronix.de> Reviewed-by: Enrico Weigelt <info@metux.net> Reviewed-by: Kate Stewart <kstewart@linuxfoundation.org> Reviewed-by: Allison Randal <allison@lohutok.net> Cc: linux-spdx@vger.kernel.org Link: https://lkml.kernel.org/r/20190604081206.933168790@linutronix.de Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2019-03-14kbuild: source include/config/auto.conf instead of ${KCONFIG_CONFIG}Masahiro Yamada
As commit 423a8155facf ("kbuild: Fix reading of .config in link-vmlinux.sh") addressed, some shells fail to perform '.' if ${KCONFIG_CONFIG} does not contain a slash at all. Instead, we can source include/config/auto.conf, which obviously contain slashes, and we do not expect its file path overridden by a user. Perhaps, the performance might be slightly better since unset CONFIG options are stripped from include/config/auto.conf. scripts/setlocalversion already works this way. Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
2018-05-17kbuild: remove CONFIG_HAVE_UNDERSCORE_SYMBOL_PREFIXMasahiro Yamada
CONFIG_HAVE_UNDERSCORE_SYMBOL_PREFIX was selected by BLACKFIN, METAG. They were removed by commit 4ba66a976072 ("arch: remove blackfin port"), commit bb6fb6dfcc17 ("metag: Remove arch/metag/"), respectively. No more architecture enables CONFIG_HAVE_UNDERSCORE_SYMBOL_PREFIX. Clean up the rest of scripts, and remove the Kconfig entry. Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com> Reviewed-by: Sam Ravnborg <sam@ravnborg.org>
2018-04-03Merge tag 'kbuild-v4.17' of ↵Linus Torvalds
git://git.kernel.org/pub/scm/linux/kernel/git/masahiroy/linux-kbuild Pull Kbuild updates from Masahiro Yamada: - add a shell script to get Clang version - improve portability of build scripts - drop always-enabled CONFIG_THIN_ARCHIVE and remove unused code - rename built-in.o which is now thin archive to built-in.a - process clean/build targets one by one to get along with -j option - simplify ld-option - improve building with CONFIG_TRIM_UNUSED_KSYMS - define KBUILD_MODNAME even for objects shared among multiple modules - avoid linking multiple instances of same objects from composite objects - move <linux/compiler_types.h> to c_flags to include it only for C files - clean-up various Makefiles * tag 'kbuild-v4.17' of git://git.kernel.org/pub/scm/linux/kernel/git/masahiroy/linux-kbuild: (29 commits) kbuild: get <linux/compiler_types.h> out of <linux/kconfig.h> kbuild: clean up link rule of composite modules kbuild: clean up archive rule of built-in.a kbuild: remove partial section mismatch detection for built-in.a net: liquidio: clean up Makefile for simpler composite object handling lib: zstd: clean up Makefile for simpler composite object handling kbuild: link $(real-obj-y) instead of $(obj-y) into built-in.a kbuild: rename real-objs-y/m to real-obj-y/m kbuild: move modname and modname-multi close to modname_flags kbuild: simplify modname calculation kbuild: fix modname for composite modules kbuild: define KBUILD_MODNAME even if multiple modules share objects kbuild: remove unnecessary $(subst $(obj)/, , ...) in modname-multi kbuild: Use ls(1) instead of stat(1) to obtain file size kbuild: link vmlinux only once for CONFIG_TRIM_UNUSED_KSYMS kbuild: move include/config/ksym/* to include/ksym/* kbuild: move CONFIG_TRIM_UNUSED_KSYMS code unneeded for external module kbuild: restore autoksyms.h touch to the top Makefile kbuild: move 'scripts' target below kbuild: remove wrong 'touch' in adjust_autoksyms.sh ...
2018-03-26kbuild: move include/config/ksym/* to include/ksym/*Masahiro Yamada
The idea of using fixdep was inspired by Kconfig, but autoksyms belongs to a different group. So, I want to move those touched files under include/config/ksym/ to include/ksym/. The directory include/ksym/ can be removed by 'make clean' because it is meaningless for the external module building. Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com> Acked-by: Nicolas Pitre <nico@linaro.org>
2018-03-26kbuild: remove wrong 'touch' in adjust_autoksyms.shMasahiro Yamada
The comment mentions it creates autoksyms.h in case it is missing, but the actual code touches it when it does exists. The build system creates it anyway because <linux/export.h> and <asm-generic/export.h> need it. The code would not have worked as intended, and people have not noticed it. This is a proof that we can simply remove it. Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com> Acked-by: Nicolas Pitre <nico@linaro.org>
2018-03-26kbuild: Improve portability of some sed invocationsMichael Forney
* Use BREs where EREs aren't necessary. * Pass -E instead of -r to use EREs. This will be standardized in the next POSIX revision[0]. GNU sed supports this since 4.2 (May 2009), and busybox since 1.22.0 (Jan 2014). * Use the [:space:] character class instead of ` \t` in bracket expressions. In bracket expressions, POSIX says that <backslash> loses its special meaning, so a conforming implementation cannot expand \t to <tab>[1]. * In BREs, use interval expressions (\{n,m\}) instead of non-standard features like \+ and \?. * Use a loop instead of -s flag. There are still plenty of other cases of non-standard sed invocations (use of ERE features in BREs, in-place editing), but this fixes some core ones. [0] http://austingroupbugs.net/view.php?id=528 [1] http://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_chap09.html#tag_09_03_05 Signed-off-by: Michael Forney <forney@google.com> Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
2018-03-21kbuild: make scripts/adjust_autoksyms.sh robust against timestamp racesNicolas Pitre
Some filesystems have timestamps with coarse precision that may allow for a recently built object file to have the same timestamp as the updated time on one of its dependency files. When that happens, the object file doesn't get rebuilt as it should. This is especially the case on filesystems that don't have sub-second time precision, such as ext3 or Ext4 with 128B inodes. Let's prevent that by making sure updated dependency files have a newer timestamp than the first file we created (i.e. autoksyms.h.tmpnew). Reported-by: Thomas Lindroth <thomas.lindroth@gmail.com> Signed-off-by: Nicolas Pitre <nico@linaro.org> Tested-by: Thomas Lindroth <thomas.lindroth@gmail.com> Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
2016-12-11kbuild: fix scripts/adjust_autoksyms.sh* for the no modules caseNicolas Pitre
When CONFIG_TRIM_UNUSED_KSYMS=y and no modules are actually selected, the adjust_autoksyms.sh script fails with: sed: can't read .tmp_versions/*.mod: No such file or directory Let's cope with that case gracefully. Signed-off-by: Nicolas Pitre <nico@linaro.org> Signed-off-by: Michal Marek <mmarek@suse.com>
2016-05-10kbuild: fix adjust_autoksyms.sh for modules that need only one symbolNicolas Pitre
When only one symbol was listed and therefore the line didn't contain any space to separate multiple symbols, that symbol got ignored. Reported-by: Arnd Bergmann <arnd@arndb.de> Signed-off-by: Nicolas Pitre <nico@linaro.org> Signed-off-by: Michal Marek <mmarek@suse.com>
2016-03-29kbuild: create/adjust generated/autoksyms.hNicolas Pitre
Given the list of exported symbols needed by all modules, we can create a header file containing preprocessor defines for each of those symbols. Also, when some symbols are added and/or removed from the list, we can update the time on the corresponding files used as build dependencies for those symbols. And finally, if any symbol did change state, the corresponding source files must be rebuilt. The insertion or removal of an EXPORT_SYMBOL() entry within a module may create or remove the need for another exported symbol. This is why this operation has to be repeated until the list of needed exported symbols becomes stable. Only then the final kernel and modules link take place. Signed-off-by: Nicolas Pitre <nico@linaro.org> Acked-by: Rusty Russell <rusty@rustcorp.com.au>