diff options
Diffstat (limited to 'tools/perf/Documentation')
29 files changed, 1252 insertions, 319 deletions
diff --git a/tools/perf/Documentation/Build.txt b/tools/perf/Documentation/Build.txt index 3766886c4bca..83dc87c662b6 100644 --- a/tools/perf/Documentation/Build.txt +++ b/tools/perf/Documentation/Build.txt @@ -71,3 +71,31 @@ supported by GCC. UBSan detects undefined behaviors of programs at runtime. $ UBSAN_OPTIONS=print_stacktrace=1 ./perf record -a If UBSan detects any problem at runtime, it outputs a “runtime error:” message. + +4) Cross compilation +==================== +As Multiarch is commonly supported in Linux distributions, we can install +libraries for multiple architectures on the same system and then cross-compile +Linux perf. For example, Aarch64 libraries and toolchains can be installed on +an x86_64 machine, allowing us to compile perf for an Aarch64 target. + +Below is the command for building the perf with dynamic linking. + + $ cd /path/to/Linux + $ make ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu- -C tools/perf + +For static linking, the option `LDFLAGS="-static"` is required. + + $ make ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu- \ + LDFLAGS="-static" -C tools/perf + +In the embedded system world, a use case is to explicitly specify the package +configuration paths for cross building: + + $ PKG_CONFIG_SYSROOT_DIR="/path/to/cross/build/sysroot" \ + PKG_CONFIG_LIBDIR="/usr/lib/:/usr/local/lib" \ + make ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu- -C tools/perf + +In this case, the variable PKG_CONFIG_SYSROOT_DIR can be used alongside the +variable PKG_CONFIG_LIBDIR or PKG_CONFIG_PATH to prepend the sysroot path to +the library paths for cross compilation. diff --git a/tools/perf/Documentation/callchain-overhead-calculation.txt b/tools/perf/Documentation/callchain-overhead-calculation.txt index 1a757927195e..e0202bf5bd1a 100644 --- a/tools/perf/Documentation/callchain-overhead-calculation.txt +++ b/tools/perf/Documentation/callchain-overhead-calculation.txt @@ -1,7 +1,8 @@ Overhead calculation -------------------- -The overhead can be shown in two columns as 'Children' and 'Self' when -perf collects callchains. The 'self' overhead is simply calculated by +The CPU overhead can be shown in two columns as 'Children' and 'Self' +when perf collects callchains (and corresponding 'Wall' columns for +wall-clock overhead). The 'self' overhead is simply calculated by adding all period values of the entry - usually a function (symbol). This is the value that perf shows traditionally and sum of all the 'self' overhead values should be 100%. diff --git a/tools/perf/Documentation/cpu-and-latency-overheads.txt b/tools/perf/Documentation/cpu-and-latency-overheads.txt new file mode 100644 index 000000000000..3b6d63705465 --- /dev/null +++ b/tools/perf/Documentation/cpu-and-latency-overheads.txt @@ -0,0 +1,85 @@ +CPU and latency overheads +------------------------- +There are two notions of time: wall-clock time and CPU time. +For a single-threaded program, or a program running on a single-core machine, +these notions are the same. However, for a multi-threaded/multi-process program +running on a multi-core machine, these notions are significantly different. +Each second of wall-clock time we have number-of-cores seconds of CPU time. +Perf can measure overhead for both of these times (shown in 'overhead' and +'latency' columns for CPU and wall-clock time correspondingly). + +Optimizing CPU overhead is useful to improve 'throughput', while optimizing +latency overhead is useful to improve 'latency'. It's important to understand +which one is useful in a concrete situation at hand. For example, the former +may be useful to improve max throughput of a CI build server that runs on 100% +CPU utilization, while the latter may be useful to improve user-perceived +latency of a single interactive program build. +These overheads may be significantly different in some cases. For example, +consider a program that executes function 'foo' for 9 seconds with 1 thread, +and then executes function 'bar' for 1 second with 128 threads (consumes +128 seconds of CPU time). The CPU overhead is: 'foo' - 6.6%, 'bar' - 93.4%. +While the latency overhead is: 'foo' - 90%, 'bar' - 10%. If we try to optimize +running time of the program looking at the (wrong in this case) CPU overhead, +we would concentrate on the function 'bar', but it can yield only 10% running +time improvement at best. + +By default, perf shows only CPU overhead. To show latency overhead, use +'perf record --latency' and 'perf report': + +----------------------------------- +Overhead Latency Command + 93.88% 25.79% cc1 + 1.90% 39.87% gzip + 0.99% 10.16% dpkg-deb + 0.57% 1.00% as + 0.40% 0.46% sh +----------------------------------- + +To sort by latency overhead, use 'perf report --latency': + +----------------------------------- +Latency Overhead Command + 39.87% 1.90% gzip + 25.79% 93.88% cc1 + 10.16% 0.99% dpkg-deb + 4.17% 0.29% git + 2.81% 0.11% objtool +----------------------------------- + +To get insight into the difference between the overheads, you may check +parallelization histogram with '--sort=latency,parallelism,comm,symbol --hierarchy' +flags. It shows fraction of (wall-clock) time the workload utilizes different +numbers of cores ('Parallelism' column). For example, in the following case +the workload utilizes only 1 core most of the time, but also has some +highly-parallel phases, which explains significant difference between +CPU and wall-clock overheads: + +----------------------------------- + Latency Overhead Parallelism / Command / Symbol ++ 56.98% 2.29% 1 ++ 16.94% 1.36% 2 ++ 4.00% 20.13% 125 ++ 3.66% 18.25% 124 ++ 3.48% 17.66% 126 ++ 3.26% 0.39% 3 ++ 2.61% 12.93% 123 +----------------------------------- + +By expanding corresponding lines, you may see what commands/functions run +at the given parallelism level: + +----------------------------------- + Latency Overhead Parallelism / Command / Symbol +- 56.98% 2.29% 1 + 32.80% 1.32% gzip + 4.46% 0.18% cc1 + 2.81% 0.11% objtool + 2.43% 0.10% dpkg-source + 2.22% 0.09% ld + 2.10% 0.08% dpkg-genchanges +----------------------------------- + +To see the normal function-level profile for particular parallelism levels +(number of threads actively running on CPUs), you may use '--parallelism' +filter. For example, to see the profile only for low parallelism phases +of a workload use '--latency --parallelism=1-2' flags. diff --git a/tools/perf/Documentation/intel-hybrid.txt b/tools/perf/Documentation/intel-hybrid.txt index e7a776ad25d7..0379903673a4 100644 --- a/tools/perf/Documentation/intel-hybrid.txt +++ b/tools/perf/Documentation/intel-hybrid.txt @@ -8,15 +8,15 @@ Part of events are available on core cpu, part of events are available on atom cpu and even part of events are available on both. Kernel exports two new cpu pmus via sysfs: -/sys/devices/cpu_core -/sys/devices/cpu_atom +/sys/bus/event_source/devices/cpu_core +/sys/bus/event_source/devices/cpu_atom The 'cpus' files are created under the directories. For example, -cat /sys/devices/cpu_core/cpus +cat /sys/bus/event_source/devices/cpu_core/cpus 0-15 -cat /sys/devices/cpu_atom/cpus +cat /sys/bus/event_source/devices/cpu_atom/cpus 16-23 It indicates cpu0-cpu15 are core cpus and cpu16-cpu23 are atom cpus. @@ -60,8 +60,8 @@ can't carry pmu information. So now this type is extended to be PMU aware type. The PMU type ID is stored at attr.config[63:32]. PMU type ID is retrieved from sysfs. -/sys/devices/cpu_atom/type -/sys/devices/cpu_core/type +/sys/bus/event_source/devices/cpu_atom/type +/sys/bus/event_source/devices/cpu_core/type The new attr.config layout for PERF_TYPE_HARDWARE: diff --git a/tools/perf/Documentation/itrace.txt b/tools/perf/Documentation/itrace.txt index 19cc179be9a7..40476b227f8d 100644 --- a/tools/perf/Documentation/itrace.txt +++ b/tools/perf/Documentation/itrace.txt @@ -1,6 +1,6 @@ i synthesize instructions events y synthesize cycles events - b synthesize branches events (branch misses for Arm SPE) + b synthesize branches events c synthesize branches events (calls only) r synthesize branches events (returns only) x synthesize transactions events diff --git a/tools/perf/Documentation/perf-amd-ibs.txt b/tools/perf/Documentation/perf-amd-ibs.txt new file mode 100644 index 000000000000..2fd31d9d7b71 --- /dev/null +++ b/tools/perf/Documentation/perf-amd-ibs.txt @@ -0,0 +1,189 @@ +perf-amd-ibs(1) +=============== + +NAME +---- +perf-amd-ibs - Support for AMD Instruction-Based Sampling (IBS) with perf tool + +SYNOPSIS +-------- +[verse] +'perf record' -e ibs_op// +'perf record' -e ibs_fetch// + +DESCRIPTION +----------- + +Instruction-Based Sampling (IBS) provides precise Instruction Pointer (IP) +profiling support on AMD platforms. IBS has two independent components: IBS +Op and IBS Fetch. IBS Op sampling provides information about instruction +execution (micro-op execution to be precise) with details like d-cache +hit/miss, d-TLB hit/miss, cache miss latency, load/store data source, branch +behavior etc. IBS Fetch sampling provides information about instruction fetch +with details like i-cache hit/miss, i-TLB hit/miss, fetch latency etc. IBS is +per-smt-thread i.e. each SMT hardware thread contains standalone IBS units. + +Both, IBS Op and IBS Fetch, are exposed as PMUs by Linux and can be exploited +using the Linux perf utility. The following files will be created at boot time +if IBS is supported by the hardware and kernel. + + /sys/bus/event_source/devices/ibs_op/ + /sys/bus/event_source/devices/ibs_fetch/ + +IBS Op PMU supports two events: cycles and micro ops. IBS Fetch PMU supports +one event: fetch ops. + +IBS PMUs do not have user/kernel filtering capability and thus it requires +CAP_SYS_ADMIN or CAP_PERFMON privilege. + +IBS VS. REGULAR CORE PMU +------------------------ + +IBS gives samples with precise IP, i.e. the IP recorded with IBS sample has +no skid. Whereas the IP recorded by regular core PMU will have some skid +(sample was generated at IP X but perf would record it at IP X+n). Hence, +regular core PMU might not help for profiling with instruction level +precision. Further, IBS provides additional information about the sample in +question. On the other hand, regular core PMU has it's own advantages like +plethora of events, counting mode (less interference), up to 6 parallel +counters, event grouping support, filtering capabilities etc. + +Three regular core PMU events are internally forwarded to IBS Op PMU when +precise_ip attribute is set: + + -e cpu-cycles:p becomes -e ibs_op// + -e r076:p becomes -e ibs_op// + -e r0C1:p becomes -e ibs_op/cnt_ctl=1/ + +EXAMPLES +-------- + +IBS Op PMU +~~~~~~~~~~ + +System-wide profile, cycles event, sampling period: 100000 + + # perf record -e ibs_op// -c 100000 -a + +Per-cpu profile (cpu10), cycles event, sampling period: 100000 + + # perf record -e ibs_op// -c 100000 -C 10 + +Per-cpu profile (cpu10), cycles event, sampling freq: 1000 + + # perf record -e ibs_op// -F 1000 -C 10 + +System-wide profile, uOps event, sampling period: 100000 + + # perf record -e ibs_op/cnt_ctl=1/ -c 100000 -a + +Same command, but also capture IBS register raw dump along with perf sample: + + # perf record -e ibs_op/cnt_ctl=1/ -c 100000 -a --raw-samples + +System-wide profile, uOps event, sampling period: 100000, L3MissOnly (Zen4 onward) + + # perf record -e ibs_op/cnt_ctl=1,l3missonly=1/ -c 100000 -a + +Per process(upstream v6.2 onward), uOps event, sampling period: 100000 + + # perf record -e ibs_op/cnt_ctl=1/ -c 100000 -p 1234 + +Per process(upstream v6.2 onward), uOps event, sampling period: 100000 + + # perf record -e ibs_op/cnt_ctl=1/ -c 100000 -- ls + +To analyse recorded profile in aggregate mode + + # perf report + /* Select a line and press 'a' to drill down at instruction level. */ + +To go over each sample + + # perf script + +Raw dump of IBS registers when profiled with --raw-samples + + # perf report -D + /* Look for PERF_RECORD_SAMPLE */ + + Example register raw dump: + + ibs_op_ctl: 000002c30006186a MaxCnt 100000 L3MissOnly 0 En 1 + Val 1 CntCtl 0=cycles CurCnt 707 + IbsOpRip: ffffffff8204aea7 + ibs_op_data: 0000010002550001 CompToRetCtr 1 TagToRetCtr 597 + BrnRet 0 RipInvalid 0 BrnFuse 0 Microcode 1 + ibs_op_data2: 0000000000000013 RmtNode 1 DataSrc 3=DRAM + ibs_op_data3: 0000000031960092 LdOp 0 StOp 1 DcL1TlbMiss 0 + DcL2TlbMiss 0 DcL1TlbHit2M 1 DcL1TlbHit1G 0 DcL2TlbHit2M 0 + DcMiss 1 DcMisAcc 0 DcWcMemAcc 0 DcUcMemAcc 0 DcLockedOp 0 + DcMissNoMabAlloc 0 DcLinAddrValid 1 DcPhyAddrValid 1 + DcL2TlbHit1G 0 L2Miss 1 SwPf 0 OpMemWidth 32 bytes + OpDcMissOpenMemReqs 12 DcMissLat 0 TlbRefillLat 0 + IbsDCLinAd: ff110008a5398920 + IbsDCPhysAd: 00000008a5398920 + +IBS applied in a real world usecase + + ~90% regression was observed in tbench with specific scheduler hint + which was counter intuitive. IBS profile of good and bad run captured + using perf helped in identifying exact cause of the problem: + + https://lore.kernel.org/r/20220921063638.2489-1-kprateek.nayak@amd.com + +IBS Fetch PMU +~~~~~~~~~~~~~ + +Similar commands can be used with Fetch PMU as well. + +System-wide profile, fetch ops event, sampling period: 100000 + + # perf record -e ibs_fetch// -c 100000 -a + +System-wide profile, fetch ops event, sampling period: 100000, Random enable + + # perf record -e ibs_fetch/rand_en=1/ -c 100000 -a + + Random enable adds small degree of variability to sample period. This + helps in cases like long running loops where PMU is tagging the same + instruction over and over because of fixed sample period. + +etc. + +PERF MEM AND PERF C2C +--------------------- + +perf mem is a memory access profiler tool and perf c2c is a shared data +cacheline analyser tool. Both of them internally uses IBS Op PMU on AMD. +Below is a simple example of the perf mem tool. + + # perf mem record -c 100000 -- make + # perf mem report + +A normal perf mem report output will provide detailed memory access profile. +However, it can also be aggregated based on output fields. For example: + + # perf mem report -F mem,sample,snoop + Samples: 3M of event 'ibs_op//', Event count (approx.): 23524876 + Memory access Samples Snoop + N/A 1903343 N/A + L1 hit 1056754 N/A + L2 hit 75231 N/A + L3 hit 9496 HitM + L3 hit 2270 N/A + RAM hit 8710 N/A + Remote node, same socket RAM hit 3241 N/A + Remote core, same node Any cache hit 1572 HitM + Remote core, same node Any cache hit 514 N/A + Remote node, same socket Any cache hit 1216 HitM + Remote node, same socket Any cache hit 350 N/A + Uncached hit 18 N/A + +Please refer to their man page for more detail. + +SEE ALSO +-------- + +linkperf:perf-record[1], linkperf:perf-script[1], linkperf:perf-report[1], +linkperf:perf-mem[1], linkperf:perf-c2c[1] diff --git a/tools/perf/Documentation/perf-annotate.txt b/tools/perf/Documentation/perf-annotate.txt index b95524bea021..46090c5b42b4 100644 --- a/tools/perf/Documentation/perf-annotate.txt +++ b/tools/perf/Documentation/perf-annotate.txt @@ -165,6 +165,13 @@ include::itrace.txt[] --type-stat:: Show stats for the data type annotation. +--skip-empty:: + Do not display empty (or dummy) events. + +--code-with-type:: + Show data type info in code annotation (for memory instructions only). + Currently it only works with --stdio option. + SEE ALSO -------- diff --git a/tools/perf/Documentation/perf-arm-spe.txt b/tools/perf/Documentation/perf-arm-spe.txt index bf03222e9a68..37afade4f1b2 100644 --- a/tools/perf/Documentation/perf-arm-spe.txt +++ b/tools/perf/Documentation/perf-arm-spe.txt @@ -116,6 +116,15 @@ Depending on CPU model, the kernel may need to be booted with page table isolati (kpti=off). If KPTI needs to be disabled, this will fail with a console message "profiling buffer inaccessible. Try passing 'kpti=off' on the kernel command line". +For the full criteria that determine whether KPTI needs to be forced off or not, see function +unmap_kernel_at_el0() in the kernel sources. Common cases where it's not required +are on the CPUs in kpti_safe_list, or on Arm v8.5+ where FEAT_E0PD is mandatory. + +The SPE interrupt must also be described by the firmware. If the module is loaded and KPTI is +disabled (or isn't required to be disabled) but the SPE PMU still doesn't show in +/sys/bus/event_source/devices/, then it's possible that the SPE interrupt isn't described by +ACPI or DT. In this case no warning will be printed by the driver. + Capturing SPE with perf command-line tools ------------------------------------------ @@ -141,6 +150,7 @@ arm_spe/load_filter=1,min_latency=10/' pct_enable=1 - collect physical timestamp instead of virtual timestamp (PMSCR.PCT) - requires privilege store_filter=1 - collect stores only (PMSFCR.ST) ts_enable=1 - enable timestamping with value of generic timer (PMSCR.TS) + discard=1 - enable SPE PMU events but don't collect sample data - see 'Discard mode' (PMBLIMITR.FM = DISCARD) +++*+++ Latency is the total latency from the point at which sampling started on that instruction, rather than only the execution latency. @@ -178,7 +188,7 @@ groups: 7 llc-access 2 tlb-miss 1K tlb-access - 36 branch-miss + 36 branch 0 remote-access 900 memory @@ -199,7 +209,8 @@ Common errors - "Cannot find PMU `arm_spe'. Missing kernel support?" - Module not built or loaded, KPTI not disabled (see above), or running on a VM + Module not built or loaded, KPTI not disabled, interrupt not described by firmware, + or running on a VM. See 'Kernel Requirements' above. - "Arm SPE CONTEXT packets not found in the traces." @@ -210,6 +221,31 @@ Common errors Increase sampling interval (see above) +PMU events +~~~~~~~~~~ + +SPE has events that can be counted on core PMUs. These are prefixed with +SAMPLE_, for example SAMPLE_POP, SAMPLE_FEED, SAMPLE_COLLISION and +SAMPLE_FEED_BR. + +These events will only count when an SPE event is running on the same core that +the PMU event is opened on, otherwise they read as 0. There are various ways to +ensure that the PMU event and SPE event are scheduled together depending on the +way the event is opened. For example opening both events as per-process events +on the same process, although it's not guaranteed that the PMU event is enabled +first when context switching. For that reason it may be better to open the PMU +event as a systemwide event and then open SPE on the process of interest. + +Discard mode +~~~~~~~~~~~~ + +SPE related (SAMPLE_* etc) core PMU events can be used without the overhead of +collecting sample data if discard mode is supported (optional from Armv8.6). +First run a system wide SPE session (or on the core of interest) using options +to minimize output. Then run perf stat: + + perf record -e arm_spe/discard/ -a -N -B --no-bpf-event -o - > /dev/null & + perf stat -e SAMPLE_FEED_LD SEE ALSO -------- diff --git a/tools/perf/Documentation/perf-check.txt b/tools/perf/Documentation/perf-check.txt new file mode 100644 index 000000000000..a764a4629220 --- /dev/null +++ b/tools/perf/Documentation/perf-check.txt @@ -0,0 +1,80 @@ +perf-check(1) +=============== + +NAME +---- +perf-check - check if features are present in perf + +SYNOPSIS +-------- +[verse] +'perf check' [<options>] +'perf check' {feature <feature_list>} [<options>] + +DESCRIPTION +----------- +With no subcommands given, 'perf check' command just prints the command +usage on the standard output. + +If the subcommand 'feature' is used, then status of feature is printed +on the standard output (unless '-q' is also passed), ie. whether it is +compiled-in/built-in or not. +Also, 'perf check feature' returns with exit status 0 if the feature +is built-in, otherwise returns with exit status 1. + +SUBCOMMANDS +----------- + +feature:: + + Print whether feature(s) is compiled-in or not, and also returns with an + exit status of 0, if passed feature(s) are compiled-in, else 1. + + It expects a feature list as an argument. There can be a single feature + name/macro, or multiple features can also be passed as a comma-separated + list, in which case the exit status will be 0 only if all of the passed + features are compiled-in. + + The feature names/macros are case-insensitive. + + Example Usage: + perf check feature libtraceevent + perf check feature HAVE_LIBTRACEEVENT + perf check feature libtraceevent,bpf + + Supported feature names/macro: + aio / HAVE_AIO_SUPPORT + bpf / HAVE_LIBBPF_SUPPORT + bpf_skeletons / HAVE_BPF_SKEL + debuginfod / HAVE_DEBUGINFOD_SUPPORT + dwarf / HAVE_LIBDW_SUPPORT + dwarf_getlocations / HAVE_LIBDW_SUPPORT + dwarf-unwind / HAVE_DWARF_UNWIND_SUPPORT + auxtrace / HAVE_AUXTRACE_SUPPORT + libbfd / HAVE_LIBBFD_SUPPORT + libcapstone / HAVE_LIBCAPSTONE_SUPPORT + libcrypto / HAVE_LIBCRYPTO_SUPPORT + libdw-dwarf-unwind / HAVE_LIBDW_SUPPORT + libelf / HAVE_LIBELF_SUPPORT + libnuma / HAVE_LIBNUMA_SUPPORT + libopencsd / HAVE_CSTRACE_SUPPORT + libperl / HAVE_LIBPERL_SUPPORT + libpfm4 / HAVE_LIBPFM + libpython / HAVE_LIBPYTHON_SUPPORT + libslang / HAVE_SLANG_SUPPORT + libtraceevent / HAVE_LIBTRACEEVENT + libunwind / HAVE_LIBUNWIND_SUPPORT + lzma / HAVE_LZMA_SUPPORT + numa_num_possible_cpus / HAVE_LIBNUMA_SUPPORT + zlib / HAVE_ZLIB_SUPPORT + zstd / HAVE_ZSTD_SUPPORT + +OPTIONS +------- +-q:: +--quiet:: + Do not print any messages or warnings + + This can be used along with subcommands such as 'perf check feature' + to hide unnecessary output in test scripts, eg. + 'perf check feature --quiet libtraceevent' diff --git a/tools/perf/Documentation/perf-config.txt b/tools/perf/Documentation/perf-config.txt index 379f9d7a8ab1..36ebebc875ea 100644 --- a/tools/perf/Documentation/perf-config.txt +++ b/tools/perf/Documentation/perf-config.txt @@ -40,7 +40,7 @@ The '$HOME/.perfconfig' file is used to store a per-user configuration. The file '$(sysconfdir)/perfconfig' can be used to store a system-wide default configuration. -One an disable reading config files by setting the PERF_CONFIG environment +One can disable reading config files by setting the PERF_CONFIG environment variable to /dev/null, or provide an alternate config file by setting that variable. @@ -247,6 +247,19 @@ annotate.*:: These are in control of addresses, jump function, source code in lines of assembly code from a specific program. + annotate.disassemblers:: + Choose the disassembler to use: "objdump", "llvm", "capstone", + if not specified it will first try, if available, the "llvm" one, + then, if it fails, "capstone", and finally the original "objdump" + based one. + + Choosing a different one is useful when handling some feature that + is known to be best support at some point by one of the options, + to compare the output when in doubt about some bug, etc. + + This can be a list, in order of preference, the first one that works + finishes the process. + annotate.addr2line:: addr2line binary to use for file names and line numbers. diff --git a/tools/perf/Documentation/perf-ftrace.txt b/tools/perf/Documentation/perf-ftrace.txt index d780b93fcf87..b77f58c4d2fd 100644 --- a/tools/perf/Documentation/perf-ftrace.txt +++ b/tools/perf/Documentation/perf-ftrace.txt @@ -9,7 +9,7 @@ perf-ftrace - simple wrapper for kernel's ftrace functionality SYNOPSIS -------- [verse] -'perf ftrace' {trace|latency} <command> +'perf ftrace' {trace|latency|profile} <command> DESCRIPTION ----------- @@ -23,6 +23,9 @@ kernel's ftrace infrastructure. 'perf ftrace latency' calculates execution latency of a given function (optionally with BPF) and display it as a histogram. + 'perf ftrace profile' show a execution profile for each function including + total, average, max time and the number of calls. + The following options apply to perf ftrace. COMMON OPTIONS @@ -125,6 +128,7 @@ OPTIONS for 'perf ftrace trace' - verbose - Show process names, PIDs, timestamps, etc. - thresh=<n> - Setup trace duration threshold in microseconds. - depth=<n> - Set max depth for function graph tracer to follow. + - tail - Print function name at the end. OPTIONS for 'perf ftrace latency' @@ -144,6 +148,67 @@ OPTIONS for 'perf ftrace latency' --use-nsec:: Use nano-second instead of micro-second as a base unit of the histogram. +--bucket-range=:: + Bucket range in ms or ns (according to -n/--use-nsec), default is log2() mode. + +--min-latency=:: + Minimum latency for the start of the first bucket, in ms or ns (according to + -n/--use-nsec). + +--max-latency=:: + Maximum latency for the start of the last bucket, in ms or ns (according to + -n/--use-nsec). The setting is ignored if the value results in more than + 22 buckets. + +OPTIONS for 'perf ftrace profile' +--------------------------------- + +-T:: +--trace-funcs=:: + Set function filter on the given function (or a glob pattern). + Multiple functions can be given by using this option more than once. + The function argument also can be a glob pattern. It will be passed + to 'set_ftrace_filter' in tracefs. + +-N:: +--notrace-funcs=:: + Do not trace functions given by the argument. Like -T option, this + can be used more than once to specify multiple functions (or glob + patterns). It will be passed to 'set_ftrace_notrace' in tracefs. + +-G:: +--graph-funcs=:: + Set graph filter on the given function (or a glob pattern). This is + useful to trace for functions executed from the given function. This + can be used more than once to specify multiple functions. It will be + passed to 'set_graph_function' in tracefs. + +-g:: +--nograph-funcs=:: + Set graph notrace filter on the given function (or a glob pattern). + Like -G option, this is useful for the function_graph tracer only and + disables tracing for function executed from the given function. This + can be used more than once to specify multiple functions. It will be + passed to 'set_graph_notrace' in tracefs. + +-m:: +--buffer-size:: + Set the size of per-cpu tracing buffer, <size> is expected to + be a number with appended unit character - B/K/M/G. + +-s:: +--sort=:: + Sort the result by the given field. Available values are: + total, avg, max, count, name. Default is 'total'. + +--graph-opts:: + List of options allowed to set: + + - nosleep-time - Measure on-CPU time only for function_graph tracer. + - noirqs - Ignore functions that happen inside interrupt. + - thresh=<n> - Setup trace duration threshold in microseconds. + - depth=<n> - Set max depth for function graph tracer to follow. + SEE ALSO -------- diff --git a/tools/perf/Documentation/perf-intel-pt.txt b/tools/perf/Documentation/perf-intel-pt.txt index 59ab1ff9d75f..cc0f37f0fa5a 100644 --- a/tools/perf/Documentation/perf-intel-pt.txt +++ b/tools/perf/Documentation/perf-intel-pt.txt @@ -151,7 +151,7 @@ displayed as follows: There are two ways that instructions-per-cycle (IPC) can be calculated depending on the recording. -If the 'cyc' config term (see config terms section below) was used, then IPC +If the 'cyc' config term (see <<_config_terms,config terms>> section below) was used, then IPC and cycle events are calculated using the cycle count from CYC packets, otherwise MTC packets are used - refer to the 'mtc' config term. When MTC is used, however, the values are less accurate because the timing is less accurate. @@ -239,7 +239,7 @@ which is the same as -e intel_pt/tsc=1,noretcomp=0/ -Note there are now new config terms - see section 'config terms' further below. +Note there are other config terms - see section <<_config_terms,config terms>> further below. The config terms are listed in /sys/devices/intel_pt/format. They are bit fields within the config member of the struct perf_event_attr which is @@ -311,218 +311,271 @@ perf_event_attr is displayed if the -vv option is used e.g. config terms ~~~~~~~~~~~~ -The June 2015 version of Intel 64 and IA-32 Architectures Software Developer -Manuals, Chapter 36 Intel Processor Trace, defined new Intel PT features. -Some of the features are reflect in new config terms. All the config terms are -described below. - -tsc Always supported. Produces TSC timestamp packets to provide - timing information. In some cases it is possible to decode - without timing information, for example a per-thread context - that does not overlap executable memory maps. - - The default config selects tsc (i.e. tsc=1). - -noretcomp Always supported. Disables "return compression" so a TIP packet - is produced when a function returns. Causes more packets to be - produced but might make decoding more reliable. - - The default config does not select noretcomp (i.e. noretcomp=0). - -psb_period Allows the frequency of PSB packets to be specified. - - The PSB packet is a synchronization packet that provides a - starting point for decoding or recovery from errors. - - Support for psb_period is indicated by: - - /sys/bus/event_source/devices/intel_pt/caps/psb_cyc - - which contains "1" if the feature is supported and "0" - otherwise. - - Valid values are given by: - - /sys/bus/event_source/devices/intel_pt/caps/psb_periods - - which contains a hexadecimal value, the bits of which represent - valid values e.g. bit 2 set means value 2 is valid. - - The psb_period value is converted to the approximate number of - trace bytes between PSB packets as: - - 2 ^ (value + 11) - - e.g. value 3 means 16KiB bytes between PSBs - - If an invalid value is entered, the error message - will give a list of valid values e.g. - - $ perf record -e intel_pt/psb_period=15/u uname - Invalid psb_period for intel_pt. Valid values are: 0-5 - - If MTC packets are selected, the default config selects a value - of 3 (i.e. psb_period=3) or the nearest lower value that is - supported (0 is always supported). Otherwise the default is 0. - - If decoding is expected to be reliable and the buffer is large - then a large PSB period can be used. - - Because a TSC packet is produced with PSB, the PSB period can - also affect the granularity to timing information in the absence - of MTC or CYC. - -mtc Produces MTC timing packets. - - MTC packets provide finer grain timestamp information than TSC - packets. MTC packets record time using the hardware crystal - clock (CTC) which is related to TSC packets using a TMA packet. - - Support for this feature is indicated by: - - /sys/bus/event_source/devices/intel_pt/caps/mtc - - which contains "1" if the feature is supported and - "0" otherwise. - - The frequency of MTC packets can also be specified - see - mtc_period below. - -mtc_period Specifies how frequently MTC packets are produced - see mtc - above for how to determine if MTC packets are supported. - - Valid values are given by: - - /sys/bus/event_source/devices/intel_pt/caps/mtc_periods - - which contains a hexadecimal value, the bits of which represent - valid values e.g. bit 2 set means value 2 is valid. - - The mtc_period value is converted to the MTC frequency as: - - CTC-frequency / (2 ^ value) - - e.g. value 3 means one eighth of CTC-frequency - - Where CTC is the hardware crystal clock, the frequency of which - can be related to TSC via values provided in cpuid leaf 0x15. - - If an invalid value is entered, the error message - will give a list of valid values e.g. - - $ perf record -e intel_pt/mtc_period=15/u uname - Invalid mtc_period for intel_pt. Valid values are: 0,3,6,9 - - The default value is 3 or the nearest lower value - that is supported (0 is always supported). - -cyc Produces CYC timing packets. - - CYC packets provide even finer grain timestamp information than - MTC and TSC packets. A CYC packet contains the number of CPU - cycles since the last CYC packet. Unlike MTC and TSC packets, - CYC packets are only sent when another packet is also sent. - - Support for this feature is indicated by: - - /sys/bus/event_source/devices/intel_pt/caps/psb_cyc - - which contains "1" if the feature is supported and - "0" otherwise. - - The number of CYC packets produced can be reduced by specifying - a threshold - see cyc_thresh below. - -cyc_thresh Specifies how frequently CYC packets are produced - see cyc - above for how to determine if CYC packets are supported. - - Valid cyc_thresh values are given by: - - /sys/bus/event_source/devices/intel_pt/caps/cycle_thresholds - - which contains a hexadecimal value, the bits of which represent - valid values e.g. bit 2 set means value 2 is valid. - - The cyc_thresh value represents the minimum number of CPU cycles - that must have passed before a CYC packet can be sent. The - number of CPU cycles is: - - 2 ^ (value - 1) - - e.g. value 4 means 8 CPU cycles must pass before a CYC packet - can be sent. Note a CYC packet is still only sent when another - packet is sent, not at, e.g. every 8 CPU cycles. - - If an invalid value is entered, the error message - will give a list of valid values e.g. - - $ perf record -e intel_pt/cyc,cyc_thresh=15/u uname - Invalid cyc_thresh for intel_pt. Valid values are: 0-12 - - CYC packets are not requested by default. - -pt Specifies pass-through which enables the 'branch' config term. - - The default config selects 'pt' if it is available, so a user will - never need to specify this term. - -branch Enable branch tracing. Branch tracing is enabled by default so to - disable branch tracing use 'branch=0'. - - The default config selects 'branch' if it is available. - -ptw Enable PTWRITE packets which are produced when a ptwrite instruction - is executed. - - Support for this feature is indicated by: - - /sys/bus/event_source/devices/intel_pt/caps/ptwrite - - which contains "1" if the feature is supported and - "0" otherwise. - - As an alternative, refer to "Emulated PTWRITE" further below. - -fup_on_ptw Enable a FUP packet to follow the PTWRITE packet. The FUP packet - provides the address of the ptwrite instruction. In the absence of - fup_on_ptw, the decoder will use the address of the previous branch - if branch tracing is enabled, otherwise the address will be zero. - Note that fup_on_ptw will work even when branch tracing is disabled. - -pwr_evt Enable power events. The power events provide information about - changes to the CPU C-state. - - Support for this feature is indicated by: - - /sys/bus/event_source/devices/intel_pt/caps/power_event_trace - - which contains "1" if the feature is supported and - "0" otherwise. - -event Enable Event Trace. The events provide information about asynchronous - events. - - Support for this feature is indicated by: - - /sys/bus/event_source/devices/intel_pt/caps/event_trace - - which contains "1" if the feature is supported and - "0" otherwise. - -notnt Disable TNT packets. Without TNT packets, it is not possible to walk - executable code to reconstruct control flow, however FUP, TIP, TIP.PGE - and TIP.PGD packets still indicate asynchronous control flow, and (if - return compression is disabled - see noretcomp) return statements. - The advantage of eliminating TNT packets is reducing the size of the - trace and corresponding tracing overhead. - - Support for this feature is indicated by: - - /sys/bus/event_source/devices/intel_pt/caps/tnt_disable - - which contains "1" if the feature is supported and - "0" otherwise. - +Config terms are parameters specified with the -e intel_pt// event option, +for example: + + -e intel_pt/cyc/ + +which selects cycle accurate mode. Each config term can have a value which +defaults to 1, so the above is the same as: + + -e intel_pt/cyc=1/ + +Some terms are set by default, so must be set to 0 to turn them off. For +example, to turn off branch tracing: + + -e intel_pt/branch=0/ + +Multiple config terms are separated by commas, for example: + + -e intel_pt/cyc,mtc_period=9/ + +There are also common config terms, see linkperf:perf-record[1] documentation. + +Intel PT config terms are described below. + +*tsc*:: +Always supported. Produces TSC timestamp packets to provide +timing information. In some cases it is possible to decode +without timing information, for example a per-thread context +that does not overlap executable memory maps. ++ +The default config selects tsc (i.e. tsc=1). + +*noretcomp*:: +Always supported. Disables "return compression" so a TIP packet +is produced when a function returns. Causes more packets to be +produced but might make decoding more reliable. ++ +The default config does not select noretcomp (i.e. noretcomp=0). + +*psb_period*:: +Allows the frequency of PSB packets to be specified. ++ +The PSB packet is a synchronization packet that provides a +starting point for decoding or recovery from errors. ++ +Support for psb_period is indicated by: ++ + /sys/bus/event_source/devices/intel_pt/caps/psb_cyc ++ +which contains "1" if the feature is supported and "0" +otherwise. ++ +Valid values are given by: ++ + /sys/bus/event_source/devices/intel_pt/caps/psb_periods ++ +which contains a hexadecimal value, the bits of which represent +valid values e.g. bit 2 set means value 2 is valid. ++ +The psb_period value is converted to the approximate number of +trace bytes between PSB packets as: ++ + 2 ^ (value + 11) ++ +e.g. value 3 means 16KiB bytes between PSBs ++ +If an invalid value is entered, the error message +will give a list of valid values e.g. ++ + $ perf record -e intel_pt/psb_period=15/u uname + Invalid psb_period for intel_pt. Valid values are: 0-5 ++ +If MTC packets are selected, the default config selects a value +of 3 (i.e. psb_period=3) or the nearest lower value that is +supported (0 is always supported). Otherwise the default is 0. ++ +If decoding is expected to be reliable and the buffer is large +then a large PSB period can be used. ++ +Because a TSC packet is produced with PSB, the PSB period can +also affect the granularity to timing information in the absence +of MTC or CYC. + +*mtc*:: +Produces MTC timing packets. ++ +MTC packets provide finer grain timestamp information than TSC +packets. MTC packets record time using the hardware crystal +clock (CTC) which is related to TSC packets using a TMA packet. ++ +Support for this feature is indicated by: ++ + /sys/bus/event_source/devices/intel_pt/caps/mtc ++ +which contains "1" if the feature is supported and +"0" otherwise. ++ +The frequency of MTC packets can also be specified - see +mtc_period below. + +*mtc_period*:: +Specifies how frequently MTC packets are produced - see mtc +above for how to determine if MTC packets are supported. ++ +Valid values are given by: ++ + /sys/bus/event_source/devices/intel_pt/caps/mtc_periods ++ +which contains a hexadecimal value, the bits of which represent +valid values e.g. bit 2 set means value 2 is valid. ++ +The mtc_period value is converted to the MTC frequency as: + + CTC-frequency / (2 ^ value) ++ +e.g. value 3 means one eighth of CTC-frequency ++ +Where CTC is the hardware crystal clock, the frequency of which +can be related to TSC via values provided in cpuid leaf 0x15. ++ +If an invalid value is entered, the error message +will give a list of valid values e.g. ++ + $ perf record -e intel_pt/mtc_period=15/u uname + Invalid mtc_period for intel_pt. Valid values are: 0,3,6,9 ++ +The default value is 3 or the nearest lower value +that is supported (0 is always supported). + +*cyc*:: +Produces CYC timing packets. ++ +CYC packets provide even finer grain timestamp information than +MTC and TSC packets. A CYC packet contains the number of CPU +cycles since the last CYC packet. Unlike MTC and TSC packets, +CYC packets are only sent when another packet is also sent. ++ +Support for this feature is indicated by: ++ + /sys/bus/event_source/devices/intel_pt/caps/psb_cyc ++ +which contains "1" if the feature is supported and +"0" otherwise. ++ +The number of CYC packets produced can be reduced by specifying +a threshold - see cyc_thresh below. + +*cyc_thresh*:: +Specifies how frequently CYC packets are produced - see cyc +above for how to determine if CYC packets are supported. ++ +Valid cyc_thresh values are given by: ++ + /sys/bus/event_source/devices/intel_pt/caps/cycle_thresholds ++ +which contains a hexadecimal value, the bits of which represent +valid values e.g. bit 2 set means value 2 is valid. ++ +The cyc_thresh value represents the minimum number of CPU cycles +that must have passed before a CYC packet can be sent. The +number of CPU cycles is: ++ + 2 ^ (value - 1) ++ +e.g. value 4 means 8 CPU cycles must pass before a CYC packet +can be sent. Note a CYC packet is still only sent when another +packet is sent, not at, e.g. every 8 CPU cycles. ++ +If an invalid value is entered, the error message +will give a list of valid values e.g. ++ + $ perf record -e intel_pt/cyc,cyc_thresh=15/u uname + Invalid cyc_thresh for intel_pt. Valid values are: 0-12 ++ +CYC packets are not requested by default. + +*pt*:: +Specifies pass-through which enables the 'branch' config term. ++ +The default config selects 'pt' if it is available, so a user will +never need to specify this term. + +*branch*:: +Enable branch tracing. Branch tracing is enabled by default so to +disable branch tracing use 'branch=0'. ++ +The default config selects 'branch' if it is available. + +*ptw*:: +Enable PTWRITE packets which are produced when a ptwrite instruction +is executed. ++ +Support for this feature is indicated by: ++ + /sys/bus/event_source/devices/intel_pt/caps/ptwrite ++ +which contains "1" if the feature is supported and +"0" otherwise. ++ +As an alternative, refer to "Emulated PTWRITE" further below. + +*fup_on_ptw*:: +Enable a FUP packet to follow the PTWRITE packet. The FUP packet +provides the address of the ptwrite instruction. In the absence of +fup_on_ptw, the decoder will use the address of the previous branch +if branch tracing is enabled, otherwise the address will be zero. +Note that fup_on_ptw will work even when branch tracing is disabled. + +*pwr_evt*:: +Enable power events. The power events provide information about +changes to the CPU C-state. ++ +Support for this feature is indicated by: ++ + /sys/bus/event_source/devices/intel_pt/caps/power_event_trace ++ +which contains "1" if the feature is supported and +"0" otherwise. + +*event*:: +Enable Event Trace. The events provide information about asynchronous +events. ++ +Support for this feature is indicated by: ++ + /sys/bus/event_source/devices/intel_pt/caps/event_trace ++ +which contains "1" if the feature is supported and +"0" otherwise. + +*notnt*:: +Disable TNT packets. Without TNT packets, it is not possible to walk +executable code to reconstruct control flow, however FUP, TIP, TIP.PGE +and TIP.PGD packets still indicate asynchronous control flow, and (if +return compression is disabled - see noretcomp) return statements. +The advantage of eliminating TNT packets is reducing the size of the +trace and corresponding tracing overhead. ++ +Support for this feature is indicated by: ++ + /sys/bus/event_source/devices/intel_pt/caps/tnt_disable ++ +which contains "1" if the feature is supported and +"0" otherwise. + +*aux-action=start-paused*:: +Start tracing paused, refer to the section <<_pause_or_resume_tracing,Pause or Resume Tracing>> + + +config terms on other events +~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Some Intel PT features work with other events, features such as AUX area sampling +and PEBS-via-PT. In those cases, the other events can have config terms below: + +*aux-sample-size*:: + Used to set the AUX area sample size, refer to the section + <<_aux_area_sampling_option,AUX area sampling option>> + +*aux-output*:: + Used to select PEBS-via-PT, refer to the + section <<_pebs_via_intel_pt,PEBS via Intel PT>> + +*aux-action*:: + Used to pause or resume tracing, refer to the section + <<_pause_or_resume_tracing,Pause or Resume Tracing>> AUX area sampling option ~~~~~~~~~~~~~~~~~~~~~~~~ @@ -596,7 +649,8 @@ The default snapshot size is the auxtrace mmap size. If neither auxtrace mmap s nor snapshot size is specified, then the default is 4MiB for privileged users (or if /proc/sys/kernel/perf_event_paranoid < 0), 128KiB for unprivileged users. If an unprivileged user does not specify mmap pages, the mmap pages will be -reduced as described in the 'new auxtrace mmap size option' section below. +reduced as described in the <<_new_auxtrace_mmap_size_option,new auxtrace mmap size option>> +section below. The snapshot size is displayed if the option -vv is used e.g. @@ -952,11 +1006,11 @@ transaction start, commit or abort. Note that "instructions", "cycles", "branches" and "transactions" events depend on code flow packets which can be disabled by using the config term -"branch=0". Refer to the config terms section above. +"branch=0". Refer to the <<_config_terms,config terms>> section above. "ptwrite" events record the payload of the ptwrite instruction and whether "fup_on_ptw" was used. "ptwrite" events depend on PTWRITE packets which are -recorded only if the "ptw" config term was used. Refer to the config terms +recorded only if the "ptw" config term was used. Refer to the <<_config_terms,config terms>> section above. perf script "synth" field displays "ptwrite" information like this: "ip: 0 payload: 0x123456789abcdef0" where "ip" is 1 if "fup_on_ptw" was used. @@ -964,7 +1018,7 @@ used. "Power" events correspond to power event packets and CBR (core-to-bus ratio) packets. While CBR packets are always recorded when tracing is enabled, power event packets are recorded only if the "pwr_evt" config term was used. Refer to -the config terms section above. The power events record information about +the <<_config_terms,config terms>> section above. The power events record information about C-state changes, whereas CBR is indicative of CPU frequency. perf script "event,synth" fields display information like this: @@ -1120,7 +1174,7 @@ What *will* be decoded with the (single) q option: - asynchronous branches such as interrupts - indirect branches - function return target address *if* the noretcomp config term (refer - config terms section) was used + <<_config_terms,config terms>> section) was used - start of (control-flow) tracing - end of (control-flow) tracing, if it is not out of context - power events, ptwrite, transaction start and abort @@ -1133,7 +1187,7 @@ Repeating the q option (double-q i.e. qq) results in even faster decoding and ev less detail. The decoder decodes only extended PSB (PSB+) packets, getting the instruction pointer if there is a FUP packet within PSB+ (i.e. between PSB and PSBEND). Note PSB packets occur regularly in the trace based on the psb_period -config term (refer config terms section). There will be a FUP packet if the +config term (refer <<_config_terms,config terms>> section). There will be a FUP packet if the PSB+ occurs while control flow is being traced. What will *not* be decoded with the qq option: @@ -1867,6 +1921,108 @@ For pipe mode, the order of events and timestamps can presumably be messed up. +Pause or Resume Tracing +----------------------- + +With newer Kernels, it is possible to use other selected events to pause +or resume Intel PT tracing. This is configured by using the "aux-action" +config term: + +"aux-action=pause" is used with events that are to pause Intel PT tracing. + +"aux-action=resume" is used with events that are to resume Intel PT tracing. + +"aux-action=start-paused" is used with the Intel PT event to start in a +paused state. + +For example, to trace only the uname system call (sys_newuname) when running the +command line utility uname: + + $ perf record --kcore -e intel_pt/aux-action=start-paused/k,syscalls:sys_enter_newuname/aux-action=resume/,syscalls:sys_exit_newuname/aux-action=pause/ uname + Linux + [ perf record: Woken up 1 times to write data ] + [ perf record: Captured and wrote 0.043 MB perf.data ] + $ perf script --call-trace + uname 30805 [000] 24001.058782799: name: 0x7ffc9c1865b0 + uname 30805 [000] 24001.058784424: psb offs: 0 + uname 30805 [000] 24001.058784424: cbr: 39 freq: 3904 MHz (139%) + uname 30805 [000] 24001.058784629: ([kernel.kallsyms]) debug_smp_processor_id + uname 30805 [000] 24001.058784629: ([kernel.kallsyms]) __x64_sys_newuname + uname 30805 [000] 24001.058784629: ([kernel.kallsyms]) down_read + uname 30805 [000] 24001.058784629: ([kernel.kallsyms]) __cond_resched + uname 30805 [000] 24001.058784629: ([kernel.kallsyms]) preempt_count_add + uname 30805 [000] 24001.058784629: ([kernel.kallsyms]) in_lock_functions + uname 30805 [000] 24001.058784629: ([kernel.kallsyms]) preempt_count_sub + uname 30805 [000] 24001.058784629: ([kernel.kallsyms]) up_read + uname 30805 [000] 24001.058784629: ([kernel.kallsyms]) preempt_count_add + uname 30805 [000] 24001.058784838: ([kernel.kallsyms]) in_lock_functions + uname 30805 [000] 24001.058784838: ([kernel.kallsyms]) preempt_count_sub + uname 30805 [000] 24001.058784838: ([kernel.kallsyms]) _copy_to_user + uname 30805 [000] 24001.058784838: ([kernel.kallsyms]) syscall_exit_to_user_mode + uname 30805 [000] 24001.058784838: ([kernel.kallsyms]) syscall_exit_work + uname 30805 [000] 24001.058784838: ([kernel.kallsyms]) perf_syscall_exit + uname 30805 [000] 24001.058784838: ([kernel.kallsyms]) debug_smp_processor_id + uname 30805 [000] 24001.058785046: ([kernel.kallsyms]) perf_trace_buf_alloc + uname 30805 [000] 24001.058785046: ([kernel.kallsyms]) perf_swevent_get_recursion_context + uname 30805 [000] 24001.058785046: ([kernel.kallsyms]) debug_smp_processor_id + uname 30805 [000] 24001.058785046: ([kernel.kallsyms]) debug_smp_processor_id + uname 30805 [000] 24001.058785046: ([kernel.kallsyms]) perf_tp_event + uname 30805 [000] 24001.058785046: ([kernel.kallsyms]) perf_trace_buf_update + uname 30805 [000] 24001.058785046: ([kernel.kallsyms]) tracing_gen_ctx_irq_test + uname 30805 [000] 24001.058785046: ([kernel.kallsyms]) perf_swevent_event + uname 30805 [000] 24001.058785046: ([kernel.kallsyms]) __perf_event_account_interrupt + uname 30805 [000] 24001.058785046: ([kernel.kallsyms]) __this_cpu_preempt_check + uname 30805 [000] 24001.058785046: ([kernel.kallsyms]) perf_event_output_forward + uname 30805 [000] 24001.058785046: ([kernel.kallsyms]) perf_event_aux_pause + uname 30805 [000] 24001.058785046: ([kernel.kallsyms]) ring_buffer_get + uname 30805 [000] 24001.058785046: ([kernel.kallsyms]) __rcu_read_lock + uname 30805 [000] 24001.058785046: ([kernel.kallsyms]) __rcu_read_unlock + uname 30805 [000] 24001.058785254: ([kernel.kallsyms]) pt_event_stop + uname 30805 [000] 24001.058785254: ([kernel.kallsyms]) debug_smp_processor_id + uname 30805 [000] 24001.058785254: ([kernel.kallsyms]) debug_smp_processor_id + uname 30805 [000] 24001.058785254: ([kernel.kallsyms]) native_write_msr + uname 30805 [000] 24001.058785463: ([kernel.kallsyms]) native_write_msr + uname 30805 [000] 24001.058785639: 0x0 + +The example above uses tracepoints, but any kind of sampled event can be used. + +For example: + + Tracing between arch_cpu_idle_enter() and arch_cpu_idle_exit() using breakpoint events: + + $ sudo cat /proc/kallsyms | sort | grep ' arch_cpu_idle_enter\| arch_cpu_idle_exit' + ffffffffb605bf60 T arch_cpu_idle_enter + ffffffffb614d8a0 W arch_cpu_idle_exit + $ sudo perf record --kcore -a -e intel_pt/aux-action=start-paused/k -e mem:0xffffffffb605bf60:x/aux-action=resume/ -e mem:0xffffffffb614d8a0:x/aux-action=pause/ -- sleep 1 + [ perf record: Woken up 1 times to write data ] + [ perf record: Captured and wrote 1.387 MB perf.data ] + + Tracing __alloc_pages() using kprobes: + + $ sudo perf probe --add '__alloc_pages order' + Added new event: probe:__alloc_pages (on __alloc_pages with order) + $ sudo perf probe --add __alloc_pages%return + Added new event: probe:__alloc_pages__return (on __alloc_pages%return) + $ sudo perf record --kcore -aR -e intel_pt/aux-action=start-paused/k -e probe:__alloc_pages/aux-action=resume/ -e probe:__alloc_pages__return/aux-action=pause/ -- sleep 1 + [ perf record: Woken up 1 times to write data ] + [ perf record: Captured and wrote 1.490 MB perf.data ] + + Tracing starting at main() using a uprobe event: + + $ sudo perf probe -x /usr/bin/uname main + Added new event: probe_uname:main (on main in /usr/bin/uname) + $ sudo perf record -e intel_pt/-aux-action=start-paused/u -e probe_uname:main/aux-action=resume/ -- uname + Linux + [ perf record: Woken up 1 times to write data ] + [ perf record: Captured and wrote 0.031 MB perf.data ] + + Tracing occasionally using cycles events with different periods: + + $ perf record --kcore -a -m,64M -e intel_pt/aux-action=start-paused/k -e cycles/aux-action=pause,period=1000000/Pk -e cycles/aux-action=resume,period=10500000/Pk -- firefox + [ perf record: Woken up 19 times to write data ] + [ perf record: Captured and wrote 16.561 MB perf.data ] + + EXAMPLE ------- diff --git a/tools/perf/Documentation/perf-kvm.txt b/tools/perf/Documentation/perf-kvm.txt index b66be66fe836..c26524d38f47 100644 --- a/tools/perf/Documentation/perf-kvm.txt +++ b/tools/perf/Documentation/perf-kvm.txt @@ -115,9 +115,9 @@ STAT LIVE OPTIONS -m:: --mmap-pages=:: - Number of mmap data pages (must be a power of two) or size - specification with appended unit character - B/K/M/G. The - size is rounded up to have nearest pages power of two value. + Number of mmap data pages (must be a power of two) or size + specification in bytes with appended unit character - B/K/M/G. + The size is rounded up to the nearest power-of-two page value. -a:: --all-cpus:: diff --git a/tools/perf/Documentation/perf-kwork.txt b/tools/perf/Documentation/perf-kwork.txt index 109ace1d5e90..21e607669d78 100644 --- a/tools/perf/Documentation/perf-kwork.txt +++ b/tools/perf/Documentation/perf-kwork.txt @@ -1,4 +1,4 @@ -perf-kowrk(1) +perf-kwork(1) ============= NAME @@ -35,7 +35,7 @@ There are several variants of 'perf kwork': perf kwork top perf kwork top -b - By default it shows the individual work events such as irq, workqeueu, + By default it shows the individual work events such as irq, workqueue, including the run time and delay (time between raise and actually entry): Runtime start Runtime end Cpu Kwork name Runtime Delaytime diff --git a/tools/perf/Documentation/perf-list.txt b/tools/perf/Documentation/perf-list.txt index 3b12595193c9..8914f12d2b85 100644 --- a/tools/perf/Documentation/perf-list.txt +++ b/tools/perf/Documentation/perf-list.txt @@ -8,7 +8,7 @@ perf-list - List all symbolic event types SYNOPSIS -------- [verse] -'perf list' [--no-desc] [--long-desc] +'perf list' [<options>] [hw|sw|cache|tracepoint|pmu|sdt|metric|metricgroup|event_glob] DESCRIPTION @@ -27,7 +27,7 @@ Don't print descriptions. -v:: --long-desc:: -Print longer event descriptions. +Print longer event descriptions and all similar PMUs with alphanumeric suffixes. --debug:: Enable debugging output. @@ -71,6 +71,8 @@ counted. The following modifiers exist: D - pin the event to the PMU W - group is weak and will fallback to non-group if not schedulable, e - group or event are exclusive and do not share the PMU + b - use BPF aggregration (see perf stat --bpf-counters) + R - retire latency value of the event The 'p' modifier can be used for specifying how precise the instruction address should be. The 'p' modifier can be specified multiple times: @@ -186,7 +188,7 @@ in the CPU vendor specific documentation. The available PMUs and their raw parameters can be listed with - ls /sys/devices/*/format + ls /sys/bus/event_source/devices/*/format For example the raw event "LSD.UOPS" core pmu event above could be specified as @@ -241,6 +243,21 @@ For accessing trace point events perf needs to have read access to /sys/kernel/tracing, even when perf_event_paranoid is in a relaxed setting. +TOOL/HWMON EVENTS +----------------- + +Some events don't have an associated PMU instead reading values +available to software without perf_event_open. As these events don't +support sampling they can only really be read by tools like perf stat. + +Tool events provide times and certain system parameters. Examples +include duration_time, user_time, system_time and num_cpus_online. + +Hwmon events provide easy access to hwmon sysfs data typically in +/sys/class/hwmon. This information includes temperatures, fan speeds +and energy usage. + + TRACING ------- diff --git a/tools/perf/Documentation/perf-lock.txt b/tools/perf/Documentation/perf-lock.txt index f5938d616d75..859dc11a7372 100644 --- a/tools/perf/Documentation/perf-lock.txt +++ b/tools/perf/Documentation/perf-lock.txt @@ -111,11 +111,11 @@ INFO OPTIONS -t:: --threads:: - dump thread list in perf.data + dump only the thread list in perf.data -m:: --map:: - dump map of lock instances (address:name table) + dump only the map of lock instances (address:name table) CONTENTION OPTIONS @@ -179,16 +179,17 @@ CONTENTION OPTIONS -o:: --lock-owner:: - Show lock contention stat by owners. Implies --threads and - requires --use-bpf. + Show lock contention stat by owners. This option can be combined with -t, + which shows owner's per thread lock stats, or -v, which shows owner's + stacktrace. Requires --use-bpf. -Y:: --type-filter=<value>:: Show lock contention only for given lock types (comma separated list). Available values are: semaphore, spinlock, rwlock, rwlock:R, rwlock:W, rwsem, rwsem:R, rwsem:W, - rtmutex, rwlock-rt, rwlock-rt:R, rwlock-rt:W, pcpu-sem, pcpu-sem:R, pcpu-sem:W, - mutex + rtmutex, rwlock-rt, rwlock-rt:R, rwlock-rt:W, percpu-rwmem, pcpu-sem, + pcpu-sem:R, pcpu-sem:W, mutex Note that RW-variant of locks have :R and :W suffix. Names without the suffix are shortcuts for the both variants. Ex) rwsem = rwsem:R + rwsem:W. diff --git a/tools/perf/Documentation/perf-mem.txt b/tools/perf/Documentation/perf-mem.txt index 19862572e3f2..8a1bd9ff0f86 100644 --- a/tools/perf/Documentation/perf-mem.txt +++ b/tools/perf/Documentation/perf-mem.txt @@ -21,22 +21,15 @@ and stores are sampled. Use the -t option to limit to loads or stores. Note that on Intel systems the memory latency reported is the use-latency, not the pure load (or store latency). Use latency includes any pipeline -queueing delays in addition to the memory subsystem latency. +queuing delays in addition to the memory subsystem latency. On Arm64 this uses SPE to sample load and store operations, therefore hardware and kernel support is required. See linkperf:perf-arm-spe[1] for a setup guide. Due to the statistical nature of SPE sampling, not every memory operation will be sampled. -OPTIONS -------- -<command>...:: - Any command you can specify in a shell. - --i:: ---input=<file>:: - Input file name. - +COMMON OPTIONS +-------------- -f:: --force:: Don't do ownership validation @@ -45,24 +38,9 @@ OPTIONS --type=<type>:: Select the memory operation type: load or store (default: load,store) --D:: ---dump-raw-samples:: - Dump the raw decoded samples on the screen in a format that is easy to parse with - one sample per line. - --x:: ---field-separator=<separator>:: - Specify the field separator used when dump raw samples (-D option). By default, - The separator is the space character. - --C:: ---cpu=<cpu>:: - Monitor only on the list of CPUs provided. Multiple CPUs can be provided as a - comma-separated list with no space: 0,1. Ranges of CPUs are specified with -: 0-2. Default - is to monitor all CPUS. --U:: ---hide-unresolved:: - Only display entries resolved to a symbol. +-v:: +--verbose:: + Be more verbose (show counter open errors, etc) -p:: --phys-data:: @@ -73,6 +51,9 @@ OPTIONS RECORD OPTIONS -------------- +<command>...:: + Any command you can specify in a shell. + -e:: --event <event>:: Event selector. Use 'perf mem record -e list' to list available events. @@ -85,14 +66,65 @@ RECORD OPTIONS --all-user:: Configure all used events to run in user space. --v:: ---verbose:: - Be more verbose (show counter open errors, etc) - --ldlat <n>:: Specify desired latency for loads event. Supported on Intel and Arm64 processors only. Ignored on other archs. +REPORT OPTIONS +-------------- +-i:: +--input=<file>:: + Input file name. + +-C:: +--cpu=<cpu>:: + Monitor only on the list of CPUs provided. Multiple CPUs can be provided as a + comma-separated list with no space: 0,1. Ranges of CPUs are specified with - + like 0-2. Default is to monitor all CPUS. + +-D:: +--dump-raw-samples:: + Dump the raw decoded samples on the screen in a format that is easy to parse with + one sample per line. + +-s:: +--sort=<key>:: + Group result by given key(s) - multiple keys can be specified + in CSV format. The keys are specific to memory samples are: + symbol_daddr, symbol_iaddr, dso_daddr, locked, tlb, mem, snoop, + dcacheline, phys_daddr, data_page_size, blocked. + + - symbol_daddr: name of data symbol being executed on at the time of sample + - symbol_iaddr: name of code symbol being executed on at the time of sample + - dso_daddr: name of library or module containing the data being executed + on at the time of the sample + - locked: whether the bus was locked at the time of the sample + - tlb: type of tlb access for the data at the time of the sample + - mem: type of memory access for the data at the time of the sample + - snoop: type of snoop (if any) for the data at the time of the sample + - dcacheline: the cacheline the data address is on at the time of the sample + - phys_daddr: physical address of data being executed on at the time of sample + - data_page_size: the data page size of data being executed on at the time of sample + - blocked: reason of blocked load access for the data at the time of the sample + + And the default sort keys are changed to local_weight, mem, sym, dso, + symbol_daddr, dso_daddr, snoop, tlb, locked, blocked, local_ins_lat. + +-T:: +--type-profile:: + Show data-type profile result instead of code symbols. This requires + the debug information and it will change the default sort keys to: + mem, snoop, tlb, type. + +-U:: +--hide-unresolved:: + Only display entries resolved to a symbol. + +-x:: +--field-separator=<separator>:: + Specify the field separator used when dump raw samples (-D option). By default, + The separator is the space character. + In addition, for report all perf report options are valid, and for record all perf record options. diff --git a/tools/perf/Documentation/perf-record.txt b/tools/perf/Documentation/perf-record.txt index 6015fdd08fb6..c7fc1ba265e2 100644 --- a/tools/perf/Documentation/perf-record.txt +++ b/tools/perf/Documentation/perf-record.txt @@ -68,6 +68,10 @@ OPTIONS like this: name=\'CPU_CLK_UNHALTED.THREAD:cmask=0x1\'. - 'aux-output': Generate AUX records instead of events. This requires that an AUX area event is also provided. + - 'aux-action': "pause" or "resume" to pause or resume an AUX + area event (the group leader) when this event occurs. + "start-paused" on an AUX area event itself, will + start in a paused state. - 'aux-sample-size': Set sample size for AUX area sampling. If the '--aux-sample' option has been used, set aux-sample-size=0 to disable AUX area sampling for the event. @@ -200,7 +204,7 @@ OPTIONS ip, id, tid, pid, cpu, time, addr, period, txn, weight, phys_addr, code_pgsz, data_pgsz, weight1, weight2, weight3, ins_lat, retire_lat, p_stage_cyc, mem_op, mem_lvl, mem_snoop, mem_remote, mem_lock, - mem_dtlb, mem_blk, mem_hops + mem_dtlb, mem_blk, mem_hops, uid, gid The <operator> can be one of: ==, !=, >, >=, <, <=, & @@ -223,6 +227,10 @@ OPTIONS '--filter' exists, the new filter expression will be combined with them by '&&'. +--latency:: + Enable data collection for latency profiling. + Use perf report --latency for latency-centric profile. + -a:: --all-cpus:: System-wide collection from all CPUs (default if no target is specified). @@ -273,10 +281,11 @@ OPTIONS -m:: --mmap-pages=:: Number of mmap data pages (must be a power of two) or size - specification with appended unit character - B/K/M/G. The - size is rounded up to have nearest pages power of two value. - Also, by adding a comma, the number of mmap pages for AUX - area tracing can be specified. + specification in bytes with appended unit character - B/K/M/G. + The size is rounded up to the nearest power-of-two page value. + By adding a comma, an additional parameter with the same + semantics used for the normal mmap areas can be specified for + AUX tracing area. -g:: Enables call-graph (stack chain/backtrace) recording for both @@ -311,7 +320,7 @@ OPTIONS User can change the size by passing the size after comma like "--call-graph dwarf,4096". - When "fp" recording is used, perf tries to save stack enties + When "fp" recording is used, perf tries to save stack entries up to the number specified in sysctl.kernel.perf_event_max_stack by default. User can change the number by passing it after comma like "--call-graph fp,32". @@ -828,6 +837,11 @@ filtered through the mask provided by -C option. only, as of now. So the applications built without the frame pointer might see bogus addresses. +--setup-filter=<action>:: + Prepare BPF filter to be used by regular users. The action should be + either "pin" or "unpin". The filter can be used after it's pinned. + + include::intel-hybrid.txt[] SEE ALSO diff --git a/tools/perf/Documentation/perf-report.txt b/tools/perf/Documentation/perf-report.txt index d8b863e01fe0..3376c4710575 100644 --- a/tools/perf/Documentation/perf-report.txt +++ b/tools/perf/Documentation/perf-report.txt @@ -44,7 +44,7 @@ OPTIONS --comms=:: Only consider symbols in these comms. CSV that understands file://filename entries. This option will affect the percentage of - the overhead column. See --percentage for more info. + the overhead and latency columns. See --percentage for more info. --pid=:: Only show events for given process ID (comma separated list). @@ -54,12 +54,12 @@ OPTIONS --dsos=:: Only consider symbols in these dsos. CSV that understands file://filename entries. This option will affect the percentage of - the overhead column. See --percentage for more info. + the overhead and latency columns. See --percentage for more info. -S:: --symbols=:: Only consider these symbols. CSV that understands file://filename entries. This option will affect the percentage of - the overhead column. See --percentage for more info. + the overhead and latency columns. See --percentage for more info. --symbol-filter=:: Only show symbols that match (partially) with this filter. @@ -68,6 +68,21 @@ OPTIONS --hide-unresolved:: Only display entries resolved to a symbol. +--parallelism:: + Only consider these parallelism levels. Parallelism level is the number + of threads that actively run on CPUs at the time of sample. The flag + accepts single number, comma-separated list, and ranges (for example: + "1", "7,8", "1,64-128"). This is useful in understanding what a program + is doing during sequential/low-parallelism phases as compared to + high-parallelism phases. This option will affect the percentage of + the overhead and latency columns. See --percentage for more info. + Also see the `CPU and latency overheads' section for more details. + +--latency:: + Show latency-centric profile rather than the default + CPU-consumption-centric profile + (requires perf record --latency flag). + -s:: --sort=:: Sort histogram entries by given key(s) - multiple keys can be specified @@ -87,6 +102,7 @@ OPTIONS entries are displayed as "[other]". - cpu: cpu number the task ran at the time of sample - socket: processor socket number the task ran at the time of sample + - parallelism: number of running threads at the time of sample - srcline: filename and line number executed at the time of sample. The DWARF debugging info must be provided. - srcfile: file name of the source file of the samples. Requires dwarf @@ -97,12 +113,14 @@ OPTIONS - cgroup_id: ID derived from cgroup namespace device and inode numbers. - cgroup: cgroup pathname in the cgroupfs. - transaction: Transaction abort flags. - - overhead: Overhead percentage of sample - - overhead_sys: Overhead percentage of sample running in system mode - - overhead_us: Overhead percentage of sample running in user mode - - overhead_guest_sys: Overhead percentage of sample running in system mode + - overhead: CPU overhead percentage of sample. + - latency: latency (wall-clock) overhead percentage of sample. + See the `CPU and latency overheads' section for more details. + - overhead_sys: CPU overhead percentage of sample running in system mode + - overhead_us: CPU overhead percentage of sample running in user mode + - overhead_guest_sys: CPU overhead percentage of sample running in system mode on guest machine - - overhead_guest_us: Overhead percentage of sample running in user mode on + - overhead_guest_us: CPU overhead percentage of sample running in user mode on guest machine - sample: Number of sample - period: Raw number of event count of sample @@ -121,9 +139,12 @@ OPTIONS - type: Data type of sample memory access. - typeoff: Offset in the data type of sample memory access. - symoff: Offset in the symbol. + - weight1: Average value of event specific weight (1st field of weight_struct). + - weight2: Average value of event specific weight (2nd field of weight_struct). + - weight3: Average value of event specific weight (3rd field of weight_struct). - By default, comm, dso and symbol keys are used. - (i.e. --sort comm,dso,symbol) + By default, overhead, comm, dso and symbol keys are used. + (i.e. --sort overhead,comm,dso,symbol). If --branch-stack option is used, following sort keys are also available: @@ -198,7 +219,11 @@ OPTIONS --fields=:: Specify output field - multiple keys can be specified in CSV format. Following fields are available: - overhead, overhead_sys, overhead_us, overhead_children, sample and period. + overhead, latency, overhead_sys, overhead_us, overhead_children, sample, + period, weight1, weight2, weight3, ins_lat, p_stage_cyc and retire_lat. + The last 3 names are alias for the corresponding weights. When the weight + fields are used, they will show the average value of the weight. + Also it can contain any sort key(s). By default, every sort keys not specified in -F will be appended @@ -282,7 +307,7 @@ OPTIONS Accumulate callchain of children to parent entry so that then can show up in the output. The output will have a new "Children" column and will be sorted on the data. It requires callchains are recorded. - See the `overhead calculation' section for more details. Enabled by + See the `Overhead calculation' section for more details. Enabled by default, disable with --no-children. --max-stack:: @@ -384,6 +409,14 @@ OPTIONS This allows to examine the path the program took to each sample. The data collection must have used -b (or -j) and -g. + Also show with some branch flags that can be: + - Predicted: display the average percentage of predicated branches. + (predicated number / total number) + - Abort: display the number of tsx aborted branches. + - Cycles: cycles in basic block. + + - iterations: display the average number of iterations in callchain list. + --addr2line=<path>:: Path to addr2line binary. @@ -427,9 +460,9 @@ OPTIONS --call-graph option for details. --percentage:: - Determine how to display the overhead percentage of filtered entries. - Filters can be applied by --comms, --dsos and/or --symbols options and - Zoom operations on the TUI (thread, dso, etc). + Determine how to display the CPU and latency overhead percentage + of filtered entries. Filters can be applied by --comms, --dsos, --symbols + and/or --parallelism options and Zoom operations on the TUI (thread, dso, etc). "relative" means it's relative to filtered entries only so that the sum of shown entries will be always 100%. "absolute" means it retains @@ -607,10 +640,13 @@ include::itrace.txt[] 'Avg Cycles%' - block average sampled cycles / sum of total block average sampled cycles 'Avg Cycles' - block average sampled cycles + 'Branch Counter' - block branch counter histogram (with -v showing the number) --skip-empty:: Do not print 0 results in the --stat output. +include::cpu-and-latency-overheads.txt[] + include::callchain-overhead-calculation.txt[] SEE ALSO diff --git a/tools/perf/Documentation/perf-sched.txt b/tools/perf/Documentation/perf-sched.txt index 5fbe42bd599b..6dbbddb6464d 100644 --- a/tools/perf/Documentation/perf-sched.txt +++ b/tools/perf/Documentation/perf-sched.txt @@ -20,6 +20,26 @@ There are several variants of 'perf sched': 'perf sched latency' to report the per task scheduling latencies and other scheduling properties of the workload. + Example usage: + perf sched record -- sleep 1 + perf sched latency + + ------------------------------------------------------------------------------------------------------------------------------------------- + Task | Runtime ms | Count | Avg delay ms | Max delay ms | Max delay start | Max delay end | + ------------------------------------------------------------------------------------------------------------------------------------------- + perf:(2) | 2.804 ms | 66 | avg: 0.524 ms | max: 1.069 ms | max start: 254752.314960 s | max end: 254752.316029 s + NetworkManager:1343 | 0.372 ms | 13 | avg: 0.008 ms | max: 0.013 ms | max start: 254751.551153 s | max end: 254751.551166 s + kworker/1:2-xfs:4649 | 0.012 ms | 1 | avg: 0.008 ms | max: 0.008 ms | max start: 254751.519807 s | max end: 254751.519815 s + kworker/3:1-xfs:388 | 0.011 ms | 1 | avg: 0.006 ms | max: 0.006 ms | max start: 254751.519809 s | max end: 254751.519815 s + sleep:147736 | 0.938 ms | 3 | avg: 0.006 ms | max: 0.007 ms | max start: 254751.313817 s | max end: 254751.313824 s + + It shows Runtime(time that a task spent actually running on the CPU), + Count(number of times a delay was calculated) and delay(time that a + task was ready to run but was kept waiting). + + Tasks with the same command name are merged and the merge count is + given within (), However if -p option is used, pid is mentioned. + 'perf sched script' to see a detailed trace of the workload that was recorded (aliased to 'perf script' for now). @@ -44,8 +64,8 @@ There are several variants of 'perf sched': By default it shows the individual schedule events, including the wait time (time between sched-out and next sched-in events for the task), the - task scheduling delay (time between wakeup and actually running) and run - time for the task: + task scheduling delay (time between runnable and actually running) and + run time for the task: time cpu task name wait time sch delay run time [tid/pid] (msec) (msec) (msec) @@ -78,6 +98,22 @@ OPTIONS --force:: Don't complain, do it. +OPTIONS for 'perf sched latency' +------------------------------- + +-C:: +--CPU <n>:: + CPU to profile on. + +-p:: +--pids:: + latency stats per pid instead of per command name. + +-s:: +--sort <key[,key2...]>:: + sort by key(s): runtime, switch, avg, max + by default it's sorted by "avg ,max ,switch ,runtime". + OPTIONS for 'perf sched map' ---------------------------- @@ -94,6 +130,16 @@ OPTIONS for 'perf sched map' --color-pids:: Highlight the given pids. +--task-name <task>:: + Map output only for the given task name(s). Separate the + task names with a comma (without whitespace). The sched-out + time is printed and is represented by '*-' for the given + task name(s). + ('-' indicates other tasks while '.' is idle). + +--fuzzy-name:: + Given task name(s) can be partially matched (fuzzy matching). + OPTIONS for 'perf sched timehist' --------------------------------- -k:: @@ -166,6 +212,30 @@ OPTIONS for 'perf sched timehist' --state:: Show task state when it switched out. +--show-prio:: + Show task priority. + +--prio:: + Only show events for given task priority(ies). Multiple priorities can be + provided as a comma-separated list with no spaces: 0,120. Ranges of + priorities are specified with -: 120-129. A combination of both can also be + provided: 0,120-129. + +-P:: +--pre-migrations:: + Show pre-migration wait time. pre-migration wait time is the time spent + by a task waiting on a runqueue but not getting the chance to run there + and is migrated to a different runqueue where it is finally run. This + time between sched_wakeup and migrate_task is the pre-migration wait + time. + +OPTIONS for 'perf sched replay' +------------------------------ + +-r:: +--repeat <n>:: + repeat the workload n times (0: infinite). Default is 10. + SEE ALSO -------- linkperf:perf-record[1] diff --git a/tools/perf/Documentation/perf-script-python.txt b/tools/perf/Documentation/perf-script-python.txt index 13e37e9385ee..27a1cac6fe76 100644 --- a/tools/perf/Documentation/perf-script-python.txt +++ b/tools/perf/Documentation/perf-script-python.txt @@ -624,7 +624,7 @@ as perf_trace_context.perf_script_context . perf_set_itrace_options(context, itrace_options) - set --itrace options if they have not been set already perf_sample_srcline(context) - returns source_file_name, line_number perf_sample_srccode(context) - returns source_file_name, line_number, source_line - + perf_config_get(config_name) - returns the value of the named config item, or None if unset Util.py Module ~~~~~~~~~~~~~~ diff --git a/tools/perf/Documentation/perf-script.txt b/tools/perf/Documentation/perf-script.txt index 005e51df855e..28bec7e78bc8 100644 --- a/tools/perf/Documentation/perf-script.txt +++ b/tools/perf/Documentation/perf-script.txt @@ -132,9 +132,9 @@ OPTIONS Comma separated list of fields to print. Options are: comm, tid, pid, time, cpu, event, trace, ip, sym, dso, dsoff, addr, symoff, srcline, period, iregs, uregs, brstack, brstacksym, flags, bpf-output, - brstackinsn, brstackinsnlen, brstackoff, callindent, insn, disasm, + brstackinsn, brstackinsnlen, brstackdisasm, brstackoff, callindent, insn, disasm, insnlen, synth, phys_addr, metric, misc, srccode, ipc, data_page_size, - code_page_size, ins_lat, machine_pid, vcpu, cgroup, retire_lat. + code_page_size, ins_lat, machine_pid, vcpu, cgroup, retire_lat, brcntr, Field list can be prepended with the type, trace, sw or hw, to indicate to which event type the field list applies. @@ -239,13 +239,22 @@ OPTIONS i.e., -F "" is not allowed. The brstack output includes branch related information with raw addresses using the - /v/v/v/v/cycles syntax in the following order: - FROM: branch source instruction - TO : branch target instruction - M/P/-: M=branch target mispredicted or branch direction was mispredicted, P=target predicted or direction predicted, -=not supported - X/- : X=branch inside a transactional region, -=not in transaction region or not supported - A/- : A=TSX abort entry, -=not aborted region or not supported - cycles + FROM/TO/EVENT/INTX/ABORT/CYCLES/TYPE/SPEC syntax in the following order: + FROM : branch source instruction + TO : branch target instruction + EVENT : M=branch target or direction was mispredicted + P=branch target or direction was predicted + N=branch not-taken + -=no event or not supported + INTX : X=branch inside a transactional region + -=branch not in transaction region or not supported + ABORT : A=TSX abort entry + -=not aborted region or not supported + CYCLES: the number of cycles that have elapsed since the last branch was recorded + TYPE : branch type: COND/UNCOND/IND/CALL/IND_CALL/RET etc. + -=not supported + SPEC : branch speculation info: SPEC_WRONG_PATH/NON_SPEC_CORRECT_PATH/SPEC_CORRECT_PATH + -=not supported The brstacksym is identical to brstack, except that the FROM and TO addresses are printed in a symbolic form if possible. @@ -257,6 +266,9 @@ OPTIONS can’t know the next sequential instruction after an unconditional branch unless you calculate that based on its length. + brstackdisasm acts like brstackinsn, but will print disassembled instructions if + perf is built with the capstone library. + The brstackoff field will print an offset into a specific dso/binary. With the metric option perf script can compute metrics for @@ -366,6 +378,9 @@ OPTIONS --demangle-kernel:: Demangle kernel symbol names to human readable form (for C++ kernels). +--addr2line=<path>:: + Path to addr2line binary. + --header Show perf.data header. diff --git a/tools/perf/Documentation/perf-stat.txt b/tools/perf/Documentation/perf-stat.txt index 29756a87ab6f..2bc063672486 100644 --- a/tools/perf/Documentation/perf-stat.txt +++ b/tools/perf/Documentation/perf-stat.txt @@ -498,6 +498,14 @@ To interpret the results it is usually needed to know on which CPUs the workload runs on. If needed the CPUs can be forced using taskset. +--record-tpebs:: +Enable automatic sampling on Intel TPEBS retire_latency events (event with :R +modifier). Without this option, perf would not capture dynamic retire_latency +at runtime. Currently, a zero value is assigned to the retire_latency event when +this option is not set. The TPEBS hardware feature starts from Intel Granite +Rapids microarchitecture. This option only exists in X86_64 and is meaningful on +Intel platforms with TPEBS feature. + --td-level:: Print the top-down statistics that equal the input level. It allows users to print the interested top-down metrics level instead of the diff --git a/tools/perf/Documentation/perf-test.txt b/tools/perf/Documentation/perf-test.txt index 951a2f262872..32da0d1fa86a 100644 --- a/tools/perf/Documentation/perf-test.txt +++ b/tools/perf/Documentation/perf-test.txt @@ -28,12 +28,44 @@ OPTIONS Tests to skip (comma separated numeric list). -v:: +-vv:: +-vvv:: --verbose:: - Be more verbose. + With a single '-v', verbose level 1, only failing test output + is displayed. With '-vv' and higher all test output is shown. + +-S:: +--sequential:: + Run all tests one after the other. By default "exclusive" + tests are run sequentially, but other tests are run in + parallel to speed execution. + +-r:: +--runs-per-test:: + Run each test the given number of times, by default once. This + option can be useful to determine if a test is flaky. -F:: --dont-fork:: - Do not fork child for each test, run all tests within single process. + Do not fork child for each test, run all tests within single process, this + sets sequential mode. --dso:: Specify a DSO for the "Symbols" test. + +-w:: +--workload=:: + Run a built-in workload, to list them use '--list-workloads', current ones include: + noploop, thloop, leafloop, sqrtloop, brstack, datasym and landlock. + + Used with the shell script regression tests. + + Some accept an extra parameter: + + seconds: leafloop, noploop, sqrtloop, thloop + nrloops: brstack + + The datasym and landlock workloads don't accept any. + +--list-workloads:: + List the available workloads to use with -w/--workload. diff --git a/tools/perf/Documentation/perf-top.txt b/tools/perf/Documentation/perf-top.txt index a754875fa5bb..af3e4230c72f 100644 --- a/tools/perf/Documentation/perf-top.txt +++ b/tools/perf/Documentation/perf-top.txt @@ -43,6 +43,10 @@ Default is to monitor all CPUS. encoding with the layout of the event control registers as described by entries in /sys/bus/event_source/devices/cpu/format/*. +--filter=<filter>:: + Event filter. This option should follow an event selector (-e). For + syntax see linkperf:perf-record[1]. + -E <entries>:: --entries=<entries>:: Display this many functions. @@ -79,8 +83,8 @@ Default is to monitor all CPUS. -m <pages>:: --mmap-pages=<pages>:: Number of mmap data pages (must be a power of two) or size - specification with appended unit character - B/K/M/G. The - size is rounded up to have nearest pages power of two value. + specification in bytes with appended unit character - B/K/M/G. + The size is rounded up to the nearest power-of-two page value. -p <pid>:: --pid=<pid>:: diff --git a/tools/perf/Documentation/perf-trace.txt b/tools/perf/Documentation/perf-trace.txt index f0da8cf63e9a..887dc37773d0 100644 --- a/tools/perf/Documentation/perf-trace.txt +++ b/tools/perf/Documentation/perf-trace.txt @@ -106,8 +106,8 @@ filter out the startup phase of the program, which is often very different. -m:: --mmap-pages=:: Number of mmap data pages (must be a power of two) or size - specification with appended unit character - B/K/M/G. The - size is rounded up to have nearest pages power of two value. + specification in bytes with appended unit character - B/K/M/G. + The size is rounded up to the nearest power-of-two page value. -C:: --cpu:: @@ -150,6 +150,10 @@ the thread executes on the designated CPUs. Default is to monitor all CPUs. To be used with -s or -S, to show stats for the errnos experienced by syscalls, using only this option will trigger --summary. +--summary-mode=mode:: + To be used with -s or -S, to select how to show summary. By default it'll + show the syscall summary by thread. Possible values are: thread, total. + --tool_stats:: Show tool stats such as number of times fd->pathname was discovered thru hooking the open syscall return + vfs_getname or via reading /proc/pid/fd, etc. @@ -241,6 +245,11 @@ the thread executes on the designated CPUs. Default is to monitor all CPUs. printing using the existing 'perf trace' syscall arg beautifiers to map integer arguments to strings (pid to comm, syscall id to syscall name, etc). +--force-btf:: + Use btf_dump to pretty print syscall argument data, instead of using hand-crafted pretty + printers. This option is intended for testing BTF integration in perf trace. btf_dump-based + pretty-printing serves as a fallback to hand-crafted pretty printers, as the latter can + better pretty-print integer flags and struct pointers. PAGEFAULTS ---------- diff --git a/tools/perf/Documentation/perf.txt b/tools/perf/Documentation/perf.txt index 09f516f3fdfb..cbcc2e4d557e 100644 --- a/tools/perf/Documentation/perf.txt +++ b/tools/perf/Documentation/perf.txt @@ -82,7 +82,8 @@ linkperf:perf-stat[1], linkperf:perf-top[1], linkperf:perf-record[1], linkperf:perf-report[1], linkperf:perf-list[1] -linkperf:perf-annotate[1],linkperf:perf-archive[1],linkperf:perf-arm-spe[1], +linkperf:perf-amd-ibs[1], linkperf:perf-annotate[1], +linkperf:perf-archive[1], linkperf:perf-arm-spe[1], linkperf:perf-bench[1], linkperf:perf-buildid-cache[1], linkperf:perf-buildid-list[1], linkperf:perf-c2c[1], linkperf:perf-config[1], linkperf:perf-data[1], linkperf:perf-diff[1], diff --git a/tools/perf/Documentation/tips.txt b/tools/perf/Documentation/tips.txt index 67b326ba0040..3fee9b2a88ea 100644 --- a/tools/perf/Documentation/tips.txt +++ b/tools/perf/Documentation/tips.txt @@ -62,3 +62,7 @@ To show context switches in perf report sample context add --switch-events to pe To show time in nanoseconds in record/report add --ns To compare hot regions in two workloads use perf record -b -o file ... ; perf diff --stream file1 file2 To compare scalability of two workload samples use perf diff -c ratio file1 file2 +For latency profiling, try: perf record/report --latency +For parallelism histogram, try: perf report --hierarchy --sort latency,parallelism,comm,symbol +To analyze particular parallelism levels, try: perf report --latency --parallelism=32-64 +To see how parallelism changes over time, try: perf report -F time,latency,parallelism --time-quantum=1s diff --git a/tools/perf/Documentation/topdown.txt b/tools/perf/Documentation/topdown.txt index ae0aee86844f..5c17fff694ee 100644 --- a/tools/perf/Documentation/topdown.txt +++ b/tools/perf/Documentation/topdown.txt @@ -325,6 +325,36 @@ other four level 2 metrics by subtracting corresponding metrics as below. Fetch_Bandwidth = Frontend_Bound - Fetch_Latency Core_Bound = Backend_Bound - Memory_Bound +TPEBS in TopDown +================ + +TPEBS (Timed PEBS) is one of the new Intel PMU features provided since Granite +Rapids microarchitecture. The TPEBS feature adds a 16 bit retire_latency field +in the Basic Info group of the PEBS record. It records the Core cycles since the +retirement of the previous instruction to the retirement of current instruction. +Please refer to Section 8.4.1 of "Intel® Architecture Instruction Set Extensions +Programming Reference" for more details about this feature. Because this feature +extends PEBS record, sampling with weight option is required to get the +retire_latency value. + + perf record -e event_name -W ... + +In the most recent release of TMA, the metrics begin to use event retire_latency +values in some of the metrics’ formulas on processors that support TPEBS feature. +For previous generations that do not support TPEBS, the values are static and +predefined per processor family by the hardware architects. Due to the diversity +of workloads in execution environments, retire_latency values measured at real +time are more accurate. Therefore, new TMA metrics that use TPEBS will provide +more accurate performance analysis results. + +To support TPEBS in TMA metrics, a new modifier :R on event is added. Perf would +capture retire_latency value of required events(event with :R in metric formula) +with perf record. The retire_latency value would be used in metric calculation. +Currently, this feature is supported through perf stat + + perf stat -M metric_name --record-tpebs ... + + [1] https://software.intel.com/en-us/top-down-microarchitecture-analysis-method-win [2] https://sites.google.com/site/analysismethods/yasin-pubs |