diff options
Diffstat (limited to 'tools/perf/Documentation')
| -rw-r--r-- | tools/perf/Documentation/callchain-overhead-calculation.txt | 5 | ||||
| -rw-r--r-- | tools/perf/Documentation/cpu-and-latency-overheads.txt | 85 | ||||
| -rw-r--r-- | tools/perf/Documentation/intel-hybrid.txt | 12 | ||||
| -rw-r--r-- | tools/perf/Documentation/perf-amd-ibs.txt | 68 | ||||
| -rw-r--r-- | tools/perf/Documentation/perf-annotate.txt | 4 | ||||
| -rw-r--r-- | tools/perf/Documentation/perf-c2c.txt | 11 | ||||
| -rw-r--r-- | tools/perf/Documentation/perf-config.txt | 4 | ||||
| -rw-r--r-- | tools/perf/Documentation/perf-list.txt | 13 | ||||
| -rw-r--r-- | tools/perf/Documentation/perf-lock.txt | 20 | ||||
| -rw-r--r-- | tools/perf/Documentation/perf-mem.txt | 82 | ||||
| -rw-r--r-- | tools/perf/Documentation/perf-record.txt | 20 | ||||
| -rw-r--r-- | tools/perf/Documentation/perf-report.txt | 55 | ||||
| -rw-r--r-- | tools/perf/Documentation/perf-script.txt | 23 | ||||
| -rw-r--r-- | tools/perf/Documentation/perf-stat.txt | 7 | ||||
| -rw-r--r-- | tools/perf/Documentation/perf-trace.txt | 11 | ||||
| -rw-r--r-- | tools/perf/Documentation/perf.data-file-format.txt | 24 | ||||
| -rw-r--r-- | tools/perf/Documentation/tips.txt | 4 |
17 files changed, 385 insertions, 63 deletions
diff --git a/tools/perf/Documentation/callchain-overhead-calculation.txt b/tools/perf/Documentation/callchain-overhead-calculation.txt index 1a757927195e..e0202bf5bd1a 100644 --- a/tools/perf/Documentation/callchain-overhead-calculation.txt +++ b/tools/perf/Documentation/callchain-overhead-calculation.txt @@ -1,7 +1,8 @@ Overhead calculation -------------------- -The overhead can be shown in two columns as 'Children' and 'Self' when -perf collects callchains. The 'self' overhead is simply calculated by +The CPU overhead can be shown in two columns as 'Children' and 'Self' +when perf collects callchains (and corresponding 'Wall' columns for +wall-clock overhead). The 'self' overhead is simply calculated by adding all period values of the entry - usually a function (symbol). This is the value that perf shows traditionally and sum of all the 'self' overhead values should be 100%. diff --git a/tools/perf/Documentation/cpu-and-latency-overheads.txt b/tools/perf/Documentation/cpu-and-latency-overheads.txt new file mode 100644 index 000000000000..3b6d63705465 --- /dev/null +++ b/tools/perf/Documentation/cpu-and-latency-overheads.txt @@ -0,0 +1,85 @@ +CPU and latency overheads +------------------------- +There are two notions of time: wall-clock time and CPU time. +For a single-threaded program, or a program running on a single-core machine, +these notions are the same. However, for a multi-threaded/multi-process program +running on a multi-core machine, these notions are significantly different. +Each second of wall-clock time we have number-of-cores seconds of CPU time. +Perf can measure overhead for both of these times (shown in 'overhead' and +'latency' columns for CPU and wall-clock time correspondingly). + +Optimizing CPU overhead is useful to improve 'throughput', while optimizing +latency overhead is useful to improve 'latency'. It's important to understand +which one is useful in a concrete situation at hand. For example, the former +may be useful to improve max throughput of a CI build server that runs on 100% +CPU utilization, while the latter may be useful to improve user-perceived +latency of a single interactive program build. +These overheads may be significantly different in some cases. For example, +consider a program that executes function 'foo' for 9 seconds with 1 thread, +and then executes function 'bar' for 1 second with 128 threads (consumes +128 seconds of CPU time). The CPU overhead is: 'foo' - 6.6%, 'bar' - 93.4%. +While the latency overhead is: 'foo' - 90%, 'bar' - 10%. If we try to optimize +running time of the program looking at the (wrong in this case) CPU overhead, +we would concentrate on the function 'bar', but it can yield only 10% running +time improvement at best. + +By default, perf shows only CPU overhead. To show latency overhead, use +'perf record --latency' and 'perf report': + +----------------------------------- +Overhead Latency Command + 93.88% 25.79% cc1 + 1.90% 39.87% gzip + 0.99% 10.16% dpkg-deb + 0.57% 1.00% as + 0.40% 0.46% sh +----------------------------------- + +To sort by latency overhead, use 'perf report --latency': + +----------------------------------- +Latency Overhead Command + 39.87% 1.90% gzip + 25.79% 93.88% cc1 + 10.16% 0.99% dpkg-deb + 4.17% 0.29% git + 2.81% 0.11% objtool +----------------------------------- + +To get insight into the difference between the overheads, you may check +parallelization histogram with '--sort=latency,parallelism,comm,symbol --hierarchy' +flags. It shows fraction of (wall-clock) time the workload utilizes different +numbers of cores ('Parallelism' column). For example, in the following case +the workload utilizes only 1 core most of the time, but also has some +highly-parallel phases, which explains significant difference between +CPU and wall-clock overheads: + +----------------------------------- + Latency Overhead Parallelism / Command / Symbol ++ 56.98% 2.29% 1 ++ 16.94% 1.36% 2 ++ 4.00% 20.13% 125 ++ 3.66% 18.25% 124 ++ 3.48% 17.66% 126 ++ 3.26% 0.39% 3 ++ 2.61% 12.93% 123 +----------------------------------- + +By expanding corresponding lines, you may see what commands/functions run +at the given parallelism level: + +----------------------------------- + Latency Overhead Parallelism / Command / Symbol +- 56.98% 2.29% 1 + 32.80% 1.32% gzip + 4.46% 0.18% cc1 + 2.81% 0.11% objtool + 2.43% 0.10% dpkg-source + 2.22% 0.09% ld + 2.10% 0.08% dpkg-genchanges +----------------------------------- + +To see the normal function-level profile for particular parallelism levels +(number of threads actively running on CPUs), you may use '--parallelism' +filter. For example, to see the profile only for low parallelism phases +of a workload use '--latency --parallelism=1-2' flags. diff --git a/tools/perf/Documentation/intel-hybrid.txt b/tools/perf/Documentation/intel-hybrid.txt index e7a776ad25d7..0379903673a4 100644 --- a/tools/perf/Documentation/intel-hybrid.txt +++ b/tools/perf/Documentation/intel-hybrid.txt @@ -8,15 +8,15 @@ Part of events are available on core cpu, part of events are available on atom cpu and even part of events are available on both. Kernel exports two new cpu pmus via sysfs: -/sys/devices/cpu_core -/sys/devices/cpu_atom +/sys/bus/event_source/devices/cpu_core +/sys/bus/event_source/devices/cpu_atom The 'cpus' files are created under the directories. For example, -cat /sys/devices/cpu_core/cpus +cat /sys/bus/event_source/devices/cpu_core/cpus 0-15 -cat /sys/devices/cpu_atom/cpus +cat /sys/bus/event_source/devices/cpu_atom/cpus 16-23 It indicates cpu0-cpu15 are core cpus and cpu16-cpu23 are atom cpus. @@ -60,8 +60,8 @@ can't carry pmu information. So now this type is extended to be PMU aware type. The PMU type ID is stored at attr.config[63:32]. PMU type ID is retrieved from sysfs. -/sys/devices/cpu_atom/type -/sys/devices/cpu_core/type +/sys/bus/event_source/devices/cpu_atom/type +/sys/bus/event_source/devices/cpu_core/type The new attr.config layout for PERF_TYPE_HARDWARE: diff --git a/tools/perf/Documentation/perf-amd-ibs.txt b/tools/perf/Documentation/perf-amd-ibs.txt index 2fd31d9d7b71..548549935760 100644 --- a/tools/perf/Documentation/perf-amd-ibs.txt +++ b/tools/perf/Documentation/perf-amd-ibs.txt @@ -85,6 +85,15 @@ System-wide profile, uOps event, sampling period: 100000, L3MissOnly (Zen4 onwar # perf record -e ibs_op/cnt_ctl=1,l3missonly=1/ -c 100000 -a +System-wide profile, cycles event, sampling period: 100000, LdLat filtering (Zen5 +onward) + + # perf record -e ibs_op/ldlat=128/ -c 100000 -a + + Supported load latency threshold values are 128 to 2048 (both inclusive). + Latency value which is a multiple of 128 incurs a little less profiling + overhead compared to other values. + Per process(upstream v6.2 onward), uOps event, sampling period: 100000 # perf record -e ibs_op/cnt_ctl=1/ -c 100000 -p 1234 @@ -162,23 +171,48 @@ Below is a simple example of the perf mem tool. # perf mem report A normal perf mem report output will provide detailed memory access profile. -However, it can also be aggregated based on output fields. For example: - - # perf mem report -F mem,sample,snoop - Samples: 3M of event 'ibs_op//', Event count (approx.): 23524876 - Memory access Samples Snoop - N/A 1903343 N/A - L1 hit 1056754 N/A - L2 hit 75231 N/A - L3 hit 9496 HitM - L3 hit 2270 N/A - RAM hit 8710 N/A - Remote node, same socket RAM hit 3241 N/A - Remote core, same node Any cache hit 1572 HitM - Remote core, same node Any cache hit 514 N/A - Remote node, same socket Any cache hit 1216 HitM - Remote node, same socket Any cache hit 350 N/A - Uncached hit 18 N/A +New output fields will show related access info together. For example: + + # perf mem report -F overhead,cache,snoop,comm + ... + # Samples: 92K of event 'ibs_op//' + # Total weight : 531104 + # + # ---------- Cache ----------- --- Snoop ---- + # Overhead L1 L2 L1-buf Other HitM Other Command + # ........ ............................ .............. .......... + # + 76.07% 5.8% 35.7% 0.0% 34.6% 23.3% 52.8% cc1 + 5.79% 0.2% 0.0% 0.0% 5.6% 0.1% 5.7% make + 5.78% 0.1% 4.4% 0.0% 1.2% 0.5% 5.3% gcc + 5.33% 0.3% 3.9% 0.0% 1.1% 0.2% 5.2% as + 5.00% 0.1% 3.8% 0.0% 1.0% 0.3% 4.7% sh + 1.56% 0.1% 0.1% 0.0% 1.4% 0.6% 0.9% ld + 0.28% 0.1% 0.0% 0.0% 0.2% 0.1% 0.2% pkg-config + 0.09% 0.0% 0.0% 0.0% 0.1% 0.0% 0.1% git + 0.03% 0.0% 0.0% 0.0% 0.0% 0.0% 0.0% rm + ... + +Also, it can be aggregated based on various memory access info using the +sort keys. For example: + + # perf mem report -s mem,snoop + ... + # Samples: 92K of event 'ibs_op//' + # Total weight : 531104 + # Sort order : mem,snoop + # + # Overhead Samples Memory access Snoop + # ........ ............ ....................................... ............ + # + 47.99% 1509 L2 hit N/A + 25.08% 338 core, same node Any cache hit HitM + 10.24% 54374 N/A N/A + 6.77% 35938 L1 hit N/A + 6.39% 101 core, same node Any cache hit N/A + 3.50% 69 RAM hit N/A + 0.03% 158 LFB/MAB hit N/A + 0.00% 2 Uncached hit N/A Please refer to their man page for more detail. diff --git a/tools/perf/Documentation/perf-annotate.txt b/tools/perf/Documentation/perf-annotate.txt index 156c5f37b051..46090c5b42b4 100644 --- a/tools/perf/Documentation/perf-annotate.txt +++ b/tools/perf/Documentation/perf-annotate.txt @@ -168,6 +168,10 @@ include::itrace.txt[] --skip-empty:: Do not display empty (or dummy) events. +--code-with-type:: + Show data type info in code annotation (for memory instructions only). + Currently it only works with --stdio option. + SEE ALSO -------- diff --git a/tools/perf/Documentation/perf-c2c.txt b/tools/perf/Documentation/perf-c2c.txt index 856f0dfb8e5a..f4af2dd6ab31 100644 --- a/tools/perf/Documentation/perf-c2c.txt +++ b/tools/perf/Documentation/perf-c2c.txt @@ -54,8 +54,15 @@ RECORD OPTIONS -l:: --ldlat:: - Configure mem-loads latency. Supported on Intel and Arm64 processors - only. Ignored on other archs. + Configure mem-loads latency. Supported on Intel, Arm64 and some AMD + processors. Ignored on other archs. + + On supported AMD processors: + - /sys/bus/event_source/devices/ibs_op/caps/ldlat file contains '1'. + - Supported latency values are 128 to 2048 (both inclusive). + - Latency value which is a multiple of 128 incurs a little less profiling + overhead compared to other values. + - Load latency filtering is disabled by default. -k:: --all-kernel:: diff --git a/tools/perf/Documentation/perf-config.txt b/tools/perf/Documentation/perf-config.txt index 36ebebc875ea..c6f335659667 100644 --- a/tools/perf/Documentation/perf-config.txt +++ b/tools/perf/Documentation/perf-config.txt @@ -708,6 +708,10 @@ intel-pt.*:: the maximum is exceeded there will be a "Never-ending loop" error. The default is 100000. + intel-pt.all-switch-events:: + If the user has permission to do so, always record all context + switch events on all CPUs. + auxtrace.*:: auxtrace.dumpdir:: diff --git a/tools/perf/Documentation/perf-list.txt b/tools/perf/Documentation/perf-list.txt index d0c65fad419a..ce0735021473 100644 --- a/tools/perf/Documentation/perf-list.txt +++ b/tools/perf/Documentation/perf-list.txt @@ -27,7 +27,7 @@ Don't print descriptions. -v:: --long-desc:: -Print longer event descriptions. +Print longer event descriptions and all similar PMUs with alphanumeric suffixes. --debug:: Enable debugging output. @@ -188,7 +188,7 @@ in the CPU vendor specific documentation. The available PMUs and their raw parameters can be listed with - ls /sys/devices/*/format + ls /sys/bus/event_source/devices/*/format For example the raw event "LSD.UOPS" core pmu event above could be specified as @@ -289,6 +289,15 @@ Sums up the event counts for all hardware threads in a core, e.g.: perf stat -e cpu/event=0,umask=0x3,percore=1/ +cpu: + +Specifies the CPU to open the event upon. The value may be repeated to +specify opening the event on multiple CPUs: + + + perf stat -e instructions/cpu=0,cpu=2/,cycles/cpu=1,cpu=2/ -a sleep 1 + perf stat -e data_read/cpu=0/,data_write/cpu=1/ -a sleep 1 + EVENT GROUPS ------------ diff --git a/tools/perf/Documentation/perf-lock.txt b/tools/perf/Documentation/perf-lock.txt index d3793054f7d3..c17b3e318169 100644 --- a/tools/perf/Documentation/perf-lock.txt +++ b/tools/perf/Documentation/perf-lock.txt @@ -179,8 +179,9 @@ CONTENTION OPTIONS -o:: --lock-owner:: - Show lock contention stat by owners. Implies --threads and - requires --use-bpf. + Show lock contention stat by owners. This option can be combined with -t, + which shows owner's per thread lock stats, or -v, which shows owner's + stacktrace. Requires --use-bpf. -Y:: --type-filter=<value>:: @@ -215,6 +216,21 @@ CONTENTION OPTIONS --cgroup-filter=<value>:: Show lock contention only in the given cgroups (comma separated list). +-J:: +--inject-delay=<time@function>:: + Add delays to the given lock. It's added to the contention-end part so + that the (new) owner of the lock will be delayed. But by slowing down + the owner, the waiters will also be delayed as well. This is working + only with -b/--use-bpf. + + The 'time' is specified in nsec but it can have a unit suffix. Available + units are "ms", "us" and "ns". Currently it accepts up to 10ms of delays + for safety reasons. + + Note that it will busy-wait after it gets the lock. Delaying locks can + have significant consequences including potential kernel crashes. Please + use it at your own risk. + SEE ALSO -------- diff --git a/tools/perf/Documentation/perf-mem.txt b/tools/perf/Documentation/perf-mem.txt index 8a1bd9ff0f86..4d164836d094 100644 --- a/tools/perf/Documentation/perf-mem.txt +++ b/tools/perf/Documentation/perf-mem.txt @@ -28,6 +28,8 @@ and kernel support is required. See linkperf:perf-arm-spe[1] for a setup guide. Due to the statistical nature of SPE sampling, not every memory operation will be sampled. +On AMD this use IBS Op PMU to sample load-store operations. + COMMON OPTIONS -------------- -f:: @@ -67,8 +69,15 @@ RECORD OPTIONS Configure all used events to run in user space. --ldlat <n>:: - Specify desired latency for loads event. Supported on Intel and Arm64 - processors only. Ignored on other archs. + Specify desired latency for loads event. Supported on Intel, Arm64 and + some AMD processors. Ignored on other archs. + + On supported AMD processors: + - /sys/bus/event_source/devices/ibs_op/caps/ldlat file contains '1'. + - Supported latency values are 128 to 2048 (both inclusive). + - Latency value which is a multiple of 128 incurs a little less profiling + overhead compared to other values. + - Load latency filtering is disabled by default. REPORT OPTIONS -------------- @@ -110,6 +119,22 @@ REPORT OPTIONS And the default sort keys are changed to local_weight, mem, sym, dso, symbol_daddr, dso_daddr, snoop, tlb, locked, blocked, local_ins_lat. +-F:: +--fields=:: + Specify output field - multiple keys can be specified in CSV format. + Please see linkperf:perf-report[1] for details. + + In addition to the default fields, 'perf mem report' will provide the + following fields to break down sample periods. + + - op: operation in the sample instruction (load, store, prefetch, ...) + - cache: location in CPU cache (L1, L2, ...) where the sample hit + - mem: location in memory or other places the sample hit + - dtlb: location in Data TLB (L1, L2) where the sample hit + - snoop: snoop result for the sampled data access + + Please take a look at the OUTPUT FIELD SELECTION section for caveats. + -T:: --type-profile:: Show data-type profile result instead of code symbols. This requires @@ -128,6 +153,59 @@ REPORT OPTIONS In addition, for report all perf report options are valid, and for record all perf record options. +OVERHEAD CALCULATION +-------------------- +Unlike linkperf:perf-report[1], which calculates overhead from the actual +sample period, perf-mem overhead is calculated using sample weight. E.g. +there are two samples in perf.data file, both with the same sample period, +but one sample with weight 180 and the other with weight 20: + + $ perf script -F period,data_src,weight,ip,sym + 100000 629080842 |OP LOAD|LVL L3 hit|... 20 7e69b93ca524 strcmp + 100000 1a29081042 |OP LOAD|LVL RAM hit|... 180 ffffffff82429168 memcpy + + $ perf report -F overhead,symbol + 50% [.] strcmp + 50% [k] memcpy + + $ perf mem report -F overhead,symbol + 90% [k] memcpy + 10% [.] strcmp + +OUTPUT FIELD SELECTION +---------------------- +"perf mem report" adds a number of new output fields specific to data source +information in the sample. Some of them have the same name with the existing +sort keys ("mem" and "snoop"). So unlike other fields and sort keys, they'll +behave differently when it's used by -F/--fields or -s/--sort. + +Using those two as output fields will aggregate samples altogether and show +breakdown. + + $ perf mem report -F mem,snoop + ... + # ------ Memory ------- --- Snoop ---- + # RAM Uncach Other HitM Other + # ..................... .............. + # + 3.5% 0.0% 96.5% 25.1% 74.9% + +But using the same name for sort keys will aggregate samples for each type +separately. + + $ perf mem report -s mem,snoop + # Overhead Samples Memory access Snoop + # ........ ............ ....................................... ............ + # + 47.99% 1509 L2 hit N/A + 25.08% 338 core, same node Any cache hit HitM + 10.24% 54374 N/A N/A + 6.77% 35938 L1 hit N/A + 6.39% 101 core, same node Any cache hit N/A + 3.50% 69 RAM hit N/A + 0.03% 158 LFB/MAB hit N/A + 0.00% 2 Uncached hit N/A + SEE ALSO -------- linkperf:perf-record[1], linkperf:perf-report[1], linkperf:perf-arm-spe[1] diff --git a/tools/perf/Documentation/perf-record.txt b/tools/perf/Documentation/perf-record.txt index 80686d590de2..612612fa2d80 100644 --- a/tools/perf/Documentation/perf-record.txt +++ b/tools/perf/Documentation/perf-record.txt @@ -227,6 +227,10 @@ OPTIONS '--filter' exists, the new filter expression will be combined with them by '&&'. +--latency:: + Enable data collection for latency profiling. + Use perf report --latency for latency-centric profile. + -a:: --all-cpus:: System-wide collection from all CPUs (default if no target is specified). @@ -336,7 +340,7 @@ OPTIONS -d:: --data:: - Record the sample virtual addresses. + Record the sample virtual addresses. Implies --sample-mem-info. --phys-data:: Record the sample physical addresses. @@ -364,6 +368,11 @@ OPTIONS the sample_type member of the struct perf_event_attr argument to the perf_event_open system call. +--sample-mem-info:: + Record the sample data source information for memory operations. + It requires hardware supports and may work on specific events only. + Please consider using 'perf mem record' instead if you're not sure. + -n:: --no-samples:: Don't sample. @@ -833,6 +842,15 @@ filtered through the mask provided by -C option. only, as of now. So the applications built without the frame pointer might see bogus addresses. + off-cpu profiling consists two types of samples: direct samples, which + share the same behavior as regular samples, and the accumulated + samples, stored in BPF stack trace map, presented after all the regular + samples. + +--off-cpu-thresh:: + Once a task's off-cpu time reaches this threshold (in milliseconds), it + generates a direct off-cpu sample. The default is 500ms. + --setup-filter=<action>:: Prepare BPF filter to be used by regular users. The action should be either "pin" or "unpin". The filter can be used after it's pinned. diff --git a/tools/perf/Documentation/perf-report.txt b/tools/perf/Documentation/perf-report.txt index 87f864519406..acef3ff4178e 100644 --- a/tools/perf/Documentation/perf-report.txt +++ b/tools/perf/Documentation/perf-report.txt @@ -44,7 +44,7 @@ OPTIONS --comms=:: Only consider symbols in these comms. CSV that understands file://filename entries. This option will affect the percentage of - the overhead column. See --percentage for more info. + the overhead and latency columns. See --percentage for more info. --pid=:: Only show events for given process ID (comma separated list). @@ -54,12 +54,12 @@ OPTIONS --dsos=:: Only consider symbols in these dsos. CSV that understands file://filename entries. This option will affect the percentage of - the overhead column. See --percentage for more info. + the overhead and latency columns. See --percentage for more info. -S:: --symbols=:: Only consider these symbols. CSV that understands file://filename entries. This option will affect the percentage of - the overhead column. See --percentage for more info. + the overhead and latency columns. See --percentage for more info. --symbol-filter=:: Only show symbols that match (partially) with this filter. @@ -68,6 +68,21 @@ OPTIONS --hide-unresolved:: Only display entries resolved to a symbol. +--parallelism:: + Only consider these parallelism levels. Parallelism level is the number + of threads that actively run on CPUs at the time of sample. The flag + accepts single number, comma-separated list, and ranges (for example: + "1", "7,8", "1,64-128"). This is useful in understanding what a program + is doing during sequential/low-parallelism phases as compared to + high-parallelism phases. This option will affect the percentage of + the overhead and latency columns. See --percentage for more info. + Also see the `CPU and latency overheads' section for more details. + +--latency:: + Show latency-centric profile rather than the default + CPU-consumption-centric profile + (requires perf record --latency flag). + -s:: --sort=:: Sort histogram entries by given key(s) - multiple keys can be specified @@ -79,6 +94,7 @@ OPTIONS - comm: command (name) of the task which can be read via /proc/<pid>/comm - pid: command and tid of the task + - tgid: command and tgid of the task - dso: name of library or module executed at the time of sample - dso_size: size of library or module executed at the time of sample - symbol: name of function executed at the time of sample @@ -87,6 +103,7 @@ OPTIONS entries are displayed as "[other]". - cpu: cpu number the task ran at the time of sample - socket: processor socket number the task ran at the time of sample + - parallelism: number of running threads at the time of sample - srcline: filename and line number executed at the time of sample. The DWARF debugging info must be provided. - srcfile: file name of the source file of the samples. Requires dwarf @@ -97,12 +114,14 @@ OPTIONS - cgroup_id: ID derived from cgroup namespace device and inode numbers. - cgroup: cgroup pathname in the cgroupfs. - transaction: Transaction abort flags. - - overhead: Overhead percentage of sample - - overhead_sys: Overhead percentage of sample running in system mode - - overhead_us: Overhead percentage of sample running in user mode - - overhead_guest_sys: Overhead percentage of sample running in system mode + - overhead: CPU overhead percentage of sample. + - latency: latency (wall-clock) overhead percentage of sample. + See the `CPU and latency overheads' section for more details. + - overhead_sys: CPU overhead percentage of sample running in system mode + - overhead_us: CPU overhead percentage of sample running in user mode + - overhead_guest_sys: CPU overhead percentage of sample running in system mode on guest machine - - overhead_guest_us: Overhead percentage of sample running in user mode on + - overhead_guest_us: CPU overhead percentage of sample running in user mode on guest machine - sample: Number of sample - period: Raw number of event count of sample @@ -125,8 +144,8 @@ OPTIONS - weight2: Average value of event specific weight (2nd field of weight_struct). - weight3: Average value of event specific weight (3rd field of weight_struct). - By default, comm, dso and symbol keys are used. - (i.e. --sort comm,dso,symbol) + By default, overhead, comm, dso and symbol keys are used. + (i.e. --sort overhead,comm,dso,symbol). If --branch-stack option is used, following sort keys are also available: @@ -201,9 +220,9 @@ OPTIONS --fields=:: Specify output field - multiple keys can be specified in CSV format. Following fields are available: - overhead, overhead_sys, overhead_us, overhead_children, sample, period, - weight1, weight2, weight3, ins_lat, p_stage_cyc and retire_lat. The - last 3 names are alias for the corresponding weights. When the weight + overhead, latency, overhead_sys, overhead_us, overhead_children, sample, + period, weight1, weight2, weight3, ins_lat, p_stage_cyc and retire_lat. + The last 3 names are alias for the corresponding weights. When the weight fields are used, they will show the average value of the weight. Also it can contain any sort key(s). @@ -289,7 +308,7 @@ OPTIONS Accumulate callchain of children to parent entry so that then can show up in the output. The output will have a new "Children" column and will be sorted on the data. It requires callchains are recorded. - See the `overhead calculation' section for more details. Enabled by + See the `Overhead calculation' section for more details. Enabled by default, disable with --no-children. --max-stack:: @@ -442,9 +461,9 @@ OPTIONS --call-graph option for details. --percentage:: - Determine how to display the overhead percentage of filtered entries. - Filters can be applied by --comms, --dsos and/or --symbols options and - Zoom operations on the TUI (thread, dso, etc). + Determine how to display the CPU and latency overhead percentage + of filtered entries. Filters can be applied by --comms, --dsos, --symbols + and/or --parallelism options and Zoom operations on the TUI (thread, dso, etc). "relative" means it's relative to filtered entries only so that the sum of shown entries will be always 100%. "absolute" means it retains @@ -627,6 +646,8 @@ include::itrace.txt[] --skip-empty:: Do not print 0 results in the --stat output. +include::cpu-and-latency-overheads.txt[] + include::callchain-overhead-calculation.txt[] SEE ALSO diff --git a/tools/perf/Documentation/perf-script.txt b/tools/perf/Documentation/perf-script.txt index b72866ef270b..28bec7e78bc8 100644 --- a/tools/perf/Documentation/perf-script.txt +++ b/tools/perf/Documentation/perf-script.txt @@ -239,13 +239,22 @@ OPTIONS i.e., -F "" is not allowed. The brstack output includes branch related information with raw addresses using the - /v/v/v/v/cycles syntax in the following order: - FROM: branch source instruction - TO : branch target instruction - M/P/-: M=branch target mispredicted or branch direction was mispredicted, P=target predicted or direction predicted, -=not supported - X/- : X=branch inside a transactional region, -=not in transaction region or not supported - A/- : A=TSX abort entry, -=not aborted region or not supported - cycles + FROM/TO/EVENT/INTX/ABORT/CYCLES/TYPE/SPEC syntax in the following order: + FROM : branch source instruction + TO : branch target instruction + EVENT : M=branch target or direction was mispredicted + P=branch target or direction was predicted + N=branch not-taken + -=no event or not supported + INTX : X=branch inside a transactional region + -=branch not in transaction region or not supported + ABORT : A=TSX abort entry + -=not aborted region or not supported + CYCLES: the number of cycles that have elapsed since the last branch was recorded + TYPE : branch type: COND/UNCOND/IND/CALL/IND_CALL/RET etc. + -=not supported + SPEC : branch speculation info: SPEC_WRONG_PATH/NON_SPEC_CORRECT_PATH/SPEC_CORRECT_PATH + -=not supported The brstacksym is identical to brstack, except that the FROM and TO addresses are printed in a symbolic form if possible. diff --git a/tools/perf/Documentation/perf-stat.txt b/tools/perf/Documentation/perf-stat.txt index 2bc063672486..61d091670dee 100644 --- a/tools/perf/Documentation/perf-stat.txt +++ b/tools/perf/Documentation/perf-stat.txt @@ -506,6 +506,13 @@ this option is not set. The TPEBS hardware feature starts from Intel Granite Rapids microarchitecture. This option only exists in X86_64 and is meaningful on Intel platforms with TPEBS feature. +--tpebs-mode=[mean|min|max|last]:: +Set how retirement latency events have their sample times +combined. The default "mean" gives the average of retirement +latency. "min" or "max" give the smallest or largest retirment latency +times respectively. "last" uses the last retirment latency sample's +time. + --td-level:: Print the top-down statistics that equal the input level. It allows users to print the interested top-down metrics level instead of the diff --git a/tools/perf/Documentation/perf-trace.txt b/tools/perf/Documentation/perf-trace.txt index fb3d2af33844..c1fb6056a0d3 100644 --- a/tools/perf/Documentation/perf-trace.txt +++ b/tools/perf/Documentation/perf-trace.txt @@ -150,6 +150,11 @@ the thread executes on the designated CPUs. Default is to monitor all CPUs. To be used with -s or -S, to show stats for the errnos experienced by syscalls, using only this option will trigger --summary. +--summary-mode=mode:: + To be used with -s or -S, to select how to show summary. By default it'll + show the syscall summary by thread. Possible values are: thread, total, + cgroup. + --tool_stats:: Show tool stats such as number of times fd->pathname was discovered thru hooking the open syscall return + vfs_getname or via reading /proc/pid/fd, etc. @@ -247,6 +252,12 @@ the thread executes on the designated CPUs. Default is to monitor all CPUs. pretty-printing serves as a fallback to hand-crafted pretty printers, as the latter can better pretty-print integer flags and struct pointers. +--bpf-summary:: + Collect system call statistics in BPF. This is only for live mode and + works well with -s/--summary option where no argument information is + required. + + PAGEFAULTS ---------- diff --git a/tools/perf/Documentation/perf.data-file-format.txt b/tools/perf/Documentation/perf.data-file-format.txt index 010a4edcd384..cd95ba09f727 100644 --- a/tools/perf/Documentation/perf.data-file-format.txt +++ b/tools/perf/Documentation/perf.data-file-format.txt @@ -370,7 +370,7 @@ struct { u32 mmap_len; }; -Indicates that trace contains records of PERF_RECORD_COMPRESSED type +Indicates that trace contains records of PERF_RECORD_COMPRESSED2 type that have perf_events records in compressed form. HEADER_CPU_PMU_CAPS = 28, @@ -602,7 +602,14 @@ struct auxtrace_error_event { Describes a header feature. These are records used in pipe-mode that contain information that otherwise would be in perf.data file's header. - PERF_RECORD_COMPRESSED = 81, + PERF_RECORD_COMPRESSED = 81, /* deprecated */ + +The header is followed by compressed data frame that can be decompressed +into array of perf trace records. The size of the entire compressed event +record including the header is limited by the max value of header.size. + +It is deprecated and new files should use PERF_RECORD_COMPRESSED2 to gurantee +8-byte alignment. struct compressed_event { struct perf_event_header header; @@ -618,10 +625,17 @@ This is used, for instance, to 'perf inject' events after init and before regular events, those emitted by the kernel, to support combining guest and host records. + PERF_RECORD_COMPRESSED2 = 83, -The header is followed by compressed data frame that can be decompressed -into array of perf trace records. The size of the entire compressed event -record including the header is limited by the max value of header.size. +8-byte aligned version of `PERF_RECORD_COMPRESSED`. `header.size` indicates the +total record size, including padding for 8-byte alignment, and `data_size` +specifies the actual size of the compressed data. + +struct perf_record_compressed2 { + struct perf_event_header header; + __u64 data_size; + char data[]; +}; Event types diff --git a/tools/perf/Documentation/tips.txt b/tools/perf/Documentation/tips.txt index 67b326ba0040..3fee9b2a88ea 100644 --- a/tools/perf/Documentation/tips.txt +++ b/tools/perf/Documentation/tips.txt @@ -62,3 +62,7 @@ To show context switches in perf report sample context add --switch-events to pe To show time in nanoseconds in record/report add --ns To compare hot regions in two workloads use perf record -b -o file ... ; perf diff --stream file1 file2 To compare scalability of two workload samples use perf diff -c ratio file1 file2 +For latency profiling, try: perf record/report --latency +For parallelism histogram, try: perf report --hierarchy --sort latency,parallelism,comm,symbol +To analyze particular parallelism levels, try: perf report --latency --parallelism=32-64 +To see how parallelism changes over time, try: perf report -F time,latency,parallelism --time-quantum=1s |